1. Target Assessment – Measures Project Team Accountability

Target assessment, which is the ability of the project team to determine how well they are doing in leading the participants to achievement of the targeted results, is the single most important factor of your program delivery.  How well you, your project team and collaborators demonstrate this will determine how effective producers will be in their ability to accomplish the risk management results you have set forth in your proposal. The producers’ ability to achieve the targeted results is the sole indicator of whether or not your project is successful and that is what we are most interested in as we review your progress throughout the program year.

The following outline contains a series of suggestions and questions that a project team may use to lead them through this process:

• Proposal as a Working Document
• Does your program delivery align with the targets listed in the proposal?
• Does your program delivery contain clear goals for helping producers achieve the risk management targets listed in the proposal?
• Be deliberate and focused with your approach, while maintaining a flexible plan of work.
• Outline steps that your project team is taking to measure accountability towards helping participants reach their risk management goals.
• Develop an evaluation tool that will assess your progress after each workshop, seminar, field tour etc.
• Measuring your progress via the Project Steps
• The Project Steps listed in your proposal should be a logical progression that leads your participants to the targeted results. It can also be thought of as your plan of work.
• Do the project steps sequentially lead the participants towards reaching the targeted results?
• Do the project steps need to be revised and/or the plan of work adjusted to accommodate barriers to reaching risk management results?
• At the end of each quarter, determine any additional steps needed to motivate audience (from improved understanding to steps for implementation of risk management strategies).

2. Verification of Producer Results – Measures Producer Achievement

The targets that you have listed in your proposal can be thought of as a series of progressive steps that will lead to improved risk management strategies for producers (producer results):

1. What the participant will learn.
1. Improved understanding, awareness and/or knowledge gain.
2. Participant commitment to apply a risk management tool and/or make a change.
1. Identifying and taking beginning steps towards initiating processes that will lead to implementation of  improved risk management strategies such as:
   1. Identify new markets.
   2. Begin to write a business/marketing plan.
   3. Set up a record keeping system.
3. What the participant will achieve and apply.   
1. Improved decision making skills.
2. Development and implementation of knowledge/skills learned and/or adoption of improved risk management strategies such as:
   1. Completion and use of a business/marketing plan.
   2. Use of a record keeping system to improve financial risk.
   3. Establish and use a communication plan as a succession planning tool.

3. Verification Methods –

How you verify your targets can occur in a variety of ways. However, the method(s) and/or tool(s) that you use must be able to show (measure) how many participants met the targets listed in your proposal and how it improved their ability to manage risk.  In short, the only measure of success is whether the producer has improved their ability to manage the financial risks associated with their agricultural business.

The following guidelines can help to improve the response rate of your participants and/or increase their willingness to participate in a survey instrument whether through an evaluation, one on one consultation, phone interview, email, and mail in survey or through another verification method of your choosing.   

• Prepare your Audience
  
  
• Obtain a pre-agreement from participants to fill out an evaluation. (Let them know that their federal dollars are working for them).
• Hand out the survey instrument 5-10 minutes before the end of the session so that participants do not feel as if they are being asked to take extra time. (Have second instructor and/or assistant distribute instrument while the first instructor finishes the presentation).
  
  
• Length and Content of Verification Tool
  
  
• The tool you choose to verify the targets should be no more than 1-2 pages in length.
• Frame the questions around topic relevance in a manner that will show whether participants will incorporate the risk management practices and/or use the risk management tools in their decision making processes.
  
  
  1. Begin with questions that determine level of awareness/ understanding of material presented.
  2. These should specifically lead to questions that ask for a direct commitment to the changes/practices you are making them aware of. (Get to the heart of the matter – what changes do they plan to make and by when?)
  3. N.B. – Any other information that you ask for in your survey is secondary and needs to placed at the end of the instrument; i.e. topics for future workshops, occupation, suggestions for improvement etc.
     
     
• Types of Verification Methods and Tools
  
  
• Pre/Post or Post Testing for short term knowledge gain (after each workshop).
• Written Evaluation/Survey that measures both short and long term knowledge gains – this could include an email or mail in survey 0-6 months after project completion.
• One on one consultation – used to determine participant commitment and assist with progress towards the development of risk management tools (writing business/marketing plans, communication/estate plans etc.).
• Phone interviews – may be conducted in the same manner as a consultation or to ask a series of questions (similar to a written evaluation).
• Face to face interviews – to ask a series of questions (similar to a written evaluation).
• Survey Monkeys – survey software for creating professional online surveys (similar to a written evaluation).
• Audience Response Systems (ARS). This is a relatively new technology using a participatory and interactive technique for gathering real time responses from participants in a classroom setting.
• DOTS – a visual assessment technique that can quickly summarize a group’s understanding of the topics being presented and discussed.
• Collecting individual stories and testimonials from participants – Throughout the course of your program delivery whether it be through workshops, seminars, field trips etc., record the participants’ stories that confirm their understanding and use of the risk management tools/practices being taught.  You may discover that participants are already implementing or have committed to additional risk management strategies that were not listed as risk management targets in your original proposal.  Subjective indicators (stories) can help to quantify producer results.

Whatever combination of verification methods and tools that you use to measure risk management results, from simple yes/no evidence, to collection of subjective data, begin with the targets in mind (what you want your participants/producers to accomplish and/or achieve).   Place as much emphasis on your process for measuring these results as the results themselves; in other words, the result is only as good as the tool that is used to measure it.

4. Examples

The following Workshop Evaluation is an example of a verification tool that can be modified for your particular audience and the risk management topics being covered.