STEP #1A. Identify needed institutional & constituency partners
> What are "institutional" partners? - Institutional partners are those organizations which share some of the same mission and needs as your own organization. Examples include:
> What are "constituency" partners? - Constituency partners are persons and organizanized groups which are not considered "institutions" but which do have a common mission and needs with your program. Some examples might be:
Involvement of these two kinds of groups does complicate and slow down a planning and development process. Any noneducators on the planning group will need to be educated to be able to contribute to the process. However, their invlovement may help you avoid later challenges to your initiative and funding, and can provide you a broader base of support when you really need it most. For example:
1. When YOU ask for funding for a new teacher support program, you can be seen as self-serving.
2. When parents understand the value of supporting the effectiveness of the new teachers which their own children may have, parents can ask your decision makers for support for the new teacher program, and that program will not be seen as self-serving.
RETURN TO STEP #1 ON THE PROGRAM PLANNER
STEP #3. Select a conceptual framework as a basis for participant & program development.
Interestingly, one of the most common reasons for ineffective programs is a lack of a solid conceptual foundation for the program. What is needed is a theoretical basis for program design decisions.
If, for example, a program intends to produced increased professional development, the program should be designed to implement some well-researched conceptual model for teacher development. The program evaluation process should collect data about participants' placement and subsequent growth on that model of development, and program changes should be designed to improve the participants' growth on that model.
Don't be too concerned about what developmental model to use. Barry can easily suggest an appropriate one to effectively meet your needs.
RETURN TO STEP #3 IN THE PROGRAM PLANNER
STEP #4 - Build links between this program and other existing improvement initiatives
DO NOT design a program that will be seen as "one more thing to do"!
Develop your program so that it serves your organizations EXISTING improvement initiatives, strategic plan, or Board of Education goals. Your intention should be to design your program so that:
If you have concerns about how to achieve this, rely on Barry Sweeny to advise you and help make it happen.
RETURN TO STEP #4 IN THE PROGRAM PLANNER
STEP #5. WHY should you do a search of relevant literature for program & participant best practices?
1. The worst way to learn anything is through a 'trial and error" approach. Using the research and expert knowledge base allows you to quickly identify the best practices that others have already discovered and that are known to produce the results you want to achieve. That saves you the time it takes to discover these through your own experience. Then you can more quickly incorporate these best practices into your program, and you will have a more effective program much sooner.
Yes, finding and using best practices in the program design phase is time-consuming, but time invested early in the process saves a great deal of time later on in trying to make your program more effective.
2. Worried about the time needed to do such a "best practice search" yourself? No problem. Barry has already done such a search, and frequently updates his knowledge of current research & literature for the areas in which he works.
> If YOU want a best practice foundation for your program - Hiring Barry to advise and guide you ensures that the current research and expert knowledge base of best practice is utilized in your program development and improvement process.
> If OTHER DECISION MAKERS want a best practice foundation for your program, then you will need to take TWO steps to save your own time:
RETURN TO STEP #5 ON THE PROGRAM PLANNER
STEP #9. Set measurable long-term program GOALS and shorter-term OBJECTIVES for each of the first three years.
> WHY THREE YEARS? - You know that change takes time, so Barry recommends three years to allow enough time to develop your staff's level of expertise and skill, to changes practices, and to collect data on some early indicators that will show your program is worth continued support. Three years is about how long most decision makers are willing to wait for that demonstration of "worth".
Even if your current support is only a 1-2 year grant, you should still plan for a three year process at minimum, to allow for the time change takes. Showing such responsible thinking demonstrates that you know effective practice, are more likely to develop an effective program, and it will actually increase your chances of getting grants.
> LONG-TERM GOALS? - If you develop plans for the next three years, set a few goals for what you reasonably expect to accomplish by the end of those three years. Be careful though. Your goals are those things for which you agree to be held ultimately accountable.
> SHORT-TERM OBJECTIVES? - Objectives should be set for the intermediate steps you will need to accomplish to eventually attain your longer-term goals. Such objectives allow you to measure annual progress in implementation, changes in knowledge, or maybe even changes in behaviors. Measuring attainment of annual objectives allows you to assess need for mid-course adjustments in your program, and to increase progress toward your goals.
RETURN TO STEP 9. ON THE PROGRAM PLANNER
STEP 11B. What are non-classroom educators?
You may consider school nurses, social workers, school psychologists, and other such staff to be educators, and want to include them as participants and in program evaluation. However, they may feel left out if your language and process target only "teachers".
RETURN TO STEP # 11B ON THE PROGRAM PLANNER
STEP #14A. An explanation of "parameters & standards"
1. "PARAMETERS" - This step is taken to set boundaries around the work to be done so your intentions are more likey to be impelmented during the process. Examples might include statements like:
2. "STANDARDS" - Standards determine the technical quality of a process, instruments, and the resulting data and decisions. The reliability, validity, and fairness of data and conclusions are examples. The more "high stakes" the use of the program evaluation, the more you need quality.
The problem is that quality requires greater effort and time, so quality costs money.
Examples of an evaluation standard might include:
RETURN TO STEP 14A. ON THE PROGRAM PLANNER
STEP 14C. Define audiences for evaluation data, conclusions & recommendations
Would you write the same letter to your mother on Valentines day as you would to your sweetheart? Of course not. Audience makes a big difference in what we say.
That's why, in planning a program evaluation or needs assessment, we need to consider to whom the data and conclusions will be shown.
If you know that you will need to defend your program to a superintendent who is focused on saving money, you may collect different data than your would if your superintendant is focused on student achievement. Knowing this early in program evaluation design can ensure that you collect all the data you will need, but no more than you need.
RETURN TO STEP 14C. ON THE PROGRAM PLANNER
STEP 14D. Discuss & select measurable indicators to evaluate
An indicator is what is actually measured. Often we want information about a factor which is not easy to measure. That's when we select an "indicator" of that factor and evaluate that instead as the best possible alternative.
For example, we may want to know how many students come from families living in poverty. Collecting that information would be very time consuming and some families would not willingly tell us their income level, so "level of income" is what we might want to know, but it may not be a good indicator of poverty for us to use. Instead schools almost always use the number of students who receive "free and reduced lunch' as the indicator of poverty to measure. The data is already provided to schools and so is easy to access and use, AND qualifying for free and reduced lunch shows that a family meets the government criteria for living in poverty, the factor we really wanted to understand in the first place.
Selecting good indicators to assess is a very important step in program evaluation because:
RETURN TO STEP 14D. OF THE PROGRAM PLANNER
STEP # 14E. Check alignment of evaluation indicators to program goals & objectives
It is very easy to create data collection tools which are too big and which discourage responses. It is also very easy to genereate more data than you actually can effectively tally and use. The goal of this step is to ensure that the evaluation process produces data which actually helps the program accomplish its goals, AND that you do not collect more data than can be effectively used.
RETURN TO STEP #14E ON THE PROGRAM PLANNER
STEP #14G . Develop targets for improvement for each indicator.
Improvement "targets" are goals you set for the desired extent of improvement within a specified time.
The targets themselves are valuable as motivators, but they are critical because of what happens in the process of defining and agreeing to measureable targets. As it turns out, that process will ensure that your targets ARE measureable. Having consensus on measureable goals will greatly increase the chances that you will actually attain the goals you set.
Another reason targets are helpful, is that they serve as hypotheses to test for what can be accomplished. The better you get at predicting accurately what you can and cannot achieve, the better your program planning will serve as a guide for implementation. In other words, eventually you will be able to actually accomplish what you predicted you could, and you will not make commitments to deliver results that you cannot attain. Acieving this is very important to the long-term success of your program.
RETURN TO STEP 14 G ON THE PROGRAM PLANNER
STEP #20. Prepare a program proposal
A program proposal is the document you will give to decision makers to explain what you intend to do and the results you expect to attain if they agree to support your program. Barry recommends that you ask for support needed for the first three years of program implementation. That should give you enough time to attain a number of desireable results. However, don't promise more than you can deliver. (Questions about that? Ask Barry for guidance.)
The more clearly you can make a case that ...
1. You know how to deliver specific desired results, and...
2. You are willing to be held accountable for delivering those results,
the greater your chances are for approval.
This strategy suggests that the program proposal should include at least:
RETURN TO STEP 20 ON THE PROGRAM PLANNER