Assignment #2

The purpose of the study presented was to decrease levels of gestational diabetes mellitus in expectant aboriginal women by increasing their physical activity.  A program took place weekly on Wednesday afternoons at the YMCA, a centrally located facility in Saskatoon.  The program had an exercise component which consisted of a warm-up, cardiovascular activities, and a cool-down.  Following the exercises there was a time for socializing where healthy snacks and information packets on a variety of subjects of interest to expectant mothers was provided.

Evaluation of this program could be done in a variety of ways, but the most effective would follow Scriven’s approach.  The evaluation would be summative since the program has been completed.  I would want the evaluator to be made aware of the program’s goals to make sure that he obtains all of the important and relevant information.  A strategy that would work in this situation is having the participants fill out a questionnaire.  If there weren’t a great number of questionnaires being returned, follow-up phone call reminders might be in order.  Another strategy would be to interview the participants.  New parents often don’t have a lot of time and a questionnaire might not be viewed as a priority, whereas much as an interview might.  Either strategy could obtain the same information.

On important aspect of the evaluation of this program would be to accurately draw conclusions based on the results obtained.  Important questions include:

• Does the sample accurately represent all expectant aboriginal women, or is the group defined more specifically?

• What aspects of the program could be improved to make it more effective?

Assignment #1

In searching for completed program evaluations online, I found one that I thought would be particularly interesting.  It is an evaluation of the Fulbright Scholar Program.  I’d been aware of the program for some time, knowing the prestige with which the program and its participants are regarded.  The program offers grants for students, scholars, teachers, professionals, scientists, and artists to study and work abroad.  Many participants have had very successful careers.  The ones I was aware of when finding the report are Julie Taymor (movie and stage director), and Renee Fleming (opera singer).  Upon researching the program further, I found many other familiar names, including many Nobel and Pulitzer Prize winners.  

The program evaluation can be found at: http://exchanges.state.gov/media/pdfs/ope/completed/execsummaries/usscholar_fulbright.pdf

The evaluation was completed by SRI International, a non-profit research institute located in California.  They sent a survey to a stratified random sample of 1004 U.S. Fulbright Scholar alumni who participated in the program between 1976 and 1999.  80% of those contacted completed the survey.  

The introduction section of the assessment report clearly states the goals of the Fulbright Program, and gives an overview of it.  As I read the goals, I considered that their broadness might make measuring success in them quite difficult.  For example, how easy is it to measure how well a program “increase[s] mutual understanding between the people of the United States and the people of other countries”?  As well, the goals of the assessment were to ascertain the broader impacts of the program and assess and document the contribution of the experience of the program on its alumni.  Upon reading the full report, I am convinced that the survey was very well designed and has successfully met its mandate.  

The evaluation does not specifically state what questions were asked on the survey, but the responses are grouped in sections that clearly relate back to the goals listed at the beginning.  In many cases, qualitative and quantitative measures are employed.  The responses lead me to believe that some questions were multiple choice questions, and others were open questions where the participants would write their answers in sentences.  Some sections contain subsections to show different ways the goals were met.  For each goal, a variety of bullet-points relating directly to that goal are listed.  In every case, the points stated appear to me to be quite comprehensive.  In the case of each goal, I am convinced that the data gathered are accurate, and that the data listed in each section is relevant to seeing that that goal is met. 

I have trouble finding too many weaknesses with this evaluation.  I wonder if part of the impressiveness of this report stems from the fact that those who would have won scholarships are likely to be very bright, well-spoken, well-written individuals who can articulate their thoughts with ease.  Although that could contribute, what impresses me the most is that the report is organized to address each of the goals of the evaluation and then shows a variety of different evidence that relates to each goal, either showing that it was met (which happened in most cases) or the degrees to which it was not.  I would consider this to be a very good example of a program evaluation. 

New Blog

This blog was created to fulfill the requirements of Graduate Studies class at the University of Saskatchewan.  The class is ECUR 809 and its focus is Program Evaluation.