Lessons Learned at Spring Brook Farm An Evaluation of the Farms For City Kids Program



Download 411.01 Kb.
Page2/10
Date conversion14.06.2018
Size411.01 Kb.
1   2   3   4   5   6   7   8   9   10


Section One:



Farms for City Kids Evaluation Portfolio Executive Summary



Introduction


Farms for City Kids is a hands-on experiential educational program for urban students that focuses on imparting life skills and practical learning as students live and work together on an operating dairy farm. The farm conducts week long residential programs for urban students, as well as day programs for local students.


This evaluation focused on the week-long, residential program. In this program, a group of students lives and eats together for a week in an on-farm dormitory. During the week they are responsible for numerous farm chores, including cleaning barns, feeding and caring for large and small animals, working in the garden, and many other activities. The students also have opportunities to participate in physical activities such as hiking. Dynamic academic lessons are interspersed throughout the experience.


According to farm literature, the intent is “…to reward these children with such lasting values as responsibility, self-confidence and the satisfaction of facing and overcoming challenges. By educating city kids about agriculture, something that is so different from their everyday lives, we hope to make an impression that will last a lifetime.”


Evaluation Process and Methods

In spring 2005, the Farms for City Kids staff approached PEER Associates, Inc. with a request to conduct a program evaluation. PEER began the evaluation process by facilitating a workshop with staff and board members to discuss the purposes of evaluation, design an overall framework for the evaluation, and begin the development of a logic model2 for the program. This process of logic modeling helped to clarify what the intended outcomes of the program were and how the staff hoped to achieve them. The logic model was later refined with ongoing input from the farm staff. Though the logic model is an evolving tool, a working draft of the Farms for City Kids logic model can be found in Appendix A.

The evaluation was designed based on the ideas organized in the logic model and ongoing conversations with board and staff, (see Appendix B for the Evaluation Overview.) In order to capture and present a wide variety of data, the evaluation took multiple approaches. The first approach was to analyze existing evaluation data that had been collected by the farm from 2002-2005. The second was to conduct a case study of a single week at the farm. Third, evaluators planned to conduct interviews with a range of teachers and students who had been to the farm. As the evaluation process progressed, staff, board, and evaluators decided to modify the approach, choosing to use surveys rather than interviews in order to contact greater numbers of students and teachers. The analysis of these data sources and recommendations derived from that analysis are presented in this Evaluation Portfolio.
This summary report provides a broad overview and synthesis of the findings from the following sources:


  • Interviews with 8 staff members, 17 students, and 2 teachers at the farm;

  • 17 Pre- and post-surveys of one group of students;

  • 125 teacher surveys collected at the farm from 2002-2005;

  • 669 student surveys collected at the farm from 2003-2004;

  • Student and teacher video interviews collected as part of a fundraising campaign;

  • 35 teacher surveys from 2006; and

  • 162 student surveys from 2006.


Findings and Discussion

The intention of the week-long farm program is to provide the students with a powerful hands-on experience that offers them numerous opportunities to develop personally, socially, and intellectually. The farm creates a safe environment where students work cooperatively and succeed repeatedly in authentic agricultural endeavors that yield tangible and skill-building results. The evaluation data gathered from a wide variety of sources detailed above demonstrate that the program is succeeding in many ways at achieving the following outcomes:



  • increased self-confidence and self-esteem;

  • improved cooperation, teamwork, and conflict resolution skills;

  • appreciation and respect for farm animals;

  • respect for each other and their teachers;

  • understanding and appreciation of agriculture and the natural environment; and

  • appreciation of a healthy lifestyle.

A succinct summary of the findings from each source is presented, followed by brief discussion synthesizing the findings.


Case study

The case study followed one group of students through their experience at the farm, collecting data at various times and through various methods. For the reader unfamiliar with the program, the case study provides a concrete, story-like context for understanding the program outcomes described in the pre-exiting and current survey data sets. Furthermore, the teacher, staff, and evaluator observations as well as student voices presented in the case study offer multiple perspectives for the overall evaluation findings.


The case study was designed to focus on three areas: changes in students’ personal and social skills, changes in their attitudes about food and health choices, and changes in their views on the environment. Students completed a survey prior to their arrival, participated in group interviews on their first and final nights at the farm, and completed a survey on their final full day at the farm. Teachers and staff were also interviewed, and filled out observation forms throughout the week. Data were also gleaned from student journals. Seven months later, the case study students and teachers completed surveys as part of a larger evaluation effort.

Evaluators, teachers, and staff observed students adapting to and thriving in the farm environment, overcoming initial fears, and quickly gaining confidence in their daily duties. The teachers offered many insights into the benefits of the program for their students, including improvements to behavior and attitudes. PS 233 students appeared to make more meaningful gains in their personal and social outcomes than in their attitudes and behaviors about healthy choices (such as diet and exercise) and the environment.


Pre-existing student and teacher surveys

As part of an ongoing, internal evaluation effort over numerous years, the Farms for City Kids staff has collected surveys from teachers and students at the end of their farm week. This pre-existing data offered an opportunity to create a preliminary analysis of program impacts before designing new surveys for the evaluation. Although the format and content of these surveys varied over the years, some questions had been consistently asked.


On the teacher surveys, the question most relevant to ongoing outcome evaluation was, “What noticeable influence did Spring Brook Farm have on your students, if any?” 116 teacher responses were coded and refined into the major themes depicted in the pie chart at left and described briefly below:


  • 41% of responses indicated that students’ social skills were improved (improved relationships, teamwork, volunteerism, etc.)

  • 28% referenced changes to students’ personal skills (self-confidence, self-esteem, responsibility, etc.)

  • 21% mentioned that students’ appreciation for agriculture improved (farm knowledge, affinity for animals, rural lifestyles, etc.)

  • 5% indicated that students’ appreciation for the environment was enhanced

  • 5% of responses included other types of influences on students

On the student surveys, the most relevant question that was available on all of the different survey formats was, “What did you learn about yourself?” 669 student responses were analyzed. The broadness of the question yielded a very wide variety of responses, which were coded, refined, and simplified into the following themes:



  • 41% of responses reported improvements in students’ self-confidence (statements about abilities to accomplish things such as “I can milk a cow.”)

  • 18% noted increases in self-esteem (assessments of self-worth such as “I learned I am a really great farmer.”)

  • 18% reported an improved appreciation for agriculture (farm knowledge, affinity for animals, rural lifestyles, etc.)

  • 16% referenced other personal and social skills (responsibility, respect, teamwork, perseverance, etc.)

  • 7% listed other responses

Although asking different questions, the themes of the responses to the student and teacher surveys are quite comparable and confirm that personal and social development are the most prominent outcomes of the program. Appreciation for and understanding of farming, farm work, and, especially, farm animals clearly emerged as the second most important outcome.


Video interviews

In 2005, students and teachers were interviewed (not as part of this evaluation) to capture testimonials for a fundraising video. In order to extract further value from the interviews and to provide additional data for triangulation of overall evaluation findings, student responses to the following relevant questions were analyzed by grouping the responses thematically:




  • What do you think is the most important lesson you learned on the farm?

  • What did you learn about yourself?

  • Why is a program like Farms for City Kids important?

  • What is one thing that you do (or feel) differently since you have returned home from the Farm?

  • What is something that you or your class can do now to continue using the new behaviors and knowledge that you learned on the Farm?

Analysis of the students’ responses suggests that caring for the farm animals left a strong impression on the students, as it was the most common response to three of the above questions (the most important lesson learned at the farm-28% of responses; why the program itself is important-44% of responses; and something they felt differently about since returning-37% of responses).

Practicing teamwork also appeared prominently as an important lesson (19% of responses) as well as a lesson learned at the farm that could be transferred back to life in the city (37% of responses).


The responses to “What did you learn about yourself?” were dominated by reflections on self-confidence (79% of responses) and self-esteem (13% of responses).


Cleaning up or protecting the environment was a theme across student responses to multiple questions, but was less prominent than those discussed above.
Student and teacher surveys

A new set of written surveys was developed to gather greater detail about the themes that evolved from the analysis of the pre-existing data. Surveys were collected and analyzed from 35 teachers and 162 students.


The first section of the teacher survey consisted of 18 scaled items asking teachers to rate their level of agreement with statements about the impacts they observed on students. Analysis of the responses suggests the following ranking of program impacts on students (note that these responses are presented from greatest to weakest):
1. Improved cooperation and teamwork

2. Increased self-esteem and self-confidence

3. Improved relationships between students

4. Improved student-teacher relationships

5. Improved conflict resolution skills


Most important lesson learned

Percent

reporting


Farm knowledge

22%

Teamwork

20%

Other

17%

Responsibility

14%

Care of animals

12%

Hard work

10%

Self-confidence

5%

The first question on the student survey asked, “What was the most important lesson you learned on the farm?” Distilled into themes (see table at right) it is evident that students are divided between the practical specifics of working on a farm (knowledge, animal care, hard work) and the underlying behaviors they practiced to get the job done (teamwork, responsibility). To clarify the meaning of the table, “22% reporting” means that twenty two percent of the total number of “most important lessons learned” by students pertained to farm knowledge.



New abilities and personal qualities discovered at the farm

Percent

reporting



Farm skills, hard work

34%

Care of animals

22%

Self-confidence

15%

Other

12%

Social skills (leadership, teamwork, etc)

9%

Self-esteem

8%

When asked While you were at the farm, did you discover new abilities or personal qualities?90% of students responded that they had. The abilities and qualities they discovered (see table at left) reflect similar themes to those in the question above. A greater percentage of students offered new abilities than personal qualities.





Personal changes from week at Spring Brook Farm

Percent reporting

Changed relationship to animals

14%

Improved personal relationships


13%

Greater sense of responsibility

12%

Other

11%

New farm knowledge

10%

Self-confidence & self-esteem

10%

Harder worker

10%

Expanded outlook

8%

Teamwork

4%

Improved behaviors

4%

Environmental stewardship

4%

Student responses to the question Do you think your week at Spring Brook Farm changed you in any way?” were less easily synthesized into larger categories (see table at right). The program clearly made an impression on students, as eighty-three percent of students indicated they had changed. Again, the themes are familiar, although this question led some students to remark on improvements in their relationships with their peers.

The recurrence of the same themes across three similar yet differently worded questions (important lessons, new abilities, and personal changes) reinforces the legitimacy of the program impacts apparent in the students.

Yes and no responses to other student survey questions are listed in the table below.




Student Survey Questions

yes

no

Do you think the experience of working in teams at the farm has helped you cooperate or work together more effectively with other kids since the trip?

77%

19%

Do you think that your trip to the farm affected the way you and your classmates get/got along socially?

75%

25%

Do you think the experience of working in teams at the farm has helped you resolve conflicts more effectively with other kids since the trip?

62%

37%

Do you think your trip to the farm affected the way you get/got along with the teacher you went with?

59%

39%

The details of students’ responses to all of the above questions, combined with the responses to open-ended questions on the teacher surveys provided a rich data set out of which many major themes and sub-themes emerged as program impacts. It should be noted that the following outline does not represent any ranked order.



  • Improved social dynamics

    • Improved relationships between students (increased individual friendships, group bonding, social mixing and greater acceptance of others, greater teamwork and cooperation, and greater respect for each other)

    • Improved conflict resolution skills

    • Improved student-teacher relationships




  • Students’ personal development

    • Increases to students’ self-confidence and self-esteem

    • Greater willingness to try new things

    • Greater compassion for animals and others




  • Success for students who struggle academically




  • Other influences

    • Broadening of horizons

    • Lesser outcomes (agricultural and environmental awareness, increased personal responsibility, and behavioral improvements)




  • Lasting value of the farm program

It is evident that the weeklong experience at Spring Brook Farm had numerous positive impacts on the students who participated. Themes emerging from both student and teacher surveys demonstrate that the program is achieving many of its stated goals.



Synthesis of findings

Data from the many sources paint a compelling and coherent picture of what is happening at Spring Brook Farm. Similar themes emerged in all of the data sets described above. There is substantial conceptual overlap in the themes, although they are kept separate to help create a more intricate understanding and analysis of the program outcomes. For example, one student stated “I learned how to milk a cow,” which simply suggests that he acquired farm skills. Answering the same question, another student reflected that “I never thought I could milk a cow but I did, and it was cool.” In addition to providing evidence about farm skills, this statement suggests that the student made gains in their self-confidence and felt greater appreciation for farming.

Despite the subtleties of thematically arranging the many observations offered by students and teachers, it is clear that the program is having a number of very important impacts on the students who participate, and on the teachers as well.

Students were experiencing personal growth in a myriad of ways. The most prominent changes appear to be to students’ perceptions of their own abilities, potential, and personal worth. Data also indicated that students develop greater personal responsibility, compassion, and work ethics.


In addition to these personal skills, students were developing socially on a number of fronts. Development of cooperation and teamwork skills was the primary theme in this realm. Other related skills and values that students developed include conflict resolution, respect for others, improved personal friendships, getting closer with their teachers, and bonding as a group.
The farm itself, and especially the animals, made a lasting impression on the program participants. Even when asked how they changed personally, many students offered examples of farm knowledge, and expressed their deep connection with the animals they worked with over the week. Indeed, it was their daily work with animals that provided for many of the gains in self-confidence and self-esteem.
Finally, less prominent outcomes emerged in the realms of environmental stewardship and appreciation for healthy lifestyles. As the evaluation did not focus great attention on these themes, it is unclear how deep or widespread the impacts are in these areas.



1   2   3   4   5   6   7   8   9   10


The database is protected by copyright ©hestories.info 2017
send message

    Main page