The evaluation of the Media Education, Arts and Literacy (MEAL) project was designed and delivered by the Michael Cohen Group (MCG), an internationally recognized firm specializing in education and youth media research.   MCG employed a quasi-experimental design for the project in which the two San Francisco program schools were compared to two match control schools.

The evaluation objectives were:

• To assess and describe the program teachers' experiences including perceived benefits and challenges in integrating a media literacy education curriculum
• To demonstrate gains over time in critical thinking and creative thinking skills of program students
• To show meaningful effects on student academic performance in language arts and mathematics

The study included 23 sixth and seventh grade program classrooms, with another 24 classrooms serving as the control group.  324 students were evaluated over the two-year program implementation period.

The final evaluation data, released in May 2007 by MCG, confirmed that implementation of media literacy education curriculum produced "significant educational benefits for students."  Utilizing the figural version of the Torrence Test Creative Thinking, a well-validated measure of divergent and creative thinking, researchers were able to show that program student showed "greater gains" and "significantly outperformed students in the control group".Researchers also found that program students showed gains in critical thinking skills. Using a media survey developed by MCG which also utilized a measure of message comprehension and analysis developed by Hobbs and Frost (1996), researchers were able to show that "children who received the Media Arts curriculum showed greater gains in endorsing attitudes that are consistent with a media-literate perspective."

The Test of Written Language (TOWL) and the California Standards Test scores were used to measure gains in language arts and mathematics skills. The evaluation data showed that program students showed "no significant advantage" over the students in the control schools in language arts or math. The researchers point out that these findings do not indicate that the program was not effective, only that "its effects were not strong enough to ‘spill over' to other, less directly-related, domains of the curriculum that are also strongly influenced by other aspects of the children's overall instruction."

Another significant finding was that "dosage" of media literacy education activities had a measurable impact on the students. The two program sites differed in both student body and staff sizes. The faculty participating in the smaller school constituted a larger proportion of the teaching staff. Additionally, the faculty in that school taught single subjects. Therefore, the students in the smaller school were exposed to the media education curricula in multiple subjects through multiple educators. Students at the larger school were most likely exposed to the media education curriculum through only a single teacher teaching one or two subjects. Also, a small group of program students (38 of the 324) from the two sites participated in the project for both implementation years.  This added another level of "doseage" to evaluate.

By comparing the data from the two program sites, researchers were able to show that "students who received the greatest dose of the media education curriculum had the most positive outcomes."

View the final program report: Evaluation of the Implementation and Educational Outcomes of a Middle School Media Arts Curriculum prepared by the Michael Cohen Group.

Read the preliminary evaluation results published in Threshold (Winter 2006), a Cable in the Classroom Publication.