Full version of the report: PDF (768K) | MS Word (2.7MB)

Executive Summary

After-school programs have grown rapidly in recent years, spurred by rising employment rates of mothers, pressure to increase academic achievement, and concerns about risks to children who are unsupervised during after-school hours. The percentage of public schools offering "extended day" programs (which include before- and after-school programs) more than tripled from 1987 to 1999, from about 13 percent to 47 percent.

The federal government's investment in after-school programs has grown rapidly as well. Funding for the 21st Century Community Learning Centers program, created in 1994, rose from $40 million in 1998 to $1 billion in 2002. The program now provides funding to 2,250 school districts to support school-based programs in 7,000 public schools.

Some studies of after-school programs have found that these programs increase academic achievement and student safety, as well as reduce negative behaviors such as drug and alcohol use. However, other studies have found that after-school programs have no effect on—and even worsen—certain outcomes, leading to debate over whether the evidence supports increased investment in after-school programs.

In 1999, the U.S. Department of Education contracted with Mathematica Policy Research, Inc., and Decision Information Resources, Inc., to evaluate the 21st Century Community Learning Centers program. The evaluation team collected student outcome data in five areas: after-school supervision, location, and activities; academic performance and achievement; behavior; personal and social development; and safety. Because the purpose of the 21st Century Community Learning Centers program by law is safe and drug-free learning environments for students that support academic achievement, this evaluation focused on student and school outcomes. It did not explore the full range of parental needs and satisfaction that might be affected by the availability of after-school programs. It did collect parent outcome data on involvement in school activities and employment status.

In its first year of data collection, the team gathered data for roughly 1,000 elementary school students in 18 schools in 7 school districts, and 4,300 middle school students in 61 schools in 32 school districts. The elementary study was based on random assignment, in which outcomes of students assigned to the program were compared with outcomes of students not assigned to the program. The middle school evaluation was based on a matched-comparison design, in which outcomes of students who participated in programs were compared with outcomes of similar students who did not. Findings from these data were presented in the study's first report (hereafter referred to as the "first report"), which was released in February 2003.

For the second year of data collection, researchers gathered additional data in two ways. First, they added more elementary school programs and students. Second, they followed middle school students for a second year, which enabled the evaluation to explore whether there were outcome differences after two years. The results are summarized in this new report, which contains findings from this second year of data collection. A third report will analyze impacts for elementary students after two years.

Key Findings From the Second Year

The findings from the second year of the study are generally consistent with those from the first year. Specifically, the study found

• Supervision by Other Adults Increased. Students in programs were more likely to be with adults who were not their parents after school and less likely to be with their parents or older siblings.
• Self-Care Was Unaffected. Participation in programs had no effect on whether students were in self-care (so-called latch-key children) after school. Multiple definitions of self-care were analyzed with similar results.
• Few Impacts on Academic Achievement. Programs did not affect reading test scores or grades for elementary students. Grades for middle school students in programs were higher in social studies relative to the comparison group but not in English, mathematics, and science. Programs did not increase whether elementary or middle school students completed their homework. Middle school students in programs missed fewer days of school and were more likely to aspire to attend college.
• Elementary Students Felt Safer. Elementary students in after-school programs reported feeling safer during after-school hours. Middle school students did not report feeling safer.
• Mixed Evidence on Negative Behavior for Middle School Students. Some estimates pointed to higher levels of negative behaviors for middle school students, while others indicated no differences between treatment and comparison groups.
• Some Impacts on Parent Outcomes. Parents of participating elementary school students were more likely to report that they attended school events. Other measures of parent involvement did not increase. There was some evidence that programs increased whether mothers of elementary students worked or looked for work. Involvement of middle school parents did not differ between the treatment and comparison groups. No employment difference was observed for mothers of middle school students.
• Few Impacts on Developmental Outcomes. Elementary students were more likely to report helping other students after school. They were no more likely to report being able to work with others on a team, believe the best of other people, or set goals and work to achieve them. Middle school students showed no differences in these outcomes.
• Low Middle School Attendance in Second Year. Two attendance patterns emerged in the study's second year. First, many students who had access to programs in the second year (53 percent) did not attend. Second, among those who did attend, average attendance was low (30 days) and similar to attendance during the first year (33 days).
• Moderate Elementary School Attendance. The first report noted that elementary school students attended programs an average of 58 days in the school year. With five additional sites and a larger student sample, average attendance was 63 days.
• Stable Program Leadership, But High Staff Turnover Between the First and Second Years. Eighty-two percent of project directors who worked in programs during the first year still worked for the programs in the second year. However, two-thirds of the line staff and one-third of center coordinators who worked in programs during the first year of the study were no longer working for the programs in the second year.

Characteristics and Impacts of Elementary School Programs

The two most common objectives of administrators of elementary school programs were to offer students a safe place after school and to help students improve academically. These goals mirror those of parents, who said they enrolled their children in the programs to help them do better in school (79 percent of parents) or to provide "a safe place for my child after school" (63 percent of parents).

Generally programs were open for three hours after school four or five days a week. A typical day included one hour for homework and a snack, one hour for another academic activity such as a computer lab, and one hour for recreational or cultural activities.

Eighty-five percent of the centers offered homework assistance, mostly by setting aside time for students to do their homework. Eighty-five percent also provided academic activities, such as teaching or tutoring, in addition to, or instead of, homework help.

Moreover, programs provided recreational, cultural, and interpersonal activities. Nearly all centers—92 percent—offered recreational opportunities, ranging from unstructured free time to organized sports. Programs also offered dance, drama, and music, and workshops on developmental topics, such as building leadership skills and resolving conflicts with peers.

Attendance at programs was moderate (Figure 1). Students attended, on average, 63 days a year, or about two days a week. Almost one-fourth of the students attended more than 100 days a year and one-fourth attended fewer than 26 days.

Supervision After School.

Students who attended after-school programs were more likely to be with adults who were not their parents, and less likely to be with their parents after school. Students also were less likely to be with an older sibling after school. Programs did not affect the frequency with which students reported "self-care," or the number of days when they were at home after school without a parent, another adult, or an older sibling. Just over one percent of both groups of students said they were in self-care three or more days in a typical week (Figure 2).

Academic Achievement. Students attending after-school programs scored no better on reading tests than their peers who did not participate; nor did their grades in English, mathematics, science, and social studies increase (Figure 2). In addition, there were no statistically significant differences between the two groups of students in time spent on homework, student effort in class, preparation for class, and absenteeism; and, according to teachers, program students were less likely to complete homework often.

Safety After School. Programs improved students' reported feelings of safety after school; 1.5 percent of participants, compared with 4.5 percent of nonparticipants, reported feeling “not at all safe” after school (Figure 2).

Developmental Outcomes. Programs had few impacts on developmental outcomes. For example, treatment group students were no more likely to report getting along with others their age, to rate themselves highly on working with others on a team, or to be able to set goals and work to achieve them than nonparticipants. Students in programs were more likely to report helping other students after school.

Negative Behaviors. Students in programs were no less likely than students in the control group to be suspended, to receive detention, or to be sent to the office for misbehaving. Students in programs were as likely as control students to report negative behaviors, such as breaking things, arguing with parents, or giving teachers a hard time.

Parent Outcomes. Parents of students in programs were more likely to attend after-school events in schools. There was no effect on parents attending parent-teacher organization meetings or school open houses, or volunteering at school. There was some evidence that programs increased whether mothers of elementary students worked or looked for work. Mothers of students in programs were more likely to be in the labor force (working full time, part time, or looking for work) than mothers of control students.

Subgroup Impacts. The study looked at subgroup impacts for elementary students but found few groups with significant impacts. Students from two-parent households had larger impacts on some outcomes than students in single-parent households, but after controlling for membership in other subgroups, many of these impacts were no longer significant.

Characteristics and Outcome Differences of Middle School Programs

During the second year of the study's data collection, program administrators indicated that their major objectives for programs serving middle school students were to help students improve academically and to provide a safe place for them after school. About 80 percent of centers offered homework sessions and 60 percent offered other types of academic assistance, such as additional help in language arts or mathematics. The emphasis on academics increased from the first to second year, according to site visitors, principals, center coordinators, and project directors. While our site visit data cannot confirm this shift, there clearly was a perception that centers were focusing more on academic activities.

Programs experienced considerable staff turnover during the 2 years of the study. Two-thirds of the staff did not return in the second year; almost one-third of the schools where centers were located had a new principal, and one-third had a new center coordinator. Only about 20 percent of programs had a new project director. Staff most commonly cited the demands on time that after-school work posed rather than pay as the reason for not returning.

Program attendance was much lower in the second year, averaging just 8.8 days. This was in large part because many students—59 percent of the program group—transferred to high schools or other middle schools that had no 21st Century programs. Among the 41 percent of the program group who had access to the program in the second year, 47 percent attended at least 1 day; for the year, their attendance averaged 30 days. This is similar to the average number of days attended in the first year (33 days). Ten percent of participating students attended more than 75 days and 59 percent of participating students attended fewer than 26 days (Figure 3). Week-to-week attendance patterns also were similar to first-year patterns.

Supervision After School. The program group was less likely to be with siblings than the comparison group. There were no differences in self-care, with roughly 19 percent of participants and nonparticipants indicating that they were not with an adult or older sibling three or more days a week after school.

Academic Achievement. There were few differences between the program and comparison groups on academic outcomes (Figure 4). The program group had higher grades in social studies. Other outcomes—including grades in mathematics, science, and English, as well as teacher reports of achievement—did not differ. The level of homework completion also did not differ.

Safety After School. There were no differences between the program group and comparison group in feelings of safety after school.

Developmental Outcomes. The program group was more likely than the comparison group—82 percent versus 79 percent—to expect to graduate from college. No differences were observed in other developmental areas.

Negative Behaviors. Findings on one of several drug-use questions indicated that the program group had a higher incidence of drug use (use for both groups was low). There were no differences on the other measures of drug use. There were mixed findings on other measures of behavior. Treatment students were more likely than comparison students to report breaking things on purpose and had higher values on an index of negative behaviors, but there were no differences on other outcomes such as punching someone, stealing, selling drugs, or getting arrested.

Parent Outcomes. No differences were found in parent involvement.

Subgroup Impacts. The study examined six subgroups: (1) grade level, (2) whether students had low or high reading test scores at baseline, (3) whether students had low or high discipline problems at baseline, (4) student race and ethnicity, (5) student gender, and (6) whether students lived in two-parent or one-parent households. None showed distinct patterns of difference, with one exception: students with low grades (at baseline) had more positive impacts than did students with high grades. Reasons for the difference were not clear.

Comparison of Findings of the First and Second Reports

The comparison below is presented separately for elementary and middle school students because the basis for differences in findings differs for the two groups. For elementary school students, differences in findings between the first and second reports are due to the addition of new sites to the study; for middle school students, differences in findings relate to an additional follow-up year.

Elementary School Students

Supervision and Location After School. Both reports found that elementary school students attending programs were less likely to be supervised by parents and siblings and more likely to be supervised by other adults. They also were more likely to be at school and less likely to be at home during after-school hours.

Academic Achievement. Both reports found that programs generally did not improve academic outcomes such as grades or test scores. In the first report, elementary school students had higher grades in social studies but not in English, mathematics, or science. In the second report, grades were not higher in any of the four subjects. Both reports found no difference in reading test scores. Both reports found homework completion was lower; the second report’s finding was statistically significant.

Safety After School. Both reports found that students reported feeling safer after school; only the second report’s finding (based on a larger sample size) was statistically significant.

Social, Emotional, and Developmental Outcomes. Both reports found that students were more likely to help other students after school. There were no differences in other outcomes, such as the extent to which students reported getting along with others or setting goals and working toward them.

Negative Behaviors. Students were equally likely to be disciplined for bad behavior, be suspended, or receive detention.

Parent Outcomes. Both reports found that parents were more likely to attend after-school events, to help their children with homework, and to ask their children about class.

Subgroup Outcomes. Neither report found noteworthy patterns of subgroup outcomes. In the second report, students from two-parent households had larger impacts on some outcomes than students from single-parent households, but these differences were no longer significant after controlling for students’ membership in other subgroups. This subgroup was not examined in the first report.

Middle School Students

Supervision and Location After School. The first report found that program students were more likely than comparison-group students to be supervised by other adults and less likely to be supervised by parents or siblings. Students also were more likely to be at school and less likely to be at home during the after-school hours. In the second report, the only significant findings were a reduction in being supervised by siblings and an increase in being at school during the after-school hours.

Academic Achievement. Both reports found few differences in academic outcomes. In the first report, students had higher grades in math but not in English, science, or social studies. In the second report, students had higher grades in social studies but not in English, math, or science. Both reports found no differences in homework completion. School absences were lower for treatment students relative to comparison students in both reports.

Safety After School. Both reports found no differences in feelings of safety after school.

Social, Emotional, and Developmental Outcomes. Both reports found an increase in students who expected to go to college.

Negative Behaviors. Both reports found mixed evidence on negative behaviors. Some estimates indicated that program students were more likely to engage in negative behaviors and others showed no difference.

Parent Outcomes. The first report indicated that parents were more likely to attend open houses, parent/teacher organization meetings, and after-school events, and more likely to volunteer at school. The second report found no differences in parent involvement.

Subgroup Outcomes. The first report found some increases in academic outcomes for black and Hispanic middle school students. The second report did not find such increases.