Part I. Five Key Themes of the CSR Program Guidance

All four of the regional symposia lasted a day and a half and had a similar agenda. The main portion of the first day's agenda addressed the requirements of the draft CSR program guidance. However, rather than walk participants page by page through each point contained within the guidance, the CSR staff organized the session around five major themes:

• CSR program purposes and possibilities
• Raising the bar for CSR
• Maintaining program integrity
• Money
• Special considerations

By approaching the requirements through these themes-with greatest emphasis on the first three-the CSR program staff sought to model, for state coordinators and other participants, a thoughtful approach to CSR that extends beyond basic compliance. The following sections summarize the portions of the guidance reviewed and discussed around each of these themes.

Theme One: CSR Program Purposes and Possibilities

Enactment of the No Child Left Behind Act (NCLB), signaled important changes for what is now known as the Comprehensive School Reform (CSR) Program. The original program, known as the Comprehensive School Reform Demonstration (CSRD) Program, was established in 1998 with $145 million in funding and governed by language in appropriations legislation and the accompanying conference report. (See A-4.) Under NCLB, however, the CSR program is no longer designated as a "demonstration" and FY2002 funding increased to $310 million. The program was authorized by Title I, Part F and the Fund for the Improvement of Education.

One element that did not change in the new law is the CSR program's primary purpose: to raise student achievement in the lowest-performing, highest-poverty schools where students and teachers are most in need of assistance. The program fulfills this mission by providing the highest-need schools with seed funding to build sustainable, schoolwide reform using the CSR framework. (See F-13.) CSR has always made high-poverty schools the priority, but now that it is authorized under Title I there is even greater emphasis on ensuring that the most needy schools are at the top of the list for the Title I portion of CSR funding. To do this, SEAs and LEAs must identify schools most in need, and must demonstrate a commitment to assisting them in their reform efforts. The program calls for CSR to be integrated and aligned at the State and local levels with other federal initiatives targeted at high-poverty, low-performing schools. For example, the CSR program requires LEAs and schools to coordinate resources from all initiatives and weave their strategies and goals into the comprehensive school reform design. (See A-2.)

The possibility inherent in the CSR program also remains consistent in the new law. As in the original program, the CSR framework of eleven components represents an approach to schoolwide improvement that is significantly different and more compelling than past reform efforts. Rather than adding another layer of reform to a school, CSR is based on the premise that "unified, coherent, integrated strategies for improvement, knitted together into a comprehensive design, will work better than the same strategies implemented in isolation from each other". (See A-1.) The eleven components, which define what makes a CSR program truly comprehensive, expand upon the nine components of the original legislation. The CSR program staff explained that the numerical order in which the eleven components (provide link to them) are presented in the legislation does not hold any particular significance, and encouraged participants to reorganize them in ways deemed most useful.

Though the fundamental purpose and possibility of the CSR program has remained consistent, the newly authorized program contains requirements that are intended to significantly strengthen how comprehensive school reform is planned, implemented, and sustained. The principal changes include the following (see A-4):

• Emphasis on scientifically based research or strong evidence of success;
• New responsibilities and accountability for SEAs;
• A revised dual priority for the award of sub-grants; and,
• An additional component in the CSR framework requiring that a comprehensive school reform program provide support for teachers, principals, administrators, school personnel staff, and other professional staff.

In response to questions at the first symposium in Washington DC, the CSR program staff prepared a glossary of terms for participants at subsequent symposia to review during the first part of the day. The "CSR Terminology" worksheet provided definitions of terms used in the program guidance:

1. Program - the purposeful and comprehensive combining of effective, research-based strategies to produce a greater impact than those same strategies could produce in isolation; the totality of what the school is doing, encompassing what happens from the time a student walks in the door until he or she leaves at the end of the day, whether it is a regular day or an extended school day or year.
2. Comprehensive Design - synonymous with "program"; an integrated, comprehensive approach to whole school reform; in the case of CSR, the design (component 2) encompasses the additional ten required components.
3. Strategy - unit of a program; an individual action or set of actions that is known to be effective in the teaching and learning process.
4. Method - a collection of strategies that comprise a regular and systematic way of doing something, such as a method of teaching reading.
5. National Model, Regional Model, or Locally Developed Model - an organized, research-based approach to school reform developed independently of a particular school or district (national model) or within a local school or district (locally developed model); a model should contain a theoretical base, articulated goals, and a fixed set of recommended strategies and methods.

In short, the terms "program" and "design" are both used to reference the full scope of a school's comprehensive school reform. "Strategies," "practices," and "methods" refer to how schools fulfill each aspect of their comprehensive school reform program. The CSR program staff has often de-emphasized the term "design" because that term is often used to describe school reform "models." Unlike the original CSRD legislation, the reauthorized CSR program contains minimal references to reform models. The CSR program staff explained that with increased emphasis on scientifically based research, it is important that schools focus on creating a comprehensive design made up of effective practices and strategies. Reform models, which in the past have been too frequently misconstrued as the "silver bullet" for comprehensive school reform, can be a part of the comprehensive design, but cannot replace it.

A "paradigm shift" is required at all levels to move away from the misconception that implementing a model equals the "fix" or the complete CSR design. In fact, there is no requirement that schools implement a model at all. If they choose, they may implement more than one model-so long as the eleven components are addressed and the pieces are knitted together in a coherent way.

The CSR program staff introduced these definitions and key aspects of change in the CSR program to help State coordinators think about them not as burdensome new requirements, but as opportunities to renew and fulfill the purpose and possibility of CSR for students in the nation's lowest-performing schools. It is crucial that CSR State coordinators understand both the letter and spirit of the law, and that they then convey these ideas to practitioners at the LEA and school levels.

In the sections that follow, these new aspects of the CSR program are explored in greater depth.

Discussions
The CSR program staff explained to symposium participants that, due to delays in the legislative process, they do not yet have answers to all questions regarding the CSR program. One "unknown" was whether CSR funds would be available under the Fund for the Improvement of Education (FIE) in the future. At the time of the symposium, no FIE funds had been allocated within the 2003 federal budget and in fact no Congressional action had occurred on the FY2003 appropriation cycle. (See "Money" for more information).

Just prior to the first symposium, however, the CSR program did learn that the new requirements take effect for all new and continuing awards on July 1, 2002. The "unknowns" raised many questions among state coordinators regarding how they should proceed with their application and award cycles. Participants expressed concerns that schools in their second or third year of CSR implementation are being asked to do too much by complying with the new requirements. In response, the CSR staff reminded coordinators that CSR is, in its essence, about continual, long-term improvement; thus, the new requirements should not be seen as additional burdens but as opportunities for SEAs and LEAs to help schools strengthen their CSR efforts.

During the discussion of program purpose and possibilities, participants also raised questions about how the CSR program and other federal programs, particularly Reading First, could be coordinated. The CSR program staff explained that programs should complement one another; for instance, SEAs can decide to apply the Reading First program standards to CSR program requirements to help raise the bar for reading instruction within CSR schools.

Participants grappled with understanding the expectations inherent in component six-the new requirement for schools to provide support for teachers and school staff. Though the guidance does not provide specific mandates on how this support must be provided, the CSR program staff pointed out that support isn't limited to money, and can include providing teachers with common planning time and distributing leadership among teachers.

To address component six, schools must identify support around all components of their comprehensive design. One way schools can do this is by considering school staff needs in the selection of external providers of technical assistance (component eight). Although many schools use model providers as a key source of technical assistance, the providers typically provide assistance only with implementation of their model, rather than for all CSR components. In such cases, applicants must identify additional technical assistance providers who will support other aspects of their school program. Model providers may also be compelled by component six to provide more comprehensive assistance to teachers and staff. In one state, model providers are considered the external technical assistance provider, but the schools also find an external technical assistance provider for the other components not addressed by the model. The CSR program staff noted that schools should be encouraged to use the standards for quality professional development, such as those as outlined in Title IX, Part A Section 9101, as a helpful reference point in addressing staff support.

While it is the school's responsibility to identify support around all components, it is the SEA's and the LEA's responsibility to hold them accountable for doing so. The CSR program staff emphasized, above all, that component six is about building capacity at the school level, and thus is critical for ensuring the sustainability of comprehensive school reform programs beyond the three-year award cycle.

Theme Two: Raising the Bar for CSR

The theme "raising the bar" as discussed in the program guidance refers to the intensified focus on student achievement and on strengthening the quality of CSR programs in the context of NCLB.

For SEAs, LEAs, and schools, new CSR program requirements were designed to help create stronger and more accountable schoolwide improvement efforts-to "raise the bar for CSR"-in three ways: by requiring research-driven decision-making; increasing accountability; and, increasing flexibility.

Research-driven decision-making
One significant change in the CSR program is the emphasis on using research that is scientifically based when adopting programs and practices. As defined in section 9101 (37) of the ESEA, scientifically based research (SBR) is "research that involves the application of rigorous, systematic and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs..."(See B-2.). Rigorous, scientifically based research and credible evidence of increased student achievement comprise the gold standard that schools and SEAs must use in weighing the value of program elements.

The emphasis on SBR appears in two distinct components of the CSR program framework. Component one requires that SBR be applied specifically to instructional methods and practices. Component one requires schools and LEAs to examine the teaching and learning practices embedded in their comprehensive school reform program, particularly in math and reading. The CSR program staff underscored that the program elements addressed in component one must be grounded in scientifically based research. If a school adopts an externally developed reform model, it is responsible for looking critically "inside" the model to ensure that it incorporates scientifically based methods and practices.

Component eleven defines a broader interpretation of the SBR requirement. It says that schools are required to implement comprehensive reform programs that either have been found to significantly improve the academic achievement of students or demonstrate strong evidence that they will do so. Thus, the school must make the case that there is sufficient research or strong evidence indicating that its CSR program-the sum of all its components-ties together to transform school practices in a way that will significantly improve student achievement.

Increased accountability
Requirements that increase accountability for SEAs are another key way in which the new CSR requirements raise the bar to ensure that quality programs are implemented. Though SEAs, LEAs and schools were required to conduct a program evaluation under the original CSRD program, the new CSR now requires SEAs to submit this evaluation to the U.S. Department of Education on an annual basis. These evaluations must address both student achievement and CSR program implementation. SEAs are thus held more accountable for ensuring that school progress is made, measured, and reported.

Annual SEA evaluations also play an important role in determining continuation funding. Some SEAs use outside evaluators who look at school structures; others conduct impact surveys to determine whether schools are meeting benchmarks; elsewhere, school liaisons give feedback. One state uses a "portfolio of readiness" system: districts use portfolios to demonstrate their progress in supporting schools, and schools create portfolios that demonstrate evidence of student achievement. The SEA's responsibility is to synthesize these various sources of data to paint a picture of change and progress.

The CSR program staff reviewed the guidance concerning CSR continuation awards. Before making these awards, SEAs must determine whether sufficient implementation progress has been made.

CSR grant funds for years two and three are awarded if the SEA determines that a school is adequately implementing its CSR plan. If the SEA determines that a school has failed to implement its CSR plan, the school can reapply for funds if certain conditions are met. The school must correct the deficiencies that led to the withdrawal of funds, complete the CSR planning process and needs assessment, and participate in the CSR funding competition on the same basis as all other applicants. (See G-15.) If the State decides progress is not sufficient to continue funding, it must use those funds to establish new CSR programs or support CSR activities at other existing CSR sites. (See F-20 and F-21.)

To evaluate implementation, SEAs may wish to use such indicators as the following: 1) measures of the school staff's awareness of and involvement in reform; 2) the amount and quality of professional development for teachers; and 3) the degree to which the reform plan is being implemented in every classroom in a CSR school. A school may, after consultation with its LEA and with the permission of the SEA, change its CSR design and keep its funding. (See G-14.) Revising a CSR plan is permissible and even encouraged if the change is the result of the school's careful review of data indicating that its current program is not working.

Increased flexibility
In addition to increasing accountability, the CSR program raises the bar by increasing flexibility. For example, States may apply for funds using the Consolidated State Application or through a CSR program application, which can be requested directly from the CSR program office. Some flexibility in timing grant competitions (see G-5 and G-9.) is also available. The CSR program also emphasizes the need for districts and schools to exercise flexibility in resource reallocation, allowing schools to design CSR programs that will access resources from multiple sources, thus increasing the likelihood that CSR awards will seed and spark school improvement that is sustained over time.

Discussions
The CSR program staff acknowledged that there are CSR program components, such as the requirement for parental and community involvement, that do not yet have strong scientific research behind them. In the absence of SBR, SEAs must demand strong evidence and take into account professional standards for quality practice.

The CSR staff noted that portions of the NCLB can also serve as an important resource for schools trying to determine the value of a program or practice that does not have SBR-quality findings to substantiate it. For example, Section 9101 of the reauthorized ESEA provides an outline of standards of quality for professional development, and Section 1118 offers criteria for quality parental involvement.

The CSR program staff reiterated that SEAs must look at places where there is "gold standard" research, such as for teaching and learning, and expect schools to draw upon this base of knowledge. When the best available evidence meets only a "silver" or "bronze" standard, SEAs should keep in mind that the objective is to begin moving practitioners towards a higher standard for making research-based decisions.

The participants discussed the implications of "raising the bar" for all levels. At the SEA level, the CSR program now must be coordinated with the State accountability plan and other federal and state initiatives. State agency staff also noted that LEAs must work more closely with schools in the planning and application process, as well as through the phases of implementing and sustaining school CSR programs.

Several participants posed questions about the elements that must be addressed in the annual evaluation. (See I-3.) The CSR program staff explained that student achievement data reported for CSR should be consistent with data reported for the Title I program evaluation. Further, while LEAs are permitted to use a compilation of school level evaluations to fulfill their evaluation requirement, it is not acceptable for SEAs to submit a collection of data gathered by schools and LEAs. SEAs must conduct and submit their own evaluation of grantees to fulfill this requirement.

State coordinators also had concerns about whether SEAs must continue to report data for the national longitudinal evaluation being conducted for the U.S. Department of Education by Westat. Coordinators wondered if these data will still be relevant now that SEAs may be using different measures for adequate yearly progress (AYP). The CSR program staff replied that, even though the Westat contract is now over, they still need data from SEAs and will continue to collect and analyze it until the study is complete. Regardless of changes in AYP measures, the achievement data will still be useful for the completion of this study.

Theme Three: Maintaining Program Integrity

Many aspects of the CSR program are intended to strengthen SEA-level commitment to maintaining the integrity of comprehensive school reform programs. By drawing out this theme within the program guidance, the CSR program staff encouraged state coordinators to see their role more broadly than that of maintaining legal and technical compliance. In addition to understanding what they must do, SEAs must also keep in mind the reasons why they are doing it; by asking questions such as, "would this proposed program really increase the likelihood of changing kids' lives for the better?" SEAs can help schools create strong CSR programs that are not only mechanically correct, but that fulfill the purpose of the CSR program.

To obtain federal CSR funds, SEA applications must describe their state's dissemination, evaluation and technical assistance strategies. SEAs must provide descriptions of the process and selection criteria they will use to structure competitive grants, and how they will ensure that only high-quality, well-defined, and well-documented programs are funded. To meet these demands, SEAs must establish structures for ensuring that awards go only to programs that are comprehensive (i.e., address all eleven components) and that use classroom practices that are supported by high-quality research and that include ongoing external technical assistance from providers with a successful track record, financial stability and the capacity to deliver high-quality materials, professional development, and implementation support. (See D-2.)

Comprehensive Program Grounded in Research

The SEA is also responsible for ensuring that funded programs are grounded in scientifically based research and in best practices identified through high quality research. Though the second part of the day was devoted to a more in-depth discussion on the topic of research, during this discussion on program integrity participants were encouraged to cultivate a straightforward message about using research to develop a CSR program that will resonate with, rather than frustrate, school practitioners. The CSR program staff suggested that when talking to schools about this component of the program, it is important not only to explain the requirement to use scientifically based research, but to ask practitioners to draw upon their sound professional judgment when "gold standard" research on a topic does not exist. It is important to emphasize that a CSR program must be grounded in the best available evidence of what works, rather than on intuition and hunches. By working together to make decisions based on the high quality research that is available, primarily that which addresses classroom practices and programs, practitioners strengthen their professionalism and investment in the implementation of their CSR program.

LEA Commitment to the Neediest Schools
Although SEAs may establish additional priorities for making awards, the new CSR program legislation requires that SEAs give priority to LEAs or consortia of schools that plan to use CSR funds in schools identified as being in need of improvement or corrective action (as defined by section 1116(c) and that have the support of their LEA in their reform effort. This priority for distributing funds exemplifies one way in which the CSR program has become more closely aligned with the new Title I accountability system.

Recognizing the low-performing schools' need for district support, SEAs must also give priority to applications that demonstrate the LEA's commitment to assisting grantee schools with budget allocation, professional development, and other strategies necessary to ensure that the comprehensive school reforms are properly implemented. (See theme four: "Money" for more detailed information regarding the requirements for sub-grant award priorities.) Though LEAs cannot, according to the legislation's definition, be counted as external technical assistance providers to schools, the intent of this provision is to recognize the importance of local support and expertise for the success of CSR. This important new requirement provides SEAs with a lever to help schools get the LEA support they need for successful CSR program implementation. (See F-2 and F-3.)

High Quality Technical Assistance
To maintain program integrity, SEAs must also ensure that external technical assistance providers identified in CSR applications possess the capacity and expertise to support the school's CSR program. To do so, SEAs must develop a system for obtaining assurances from technical assistance providers that they are financially stable, have a track record of success in similar schools, and possess the capacity to deliver high-quality materials, professional development and implementation support to the school. (See D-3.) SEAs must also verify that a funded school uses CSR funds only to support the costs associated with implementing its school reform program.

Discussions

Theme Four: Money

Though the program guidance provides clear information on requirements regarding the use of CSR monies, the CSR staff did want to convey several important ideas not immediately evident in the guidance. First, the CSR staff encouraged state coordinators to see the CSR program as a significant federal investment that directly impacts reform at the school level. Further, a clearly defined priority in the new legislation provides an extremely important driver for working with those schools most in need of improvement.

The $310 million FY2002 budget for the CSR program was appropriated under two separate authorities, or funding streams: under Title I, Part F, and under the Fund for the Improvement of Education (FIE). The intent of the legislation is clear that the bulk of CSR funds are to be used only in Title I-eligible schools. For sub-grant awards under the Title I portion of CSR funds, coordinators must think of their mandate as a single priority with two parts: LEAs must promise that 1) they will use the funds in schools identified as in need of improvement or corrective action, and 2) they are committed to assisting those schools in their reform efforts. (See F-2.)

CSR funds become available to SEAs beginning on July 1 of each year, once their State applications have been approved. The SEA may reserve up to five percent of its CSR funds for administration, evaluation and technical assistance. (See C-3, E-1, F-6 and G-6.) The minimum annual CSR award to LEAs is $50,000. CSR staff noted that, as before, CSR program funds are still intended only for schools initiating reform and that CSR award schools cannot reapply for funds once the three-year funding cycle is complete-regardless of the success or failure of their reform efforts. (See G-3.)

CSR staff made two important points of clarification regarding Title-I eligible schools. First, even if a school is not receiving Title I funds, it is eligible for CSR funds so long as it is Title-I eligible. This is an important distinction because many high-poverty, low-performing high schools and middle schools do not receive Title I funds due to LEA policies that concentrate improvement activities at the elementary level. Thus, secondary schools with great need for reform need not be left out of the CSR program.

The CSR program staff also explained that the size of state CSR awards is determined by the formulas established for Title I and FIE eligibility (the number of school-age children aged 5-17). Thus, the proportion of CSR funds allocated in each State will change if there are changes in the relative numbers of their students living in poverty or total student population.

The CSR staff responded to the oft-heard complaint that the $50,000 minimum award does not come close to covering the cost of full implementation of many national reform models. While acknowledging this reality, they asked coordinators to understand and make clear to LEAs an important distinction: this amount is not intended to subsidize implementation of a model at a school, but rather to seed-fund and leverage the implementation of a school's comprehensive reform program. Coordinators were also reminded that SEAs may fund schools at higher levels, and urged them to base their decisions on levels of need in addition to the quality of the application. Because of varying levels of need, SEAs were also urged to consider funding schools at varying levels instead of determining a common award amount for every school.

Discussions
In response to a question about the minimum amount of funding an SEA may provide (see F-6), the CSR staff explained that LEAs can apply to the State for additional administrative funds. In granting such requests, SEAs can use funds from their 5 percent allocation for administrative expenses, but not from funds allocated for CSR program awards to schools.

The second point of clarification was to reiterate that the FIE portion of the total CSR funds is designed to allow non-Title I eligible schools to apply for and receive Comprehensive School Reform program funding. Although SEAs can give priority points to schools with a high level of poverty, they may not create a priority that totally excludes non-Title I schools.

Participants asked the CSR office to clarify two additional points regarding SEA priorities: whether States can create additional priorities, and whether States can limit competitions to schools that serve only elementary or only secondary students. States are permitted to create priorities in addition to the one required in the legislation; for instance, States can create a competitive priority for elementary schools or for secondary schools. States can also request a waiver to limit CSR competitions to specific levels. However, SEAs were cautioned not to make competitive priority points so high for one priority type that other high-quality applications are eliminated from the competition.

Theme Five: Special Considerations

Though CSR staff did not go into great depth on this aspect of the program guidance, "special considerations" is an important theme. Among these special considerations are requirements that affect small schools in particular. (See F-7, F-8, F-9 and F-10.) The CSR guidance also outlines the implications of the new law for charter schools and private schools. (See F-16, F-17 and F-18.) Schools funded through the Bureau of Indian Affairs (BIA) are also eligible to compete for CSR funds, and program details specific to those schools are provided. (See F-19).

Discussions
One participant asked whether there are any requirements unique to the Bureau of Indian Affairs (BIA). CSR program staff noted that BIA-funded schools can apply to the BIA, to their SEA, or to both, but an individual school can receive funding from only one of those sources. If the award comes from the SEA, the funds come from the SEA, not from the BIA. In addition, because the BIA functions as both the LEA and the SEA, it must abide by the requirements governing each. (C-2, F-20).