The following is a Power Point presentation consisting of 24 slides prepared by Grover J. Whitehurst, Assistant Secretary, Office of Educational Research and Improvement.

Slide 1: Evidence-Based Education (EBE)

Grover J. (Russ) Whitehurst
Assistant Secretary
Educational Research and Improvement
United States Department of Education

Slide 2: What Is EBE?

The integration of professional wisdom with the best available empirical evidence in making decisions about how to deliver instruction

Slide 3: Evidence-Based Education

This slide shows a chart of sources for evidence-based education. The first source is empirical evidence. One type of such evidence is scientifically based research, which can focus on practices or on programs. The second type of empirical evidence is objective measures, which can consist of benchmarks or local data. The second source for evidence-based education is professional wisdom, which can include personal experience and consensus views.

Slide 4: What Is Professional Wisdom?

• The judgment that individuals acquire through experience
• Increased professional wisdom is reflected in numerous ways, including the effective identification and incorporation of local circumstances into instruction

Slide 5: What Is Empirical Evidence?

• Scientifically based research from fields such as psychology, sociology, economics, and neuroscience, and especially from research in educational settings
• Objective measures of performance used to compare, evaluate, and monitor progress

Slide 6: Why Are Both Needed?

• Without professional wisdom education cannot>
  • adapt to local circumstances
  • operate intelligently in the many areas in which research evidence is absent or incomplete.
• Without empirical evidence education cannot
  • resolve competing approaches
  • generate cumulative knowledge
  • avoid fad, fancy, and personal bias

Slide 7: Scientifically Based Research

• Quality
• Relevance

Slide 8: Quality: Levels of Evidence

• All evidence is NOT created equal
  • Randomized trial
  • Quasi-experiment, including before & after
  • Correlational study with statistical controls
  • Correlational study w/o statistical controls
  • Case studies

Slide 9: Randomized Trials: The Gold Standard

• Claim about the effects of an educational intervention on outcomes
• Two or more conditions that differ in levels of exposure to the educational intervention
• Random assignment to conditions
• Tests for differences in outcomes

Slide 10: Why Is Randomization Critical?

• Assures that the participants being compared have the same characteristics across the conditions
• Rules of chance mean that the smart, motivated, experienced, etc. have the same probability of being in condition 1 as in condition 2
• Without randomization, differences between two conditions may result from pre-existing difference in the participants, e.g., more smart ones in condition 1

Slide 11: Why Is Randomization Critical?

Without randomization, simple associations such as between Internet use and science grades have many different interpretations. This slide shows a graph that compares the average science scores of 4^th-graders, 8^th-graders, and 12^th-graders who used the Internet at home versus those who did not. On a scale of 0-300, 4^th-graders who used the Internet scored, on average, 156, while non-Internet users scored 143. Eighth-grade Internet users had an average score of 159, while their nonuser peers averaged 140. For 12^th-graders the scores were 153 for Internet users and 136 for nonusers.

Slide 12: Relevance

• Does the study involve a similar intervention and outcome to those of interest?
• Were the participants and settings representative of those of interest?
• Were enough participants involved to justify generalization?

Slide 13: EBE: How To Use Existing Science

• Search literature (ERIC, Campbell Collaboration, PsychInfo, etc.)
• Screen literature
  • Relevance
  • Quality
• Search for pre-digested evidence
  • Narrative reviews (ERIC digests)
  • Systematic reviews (meta-analysis)

Slide 14: Screening Research-Cautions

• Unconditional conclusions
• Conclusions involving hypotheticals
• Conclusions that diverge from evidence
• Strong calls to action
• Mixtures of opinions with evidence
• Low prestige publication outlet
• Publication outlet with ideological agenda

Slide 15: Evidence Will Not Make the Decision

• Be skeptical
• Consider other ways of achieving goal
• Consider consequences and local circumstances
• Consult with experts who understand evidence before making costly decisions (This is different from consulting authorities who may know the subject area but not rules of evidence)

Slide 16: EBE: How To Use Objective Measures

• Find or develop local data
  • measure and measure frequently
  • stagger the introduction of an innovation across sites
  • compare between and within
• Search state and national databases for benchmarks
  • www.edtrust.org
  • www.greatschools.net
  • www.schoolmatters.com
  • www.just4kids.org

Slide 17: Example of Using Local Evidence

• Benchmarking using the http://www.edtrust.org website to identify high flying schools in your state with similar demographics

Slides 18 and 19: Web Pages from The Education Trust

These Web pages are an example of what slide 17 describes. They depict those Texas schools that have 90 percent of their students eligible for free/reduced lunch and have also scored in the top 10 percent of all schools over the past 2 years in percentage of 4^th-graders passing the English proficiency test.

Slide 20: EBE-Where Are We?

This slide consists of a pie chart in which most of the pie (90+ percent) is "professional wisdom" and a small piece is "external evidence".

Slide 21: Education Lags Behind

While the total number of articles about randomized field trials in other areas of social science research has steadily grown, the number in education research has trailed behind. The graph on this slide measures the growth of randomized field trials from 1950 to the present in the areas of criminology, social policy, psychology, and education. It shows that the most rapid growth has been in criminology, followed by comparable rates of growth in social policy and psychology, with education having the least amount of growth. Source for the graph: Robert Boruch, Dorothy de Moya, and Brooke Snyder, 2001.

Slide 22: Where the Research Dollars Flow

This slide introduces a table by stating: "Of 84 program evaluations and studies planned by the Department of Education for fiscal year 2000, just one involved a randomized field trial." The table shows that in fiscal year 2000 the Department conducted 51 surveys of need, 49 studies on program implementation/monitoring, 15 non-randomized impact evaluations, and only 1 randomized field trial. Source for the chart: Robert Boruch, Dorothy de Moya, and Brooke Snyder, in Robert Boruch and Frederick Mosteller, eds., Evidence Matters (Brookings, 2001).

Slide 23: What ED will do

• The What Works Clearinghouse
  • interventions linked to evidentiary support
  • systematic reviews
  • standards for & providers of evaluations
• Preschool Curriculum Evaluation Research
• Well designed, timely, & nonpartisan evaluations of ED's own programs
• Funding for evaluations of promising innovations in the field
• Build capacity

Slide 24: Goals

• ED will provide the tools, information, research, and training to support the development of evidence-based education
• The practice of evidence-based education will become routine
• Education across the nation will be continuously improved
• Wide variation in performance across schools and classrooms will be eliminated

[ BACK ]

Last modified March 21, 2002 (jca)