Pre-Application Meeting Videos and TranscriptsApril 22, 2002
DR. SWEET: Let me give you a little bit of background before I launch into the specifics of this program because it took a lot of planning. We, at OERI of course know that we know a lot about early reading and one of the ways that we know that is we went back and reviewed the synthesis, of course, on the NAS's Preventing Reading Difficulties in Young Children and of course what we realized is, is though we know a lot about early reading, we know less about reading comprehension.
But we were very fortunate because we have new landmark reports that we look back to and Valerie and Russ have mentioned those to you. The first was the report of the National Reading Panel and we're very fortunate today, by the way, to have Tom Trabasso with us who is going to render a special presentation later on. So, we're pleased that he's here to do that for us.
And next, after we reviewed that report by the National Reading Panel, it's called Teaching Children to Read, we went on and we reviewed the report of the Rand Reading Study Group which it's in its quote, unquote, final phase. Rand is reluctant to stamp it as final because we want to continue a conversation. So, it will never be final, it will be evolving over time and we welcome your input. And I keep saying the we. I recently returned from Rand and was privileged to work with Catherine Snow at Harvard University and other study group members there on developing their report which is called Reading for Understanding. It's out in hard copy, just came out last week.
We recognized after looking at these materials that critical frontiers for reading have barely been broached. Okay, we've got that. Let me give you an example of what we're talking about here. Deriving empirically grounded theories of comprehension development and reading instruction across a full range of ages and race, there are still some holes there.
Now, I want to acknowledge right up front that we have very deep pockets of knowing in this area of comprehension and one good example of that, of course, is in the area of instructional strategies. But even there we need to study strategies in terms of across multiple strategies, across a range of variation such as reader differences, text types, instruction context and the like.
So, although we know a lot there's still a lot we need to learn. We have a very definite purpose to this program and it is to expand scientific knowledge of how students develop proficient levels of reading comprehension, how it can be taught most optimally and how it can be assessed in ways that reflect as well as advance our current understanding of reading comprehension and its development.
We have an overarching goal. That goal is to obtain converging empirical evidence on the development and assessment of comprehension that coheres with scientifically supported theories of the processes involved in reading comprehension and, of course, we wish to provide a scientific foundation for approaches to comprehension instruction that allows students to achieve proficient levels of comprehension across a range of texts and subjects.
Now, here I'd like to emphasize the importance of this goal, most particularly the part where we point to comprehension across a range of texts and subjects. This is indeed the heart of the PRRC initiative, but I hasten to add that in this overarching goal we allow for, we encourage a developmental view given that you will find that there are no age/grade restrictions that apply and that's quite purposeful on our part.
So, the PRRC Program is being launched and we're very excited about it here at OERI. For this year, FY '02, we have approximately $4.5 million. We will have grant awards of varying size roughly between 75 and $500,000 per. They may range in terms of length between one year and three years. So, anywhere up to 36 months.
A little more background here. Many of you already know that we had a notice of proposed priority that was released in January. The end of January the Secretary had this published in the Federal Register and for the ensuing month, up through February the 22nd, we solicited comments from the universe here in the United States and actually we got three. And since there are only three I'm going to let you know what they are.
But the bottom line here is because these three were actually covered under the Notice of Proposed Priority you will have noticed, and it's in your packet of information, that the final priority in the proposed priority read the same. For example, we got one comment where the commentor thought it was critical to examine where reading comprehension should be taught, who should be teaching it and to whom it should be taught, especially at the middle and high school levels. And, of course, this is part of the call, part of the priority. We would, of course, like to receive studies of this nature.
The second comment had to do with the wish for additional research and that it be supported in terms of the role of one to one mentoring using trained community volunteers as an intervention strategy for struggling readers in the area of reading comprehension. Here, again, covered by the priority.
And the third commentor had a concern that research on the nature of expository text and social studies and probably in other areas should be included in the priority and, furthermore, the same commentor argued that a research effort of this nature would involve expert analysis of currently published expository text. Well, here again, not only is this permissible under the priority, we welcome it, it's highly desirable.
So, just to give you a flavor of what we're covering here, we have an absolute priority that's been established. We use the word absolute because it's the opposite of invitational. There are such priorities, this is not one of them. It's a very targeted competition we have here and we specifically wish to build a scientific foundation for educational practice by supporting rigorous research and reading comprehension and we also wish to produce findings that will change instructional practice and promote academic achievement. Now, of course, we know that findings, per se, can't change instructional practice, but most assuredly they can and will provide a solid foundation for instructional practices in the classroom.
Let me add a footnote here. The information from here on will appear to some of you as basic, but then again we expect that applicants will, some will be very experienced and others will be just tripping into these waters for the first time.
So, if you are an applicant for this program there are certain things that you need to do and that is to construct a study that focuses on one or more of the three areas of inquiry that I'm going to discuss. The first is developmental patterns of students' comprehension, the second is instruction interventions for reading comprehension and the third is measures of reading comprehension that reflect empirically justified dimensions, distinguished reader differences and are sensitive to instructional goals.
And I want to point out to you that in this program of research we're calling for the development of measures that service instruction as opposed to those designed more broadly for the purposes of accountability.
And by this I mean deriving measures or devising measures that can be used both to inform instruction and also as research tools to detect subtle changes and not so subtle changes in instructional outcomes as a result of instruction interventions. So, these kinds of measures would be complimentary to accountability measures.
In your proposal you need to indicate which approach you're going to use. Now, we recognize that contingent upon the scope of your study that you're proposing you may, in fact, end up using multiple methods but, however, there's probably a main method that you're going to use. You should describe them all. If it's an experiment, it could be a quasi-experiment, correlational, other or quantitative such as a simulation, it could be a descriptive study using qualitative techniques, any one or combination thereof you need to let us know specifically. The bottom line here is the design of these studies must be clear and I'm going to speak a little bit more about these in a minute.
There are selection criteria in this program. There are four of them, national significance, quality of the project design, quality of potential contributions of personnel and adequacy of resources. And the one thing I would like for you to note is that they are weighted criteria and I will address this again in a couple of minutes.
I'd like to draw your attention to the application page limits. Now, all of this information is included in the Closing Date Notice, but those of you who have applied for grants to OERI in the past may note a significant change here. The page limit here is definitively much lower than it used to be. It's 25 pages, but that's quite ample. 25 pages for your research narrative and the appendix should not exceed 20 pages.
We clearly state, and you can go back to the Closing Date Notice, again this is in your packet of materials, that the page limit does not apply to the Application for Federal Assistance, the one-page Abstract, the Budget Summary Form, the Narrative Budget Justification, the Curriculum Vitae, the literature you cite nor to the assurances or justifications.
Let's review time line. The e-Applications, which by the way are mandatory for this competition, became available on the Web April 10. Letters of Intent are due April 29. Your Proposal is due to us by May 31 and we wish to have awards made by September.
Now, I've mentioned to you about e-Applications. This is the first time here at OERI that we're actually requiring the use of e-Applications. I hasten to add that there is such a thing as a waiver and if, for some obscure reason, you are not able to apply electronically you are eligible to apply for a waiver. And the way you can do that is to send a one page justification to us and we'll decide on a case by case basis. This fax number is in your packet of materials and in the published Register notices, but you should fax your request to 202-219-2135 or you can e-mail either me or Rita Foy Moss, who is a colleague who is unfortunately not able to be with us today. You've seen her name in the Federal Register, as well. We will act on those as expeditiously as possible. We're encouraging you to use these alternate modes because the regular mail is still somewhat troublesome for us here in D.C., they are still slow-ups due to the Anthrax situation we had.
I'd like to talk to you now about the different parts of the application and with regard to the Abstract you need to do certain things. Convey the specific goals of your study, the specific goals of the research that you're proposing. You need to use language that's understandable by a range of audiences and I'll tell you why. Should you be fortunate enough to receive a grant, and I know that everyone in this room probably will, I'm hoping that you will, your Abstract is going to be used for multiple purposes and we will use it to publicize your study. So, please, clarity is a very, very important ingredient when you're constructing your Abstract.
When you're preparing it you should include, of course, the theoretical and conceptual background of your study and by this, of course, we're referring to prior research that this investigation builds upon and that provides a compelling rationale for the study that you're proposing. You, of course, need to lay out your research issues, your hypotheses, your questions in a clear way. With regard to your study design you should include a brief description of the sample including the sample size, methods, principal dependent, independent and controlled variables and your approach to data analysis. And finally, of course, you need to speak to the significance and the implications of the study that you are proposing in terms of what's expected from this work with regard to its potential impact on research, policy, practice, et cetera, and I know that you can craft a very cogent summary in this way.
Getting to the Research Narrative you need to clearly identify, not only identify but speak to the four criteria that I mentioned to you earlier because if yours is going to be a strong application, and I know that it will, you're going to address each of these criteria in the Research Narrative. With regard to national significance you're going to make a well reasoned and compelling case for national significance of the problems or issues that are going to be the subject of your proposed research. With regard to the quality of the project design you're going to present a design that's complete, that's clearly delineated and that incorporates some research methods as Dr. Reyna referred to earlier. With regard to the quality and potential contributions of personnel you're going to ensure that your personnel descriptions make it apparent that the project director, the PI's, all the key personnel, that they all possess the training and experience that's commensurate with the duties that they're going to assume in the study that you're going to conduct for us.
In this same Narrative you're going to include, of course, a Literature Review and in so doing you're going to make it clear that you've taken the current research into account. You're going to present your review logically and explicitly. And, here again, use this as an opportunity to guide the reviewers through your proposal because there are going to be, and there is going to be a panel of your peers and they're going to be evaluating these applications. So, please help them through it.
So, you should, of course, write a rigorous research plan in simple and clear language. You need, of course, to do a rationale for your study and you need to, of course, address theoretical and empirical motivations that you have. I mean, you need to make these crystal clear.
Now, when you're looking at your study, of course, articulate your conceptual framework and, again, all of the relevant prior empirical evidence. You need to develop, of course, hypotheses or research questions that are testable and you need to indicate not only your research method, but why the approach that you chose optimally addresses the questions that you're posing and underscore clarifying your design. You know, the reviewers are going to be going through your applications and those of others and clarity will gain you everything.
Again, looking at the studies that you're going to be putting together you need to distinguish independent and dependent or predictory criteria variables. You need to be explicit in doing this. Don't make us hunt for them. You need to select your measures carefully and justify their use and, of course, you should include a data analysis plan.
For example, if you are doing an experiment, and I know that many of you will be contemplating doing so, you should develop concise hypotheses or research questions, describe your sample, describe control and intervention conditions, explain procedures for random assignment, discuss attrition and how you will deal with it and describe any incentives that you may have to use to encourage participation in your experiment. And, also, if there are any variations that you need to account for such as if you're looking at several programs with different structures, you need to account for those variations in your design and make it explicit how you're going to handle these things.
A word about the Appendix. It's a very, very bad idea to put things in the Appendix that are critical to the evaluation of your study. So, I want to point this out to you. Put all of the required details up front. Do not put required details in the Appendix. It's for a specific purpose, the purpose being for items such as assessment or survey instruments, letters of support that document any collaborations you have or other relationships that you're developing, but please be careful in doing so.
When you're speaking about, in your application, the quality of personnel you need to explain to us why your team is best qualified to do the proposed work. We suggest that you use charts, diagrams, tables, anything else to show time lines, tasks and staff responsibilities because these really need to jump out at us. You should provide names, resumes and responsibilities for consultants and why it is that you chose those consultants. And you should, of course, include letters of commitment and the appropriate place for these is in the Appendix.
When you're preparing your Budget you should put key staff in your Budget along with their salaries, time commitments and fringe benefits. You should clarify how equipment, supplies, incentives for participants that are going to be used as well as your travel requests. And this may sound elementary, but it's not, add the numbers correctly. I can't tell you the number of times we've had proposals come in here and, unfortunately, in one recent case the bottom line figure was less than what the principal investigator proposed in the subsections.
And I know that at most institutions of higher ed there's a Grants Office and then there's your office. So, I would recommend that you do your independent bottom line number in need and that it match the one that's going to come out of your Grants Office. It's as simple as that. And you need, of course, to include that Budget Summary Form in your Budget Narrative.
Just a word on the preparation of the protection of human subjects. Of course, the first thing you need to do is to determine whether or not your proposed work is exempt. And we encourage you very, very strongly to complete these required forms prior to submitting your application, the reason being that the IRB process is a very long one and it actually has the potential to hold up grant awards and this has happened in the past to your colleagues. So, please don't let that be you.
Finally, there's an application checklist. Again, it's in your packet of materials. It's also embedded in the Closing Date Notices, et cetera, but be sure that you tend to each and every one of these. A colleague, Cathy Perkinson, is going to bring you through the e-Application software and you'll see if you are using this you're not going to leave any of these out because it prompts you if you do.
Before I turn this over to Cathy I'm going to ask Val, any other comments before we swing into e-Application?
MS. REYNA: I would say given the length of the application, and take this as a suggestion rather than a mandate, I would immediately, it is a good idea to be clear, but I would be technical and I would be technical immediately. I would anticipate that your reviewers will be experts in the field, who have published in the field. So, assume that and don't, with 25 pages double spaced you really don't have a lot of time to give a lot of preamble. So, I would jump right into the technical material right away.
DR. SWEET: Good advice, thanks.
Pre-Application Meeting Videos and Transcripts
This page last modified May 30, 2002 (jer)