About the Study

The study was funded by U.S. taxpayers by the Department of Education through the Institute of Education Sciences. The SERP team was committed to standards of rigor that will allow us, as a nation, to learn whether these programs can make a difference in what students learn. We have been inspired by the enthusiasm with which students engage in Word Generation discussions, and by the profound effect we see in classrooms where students and teachers spend part of their day listening to each other reason about important ideas. We rejoice when students in STARI classes, who are several grade levels behind in their reading, display depth in their discussion of important ideas and cheer for their partners as they get a little faster in their timed reading each day. We are so thankful for the opportunity to help make a difference in the lives of children.

Word Generation

Schools that volunteered for the Word Generation study were entered into a lottery that determined whether the school used the program in the first two academic years (beginning in fall 2012), or whether they would be deferred until the third year (fall 2014).

What participating schools were promised:

• All materials were provided free of charge to schools lotteried into the program in the first two years. Schools lotteried into the deferred condition received a $3,000 stipend for each of the first two years and received all materials free of charge in year three.
• A summer institute for professional development was provided to teachers in advance of the first year of teaching the curriculum.
• Coaching was provided to teachers during study years.

What participating schools promised:

Whether the school was using the program materials during the first two years or was deferred until year 3, multiple assessments were administered to students at the beginning and end of the year. The assessments included:

• The RISE reading assessment that is already in use in many schools.
• A reflective judgement test designed to judge the sophistication of students' reasoning on a topic of interest (such as how to resolve a disagreement among adults about student exposure to violence on television).
• An academic language test that explores students' familiarity with the difference between everyday language and the more structured language used in texts or in more formal situations.
• A perspective-taking measure that captures the sophistication of students' capacity to consider and analyze different points of view.
• A vocabulary assessment that measures knowledge of the Word Generation words.

Strategic Adolescent Reading Intervention

All schools that participated in the STARI study used the curriculum in their schools during the first two years. Students who are eligible for the program were selected based on a combination of MCAS and RISE scores. Students were then chosen by lottery for STARI classrooms. Those not chosen stayed in the classes in which the school would have placed them, if the study were not underway.

What participating schools were promised:

• All STARI materials were provided to the school free-of-charge, including the novels.
• A summer institute was provided for first-year STARI teachers.
• Regular coaching was provided to STARI teachers during the school year, and first year teachers were given an opportunity to take an adolescent literacy course for graduate credits.
• STARI teachers participated in a Professional Learning Community that met regularly. In these meetings, teachers shared their expertise and their experiences with the curriculum.
• SERP staff were on-call to help troubleshoot throughout the study.

What participating schools promised:

• Schools worked with the SERP team to schedule students early in the year. Once students were placed in the class by lottery, it was essential that they not be moved to a different class once the semester was underway.
• All eligible students (whether or not they were assigned to a STARI class by the lottery) were administered assessments at the beginning and end of the year. Assessments used for STARI were the same as the assessments used for Word Generation.
• Schools limited STARI class sizes to ~15 students. While schools may need to place a few additional students in the class, too large a size would undermine the effectiveness of the program.

Additional Details

Observers visited classrooms in all schools during the first two years and recorded discussions on video (students whose parents did not consent were blurred out of video recordings and did not appear in any film used by researchers). Video recordings allowed us to study the development of discussion and to study the ways in which discussion can be supported in many different classrooms and grade levels, in order to improve our coaching. By design, CCDD research assumed a partnership between academics and practitioners. Beyond implementing the intervention curriculum, teachers and principals participated in the data collection process. In the STARI program, a fair selection process was used in order to ensure every eligible student had an equal chance at placement in an intervention classroom. SERP will always strive to ensure that the requirements of the research components of CCDD are worked out in partnership with teachers and principals. Assessments and observations were not used to evaluate the teacher in any way, nor did they impact the students’ grades. The Word Generation and STARI coaches supported teacher participants during the process and, because their perspective and input is so important, teachers were routinely asked to reflect on student engagement and learning. Assessments and other hard-copy data are housed in a locked, secure facility. Electronic data is stored on a secure, non-networked, computer. Data are only handled by project staff that have signed security pledges and non-disclosure forms. All of the data collected was subject to the strict handling protocols established by the Harvard University Institutional Review Board.

During the study years, all curriculum materials were provided to participating schools with funding from the grant. The funding also covered the costs of professional development, teacher stipends, and coaching. After data collection concluded, materials were revised based on feedback from teachers and students, and all materials were made available for free to teachers nationwide. Pre-printed materials for both programs are available through a third-party printing partner for convenience.

Video 1 of 2: Participation in School-based Educational Research: A Significant Commitment on the Part of the Schools

Meenakshi Khanna shares:

schools need to be fully informed about the research process so they understand the nature of the commitment •  many of the schools we partner with are already required to administer several assessments  •  baseline data is essential to understanding whether an intervention has an impact

Jennifer Winsor discusses:

the impact that five assessments have on a school, including scheduling and loss of instructional time  •  schools recognize that participating in a research study provides opportunities and resources that they would not otherwise receive  •  it’s important to have a person other than the principal to help coordinate assessments and materials distribution

Erin Ruegg shares:

the valuable contribution to the profession as a result of participating in a research study

​

Video 2 of 2: Special considerations for STARI Research Participants

Lowry Hemphill shares:

challenges of implementation of STARI program  •  STARI is a reading intervention class designed for students who are scoring “warning” or “needs improvement” in reading comprehension  •  STARI may not be the appropriate intervention for many struggling readers, and SERP staff can help schools identify the most appropriate interventions  •  teachers selected for STARI should have several years of experience and be confident in their classroom management  •  funded by the federal government, and teachers must be willing to follow the guidelines that come with the funding  •  not all students for whom STARI is appropriate will receive the curriculum due to the randomization requirement for the study  •  if possible, class size should be in the range of 14-20 students  •  research requires classroom visits, videotaping, and pre- and post-assessments.

​