New tests developed by two state consortia more effectively target and assess essential college- and career-readiness skills than some other tests, according to two new reports.
The reports, produced by the Thomas B. Fordham Institute and the Human Resources Research Organization (HumRRO), evaluate the quality of four standardized tests aligned with new rigorous college- and career-ready standards: the Massachusetts Comprehensive Assessment System (MCAS), ACT Aspire, and tests developed by the Partnership for Assessment of Readiness for College and Careers (PARCC) and the Smarter Balanced Assessment Consortium (SBAC). The reports find that the PARCC and SBAC tests emphasize the most important content students need for college and career readiness and more effectively assess students’ higher-order thinking skills than MCAS and ACT Aspire.
“[T]he next generation assessments that were developed with the Common Core in mind have largely delivered on their promises,” according to the Fordham report. “[T]hey tend to reflect the content deemed essential in the Common Core standards and demand much from students cognitively. They are, in fact, the kind of tests that many teachers have asked state officials to build for years. Now they have them.”
For the reports, Fordham and HumRRO compared the English language arts/literacy (ELA) and math assessments for each test provider against criteria developed by the Council of Chief State School Officers (CCSSO) to measure test quality. The evaluations focus on two key components of each test: (1) how well each test assesses the content students’ need most for college and career readiness and (2) how well each test assesses the depth of knowledge necessary for college and career readiness, as demonstrated through measurements of students’ thinking skills. The reports evaluate how well each test aligns with the CCSSO test quality criteria and assign ratings of excellent, good, limited/uneven, or weak for each criterion. The Fordham report evaluates the fifth- and eighth-grade tests from each provider, while the HumRRO report evaluates the high school assessments.
The PARCC and SBAC fifth- and eighth-grade ELA tests earned twice as many good and excellent ratings as the ACT Aspire and MCAS tests, according to the Fordham report Evaluating the Content of Quality Next Generation Assessments. The four testing programs earned similar ratings on their math assessments; although, PARCC was the only math assessment that earned all good or excellent ratings, according to the Fordham report.
“These reports definitively confirm what we’ve been saying for some time—PARCC is the highest-quality, best-in-class assessment available,” says Hanna Skandera, secretary of education for the New Mexico Public Education Department and chairwoman of the PARCC governing board, in a statement about the findings.
On the high school math tests, both PARCC and SBAC earned all excellent and good ratings for their effectiveness in measuring the content and depth of knowledge necessary for college and career readiness, as shown in the table below from the HumRRO report Evaluating the Content and Quality of Next Generation High School Assessments. Meanwhile, on the ELA tests, PARCC and SBAC each earned excellent ratings on five out of the six content criteria, while ACT Aspire and MCAS each earned mostly limited/uneven and weak ratings for content. On depth of ELA knowledge assessed, SBAC was the only test to earn all excellent and good ratings, according to the HumRRO report.
“These evaluations confirm the work of educators in our member states, who built this system from the ground up to measure students’ college and career readiness,” says SBAC Executive Director Tony Alpert, in a statement responding to the Fordham and HumRRO reports. “We are confident our assessment system will continue to be recognized as an historic and groundbreaking system to improve teaching and learning.”
The Fordham and HumRRO reports give the PARCC and SBAC assessments top ratings for the quality, complexity, variety, and level of rigor of the test items they include. But a separate study by the American Institutes for Research (AIR) finds that the achievement standards (or cut scores) these testing providers have set do not necessarily indicate that students have mastered college- and career-ready content equally.
“State achievement standards represent how much the state expects their [sic] students to learn in order to reach various levels of academic proficiency,” according to the AIR report National Benchmarks for State Achievement Standards. SBAC, for instance, groups student test scores into four achievement levels and indicates that a score at or above Level 3 suggests a student is ready for college-level course work. PARCC, meanwhile, groups student test scores into five achievement levels and classifies scores at or above Level 4 as a sign of college readiness.
The AIR report finds that the PARCC and SBAC classifications of “college readiness” are not equal. “Smarter Balanced college-ready standards (Level 3) are significantly below PARCC college-ready standards (Level 4) by about one-quarter of a standard deviation,” meaning the PARCC tests expect students to reach a higher level of mastery before designating them “college ready.”
Furthermore, in most cases, the proficiency standards for the two testing consortia fall short of the level students must achieve to reach proficiency on the National Assessment of Educational Progress (NAEP), which many analysts consider the “gold standard” for measuring student achievement.
In the study, AIR compares the fourth- and eighth-grade reading and math achievement standards for PARCC, SBAC, ACT Aspire, and several individual states with those for NAEP. The study finds that in both ELA and math in both grades, a Level 3 score on SBAC compares in difficulty to the basic level on NAEP. Meanwhile, SBAC’s Level 4, the consortium’s highest achievement standard, aligns with NAEP’s proficient level in ELA and math for both grades.
PARCC’s Level 4 college-ready standard aligns with the NAEP basic level for ELA in fourth- and eighth-grade. In math, however, PARCC’s Level 4 compares to NAEP’s proficient level, according to the report. Scores at PARCC’s Level 5, the test’s highest achievement standard, align with NAEP’s advanced level in fourth- and eighth-grade math and fourth-grade language arts. Level 5 scores on PARCC’s eighth-grade ELA test align with NAEP’s proficient level. The report notes several caveats to these comparisons. For PARCC and SBAC, the researchers compare the consortia’s ELA standards, which include writing, to NAEP reading standards, which do not include writing. Furthermore, NAEP does not measure identical content as PARCC and SBAC, so aligning the scoring scales for the three tests can only estimate how test scores compare since the scores do not necessarily equate directly.
In addition to these reports on the PARCC and SBAC assessments, the Center for American Progress (CAP), an independent nonpartisan policy institute, also recently released Better Tests, Fewer Barriers: Advances in Accessibility Through PARCC and Smarter Balanced. The CAP report highlights the positive advancements the two consortia have made designing built-in test features and accommodations to support English language learners and students with special needs.
The Fordham report Evaluating the Content and Quality of Next Generation Assessments is available at http://edexcellence.net/publications/evaluating-the-content-and-quality-of-next-generation-assessments.
The HumRRO report Evaluating the Content and Quality of Next Generation High School Assessments is available at https://www.humrro.org/corpsite/sites/default/files/
AIR’s report National Benchmarks for State Achievement Standards is available at http://www.air.org/resource/national-benchmarks-state-achievement-standards.
CAP’s report Better Tests, Fewer Barriers: Advances in Accessibility Through PARCC and Smarter Balanced is available at https://www.americanprogress.org/issues/education/