Improving Implementation of Innovative Assessments

All4Ed submitted comments to the U.S. Department of Education’s request for information regarding the Innovative Assessment Demonstration Authority (IADA) under the Every Student Succeeds Act (ESSA) to help the agency encourage states to pursue testing innovations and improve the quality of their statewide assessments. Download or read our suggestions below.

Thank you for the opportunity to respond to the request for information (RFI) regarding the Innovative Assessment Demonstration Authority (IADA) under the Elementary and Secondary Education Act (ESEA) (Docket ID ED-2023-OESE-0043). We agree with the U.S. Department of Education (ED) that the statewide assessments required under the ESEA can be effective tools for advancing student learning, supporting equity, and ensuring families, educators, and the public have access to data on student opportunities and progress. For individual students, this data is critically important as parents, teachers, school leaders, and administrators work together to accelerate learning coming out of the COVID-19 pandemic. Further, at a systems level, we continue to believe in the value of states measuring—in a comparable, valid, and reliable way—whether all students and individual groups of students are meeting grade-level expectations and are on-track to graduate college- and career-ready, so that resources may be allocated to the students and communities that need them most.

Given these important uses of statewide assessments, we also support and appreciate states’ efforts to continuously improve, innovate, and bolster the quality and accessibility of their assessment systems—and acknowledge the role ED plays in supporting states’ efforts. This includes leveraging federal funds available through formula and competitive grants for state assessments (CGSA) and federal flexibility available through the IADA, as well as the technical assistance and guidance states receive through the federal assessment peer review process. States need the (1) resources, (2) time and flexibility, and (3) knowledge and capacity to develop and incorporate more sophisticated, innovative test designs and test items that can measure higher-order thinking skills aligned with the state’s challenging academic standards; to ensure assessments are culturally and linguistically relevant and free of bias; and to provide more timely and relevant information to parents and educators.

We believe ED should think broadly about how it can best support the development of innovative statewide assessment systems, and our comments below reflect that participation in the IADA may not always be the sole or most effective route a state could take to achieve this goal.

1. Resources. ED should use the Competitive Grants for State Assessments (CGSA) program to provide funding to state educational agencies (SEAs) to develop innovative statewide assessments systems that could be used to meet current federal testing requirements in Title I, Part A.

Although we appreciate ED’s interest in increasing the number of states using IADA, we believe an even more critical need of states seeking to develop innovative statewide assessments is financial assistance. With existing state resources committed to maintaining the current assessment system (which is critical until a new innovative assessment is ready for full operation statewide), SEAs lack the funds to develop a new, more innovative approach. States need supplemental resources to engage with and seek input from stakeholders; develop the overall assessment design, blueprints, and innovative assessment items; and contract and collaborate with assessment vendors and technical experts, as needed. For this reason, we encourage ED, whenever sufficient funds have been appropriated, to run a future CGSA competition(s) solely focused on innovative statewide assessment systems. ED could do so by proposing an absolute competitive priority for SEAs seeking to develop an innovative statewide assessment model to meet ESEA assessment requirements in section 1111(b)(2)(B). These funds should be available to all states, not just those participating in the IADA, as states could field test and scale their new assessment without seeking that authority.

Past CGSA competitions have funded relatively small projects across a range of priorities, rather than focusing on more transformative changes in a single priority area. In the past, the diverse nature of CGSA grantees (and the small grant size) has limited the program’s ability to have a significant impact in improving state assessments required by ESEA. However, Congress has recently increased CGSA appropriations, making it more feasible for these funds to support larger innovations in state assessment systems. Likewise, there is precedent (i.e., the Race to the Top Assessments Grants program) in successfully leveraging federal competitive dollars to make significant progress in large-scale testing – using funds to develop, pilot, and implement transformative approaches, including with general assessments in English/language arts and math, alternate assessments for students with the most significant cognitive disabilities, and English language proficiency assessments. Unlike the Race to the Top Assessment Grants program, which required a very large federal investment, a CGSA competition focused on innovative statewide assessments could be more modest in scope, as grants could support a single SEA rather than consortia of states.

A CGSA competition focused on developing innovative statewide assessment systems would provide states with resources and time to collaborate with stakeholders to plan their new assessment approach; solicit proposals from vendors and other technical and psychometric experts to facilitate and guide the work; develop test blueprints and assessment items; pilot assessment platforms and items; screen and correct for racial and cultural bias; provide professional development for educators and other school and district staff; and gather feedback and make course corrections. At the end of the grant period, states would be positioned to field test and scale their new, innovative statewide assessment—and could then seek flexibility from ED (via a field test flexibility waiver or IADA) to avoid double testing students during field testing.

2. Time and Flexibility. States need dedicated time for planning and developing a new, high-quality innovative assessment apart from time to conduct field testing of the new assessment, but only need statutory flexibility during the field test phase. Moreover, given equity concerns during extended field testing, we encourage ED to not only clarify IADA’s “comparability” requirements, but also work with states to minimize the time required for field testing and scaling the innovative assessment statewide.

Planning Time

While ED seeks input in the RFI on allowing for planning time during the IADA, we note that, per section 1204(e)(2)(A)(i) of the ESEA, as amended by the Every Student Succeeds Act (ESSA), SEAs that receive IADA have flexibility from ESEA’s requirement to assess all students using the same assessment from the beginning of the demonstration authority period. Thus, if an SEA is not ready to field test a complete innovative assessment, it is inappropriate – and unnecessary – for the state to seek IADA, as the flexibility from these ESEA requirements is the primary benefit of IADA participation. It would be more effective for states to use CGSA, and the funding it would provide, as a time for planning than to create a planning period, without the benefit of grant funding, within the IADA’s five-year demonstration authority period. 

That said, if ED were to create a planning period for SEAs within the IADA, even though no additional funding would be available, we would recommend:

• Conditioning the allowance for planning time within IADA on continuing to administer the same assessment (i.e., the existing statewide assessment) to all students in all required grades and subject areas in the state during the planning period. In other words, the flexibility afforded to states under IADA in ESEA section 1204(e)(2)(A)(i) would not apply during planning years.

• Counting the planning years toward the demonstration authority period in ESEA section 1204(b)(2). For example, a state could take the first three years of the demonstration authority period for planning and development (while continuing to administer the existing statewide assessment to all students in the state), and then use the final two years of the demonstration period to field test and scale the assessment to statewide use by year five. An SEA could continue to request an IADA extension for one to two additional years beyond that, if needed.

Comparability

In general, we encourage ED to find ways to provide support and incentives for states to scale their innovative assessment systems more quickly and minimize the number of years that flexibility for field testing is required. This is because more equity concerns and “comparability” considerations arise as SEAs extend the number of years for field testing (i.e., the time period during which both the existing statewide assessment and the new innovative statewide assessment are administered).

For instance, if the new innovative assessment is a more authentic measure of student learning and probes more deeply into critical areas of the state’s standards, it is concerning for certain students to be excluded from that assessment for five years – or up to seven years if an SEA receives an IADA extension. Likewise, as we saw with the New Hampshire IADA pilot, the longer multiple assessment systems are administered within a state, the more challenging it is for the SEA to have student achievement data that can be compared statewide as required by ESEA; to communicate that data consistently and clearly to parents, educators, and the public; and to identify schools fairly and accurately to receive additional support and resources (including school improvement funds).

In our view, these equity concerns undergird the “comparability” requirements in ESEA section 1204(e)(2)A)(iv) and (x) and corresponding regulations: the innovative assessment system must  “generate results that are valid and reliable, and comparable, for all students and for each subgroup of students…, as compared to the results for such students” on the existing state assessment and must “generate an annual, summative achievement determination, based on the aligned State academic achievement standards… and based on annual data, for each individual student.”

In the past, ED has permitted SEAs transitioning to new assessment systems to have one year of field testing flexibility, waiving many assessment, accountability, and reporting requirements. Under these waivers, SEAs have been able to forgo reporting of results from field testing a new assessment, as the standards setting process does not occur until after the field test and since states may wish to raise the achievement standards on the new test relative to the old one (as was the case for states transitioning to assessments aligned with college- and career-ready standards). In fact, revising academic achievement standards to better reflect the deeper learning skills and knowledge students need to demonstrate to be on-track for success after high school may be one key factor behind developing a new assessment. Thus, one-year field test flexibility waivers, reasonably, have no requirements for SEAs to demonstrate “comparability” between the old and new assessments. Instead, the waiver enables SEAs to focus on ensuring the new assessment will meet federal requirements and pass peer review once it is operational and administered statewide in the following year.

On the other hand, because SEAs receive flexibility through IADA to administer and use two different assessments for five years or more, IADA’s statutory and regulatory language requires the new innovative assessment to be built with the prior assessment’s achievement standards in mind and for states to find a way to validly, reliably, and credibly use results from both assessments in order to maintain transparent reporting for parents and the public; fairness for schools in the accountability system; and consistent expectations for all students during extended field testing. ED cannot ignore these statutory and regulatory provisions for “comparability” and the reasons behind them, even if they complicate the process an SEA typically undertakes to develop a new assessment.

Given our equity concerns associated with multiple years of field testing (as permitted under IADA) and the proven approach to developing more innovative statewide assessments using a one-year field test flexibility waiver, we recommend ED highlight for states how they could pair a CGSA grant with a one-year field test flexibility waiver to plan, develop, pilot, and implement an innovative statewide assessment system as an alternative path to IADA without “comparability” requirements. Under such an approach, SEAs would be required to submit their innovative assessment for peer review following the first full operational assessment, but would not have to submit a peer-reviewed application in order to begin field testing (which is the case with IADA). These SEAs could instead submit a waiver request directly to ED to avoid double testing students during the field test.

For states that prefer multiple years to field test their innovative assessment, we recommend ED continue to offer the chance to apply for IADA—and that ED use the recently published progress report described in ESEA section 1204(c) to inform expansion of the authority beyond seven SEAs. For these states, some demonstration of “comparability” will continue to be necessary, both because it is statutorily required and because of the valuable uses of state assessment data. That said, ED could clarify the “comparability” requirements in several key ways:

• We do not believe the IADA requirement for the innovative assessment to “generate results that are valid and reliable, and comparable, for all students and for each subgroup of students…, as compared to the results for such students” (ESEA section 1204(e)(2)A)(iv)) means that an individual student’s result on the old and new assessment would need to be the same. Instead, “comparable” should mean the results are “able to be compared.” Specifically, states need to be able to compare and interpret student results from both assessments in order to use that data during the demonstration period for several purposes required by Title I, Part A: reporting to parents and educators on individual student progress, reporting to the public on state and local report cards, and identifying and supporting school improvement through its accountability system. The ability to compare the results to use them for these purposes goes beyond simply comparing the quality of the two assessment systems, and ED could issue guidance or a Dear Colleague Letter to clarify this interpretation of “comparable” results.

• Based on its experience implementing the IADA and the interpretation above, ED could also provide examples of ways SEAs participating in IADA could show “comparability” between the existing assessment and innovative assessment that would meet the regulatory requirements in 34 CFR §200.105(b)(4)(E): “An alternative method for demonstrating comparability that an SEA can demonstrate will provide for an equally rigorous and statistically valid comparison between student performance on the innovative assessment and the statewide assessment, including for each subgroup of students.”

• Relatedly, ED could clarify whether the academic achievement standards on the innovative assessment system must be the same as the existing statewide assessment, or whether states could demonstrate they are setting more rigorous achievement standards on the new assessment. If an SEA desires to set higher academic achievement standards on its innovative assessment, guidance could clarify that this would be permissible under IADA, so long as the SEA also issued an annual summative determination for all students participating in the innovative assessment aligned with the academic achievement standards used for the current statewide assessment (per ESEA section 1204(e)(2)A)(x)).

In other words, during the demonstration period, student results from the innovative assessment would be expressed in two ways: (1) using the new, more rigorous achievement standards and (2) equating or comparing that result to align with the achievement standards used for the current statewide assessment. Once the innovative assessment is administered to all students statewide and used for meeting ESEA’s requirements in ESEA section 1111(b)(2), it would no longer be necessary to refer to and report data using the prior academic achievement standards. At that time, the new, more rigorous academic achievement standards would be applied statewide to meet the requirements of ESEA section 1111(b)(1) and would be referenced when the SEA submits the innovative assessment for federal peer review.

3. Knowledge and Capacity. In order to successfully scale innovative assessment approaches, state educational agencies need better guidance and support from the Department regarding innovations in statewide assessments that would be consistent with ESEA requirements and federal assessment peer review.

ESEA, as amended by ESSA, maintained most of the core assessment provisions from the No Child Left Behind Act, but made several adjustments to support SEAs in improving the quality of their assessment systems. This includes highlighting that state assessments may:

1. “involve multiple up-to-date measures of student academic achievement, including measures that assess higher-order thinking skills and understanding, which may include measures of student academic growth and may be partially delivered in the form of portfolios, projects, or extended performance tasks.” (ESEA section 1111(b)(2)(B)(vi))
2. “be administered through multiple statewide interim assessments during the course of the academic year that result in a single summative score that provides valid, reliable, and transparent information on student achievement or growth.” (ESEA section 1111(b)(2)(B)(viii)III))
3. “develop and administer computer adaptive assessments…, provided the computer adaptive assessments meet the requirements.” (ESEA section 1111(b)(2)(J)(i))

States do not need to participate in the IADA or pursue a field test flexibility waiver in order to incorporate these features into their statewide assessments, and ED could emphasize this more in non-regulatory guidance or a Dear Colleague Letter to state chief school officers and their staff.

Likewise, IADA was added to the ESEA as amended by ESSA to support SEAs pursuing more innovative assessment designs, such as “competency-based assessments, instructionally embedded assessments, interim assessments, cumulative year-end assessments, or performance-based assessments” (ESEA section 1204(a)(1)). Statutory priorities within the CGSA also include “measuring student academic achievement using multiple measures of student academic achievement from multiple sources” and “evaluating student academic achievement through the development of comprehensive academic assessment instruments (such as performance and technology-based academic assessments, computer adaptive assessments, projects, or extended performance task assessments) that emphasize the mastery of standards and aligned competencies” (ESEA sections 1201(a)(2)(K)-(L) and 1203(b)(1)(A)).

Despite these statutory changes, SEAs may still lack clarity on how they could develop an assessment system with these features that would meet federal assessment peer review requirements. We recommend revising and updating the current federal assessment peer review guidance with additional examples of how SEAs could submit satisfactory evidence in cases where they are using a computer adaptive assessment, multiple assessments throughout the year (i.e., “through-year” assessment), performance tasks, and other innovative assessment designs highlighted in ESSA. Without more explicit guidance, SEAs may still believe they cannot incorporate adaptive items, for example, because they would not include sufficient grade-level assessment items.

In addition to updated peer review guidance, ED could produce other non-regulatory guidance, such as Frequently Asked Questions, to dispel common myths regarding whether more innovative assessment designs – like those cited in the ESEA statute – are permissible. In particular, it would be helpful for additional guidance documents to include specific examples of states that have adopted more innovative approaches and designs for the required federal assessments and submitted those assessments successfully through the peer review process.

Upon revising existing guidance or issuing new guidance, it will also be important for ED to offer technical assistance and conduct outreach to state assessment and Title I directors, state assessment technical advisory committees, assessment vendors, psychometricians and their professional organizations, and other stakeholders to explain the new guidance, show ED’s commitment to improvement and innovation in the field of large-scale student assessment, and highlight how modern assessment designs and techniques align with ESSA’s requirements.

It will be especially critical to educate not only state assessment leaders responsible for developing these systems, but also individuals who may serve as federal peer reviewers. This will help ensure there is broad understanding in the field – from those responsible for procurement and development to those responsible for evaluation and oversight – of how innovative assessment designs can be used consistent with ESSA’s requirements. Thank you again for the opportunity to provide feedback on how the Department can better support states’ efforts to implement innovative assessments, particularly through the IADA. We would be happy to discuss any of the recommendations offered in these comments further; please contact Anne Hyslop, Director of Policy Development (ahyslop@all4ed.org). We appreciate your leadership in calling attention to the important roles assessments play and the need to continuously improve these systems so that they are serving all students well and providing stakeholders with more useful data to inform and advance student learning.