Building a Stronger Future for Evidence-Based Improvement in ESSA Implementation
Building a Stronger Future for Evidence-Based Improvement in ESSA Implementation
Maria Ferguson, Executive Director, Center on Education Policy
Anne Hyslop, Assistant Director, Policy Development and Government Relations, Alliance for Excellent Education
Seng-Dao Yang Keo, Director, Office of Student and School Supports, Nevada Department of Education
Candice McQueen, Chief Executive Officer, National Institute for Excellence in Teaching
Diane Stark Rentner, Deputy Director, Center on Education Policy
On January 28, 2020, The Center on Education Policy (CEP) at The George Washington University and the Alliance for Excellent Education held a webinar to better understand CEP’s implementation research on the Every Student Succeeds Act (ESSA). This webinar included CEP’s latest report A Stronger Future for Evidence-Based School Improvement in ESSA. It focused on the challenges and opportunities for states and districts as they implement evidence-based interventions to support low-performing schools under ESSA.
Unlike the policies that preceded it, ESSA does not require identified schools to adopt a particular set of strategies to improve. But it does require that the interventions schools choose be backed by evidence of their effectiveness. While this is a more flexible approach, the new requirement to rely on evidence-based strategies has created its own set of implementation issues. This webinar discussed how states, districts, external partners and vendors, researchers, and education advocates have responded to ESSA’s evidence requirements and the systems, capacity, supports, and other resources districts need to find, develop, and implement evidence-based approaches.
Please direct questions concerning the webinar to firstname.lastname@example.org. If you are unable to watch the webinar live, an archived version will be available at all4ed.org/webinars.
The Center on Education Policy is a national, independent source for research and information about public education. The Center helps Americans better understand the role of public education in a democracy and the need to improve the academic quality of public schools. cep-dc.org/
The Alliance for Excellent Education (All4Ed) is a Washington, DC-based national policy, practice, and advocacy organization committed to improving educational outcomes—and lives—of students, with a focus on those in middle and high school. We embrace diversity, equity, and inclusion and specifically advocate on behalf of all students who are historically underserved or marginalized. all4ed.org
If you are interested in renting the Alliance’s facilities for your next meeting or webinar, please visit our facilities page to learn more.
Anne Hyslop: [Music playing]
Hello and welcome to today’s webinar on building a stronger future for evidence-based school improvement. I’m Anne Hyslop assistant director for policy and government relations here at the Alliance for Excellent Education. We are thrilled to be cohosting today’s webinar with the Center on Education Policy at George Washington University. Since the passage of the Every Student Succeeds ACT, or ESSA, our organizations have been working with states, districts, researchers, and advocates, to better understand how the new law is working, especially for historically underserved students. Today’s webinar highlights one of the biggest challenges states and districts faces in ESSA implementation: school improvement.
As a reminder, unlike the policies that preceded it, ESSA does not require low-performing schools to adopt any particular set of strategies to improve. But it does require that the intervention schools choose be informed by a needs assessment, and be backed by evidence of their effectiveness. While this is a more flexible approach, the new requirement to rely on evidence-based strategies has created its own set of implementation issues. We’ll be discussing some of those issues, and how states and districts are responding to them, during today’s webinar.
I’m honored to be joined by an all-star panel. In the studio, I’m flanked by Maria Ferguson and Diane Stark Rentner, the executive director and deputy director at the Center on Education Policy at George Washington University. Maria and Diane have just released the final report in a series on ESSA’s evidence requirements, and will be sharing their findings with us, in a moment. To download the report and other work in this series, visit www.cep-dc.org.
We also have a few guests joining us remotely. I’d like to welcome Candice McQueen, CEO at the National Institute for Excellence in Teaching, and former commissioner of education in Tennessee. And Seng-Dao Yang Keo, director of the Office of Student and School Supports for the Nevada Department of Education. Both Seng and Candice can speak to how they, in partnership with school districts, responded to ESSA’s evidence requirements and the systems supports and other resources districts need, to find and implement evidence-based approaches.
Before kicking off our discussion, a few details. If you’re following along on Twitter, please tweet us @all4ed and @cepdc, to join the conversation, and share your thoughts, comments, and questions, using the hashtag #evidenceinessa. Our teams and expert guests will do their best to respond to you in real-time.
And with that, I’ll turn it over to Maria.
Maria Ferguson: Thank you, Anne. Thanks, everyone – glad to be here. Candice and Seng, thank you so much for joining us. We’re gonna start having moderated conversation, but keep it kind of casual, so everyone has a chance to chime in with their experience on using evidence-based requirements in ESSA. Anne, I wanna start with you. One of the major goals of ESSA is spur evidence-based improvement in schools identified for comprehensive or targeted support. Can you walk us through some of the variations we’ve seen and how this impacts evidence-based school improvement? I know the Alliance has done some interesting work on this, so.
Anne Hyslop: Sure, yeah. I think that while most of our webinar will probably focus on what’s happening on the intervention side, it’s important to get some context about how the accountability systems in ESSA affect what then happens when it comes to which schools need to find these interventions, and which districts and how states are affected by it. So, you know, ESSA created a much more flexible accountability system than No Child Left Behind before it. But what is required is that, in any school that’s identified, whether it’s identified on the basis of the whole school struggling or having low graduation rates schoolwide, or whether it just has one or two student subgroups that need extra support, all schools need to have in place an improvement plan that includes at least one evidence-based intervention. Evidence-based could be at multiple different levels. The law defines these four tiers from strong evidence at the highest tier, then moderate, promising.
And finally, there’s sort of this fourth tier that’s called “demonstrates a rationale,” which [laughs]is a kind of wonky name but basically just means there’s some logic behind this intervention, but not yet really a robust scientific study of any kind, and more research is needed to be done to see if that works. So, all schools need to have at least one of these interventions in place. What’s really interesting, though, under ESSA – and I think CEP did a great report that found this – is that states have simply set up very different accountability systems, which has resulted in some states identifying, you know, five percent of their schools for one of these two types of improvement. And other states identifying two-thirds or more of their schools. So, the size of your school improvement portfolio varies tremendously, and the capacity, then, that you need at the state or district level to manage that is also just much greater in some states than in others.
You know, it’s to find the vendors, to just simply approve school improvement plans, and follow-up with districts and with schools to see if they need TA, or coaching, or to evaluate whether it’s working, that’s much easier to do if you have 5 percent of schools that are improvement than if you have 65 percent of schools in improvement. I think another sort of related challenge that this brings is that, ESSA does have a set aside for school improvement funding. So, every state sets aside seven percent of their Title I funds, to support these schools. Seven percent can go a lot further in 5 percent of schools than in 65 percent of schools. So you have some states where those school improvement dollars are just being spread really thinly, and, frankly, aren’t enough to really implement a robust evidence-based intervention.
And finally, I think the sort of, just the way the accountability systems are set up, you have a bunch of schools that are being identified that have particular needs. I think evidence-based interventions have been especially challenging in rural or more, like, small-city districts, where, you know, maybe these are low-graduation rate high schools that haven’t been identified, before. And we don’t really have a good bank of evidence-based interventions or providers ready to step up and help those schools. Likewise, on the schools that are identified maybe just because they have a single subgroup or two that are struggling, that’s often students with disabilities or English learners. And again, we don’t have a great sense of evidence-based interventions, yet, to really meet the needs of those students. So there’s definitely – it’s highlighting, I think, some needs from the research and provider community to kind of step in and respond to how the accountability systems are shaping which schools are identified.
Maria Ferguson: Everything that you’re saying really does comport with the series of reports that we’ve done, you know, from how it starts at the identification process, how it goes down to using the interventions and all the associated, you know, sort of opportunities and challenges along the way. CEP released the final report in its series on ESSA, in December, called “A Stronger Future for Evidence-Based Support in ESSA.” And, Diane, I was hoping that you could sort of take us through some of the major takeaways that we learned from that report?
Diane Stark Rentner: Sure. So, briefly, by way of background on the findings, we interviewed officials from seven state education agencies and five school districts, in the winter and spring of 2019. And there was a balance of geographic concerns, of state control, of size of district, rural district, county districts, urban centers, so we got a pretty good mix, even though it’s a small sample, pretty representative of what’s going on out there in the United States. And then the final set of work we did was, we convened a meeting, in July, of educators, researchers, and policy experts, to talk about our findings from our interviews, but also to think through some ways that researchers can better support educators in trying to improve schools, and just make this a better process. But for the first major takeaway from our interviews, I would say that we learned that the process of selecting evidence-based interventions did have some challenges.
Both the state and district interviewees noted that they and school principals are not academic researchers. And so, understanding the complexity of some of the research behind the strategies and interventions was, in itself, challenging. And related to that, there were access issues; they might find some intervention that they’re interested in, only to go to the website and find that there’s a paywall behind the publication where the research is published. So, they didn’t expect that, and that created additional hurdles. And finally, regarding the selection, nearly all the district interviewees said that they did have some struggles in finding an appropriate intervention for their unique context, setting, or the group that they were targeting.
And this was most pronounced in rural school districts. As Anne mentioned, they have a particular challenge in that not a lot of research is done in rural districts, yet, there’s a lot of need. The second big takeaway is vendors are playing a big role in this process, and they’re partners with districts in implementing these strategies. And one point I wanna make is, I think when we talk about vendors, people always think it’s some big for-profit corporation. But in reality, when you look at the state lists of approved vendors, some are the national for-profit corporations; others are nonprofit research groups. There are state-level groups, even some local-level groups.
So, it’s not one monolithic big for-profit enterprise; it’s a bunch of different entities that are helping school districts in this process. The third takeaway is that we found that there’s already having an impact, the evidence requirements are already having an impact on school systems, in a variety of ways. First, district interviewees noted that they are paying much more attention to the evidence behind strategies, interventions, or products that they are purchasing. They are asking for the research basis, as required under ESSA, for their federally-funded stuff, but we also heard that it’s spread to all of their activities, that they’re asking these questions. And a side effect of that has been that some vendors’ contracts have been terminated or not restarted again, because they could not produce the evidence.
And we heard that it was painful for some systems, because they’ve had these longtime partners, but given that they couldn’t produce the evidence, they had to find something else to do. And then, finally, a big sea change is that the school improvement plans are actually a functioning guiding document now. District folks said that, in the past, the principal would do the school improvement plan, it would get approved, and then it would sit on the shelf until the next time they had to do another school improvement plan. But under ESSA, with the selection of these strategies and interventions, those documents are guiding the process. And it’s also making the districts coordinate their efforts with the plan, such as when’s the appropriate time to provide professional development, what other supports do we need to help the school succeed in doing this intervention.
So that’s a big sea change that I think is a positive one. With regard to our meeting in July, just wanted to talk about a couple of things. Obviously, there was discussion about how you can provide educators better access to research. And some attendees suggested, to be on this whole paywall issue and the complexity of research, that maybe the researchers could, you know, do what they do, which they need to be published, we understand that. But then they could create a publicly-available short summary of the research, that has, you know, what the intervention is, who it was targeted to, what the outcome was, and which ESSA evidence tier it falls under. And then, educators could access that, read that, and if they’re more interested, then they could decide that they want to pursue that more in-depth and purchase the article.
And then, finally, the big takeaway is that, everyone agreed that school improvement efforts would be better supported if the relationship of researchers and educators was one of mutual respect and benefit. And what we mean, here, one of the attendees at the meeting said this, and I think she captured it perfectly: “A lot of the time, research is done to education communities, not with and not for education communities.” And what she means by that is, they understand that being part of research is great, it adds to the knowledge base, but often, the educators are not at the table saying, “You know, this would be really helpful to us, if we could do that.” So, what they want is a more mutually beneficial relationship, where perhaps the researcher comes in and does their study, but that the same time helps the school system with something that they want studied.
Maria Ferguson: Yeah, it’s a great summary. By the way, there’s something incredibly valuable about bringing people together in a room and letting them talk about, you know, common goals around something, even though the come from different perspectives.
Candice and Seng, you’ve been very patient. Candice, I wanna bring you into the conversation. In all sectors, not just education, the term “evidence-based policymaking” gets bandied about, lately, and I think some worry that the definition and what it really means is getting a little muddled. So, Candice, in your capacity now as CEO of the National Institute for Excellence in Teaching, and certainly, in the past, as commissioner of the state of Tennessee, can you give us your sense of, what’s the current state of evidence-based policymaking in education, from where you sit?
Candice McQueen: Well, I do think there are reasons to be optimistic about where we are, and I’ll also point to some challenges that I see across the state that I’ve been in over the last year. The reason for optimism is probably twofold. I mean, first, we are actually talking about evidence, we’re talking about data, and we’re talking about it in a way that we have never talked about it before. When you actually have an opportunity, now, to discuss under what conditions, with whom, where did this particular strategy or this particular piece of work actually have impact, it’s a new day. Usually, a partner or a vendor was matched up with a district for school improvement because they had a relationship.
They knew each other, quite frankly, from a local superintendent organization where they met, maybe this group sponsored them and they somehow connected, and now they’re serving as that person’s partner. As opposed to the reality that we should be actually vetting that particular individual and that particular organization, to make sure that that partnership fits the school or the district needs. That is happening more and more, and so we do need to be optimistic. I see a good number of states – not enough, but a good number of states – actually taking on the vetting process themselves, and saying, “We’re going to put out an RFP. We’re gonna clearly state what are the evidence requirements that are particular for our school improvement needs, and then they create a pathway to have these partners actually engage directly with districts and schools.”
That is new. It’s a _____ opportunity that has not existed, historically. Unfortunately, I think the challenges that we see on that front is we don’t enough states doing that. It’s still a handful that are really taking on that responsibility; much of that has still been left to the local communities to make sense of all the partners and the evidence that’s out there. And I think our other challenge that I would just mention, briefly, is that we have the challenge, quite frankly, of how people are interpreting what the evidence means. Is it a theory of action of evidence that actually systematically improves schools, or is it, there’s evidence around this coaching model, or there’s evidence around this particular literacy curriculum, and so that’s what I’m going to choose.
As opposed to a really systematic, you know, approach that brings all of that together in a theory of action that is also evidence-based. So there’s been a lot of confusion around what that looks like on the ground. But I’ll end with an optimistic moment. I did get an opportunity to meet with the Department of Education, right there in D.C., to talk about evidence with a group of folks that were internal staff to the USDOE. And they had an actual evidence summit, and they brought in folks from across the country to give them feedback about what evidence _____ look like on the ground. And they continued to support this work federally. I think that’s encouraging, that we have all _____ federal, state, and local really thinking about, “How do we work together to interpret this for the people on the ground, in different contexts, who are doing the work?”
Maria Ferguson: Great – that’s good to know.
Seng, I wanna bring you into the conversation. Nevada has received a lot of attention, much deserved attention, for state leadership and sort of a comprehensive approach to using evidence to improve policy and practice. For those who are watching, who are maybe less familiar with all the good stuff that’s going on in Nevada, can you describe the approach the state has taken for implementing evidence, ESSA’s evidence-based requirements? And then, maybe _____ any big lessons you’ve learned during the course of this intervention and improvement.
Seng-Dao Yang Keo: Yeah, so, within the state of Nevada, we actually leverage not just federal grants, but we leverage state grants. So, within the Office of Student and School Supports, we house and lead and administer most of the federal programs, as well as most of the large state programs, and _____ serve as the _____ arm for the Nevada Department of Education. That made it very easy, then, to make a switch _____ every grant. That essentially meant every grant that was within the Office of Student and School Supports was being leveraged to then prioritize evidence-based interventions. What that looks like, in practice, is leveraging every grant that could be competitive, and if it wasn’t competitive at the time but we had the option of making that grant competitive, then we went 100 percent competitive with prioritizing evidence-based interventions.
So, what that looked like is, essentially, a few years ago, having grants that we essentially just approved [glitch interferes with audio] transition into grants where we said, “We actually want to see the evidence to demonstrate the interventions you requested funding for now meet these rigorous ESSA evidence requirements.” And that really shifted how money was spent across the state, over the course of three years. So, by our estimation, that looks like, our state spending, roughly, specifically, for Title I [glitch interferes with audio] spending 15 percent of those dollars that we get, every year, on evidence-based interventions. To now we can say, for the past 3 years, 100 percent of that funding stream is now spent on evidence-based interventions. So that’s what we did within the office.
And we also placed our best on [glitch interferes with audio]. So, with Title I 1000 _____ funds, everything, 100 percent of the interventions [glitch interferes with audio] evidence levels one, two, [glitch interferes with audio]. But for every other grant, we also chose two _____, and our _____ were supplemental curriculum and professional development. Those also had to meet ESSA evidence levels one, two, and three, and then in addition, we hosted an equity and evidence convening, and also a speed dating event, when we launched this policy shift across the state. That provided an opportunity for partners and vendors to understand the state goals and the priorities, and the rationale for why we were doing this.
Which is really important to know, because then school leaders and district leaders had an opportunity to meet vendors whose evidence levels for their intervention met _____ _____ requirements. And it also gave everybody an opportunity to hear, at the same time, the rationale for why we were doing this and what our goals were. And that really speaks to being clear about leading for equity, because our state views the ESSA evidence requirements, _____ rigorous requirements from _____ _____ and ESSA, as an opportunity to lead for equity. This is really our opportunity to say, you know, “Spend your money on what works.” Especially in those schools that need it the most.
So in terms of quick lessons that we have learned, in the past three years that we have implemented this work, one thing that I will say is that it’s really important to have a shared understanding of what the evidence [glitch interferes with audio]. And so, in particular, what really came to light was that many folks would say, “Our intervention is based on what the research body says we should be doing.” And we shifted that conversation to say, “Yes, and it’s really important for us to focus on outcomes and results. So, not only should your intervention be based on what your needs assessment is _____ _____ for you to address, and what the research _____ _____ is saying for you to do, but we’re really focused on your implementation of this intervention, and what the outcomes are.
“What are the results? What are the outcomes? And what is the evidence to demonstrate that this is effective, that this is significantly and positively improving school outcomes?” In addition, the other lesson that we’ve learned is, you have to differentiate based on the needs of districts. We have large urban districts; we also have very small rural districts across the state. And so, that was important for us to make sure that we were meeting [glitch interferes with audio].
And then, I also think it’s important to highlight that you really need courageous leadership, because there is a – there was, and continues to be, some pockets of resistance. And the resistance comes from a resistance to change. Not that people don’t all want the same thing; we wall want the same thing: we want schools to be highly effective, and we want schools to be equitable for all kids, and we want them to be _____. How we get there is different. And so, I think knowing that you need to support leaders, and you might have to give district leaders and school leaders some coverage, and so that takes courageous leadership at all levels.
And that – it also means building local capacity, because those organizations that have the research base, that have money to then have rigorous evaluations conducted, that will easily allow them to slide in to say, “Yes, we meet the rigorous ESSA evidence requirements,” those organizations tend not to be the local organizations. Those tend to be, already, the large national organizations that have many resources. And so, being a state that really values local innovation and values local organizations, it was really important for us to also make sure that we have that balance.
Maria Ferguson: That’s great. I’m gonna pivot from there, if I can, to a question for Candice. You know, it’s hard to argue that a skilled educator and strong leadership, as Seng just talked about, can really be impactful when it comes to school improvement. From the federal level, previous school improvement policy has had a little bit of a different take on how you deal with educators, especially ineffective educators. ESSA is quite different in that regard. Candice, can you share some examples of how ESSA’s evidence-based requirements can actually empower teachers, and foster what we call a continuous improvement environment? It would be great to get your perspective on how teachers are viewing this, and the important role that they play.
Candice McQueen: _____ I think ESSA’s new requirements empower teachers in two ways. First, it empowers teachers to be a part of the solution, as these solutions and partnerships are being done with teachers, not to teachers, or sometimes against two teachers. And the second way the ESSA strategies have really encouraged teachers to now not have to guess whether something works. It’s reducing that variability and the unknown, because we do have examples and vetted research that shows that these evidence components actually do work in their context – in many cases. That’s not perfect, because we still need to do more research, that was mentioned earlier, I think Diane mentioned, that this ability to actually have that connection between the practitioners and the researchers to do that deeper work.
But we have lots of promising practices and more work being done in that area. So, first, when teachers are empowered with these strategies, meaning, they’re part of the solution. It’s not a, “Well, let’s get rid of all the teachers, and start over,” or, “Let’s actually bring in a program, a software package.” We know that those things don’t work, but when teachers are part of the solution, through their own professional learning, they’re part of the buy-in, they get to actually have these continuous feedback and improvement loops alongside a particular product or program that are brought in, and it’s part of a systematic approach to school improvement where professional learning is at the center.
A great example of that is when you see leaders actually encouraging their teachers to be part of the process of the selection of multiple evidence-based partners. And then after that selection, there are structures where teachers are empowered to be teacher leaders, to then help with the implementation. And then, on the flipside, they’re also part of the professional learning communities that then work to ensure all teachers have what they need to be successful in that new evidence-based partnership. And then finally, they’re part of the feedback loop on, “Did it work? Did it not? What data was I using?” And how can we make, you know, a better decision, if it was not working.
When teachers are part of all of those loops, you end up having growth not only of the teacher knowledge and the teacher skills, but that translates into the classroom, so you have that knowledge base for your students’ growth and student ownership. And then, that second aspect of making sure that we’re empowering teachers with partnerships that work, we hear more and more teachers across the country say, “I’m working harder than ever.” But they’re not necessarily feeling like they’re as effective as they could be. They want a basis of, whether it’s a curriculum _____ that they’re using or whether it’s a professional learning partnership that they’re in, they wanna know that it works in the context that they’re in. And when they have that type of relationship, there is a relief that, “I actually now am in a partnership where I don’t have to guess, I don’t have to find things, I don’t have to piece things together,” as teachers have been doing for years.
They can say, “I have the comfort level that this curriculum works, and this professional learning strategy works. And now, I just have to hone my own practice within that, and do it so I can offer what I have to all students.” _____ _____ _____ _____ equity comment, I would wholeheartedly agree, making sure that all students are served through that partnership is the job of the teacher. And then they have these evidence-based partnerships, that can then be job one, as opposed to, “I’m struggling just to keep my head above water.” And so, I’m excited that teachers are part of those parts of that conversation.
Maria Ferguson: That’s great. I wanna stay focused on local, for a minute, here. Diane, in our conversation and our reports that we’ve done, we’ve talked a lot about the delicate balance between implementing interventions with fidelity versus making sure those interventions are appropriate and work within a local context. Can you just talk briefly about, a little bit, and give folks the sense of what our discussion was like? And then, Seng, after Diane talks, I wanna hear from you in terms of a little bit of how you are managing that tension in Nevada, as well.
Diane Stark Rentner: Sure. We had a great discussion, at our July meeting, on this very topic. We had a panel of educators, and they all were saying that, you know, “We’re so excited, our schools have these interventions and these plans, and we’re following them through. And we keep telling them if you implement with fidelity, you know, we’re gonna see results.'” And then, the next discussion was _____ the researchers, and they said, “Whoa, I think you guys are too sold on implementing with fidelity.” They said, “You need to be more flexible and adjust to the situations as they arise.”
And so, that’s another unique tension that you’re gonna face in this, you know, new world of evidence-based school improvement, is that I think the school people are completely sold on these interventions as being the problem solvers. And then, the researchers are a little bit more practical saying, “You know, you should just be aware that it might not work exactly as it worked where it was studied, and just have to be nimble and adjust your course as needed.” So, that’ll be an interesting thing to watch, in this coming year, how it plays out.
Maria Ferguson: Before we go to Seng, Anne, do you have anything you wanna add on that point?
Anne Hyslop: Yeah, I mean, I would say that one of the other interesting things, that I think came up in your research and that we’ve heard, also, is this sort of, this tension between choosing an evidence-based intervention that meets one of the higher tiers, but maybe isn’t as suited to your particular context, locally. Versus going with one that’s maybe promising or even just demonstrates the rationale, but feels like is actually perfectly suited to your community and the students that you serve. As Seng mentioned, you know, the sort of Title I school improvement funds have to be spent on one of the top three tiers of evidence; you can’t use those funds to demonstrate _____ rationale approach. But if you’re not using those funds, you could sort of pair, say, a moderate or a promising intervention with some things maybe around that tier four level, but might be better-suited to your local context.
Maria Ferguson: And, Seng, as Candice was talking about, I mean, teachers know their students better than anyone. Can you talk a little bit about how the state, in Nevada, has helped school leaders identify strategies that are really well-suited to their student populations? And to understand that not all interventions are created the same?
Seng-Dao Yang Keo: Yes, so, we actually started with prioritizing funding. And the reason why that’s important to note is, in thinking about what actually happens within a classroom, and providing support within a classroom and within a specific school, to make these large mind shifts and behavioral shifts _____ is going to require significant investment. Not just time, but we have to put some money into it. And the other thing is, really, we focused on, really, capacity, so we had to assess what was the capacity at the school level, at the district level, and at the state level, in terms of really providing system level coherence and alignment around this work. And so, one of the things that we ended up doing as a state is, we partnered with outside organizations to help build our capacity at the state level, to support districts.
Encouraged these partners to also work with the [glitch interferes with audio] to then provide that support at the school level, to school leaders and, of course, the teachers [glitch interferes with audio]. In addition, created resources for not just [glitch interferes with audio] and school leaders, but, really, resources that we could then engage teachers with. And we’ll be, hopefully, rolling that out, actually, pretty soon. We developed an evidence-based list for the state of Nevada, and that’s in partnership with two research organizations. We work with Dr. _____ _____ _____, and then, currently, we are working with _____ _____ to then help us vet the evidence. So that it makes it easier for community leaders, school leaders, teacher leaders to then identify different evidence-based interventions that might meet the unique needs of the schools.
And then, what I would say is, this also meant being very explicit about valuing rigorous evaluation [glitch interferes with audio] being done. Because what we often heard from school and district leaders was focused on saying, “We really believe that the dollars need to be spent on intervention [glitch interferes with audio] needs to be spent on students.” And one of the pushbacks that our team had with them was, basically, “How do you know if these interventions are working?” Right? We’ve been implementing these interventions for several years, and, yes, we agree that, you know, interventions have to be the focus of this. But how do you know [glitch interferes with audio] collect the data, collect the evidence.
What are hope you are hearing, in the Nevada story, is that we did everything we could to change our own practices at the state level. And I think that’s really important for people to hear: we had to change how we did business at the Nevada Department of Education. We had to change what we thought we knew, and what supports we thought we needed to give to districts and schools. We needed to listen more and be flexible, so that we could hear the needs of the districts and schools. We needed to be nimble and be able to adjust, in even in real-time, how we were providing support and the supports that we were providing, so that we could really empower and support educators across the state, especially teachers, especially school leaders, and district leaders.
And so, what that looked like is changing our day-to-day. So, for example, our team had several one-on-one meetings with district leaders, with school leaders, even, where we would say, “We are going to carve out, you know, an hour, so that we can have office hours with these, you know, school leaders.” And that’s all we did, and were did that frequently. And so, that’s really how we saw our role, and just being humble about saying, “We need help to make this happen, and we don’t know all the answers. And we are going to need district and school leaders to be at the table, as we are developing [glitch interferes with audio].”
Maria Ferguson: That’s great – thank you. I wanna go a little bit broader, here, and talk comprehensively about the difference between a standalone intervention and a continuous improvement process. In our work, we have heard conversations and concerns that ESSA’s evidence-based requirements may turn into more of a compliance exercise, as opposed to something that really does foster continuous improvement. And, Candice, from your perspective, I’m wondering what makes for a continuous improvement mindset. Can you talk a little bit about how leaders can support using ESSA’s evidence-based requirements as a jumping-off point for sort of a larger worldview about how we view continuous improvement?
Candice McQueen: Yeah, in Tennessee, we had that very concern, and I appreciate how you verbalized _____ _____ that this would become a checklist of _____ _____ _____ products or services that we said, “Yes, those are evidence-based, and so you need to do those,” and somehow that became the magic bullet. So, we had to step back and actually say, “We want you to create a theory of action and a systematic continuous improvement model, that these components might fit into, that are evidence-based. But how are you creating systematic and long-term sustained change using that?” _____ _____ _____ Nevada was one that we looked to, to how to really think about those partnerships.
And then we thought about it in our context of how do we use what we’re doing _____ _____ state, and change our practices, just like it was described in Nevada. To create that continuous improvement mindset at the state level, which we hoped would then flow into how the districts thought about their systems of support, and then how the schools did, as well. A great example of that is how we, as a state department, look at our regional offices in Tennessee. So, as commissioner, we had really changed, over a period of time, of our regional offices, we have eight of them across the state, from a technical service center to really more of a support center that was elevated in how we thought about the instructional core.
How do we think about curriculum approaches? How do we think about instruction and evaluation? How do we help connect that to assessment, and create systems that support the instructional core? So, when ESSA’s evidence-based requirements came around, we knew we had to even do more, there. Where, instead of doing one-off workshops through our core centers, or one-off topics, how would we ensure that all eight of those offices had their own continuous improvement mindset. So we adopted improvement science work, and that improvement science work then helped make sure our folks could work directly with districts on identifying, you know, “Why are they doing this? What are the drivers? What data do they have now? What particular service, partnership, product would actually help define that?
“And now that you have that, what’s next?” Well, it’s that continued professional learning, what are the systems of support that support that, and then how do you know it’s working. And how do you know when it’s not and you need to step back and make another decision within the funding decision that you just made, right, how does that fit into it. And we needed to do that with what called cycles of improvement, where they were smaller cycles of improvement that really tested out, “Are these products, partnerships, services working on smaller – ” in smaller increments of time.
We had always looked at these at a yearlong model. We needed to stop and look at it more quarterly, _____ what do we know, how do we know it, what’s the evidence telling us. And we used those service centers, which we call our, you know, regional centers of excellence, to actually help with that process. What that did on the ground, then, at the district level was when districts would say, “You know what, we need better PLC, so we’re going to send everybody to a PLC conference, and have them create PLCs,” _____ _____ _____, “Well, how does that fit into your larger work that you are trying to do?”
Yes, PLCs might be a great partnership that you want to bring in, a great model, but it does not necessarily do anything by itself. So what’s driving why you wanna use PLCs, how are you going to use, then, the partnership that you wanna bring in on that, to actually support the whole system, teacher leaders, and all the systems that actually then would engage with PLCs? And then, what do you hope to get out of them, and how will you know? We have to teach our practitioners and teach the people on the ground how to think about their work _____ _____ action research, and to do it in a way that creates that continuous improvement mindset.
And _____ _____ _____ saying what Diane said about that tension between, “I have this thing, this service, this partnership, and it’s going to be the magic bullet.” And I love that researchers said, “Wait a minute, you’re gonna have to think about those iterations on that for your unique context.” That is exactly what we were trying to do in Tennessee, to say how do you marry those two. You’ve got a great partnership, but you’re gonna have to iterate on that, and let’s use this continuous improvement scientific way that you think about improvement alongside it, teach our own people how to do that. Which then allows you, we believe, to grow faster, and certainly have a sustained effort with whatever evidence-based partnership you’ve brought in.
Maria Ferguson: That’s great. And for the acronym-averse, I wanna point out that “PLC” means professional learning communities – most you know that, but just in case, always wanna throw that in there. So, we’re on four years of ESSA, now, is that correct, yeah? So, and, Anne, you were actually at the department when ESSA was passed. So I wanna bring it to both Anne and Diane, here, briefly, as we get ready to close this, to talk about what recommendations, moving forward, can we make to make ESSA’s evidence-based requirements more effective, even better, even spur more, lead more to that kind of continuous improvement that Candice was talking about.
Anne Hyslop: So, I can start [crosstalk]. Yeah, I think it’s important, first, to think that, yes, ESSA has been in place for four years, but, you know, there was at least a year of that, that was just the department putting regulations in place, or trying to put regulations in place, and issuing guidance. And then, a year for states to be writing their plans. And then, okay, finally, two years in, maybe we’re starting to identify some schools that might need to adopt these interventions that are evidence-based, and identify evidence-based interventions. But, you know, we’re now in year-four, you know, so we’ve had these two years of planning and getting ready to do things.
So sometimes, you know, this is really kind of year-one, the 2019-2020 schoolyear. So, I think there are certainly some things that we can already see that could be improved in the underlying law, but I think in the coming years, there’ll be a whole lot more that we’re going to be learning, and we need to continually be doing research like CEP has done, to continue to build our knowledge base of how evidence is playing out in ESSA implementation. One of the things that I think, you know, and I hope that Seng and Candice would agree with this, is that, I think ESSA maybe made a slight miscalculation in the capacity of school districts to manage this work. So, a lot of the sort of onus of approving evidence-based interventions, and finding them, is falling on school districts, some of which are LAUSD and have a lot of capacity, and some of which are tiny small rural and isolated school districts.
Maria Ferguson: It’s a little bit the bete noir of federal policy, in some ways [laughter] [crosstalk].
Anne Hyslop: And so, finding a way, I think, to get more money and more capacity, particularly to state education agencies, who we are asking to transform themselves from compliance-oriented agencies into more support agencies that are really partnering with districts in this work. So, you know, to have the ability of a state to set up a research partnership with a major university or two in their state, where they can be not just approving existing interventions to see if they’re evidence-based, but also building that evidence-based and really studying their interventions. I think there’s more that we can do on the research side, so, making sure that, you know, the new I3 grant program, Investing In Innovation, now called Education Innovation Research – it’s EIR – I’m probably butchering the acronym, as well. [Laughs]
Maria Ferguson: At least you explained the acronym, that’s good.
Anne Hyslop: The EIR grants, like, expanding funding for that, and really tailoring that to, “Well, we wanna use this fund for school improvement.” But I think finding a way so that some of the school improvement funds can actually stay at the state level, to build the capacity of offices like Seng’s, in Nevada, to manage and do this work. ‘Cause I think we’re just not going to possibly succeed, especially in those states that are identifying large numbers of schools, if we can’t build the capacity of our state agencies.
Maria Ferguson: Great points. Diane, do you wanna add anything to it?
Diane Stark Rentner: Oh, I agree with everything that Anne said. I would just make a plea on the research side of things, you know, the US Department of Ed, they should make sure that every grant that they give for research, that those findings are publicly available. Same with foundations, I know the Hewlett Foundation and some others have open access rules with regard to the research that they fund, but that goes a long way to help the educators actually get the information. Again, as I said, they should package it in a way that educators can access and understand. And this is a technical point, but we heard it again and again, to the extent that, on the research side of things, that people can start to, like, tag it to the evidence tier, that really helps the educators.
‘Cause often they would _____ _____ read it and go, “Well, I don’t know what this is. I hope it’s one of tier one, two, or three, but I don’t know.” So, those small things can help these under-capacity school districts make some great strides in this area.
Maria Ferguson: So, before we close out, I wanna just punch it back to Seng and Candice, for, very quickly, can you both share with us what you think would be helpful, moving forward, to make ESSA even more effective, very quickly, before Anne does our wrap-up. Seng, why don’t we start with you, and then, Candice, you can have the final word on this topic.
Seng-Dao Yang Keo: _____ thank you. So, one thing I would say is, we need to address the resistance to change by modeling the continuous improvements that we are seeking within schools and within districts. And so, really leveraging research, data, and evidence at every level to drive policy decisions, to just, to also drive daily decisions that teachers make, that school leaders make [glitch interferes with audio]. And one other thing that I would add to this is, go back to leading for equity, because at the heart of ESSA, at the heart of the ESSA evidence levels and tiers is the need for us to make sure that we are servicing all students. And that we are meeting the needs of the local contexts. And so, I think in knowing that, not only focusing on the program side of the work but also thinking about these stakeholders that also play a role in this.
So, I will just note that the grants team and the _____ teams within districts and within the state agencies, they are often left out of these conversations. And it’s really important for them to understand, when they’re writing grants, when they are trying to balance budgets [laughs], really helping them translate what those numbers really mean. And for us that would mean improved school and student outcomes [glitch interferes with audio].
Maria Ferguson: That’s great – thanks. How about you, Candice?
Candice McQueen: Yeah, I would add just to encourage our states, in particular, but I would include all levels in this, too, to take full advantage of the leverage points they have under ESSA. I have been surprised at how many states have not taken their role in talking about what these different evidence levels mean, how they can be a partner in creating, you know, a pathway to the strong partnership, introducing these partnerships to districts. And then using, as was mentioned earlier, that school improvement plan as that lever and that MOU that allows you to really push for change. And to take full advantage of this opportunity that you’ve been _____ _____ the state department appropriately, on the research and policy side, to make sure that this is done well and that you have the technical support to do it.
And then, I would say the same thing on the federal side, take full advantage of what this says and support it well. And do the things that are going to need to be done to make sure that, while we’re just at the beginning of implementation, that it doesn’t flounder. That this is till kept front-and-center, as we continue to do this work on the ground.
Maria Ferguson: You guys have all made such good and distinct points about this, so I thank you for that. Just quickly, before I give it back to Anne for the wrap-up, I do wanna just note that, a note of thanks to the Hewlett Foundation for supporting this work that CEP has done, for supporting this webinar. They’ve been great partners in this and we’re very grateful for that, and have really been important in sharing the lessons learned from ESSA, which is always gonna be a useful thing for anyone in the field. So thank you, Hewlett Foundation, for your support of this.
Anne, bring it on home.
Anne Hyslop: Well, this has just been such a great discussion. Thanks to Candice and to Seng for sharing their insights, and to Maria, Diane, and the CEP team for cohosting with us.
If you missed any of today’s webinar or you want to share it with your colleagues, you can access archived video from the webinar at all4ed.org/webinars. For more resources and information about evidence-based interventions, and the state of ESSA implementation, you can visit our websites at www.all4ed.org/essa and www.cep-dc.org. In particular, be sure to check out each of the reports in CEP’s series on ESSA evidence and school improvement.
Thanks again, and have a great day.
[End of Audio]
Welcome to the Alliance for Excellent Education’s Action Academy, an online learning community of education advocates. We invite you to create an account, expand your knowledge on the most pressing issues in education, and communicate with others who share your interests in education reform.
or register for Action Academy below: