October Board of Elementary and Secondary Education: MCAS and NAEP

 And the presentation is posted here

and if you're watching the video, this starts about 29 minutes in

Curtin: notes the public release of MCAS and accountability data last month, NAEP results released for the country

MCAS: important to level-set where we've been in terms of the administration; have had four different kinds of administration over four school years

• 2019: full administration to all
• 2020: no tests administered (to anyone)
• 2021: half the test administered to 3-8; full test to grade 10
• 2022: full test to all 

2022: mixed results compared to 2021: math scores increased; ELA scores declined 'for the second year in a row"; science scores increased slightly

compared to 2019 "we still have a ways to go" across all subject areas

two focus areas in ELA: early literacy challenges; lower writing scores

"not surprisingly" student absenteeism over the past three years has played a significant role

Math: the best news
"I'm not a huge fan of the terms 'loss' and 'recovery' being used, but they're certainly out there being used in the education world"
saw recovery across all grades: about 50% of the loss in 2021 was recovered in grades 3 and 4
grades 3-8 big dip from 19-21, but big recovery in 22
rate of loss has slowed considerably in grade 10 between grades 3-8 and 10, same percentage point loss from 19-22 across all 
Hills: how much comfort do we take in that given the 2021 administration?
Curtin: half the test provides half the information; "ultimately, I think we should be looking at where are we now, and what's the long sort of result versus pre-pandemic"
"but it is important to note the efforts of our students and our educators in 2022 to show the increases that we did show"
"certainly our new baseline is the difference between 2022 and 2019"
by race and ethnicity: question has been disparate impacts
"there are two stories here"
Achievement for African-American and Hispanic students remains low
"but it's important to note that the gaps...between students...have not necessarily increased over the time"
still gaps that exist and this is work moving forward

ELA: different story
variable loses across grades 3-7, 8 shows signs of recovery
3-8 another 5% loss, 11% 19-22
grade 10: "an interesting situation"
2021 had a slight increase; 2022 lost ground compared to 2021
3% less 19-22
interesting note on grade 10 students: didn't have any data on those students on those students since they were in grade 7 in 2019
"first look at a group that we actually didn't have any data on since 2019"
race and ethnicity: "achievement gaps have not changed" much
grade 10 African-American and Hispanic students gap closing with white students in ELA

within MCAS data, can dig down and try to isolate, especially in ELA
much of the ELA challenge to early grades; loss accelerated in grades 3-5, rather than grades 6-8 where it slowed; thus early literacy

every student has at least one required essay; average points per essay decreased in 3-8
declines larger in grades 3-5
also a loss in middle school, challenges exist in early grades
"will have to keep a real eye and redouble our focus"
only get more students who have been impacted by the pandemic as time goes along

science
small recovery in grades 5 and 8
the first administration of physics and biology (next gen test) is not comparable to prior years
new baseline: 47% M/E for 10th grade science

impact of student absenteeism over the past three years

"students have attended less school--I'm not breaking news in any way there"
average student in MA missed 11 days of school in '21 and 15 days in '22, thus 26 days of school over the past two years
chronic absenteeism in grades 3-8 increased by 138% between 2022 and 2019; nearly 100K students were chronically absent last year
chronically absent is 10% of days of school
18% of all students missed 18 or more days in 2021; 28% missed 18 or more days in 2022
in urban districts, nearly 40% of students were considered to be chronically absent in 2022
"easy to see the loss of instruction due to missing school"
"certainly helps to inform some of what we're seeing today"
"obviously, the main driver of this was COVID"
the number of positive cases had in schools in 2022 and take the minimum amount of time that students were out of school, that results in 1.7 million days of missed school due to positive cases
"hopefully, knock on all kinds of wood, we will not have to experience this year"
is the state going to actively DO anything to PREVENT that? Because they aren't so far!
Moriarty: very broad high-level overview
"just really want to lean into the urgency that this really does call out"
"looking at close to half of kids not meeting proficiency across the grades before the pandemic"
"and now we're seeing this accelerated in a huge way"
not back to 2019 but how do we change the work that we're doing "so these trajectories are not on a baseline that was unacceptable to begin with"
think should look at increase in students that are not meeting expectations "students who are very very far behind"
"think it is worth looking at the places where the majority of students are not learning to read on time...believe the whole school culture is effecting when most kids are behind"
"over 300 schools that didn't teach kids to read in 2019 based on third grade MCAS...over 500 this time around, over 60% of our schools"
that is not what that data means. at all.
Stewart: asking about the data that the Department is collecting around family engagement
Curtin: I don't know if we're getting information around family engagement explicitly; something we'll have to get back to you 
Stewart: authentic engagement of families help to close the gap

Curtin: on accountability, using 2022 to reset the baseline moving forward
"did release a wealth of information"
all data for accountability data, all school and student group percentiles; federal or state designation for a small number of schools
No release of determination of a school's need for assistance or determination; no targets or measures of targets; back to that in 2023
current underperforming or chronically underperforming maintained in public release; review happening now and any changes will be reported in the coming days

Monday was NAEP day
"important to understand who is being talked about in these results"
representative sample of 4th and 8th grades in reading (just reading; not writing) and math
all states contribute to the nationwide sample; had about 2500 students sampled per test; in total 10,200 7% of students in 4th and 8th grade 
sampled in 262 schools in MA, which is 20% of schools with a 4th or 8th grade
MA was top performing school in the 2022 NAEP administration, statistically tied for top-ranking on all four tests
"yesterday is a big day, with the score release; it's a researcher's dream...it's a headline writer's dream, in some ways when the results come out"
"other than the relative standing of Massachusetts compared to other states...there wasn't a lot of us to learn from the NAEP results that we didn't already know from MCAS"
"really a testament to the quality of our test"

"across three of the four tests, statistically, the results were spot on; the eighth grade math results had a little bit of variance"
"generally speaking, the results told us exactly what the MCAS results told us"

Hills: "we may still be at the top or near the top compared to other states, but we're not in a good position compared to 2019 or where we should be in general"
Curtin: "that's absolutely true, but they're telling us the exact same story that the MCAS data" told us
relative standing hasn't changed for the last decade plus
for the last six administrations, we have either been first by ourselves or tied for first
"that doesn't mean our level of attainment hasn't changed, and it has"
an arc over the past twenty years, 2022 results put us where we were back in 2003, a little lower in 8th grade math (though reading has been more of a straight line)
not inconsistent with MCAS results
MA against all national public school students; still performing at a higher level than the nation
high level overview of MCAS and NAEP

West: comment not question
"really we got a very consistent signal from the NAEP" compared to MCAS
"want to praise you and your team...that was really not the case nationwide"
"that's what we learned...it's an audit on our results"
"as a member of the National Assessment Governing Board, I can't have you saying we didn't learn anything from the results" (laughter: Curtin: "Fair enough...well said")
two signals that we can be very confident in
both of them are conveying that we need to bring a sense of urgency to academic recovery in the wake of the pandemic
although Massachusetts is the highest performing state overall, it is generally not the highest performing state if you look at the performance of subgroups; often in the top tier of states, but there are some states that are better at serving particular groups of students
Stewart: urge a renewed urgency around engaging families around this; authentic engagement improves schools and student proficiency
Lombos: process how to do engagement around this, but also shifting programs to meet the need
Commissioner: kicked off a vision session "what's next"
started pandemic recovery last year, but this is about larger education vision for the state: a multifaceted approach including family engagement, mental health supports, research-based approach
Lombos: love to know how to be supportive and helpful
Hills: how you think of goals for the state overall
related to Commissioner's goals; "not saying 'what's a number,' but when you think of these kinds of recovery needs...how do you think about the period of time and the rate at which recovery takes place"
Curtin: a big challenge for all of us
target setting in the past came up on methodology based on improvement we'd seen from like-performing schools; based on historic data
pandemic has changed the course of the trajectory of our data
try to think about our methodology, that sets goals for state and districts and schools BUT is also cognizant of what has happened over the past three years
finding that balance will be difficult, not going to sugarcoat that, but working on that right now
Hills: lack of a a reasonable rate of recovery
"but goals can also be used to say this is where we believe we need to be one, two, and three years overall...and a year from now we've fallen short, it's a FAILURE...we need to pivot and recalibrate our resources"
"I think in the absence of something like that--with the resources going in--but we won't really know if that rate of improvement is what we think as of now is what it should be"
Curtin: certainly we have the history of No Child Left Behind saying 100% of students should be proficient; "the important thing to find is the intersection of ambition and achievability" 
that's both a good answer and nicely put
"happy to come back and talk to you when we release those goals and targets for the districts, we would be happy to come back and talk to you"
Hills: "I'd ask that you come back for a CONVERSATION with the Board before you've locked into the goals and released them"
that's a truly terrible idea, and totally not their job 
doesn't want to approve targets and goals for every single district
"everything else over the course of the year could go well, but if we're not seeing progress one, two, three years from now" that's the issue
"but get the input before you're at the end of the process"
Canavan: "goals to me help with engagement"
if we know what we're aiming for "even if it's a moonshot" hopefully we can use the data to rally people to be more part of that process
that isn't how this works
I think it'd be an exciting process and exciting information to share publicly
Mohammed: trajectory of data that has a trendline
how do we think about how much any of those things tie to an increase in scores?
how do we think about those as we design interventions?
Curtin: evaluations of programs is important, group at the department that works on this
seeing degree of effectiveness
continue that going forward with recovery methods
West: concern that's hanging over me
"if we're over ambitious in the goals we set...there's always the danger that other students become a distraction and all the resources go into those measurable outcomes"
there are students in high school that won't matter for district's accountability going forward, but who were clearly impacted by the pandemic
look beyond our measured outcomes
"in some sense, the task is most urgent for the students who are high school sophomores now who are about to be through any stage they're contributing to MCAS scores but we don't want schools and districts to be ignoring their needs; to some extend, we want them to be prioritizing their needs"
a comment on the limited framework on in our 
Hills (he is acting as chair, had said he'd call on Rouhanifard, but instead jumped in himself; wondering if anyone is going to call him on a point of order if he continues to do this every single meeting when he is chairing) just a follow-up thought to that: not a believer that everything needs to be measured; judgement is needed
"I agree with what you're saying; you come back and tell us, at least for some of the key areas, if scores are X, are we feeling happy or are we not feeling happy"
Riley: "I think that's a nice segue"
Moriarty: one of the strengths of this presentation was including the chronic absentee data; helpful to put that in this context to start asking the right questions
"clearly we have a lot of families and students who have been disengaged from school in a greater way than we have ever seen before. None of this tells us how to re-engage"