Board of Ed for December: accountability system

Wulfson: why we're doing this work
believe state has an obligation to identify schools in need of intervention and to work with those schools
1800 schools in the Commonwealth and most are doing a really good job
believe we have an obligation to identify and shine a spotlight on the schools that are doing really well
believe we have an obligation to share information with parents
believe we have an obligation to provide information to the taxpayers of the Commonwealth
required by both federal and state law to hold districts accountable and report on their performance

ESSA gives us a "great opportunity to revise" accountability system
lots of feedback over that time to help us think about how we can do this important work even better
a few overriding goals and considerations

1. a system that works for Massachusetts but meets federal requirements (a single system)
2. testing is only one element; must have multiple measures of school and district performance, 'though more measures is more complexity
3. not all that is important can be measured in a quantitative way (thus report cards)
4. very sensitive to labeling of schools; any such thing needs to be done for a clear purpose
5. aware of comments of correlation between achievement and demographics; measures that allow schools to be recognized despite or even because of demographics; but also give students what they need to succeed after high school 

appreciative of feedback from stakeholder groups

coming back next month for discussion on achievement versus growth

later in year will be implementing new system through new regulations 

some discussion of how detailed want regulations to be

Johnston: public discussion of design of accountability system

ongoing discussion

Curtin: what's included in the framework

will not be talking about weighing today 

"learned our lesson from the last time we had that discussion..it tends to take on a life of its own

Here’s what is included #MAEdu pic.twitter.com/WHwSYjf71x

— Tracy O'Connell Novick (@TracyNovick) December 19, 2017

achievement in ELA, math, and science
student growth percentile
"eventually we might want to look at growth as a standard measure" but need more data
high school completion (four year, extended engagement, annual dropout)
dropout "much more current" as high school rate is a lagged measure
five year graduation rate plus the percentage of students that you still have enrolled in your schools
"we want to incentive bringing those kids back into the school culture"
extended engagement is replacing the five year rate
Wulfson: "it's up to the school committee to decide" the age of students readmitted and we'll respect that
English proficiency   (as required by ESSA): progress towards attaining proficiency
setting targets at six year mark, will depend on where students are coming in
Fernandez: what about actual attainment Moriarty is concerned about if that's appropriate (clearly he thinks it's too long)
Peyser: is attainment included in this? yes, counts towards progress
additional indicators: chronic absenteeism (all schools) percent of students that miss more than 10% of days
applies across all grades, not just testing grades
                                    percentage of students passing 9th grade
"More than four times likely to drop out of high school if you fail at least one course in 9th grade"
probably has gained the greatest amount of concern from stakeholders
concern that it will be gamed
also concern that middle school is the lack that high school becomes responsible for
West: perhaps return to weighing
perhaps transparency without weight
Curtin: perhaps reporting on district report card; West: "that's the zero weight"
McKenna: it's usually because you're behind by the time you enter ninth grade; it's really a status of your entering ninth grade; "really a reflection of middle school, not high school"
Curitn: point taken, "I think we're starting to build a little bit of a mountain of constructive criticism around this element"
Trimarchi: or seventh grade for elementary school
Curtin: more variance in grading systems of middle school than in high
Wulfson: agree that it's always the year before, but also every school has to take students where they are
Moriarty: would not want to create anything that just passes students through
Sagan: "the mountain grows..."
Curtin: objection around chronic absenteeism; to what extend districts can do anything about it
percentage of students completing advanced coursework (IB, AP, dual enrollment); we have more students ready than are taking them
"have some pretty sizable equity gaps"
Peyser: MassCore potentially another an indicator
McKenna: are there superintendents saying that this is a resource issue?
Curtin: that's why it's broad; an economic component to AP
Moriarty: hope that history and social science would be included once they're required by the state
Wulfson agrees that's the plan
Doherty: art and music, school culture: anything included
Curtin: view this as one that grows over time; start on school and district report cards
"report card will complement accountability system"
Wulfson: early on surveys on school climate; making sure it isn't or can't be gamed
I missed a question from Morton there
McKenna: are we asking schools to report on what we're doing with the arts?
Curtin; we have the course schedule for all publicly funded students in the Commonwealth
Fernandez: family engagement
Curtin: having a system that grows over time
"that was the easy one" (to laughter)

"we believe in both" using relative and target components
In other words they get measured against themselves as well as absolute measures
accountability percentile calculated using all available indicators for a school for both schools and subgroups
"continue to calculate a school percentile"
next generation MCAS allows comparison across grade spans (at this point high school still transitional)
West: why is it a virtue to compare across all grade levels? May be variation at tails
could have bottom five percent dominated by elementary schools
Curtin have run some simulations on this: lowest performing schools on old test tend to be lowest performing on new test
middle schools may do well "for middle schools" but not well across all
Peyser: any pattern for small versus large?
Curtin: both tails of the distribution have been the same; some change in the middle
"it's more about the universe than about the grades"
Peyser: also eventually, there will be more than one year
Curtin: agrees
Wulfson: this is not the same as the Globe publishing percentiles; "we obviously can't stop them from doing that"
but this includes more than that

Curtin: "want to put a focus on closing the achievement gap by raising the achievement floor"
found that a lot of gap closing was by the highest performing group doing worse
setting targets for all students on all of the indicators
"we are proposing to take a different tack than we have in the past"
"a group of the lowest 25% of students" on all indicators who have been in the school for more than one year
controlling for transiency
"we want schools to intentionally focus on the performance of their lowest performing students" ON ALL INDICATORS who they have had over time
Morton: now there is not a disincentive to welcome students; speaks of Puerto Rico
target for all students; target for their lowest performing 25%
schools might decline, not change, improve, met target, or exceed target
"right now everybody has a school percentile and everyone has a group of possible points"
West: subgroups?
Curtin: "continuing our fine tradition of posting an inordinate amount of data on our website" yes, that will continue
Peyser: could be steps along the way on weighing measures; also what our parameters are, as well as the signals we're trying to emphasize or send

"we will no longer be placing schools in a vertical hierarchy of schools level 1-5"
a lot of angst on level 1 and 2
weren't really differentiating assistance between levels 1 and 2
"what do we want the system to do"
we want to identify our lowest performing schools" for assistance
we want to identifying our highest performing schools for learning from
in the middle are "a lot of very good schools that we don't necessarily need to label"
complaint that lowest 20% was identified
"going to normatively place only lowest 10% of schools"
"we were choosing level 4 schools primarily out of lowest 10% of schools"
have thought about weighting as a 50/50 split
if that's true, most schools will have half of their categorization based on students who have been there for at least two years

So that can look like this #MAEdu pic.twitter.com/3Fi3ecnF4e

— Tracy O'Connell Novick (@TracyNovick) December 19, 2017

two year participation measure of testing (to mitigate a single student or small handful throwing it)
schools requiring assistance is going to contain not only the lowest 10%, but also schools with low graduation rates, and schools with low participation rates over two years: estimation of 15%
Peyser: so Boston Latin moved to Level 2 due to non-participation: this would put them in the bottom category?
what would the assistance or intervention be?
"are we in essence just trying to call them out?"
"yes."
West: a ton of information here
haven't seen this level of weight placed on 25% group in other states
going to be hard to look good with that level of emphasis
Curtin: not sure I'd agree with that; it's a growth measure
West: only on criterion referenced
aka: those 25% are measured using growth
McKenna: DESE should be where people come to get assistance and support
Wulfson: adult consequences
Curtin: nothing here takes away from authority of board or of Commissioner
Peyser: great deal of pressure and attention on targets
being able to look at turning the dials in different ways to understand the implication of emphasizing
Curtin: not planning on a jumping into a five or six year target
important to have a second year
will set target based on previous year; longer term targets after two years of data

Issues
also redesigning school and district report card (take the survey!): that includes other information (including access to arts, social emotional, school climate)
high school assessment transition coming up next year
administering two kinds of MCAS at the same time and then what that looks like
district accountability will no longer be based on the performance of its lowest performing school
Board discussion on January 23 to discuss system refinements and weighing; "will be looking for direction and feedback"
February 27: proposed amendments to state accountability regulations
have met with some superintendents, "will meet with frankly any organization that wants to meet with us" but are looking to do broader rollout
Johnston: "underperformance is not destiny" and schools that are improving are shown as such in this system as proposed