2015 Accountability Levels: everything's made up, but the points matter

It's time once again for the release of the Massachusetts statewide accountability levels!
The big news for Worcester yesterday is that Burncoat Prep, in its first year that it was eligible to do so, left Level 4 status. It should be noted that both of our previous Level 4 schools, Chandler Elementary and Union Hill, are Level 1 this year for closing achievement gaps. Elm Park remains Level 4 (it was always going to, as turnaround plans are for three years).
Statewide, the big news was that Madison Park Vocational in Boston has been declared Level 4; that, despite having a turnaround plan already.

You can find all of the school levels on this list from DESE. Schools have to have a PPI) of 75 or higher for both their entire student population and their subgroups in order to be Level 1. (You don't have to have all high scores to be Level 1, just break 75 on both.) Slip on either, or on any subgroups, and you're Level 2.
This gets a little weird when the state starts trumpeting districts that are all Level 1 and 2, as it did yesterday. A subgroup has to have 25 students to "count." Thus a school that has (for example) 23 ELL students doesn't have an ELL subgroup. Likewise with any such subgroup. Beyond the myriad of other ways in which this system is stacked, this is another one in which a few kids in your student population can change all the results.
This gets even more odd when it comes to Level 3 this year. Level 3 schools are the bottom 20% of schools in the state. Because there will always be a bottom 20%, for every school that goes up a level, one must come down a level. However, schools that took the PARCC this past year were held harmless--they could not slip a level down. Thus the only schools that newly fell into Level 3 were schools that took the MCAS, and their chances of doing so (since they weren't--I hate using this verb but it is sadly accurate--competing against all schools, but only against those taking MCAS) were proportionally that much greater.
And note that this will be even more the case this coming year, assuming that more schools switch to PARCC: only MCAS schools can fall, so proportionally more MCAS schools will fall.

Last year, I ran through the way in which Level 4 schools are declared, pointing out that this isn't based on lowest test scores, lowest PPI, or any such thing, but solely at the Commissioner's discretion. It took some digging, since they've moved these charts over to the Executive Office of Education's website* (which is kind of a mess, to be kind), but I found the spreadsheet that lays this all out again. If you remember from last year, this is the data that rank orders schools based on their combined PPI's (the full student body and the subgroups). This year, the column with the percentile rankings is column O. Thus the following is based off sorting by column O.
First, there are 309 schools that have "insufficient data" to have a percentile ranking. This includes all of the early childhood centers (if you only teach kids in grade 2 and younger, you don't have test data), but it also includes a good number of full elementary schools and high schools, and it has a mix of schools that took MCAS and took PARCC. I don't know what that means.
But again, the schools that are lowest in percentages are not the ones that are, or are being declared, Level 4. There are schools that are at 1%; Madison Park is at 2%. And again, that isn't a one-year thing; PPI is based on multiple years of data.
No doubt we'll hear on Tuesday that the Commissioner has been "very concerned" and probably "for some time" on Madison Park; note that Southbridge is also on the agenda, and the lowest percentage any Southbridge school has is a 3 (there are 28 schools that are 1's and 2's).

All of which is to say, again: this isn't a clean, statistical system. That was true of Level 4's and 5's already; it's now true even of Level 3's.
And that makes it pretty important that under the Every Student Succeeds Act that President Obama signed today, states are only required to do something (and what is left fairly vague) for their bottom 5% of schools.
It's time for Massachusetts to change the law.


________________________
*to find it: EoE website>>Departments and Boards>>Board of Elementary and Secondary Education>>A-Z Programs and Topics>>Accountability>>Reports>>Accountability Lists, Materials, and Tools>>Lists (which is the 2015 Excel spreadsheet)
Really. You don't want to know how long it took me to find that.