Friday, February 24, 2012

Family income = a top ten list

Everyone would like an easy answer in education.
We want an easy way to tell the good schools from the lousy ones. Thus the top ten list.
Lists give us the illusion of control over something that is complicated. They're neat, and they feed into our human wish for a narrative of winners and losers.
It is, however, an illusion, and this recent version is again based on factors that tell us very little about education in those schools.
GoLocalWorcester creates their top high school list through three factors:
  • SAT scores (broken apart by math and English)
  • MCAS scores (broken apart by math and English)
  • per pupil spending
Let's take these backwards: per pupil spending in Massachusetts is (as I've gone over here before) at least partly determined by the foundation budget (here's Luc Schuster for a refresher, should you need it). The minimum per pupil funding required is going to average out to be higher for a community with more special education pupils, or, more tellingly, for vocational pupils. To ignore this is to ignore the heart of how education is funded in Massachusetts.
Now, clearly, every community on here is not settling for minimum funding of education, and certainly that matters. A community that chooses to fund education over what the state requires is choosing to make education a priority*
However, that does go along with two other things: first, it argues that the community, and that community's parents, prioritize education over other choices, and perhaps have the freedom to do so. A public community choice argues that private choices--around how family funds are spent, around how family free time is spent--may well also be educationally focused, and that the parents of children in that community can make those choices (are not, for example, working several jobs to make ends meet, or are adjusting to a new culture, or are not present for the child at all).
It also goes along with the communities that have resources they feel can be freed for education. A quick glance down this list bears that out. These are communities with high property values and higher family incomes.

And that brings us to the other two data points used.

Tests are numbers and people believe numbers. We have a tendency to think that numbers exist in a vacuum, swayed by nothing outside.
SAT's, however, clearly correlate with family income**. You can see here for the latest round of this analysis. This, along with other troubling correlations including race, is why a growing number of colleges have stopped using them (well, that, and they're not a great predictor of college success, either).
The MCAS suffers from the same correlations; as W. James Popham points out, it's really difficult to design a standardized test that does not (more from him here; see also Alfie Kohn).
Thus we've got three shots at family income, plus a dash of the foundation formula measuring what makes great central Massachusetts high schools.
Unfortunately, this does nothing to help us discover what those schools do with the students who walk in the door, whatever their family income or other background.
This just isn't a useful measure of education.

*I'm ignoring here the school GLW declared number one, Mass Academy of Math and Science. MAMS is unlike any other school in the state, in that it is funded directly from Boston. To be honest, I have no idea how their per pupil funding is calculated. They also are a school of only 11th and 12th graders that has strict admission standards, requires pupils to provide their own transportation, and draws from all over central Mass, so they're an outlier all around. I'd put Advanced Math and Science Academy in a similar category, except for the funding, as they, too, have control over their school population (as a charter).
**If you follow the links, you'll see there's a correlation with parent education level, as well, but it is outweighed by family income.
And yes, post hoc ergo propter hoc.