Charters may be good on test scores, but maybe not so much on everything after

No doubt you saw the coverage swirling yesterday of the NBER paper raising some serious questions about what charter schools are long term doing for their students. To quote from the abstract:

No Excuses charter schools increase test scores and four-year college enrollment, but have a small and statistically insignificant impact on earnings, while other types of charter schools decrease test scores, four-year college enrollment, and earnings. Moving to school-level estimates, we find that charter schools that decrease test scores also tend to decrease earnings, while charter schools that increase test scores have no discernible impact on earnings.

This of course raises some really troubling questions on what we're doing with charter schools, then, since test scores aren't intended to be the end all of education; it's supposed to prepare you for the rest of your life!
Obligatory interruption here to point out (again) that the Massachusetts Constitution in fact creates public education in the Commonwealth for something EVEN MORE IMPORTANT than "earnings."
It's for the preservation and continuation of a democratic republic.

Okay, so far, so...lousy, actually, because that's pretty horrifying, but at least we know where we are. I think it's crucial to point out a few things that didn't get a lot of attention yesterday, though.

First, one of the authors of this paper is Roland Fryer, best known around here for his seat on the Massachusetts Board of Education, but here in his professional capacity at Harvard. Fryer's a charter supporter, no question, so the research that is negative on charters coming from him is unexpected.
There have, however, been some questions raised about some of Fryer's work in the past (see, for example, this from Vox on research released earlier this year on police shootings), in terms of methodology, sample size, and so forth, and those can be issued here as well: the study doesn't have exact matches on the public side for the charter students; it only looks at Texas, and so on.
It's also worth nothing that this isn't a completed research piece: what comes out this way isn't a peer-reviewed piece of work, analyzed by others. There are reasons for that having to do with the work Fryer does, but it also is worth nothing.
This isn't to take away from the conclusion drawn, within its parameters; just note the parameters.