Showing posts with label research. Show all posts
Showing posts with label research. Show all posts

Wednesday, July 15, 2026

Things I am reading this week

  •  Please add me to those not at all shocked that little kids are finding their way around various kinds of restrictions on school laptops to get up to all sorts of nefariousness. 
    Kids have been off task and distracted in school long before Chromebooks or iPads existed. But some parents and experts say devices only make it easier to engage in non-academic, inappropriate and even dangerous content.
    This is especially challenging for young children whose brains, self-regulation skills and self-control are still developing.

  • The storied Financial Times of London is out this week with an article on employers pulling back on their encouragement or even insistence on employee use of AI. In an excellent example of Goodhart's Law, when employers made massive uses of AI something that mattered, employees made maximizing use, rather than doing actual good work, their focus.
    I really hope that educators are watching how quickly things are changing in the workforce on such usage, as that keeps being the argument for why we MUST have students using it in schools. We are very much watching a tech bubble in real time. It's unwise to train kids for the bubble. 

  • And speaking of educators and AI use, please do yourself a favor and review this piece of "meta-meta-analysis" from Patrick E. O’Neil, looking at the major flaws in significant parts of the research propping up AI use in schools. From his introduction: 
    The audit found that none of the examined meta-analyses provided a valid basis for the claims they advanced: none had a coherent construct, none sufficiently assessed publication bias, and all had severe heterogeneity. Statistics were misapplied, miscalculated, and misinterpreted. A majority (61%) of the randomly vetted primary studies were problematic, most commonly because the outcome measured didn’t match the meta-analysis.
    I await any of the educational publications I read, all of which have ongoingly significantly boosted this breathless urgency of such use in education, to put this one of their front page. 

  • And speaking of research, I took a look at this research into (essentially) what districts did with their ESSER funding, specifically looking at if such grants were used to offset local property taxes. The piece found it did, which might be what you end up seeing circulating.
    What matters a lot though, is what districts they looked at: in order to have groups to compare, the researchers used the 5% poverty rate used for Title I as their cutoff, looking at districts just to either side of the line.
    What that means is they looked at what wealthy districts did with ESSER that got it.; 5% is a very low rate of poverty in a district; as best as I can find, the national average is closer to 13%, and that would leave out all of the very largest districts in the country.
    That's not really the same thing as saying "here's what happened with ESSER funding" at all. 

Tuesday, May 19, 2026

Board of Ed for May: Research and evaluation

 memo is here

Craven has left; Hills now chairing

speaking on this, per the memo: Rob Curtin, Deputy Commissioner, Matt Deninger, Associate Commissioner of Planning and Research, and Kendra Winner, Research and Evaluation Coordinator

Deninger: as a former high school English teacher, it is always good to be back in a high school

Monday, May 11, 2026

Recess for all

 New this morning, the American Academy of Pediatrics has issued today their first updated guidance on recess, and they're doubling down on its importance. From the AP's coverage: 

The group “has always supported play – free play for kids – but it’s been increasingly threatened over time,” partly by the drive for higher test scores, said Dr. Robert Murray, a lead author. “It has a very powerful benefit if it’s used to the fullest.”

The new guidance, published Monday in the journal Pediatrics, is similar to the previous policy statement but cites the latest research on why these breaks are essential for kids’ academic success and mental, physical, social and emotional growth.

You can read their full publication, which includes links to relevant research, here.  

I will opine only as I have before that there is a solid argument for Massachusetts counting recess as instructional time, which is the main holdup on increasing such time. That would take the Board of Ed passing a change to the time on learning regulations, 603 CMR 27.04.

Wednesday, April 8, 2026

"risk eroding the very human capabilities they are meant to support"

Is there bad AI news this week?
Friends, there is ALWAYS bad AI news!

Two things to review: 

  • A paper released by Cornell University looking at the "consequences of AI assistance" using mathematic reasoning and reading comprehension (that is, two of the core competencies of education!). They found: 
    AI assistance reduces persistence and impairs independent performance: After brief AI-assisted sessions (~10 minutes), participants were significantly more likely to give up on problems and performed significantly worse once the AI was removed, compared to participants who never had AI assistance.
    Effects are concentrated among users who seek direct solutions: Persistence costs were concentrated among participants who prompted AI to solve tasks for them directly. Using AI for hints or clarifications did not produce significant impairments.
    Effects generalize across domains: Effects replicated across fraction arithmetic and reading comprehension, suggesting it is a general consequence of AI-assisted problem solving, not specific to any particular task.

    Their conclusion?

    These findings raise urgent questions about the cumulative effects of daily AI use on human persistence and reasoning. We caution that if such effects accumulate with sustained AI use, current AI systems — optimized only for short-term helpfulness — risk eroding the very human capabilities they are meant to support.

    I found this page which gives visuals for their findings very accessible; I include one below. 


  • Sam Altman, the head of OpenAI, is the subject of an in-depth profile in The New Yorker . It reads almost as a parable of hubris: you can trace the loss over time within the company of values expressed early on.
    That he has as much power as he does is horrifying.


Visual from "AI Assistance Reduces Persistence and Hurts Independent Performance"


Thursday, March 12, 2026

You will shift towards the bias in AI

 From recent research from Cornell: 

In every experiment, the researchers found that participants’ views shifted in the direction of the AI bias. The biggest surprise, Naaman said, was that mitigation measures did not work.

“We told people before, and after, to be careful, that the AI is going to be (or was) biased, and nothing helped,” Naaman said. “Their attitudes about the issues still shifted.”


The research itself is here.  
It cannot legitimately be used as a "thought partner" or even to "clean up writing" as I've heard people say even in the last day.

Don't use it. 

Friday, January 30, 2026

To read on snow days

 Once we got through this year's/week's round of "how do superintendents decide on snow days, anyway" followed by "here are people who disagree with whatever this week's decision was," we got--thanks, I think, to how BIG last weekend's storm was!--a round of "what do snow days do for learning?" And in two cases, they found Professor Joshua Goodman's 2014 paper "Flaking Out: Student Absences and Snow Days as Disruptions of Instructional Time" (good title!). He was interviewed by NPR here and is included in Chalkbeat's newsletter here, from which I quote: 

Goodman compared test scores in years where there was more or less snow. A year with lots of heavy snow (days with more than 10 inches) had no effect. Yet math scores were lower in years with more days of moderate (4-10 inches) snow.

These counterintuitive results can be explained by how schools responded to different levels of snow. Heavy snow meant more snow days. Moderate snow meant more days when schools stayed open but attendance was sharply down, particularly among low-income students.

In other words, snow days did not seem to set students back much; missing regular school days due to snowy conditions did.

The takeaway is not that schools can close indefinitely without consequence, and Goodman did find some small negative effects of closures for high-poverty schools. But the study suggests that a few snow days won’t be terribly harmful because teachers figure out ways to adapt.

Yet when schools are open and a big chunk of students are missing, teachers are in a tough spot. Some may continue teaching new material, and then try to help students who missed the original lesson catch up. This creates a dual challenge, though: The makeup work isn’t quite as effective, and teachers’ time and energy is diverted from moving everyone else forward.

I will be interested in seeing if anyone cites this in future decisions!

PS: Don't miss this cartoon from The New Yorker.


Wednesday, January 14, 2026

AI risks outweigh benefits in schools

Rhetoric arguing that technology adoption in and of itself represents innovation and progress is not only false but undermines society’s ability to discern how to effectively harness AI to advance children’s education.

...though I would dispute that the last is possible.

Headline from NPR this morning: "The risks of AI in schools outweigh the benefits, report says" in one that probably landed on your screen. It's from a Brookings release, and while the full report is over 200 pages (add it to your "to be read" list), the summary (from which the above quote is taken) is certainly digestible.

Wednesday, October 8, 2025

Report on school construction from Mass Inc and WRRB

 This was released yesterday

How great is the clever title?
A building foundation...the foundation budget?

The upshot is that if Shrewsbury has nine schools, and Worcester has fifty schools, and each can build one new school at a time, when we get to the end of any set of time, Shrewsbury can rebuild all of its schools before Worcester has rebuilt a fifth of theirs.

Thursday, September 25, 2025

new report on MA school building funding equity coming

 ...and while I cannot go to the release, some of you may wish to!



Worcester Regional Research Bureau and MassInc are coming out with a joint report entitled "Fixing the Foundation: Uneven Access to Modern Schools and a Blueprint for a More Equitable Future." It's being released in an event October 7 from 10 AM to 12 at the State House.

You can find all the information and a way to RSVP here. 

I'll be sure and share the report once it is out! 

Thursday, September 18, 2025

Before you paint all access to social media for young people with the same brush

 ...please be sure and review the results of the recent "Online Experiences and Mental Health of LGBTQ+ Young People" from The Trevor Project. Opening line: 

Most LGBTQ+ young people agreed that they go online to connect with people because it is difficult finding others to relate to and connect with in their daily lives.


Wednesday, September 17, 2025

Tuesday, September 16, 2025

Is there bad news on AI in education this week?

 THERE IS ALWAYS BAD NEWS ON AI IN EDUCATION!

  • Okay, but first some good news: students are pushing back.

  • Can teachers set up "simulated students" to trust in figuring out what practice or assessments to give their students? 
    “We were interested in finding out whether we can actually trust the models when we try to simulate any specific types of students. What we are showing is that the answer is in many cases, no,” said Ekaterina Kochmar, co-author of the study and an assistant professor of natural-language processing at the Mohamed bin Zayed University of Artificial Intelligence in the United Arab Emirates, the first university dedicated entirely to AI research.
    NO! Somewhat darkly amusingly, it's because of how they work:
    The LLMs that underlie AI tools do not think but generate the most likely next word in a given context based on massive pools of training data, which might include real test items, state standards, and transcripts of lessons. By and large, Kochmar said, the models are trained to favor correct answers.
    “In any context, for any task, [LLMs] are actually much more strongly primed to answer it correctly,” Kochmar said. “That’s why it’s very difficult to force them to answer anything incorrectly. And we’re asking them to not only answer incorrectly but fall in a particular pattern—and then it becomes even harder.”

     



Thursday, August 7, 2025

Is there bad news on AI in education? Edition whatever

 Yes, friends, there is ALWAYS bad news in AI education!

First up, Benjamin Riley is as frustrated as I am about AI in education and this nonsense "moonshot" notion, in a post that I urge you to read from first to last.

Some schools have been using AI surveillance software, and it not only is triggering false alarms; it has resulted in not only student discipline, but also students being arrested.

And yesterday, we had another release of research which tells us the same things we find every time we ask: 

Asked to generate intervention plans for struggling students, AI teacher assistants recommended more-punitive measures for hypothetical students with Black-coded names and more supportive approaches for students the platforms perceived as white, a new study shows...

Common Sense Media found that while these tools could help teachers save time and streamline routine paperwork, AI-generated content could also promote bias in lesson planning and classroom management recommendations.

Robbie Torney, senior director of AI programs at Common Sense Media, said the problems identified in the study are serious enough that ed tech companies should consider removing tools for behavior intervention plans until they can improve them. That’s significant because writing intervention plans of various sorts is a relatively common way teachers use AI.

The report from Common Sense Media can be found here. 

I will echo Riley's post then, here: 

Add to that all of the generative AI systems. 

Wednesday, August 6, 2025

There's actually some really interesting points on enrollment in this report

 ...but friends, you are not going to glean them from reading the Boston Globe's coverage.

Please enjoy these lovely flowers from Montreal,
which have nothing to do with this post otherwise.


In June, Joshua Goodman, a professor at BU, and Abigail Francis, who is pursuing her doctorate there, released a study looking at Massachusetts school enrollment from pre-pandemic levels to now. Massachusetts is a handy one for this because the state tracks not only public school enrollment but private and homeschool enrollment (not all states do). 

Their research uncovers three main points: 

Monday, June 16, 2025

There is always more to read on generative AI

 And this week, you don't want to miss: 

  • This piece in the New York Times on people 1whose interactions with generative AI sent them down conspiratorial rabbit holes: 
    Generative A.I. chatbots are “giant masses of inscrutable numbers,” Mr. Yudkowsky said, and the companies making them don’t know exactly why they behave the way that they do. This potentially makes this problem a hard one to solve. “Some tiny fraction of the population is the most susceptible to being shoved around by A.I.,” Mr. Yudkowsky said, and they are the ones sending “crank emails” about the discoveries they’re making with chatbots. But, he noted, there may be other people “being driven more quietly insane in other ways.”

    That's a gift link above, so do read the whole thing. Benjamin Riley relates a similar situation closer at hand in his post here.

  •  On the ongoing theme of "this is doing the opposite of what you want to do to people you're educating," we have this piece of research from Cornell which came out last week, which finds: 

     Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels. These results raise concerns about the long-term educational implications of LLM reliance and underscore the need for deeper inquiry into AI's role in learning.

    That's in addition to use of generative AI being linked to erosion of critical thinking skills, memory loss and procrastination.

I await those in the education press to give this even 1/10th the coverage they give them "WE HAVE TO DO THIS NOW" nonsense they pedal daily.

___________________
1 as the first individual quoted started by using it for "to make financial spreadsheets and to get legal advice," given what we know, he was already in trouble. 

Wednesday, May 28, 2025

Recent things to read on AI

Friday, September 6, 2024

Two to read on the MCAS ballot question

 Without commenting further at this time--though I have many thoughts!--I did want to share two things that I think are below the radar to read on the ballot question ending the use of the high school MCAS as a graduation requirement: 

  • The Center for State Policy Analysis at Tufts University is doing a series on the Massachusetts state ballot questions; this is the one on the MCAS question. It's really well done, and it had me finding their other ones.

  • I ongoingly lament the lack of attention to the actual work being done on and around the MCAS, and the actual research (using decades of data) of John Papay at the Annenberg Institute at Brown University is something that periodically gets reported out at the Board of Ed, and otherwise doesn't seem to capture much attention (I think in part because many of those who spend a lot of time talking about the MCAS don't like the information being arrived at). He and others published this back in July.

Thursday, February 22, 2024

The new literacy crusade

And I mean that with all the negative baggage that carries, yes. And a periodic reminder that the only person I speak for here is me  

xkcd 1167, acknowledging that the world may not need more written on this

I had a friend ask me earlier this month what I knew about what Governor Healey announced in the State of the State as her administration's "Literacy Launch" and more largely about what Massachusetts is doing on literacy and public education.

If you read the linked article, you'll see that Secretary Tutwiler outlines a $30M effort from the state that seems reasonable. They're planning to fund professional development and grants for districts to purchase curriculum that more strongly supports kids learning to read. That seems fine, and it's the role of the state to support districts in this way.

Let me be clear: while teaching kids to read is not my professional background, it's demonstrable that:

  • kids need different things in order to learn to read; not every kind of instruction works with every child (kind of like everything else)

As a local side note: this is where Worcester was. It wasn't until we got a new superintendent, and the leadership that resisted getting teachers the curriculum they needed was gone, that we made a change. $7M or so in ESSER funds went to a new elementary reading curriculum; Worcester's elementary schools now have Core Knowledge for Language Arts or CKLA.

Where this becomes something else, as too often in education, is when it becomes a crusade. 

And all, we are there in spades. 

I have been struggling for weeks on how to tackle this issue on here, because it is so big, so heated, and, at ground, so breathtakingly missing the mark. Yesterday, though, I had shared with me the third of Maren Aukerman's of the University of Calgary's three part series on the Literary Research Association site "The Science of Reading and the Media" which hits the mark; you can find the first part here, and the second here.

I particularly found useful how she frames the state of coverage, which will sound familiar to any readers of the Boston Globe:

From how much of the media tells it, a war rages in the field of early literacy instruction. The story is frequently some version of a conflict narrative relying on the following problematic suppositions:

a) science has proved that there is just one way of teaching reading effectively to all kids – using a systematic, highly structured approach to teaching phonics;

b) most teachers rely instead on an approach called balanced literacy, spurred on by shoddy teacher education programs;

c) therefore, teachers incorporate very little phonics and encourage kids to guess at words;

d) balanced literacy and teacher education are thus at fault for large numbers of children not learning to read well.

The upshot? "Unfortunately, these suppositions turn out to be highly misleading."

In the first part, Professor Aukerman starts by asking if reporting is biased, using as an exemplar Dana Goldstein's piece from 2022 on Lucy Calkins. Aukerman walks us through the lack of balance, use of straw man arguments, myopic lens fetishizing phonics instruction, and logical fallacies. One could--and I would argue, we need to!-- do the same here in Massachusetts with the Globe's three part much-touted series on literacy. 

In the second part, Aukerman looks at if media is using high quality research. The lack of deep understanding and use of research across education (and I dare say, in other fields) is an ongoing issue of the current state of media, and we certainly see it here in Massachusetts. Even something so basic as the basis of the argument in their literacy series, which the Globe has based on ELA MCAS scores reveals this sloppiness, as MCAS scores do not tell us much about literacy; ELA MCAS tests much more than literacy and is not, on its own, a test of literacy. 
But more broadly, you can see this in the coverage:

By drawing mostly on vociferous advocates of one approach and bolstering their claims primarily with other journalism, journalists create an echo chamber which itself is disconnected from reading research. 

And would you guess this from reading the popular coverage?

 ...there is insufficient evidence to conclude that any single approach, including the particular systematic phonics approach often elided with “the science of reading,” is most effective.

Nearly all coverage also lacks both historical context:

 The idea that phonics can fix children’s reading ills is at least 70 years old, yet results from other large-scale phonics reforms have also yielded disappointing results, including during the Reading First era in the U.S. and as England’s recent national curriculum mandates have played out

And, I would argue, a disturbing ignorance about how curriculum works in the classroom. The Globe, for example, waved their hand and deemed "outdated" curricula that they found in their survey that districts are using. But that both a) leans on the above poor understanding of what is quality curricula and b) understands "curricula" as if it is taken from a box and inserted into children's heads without going through educators who always use a variety of resources to best meet the needs of children.

The third part of Aukerman's series brings us to where I am concerned we are in Massachusetts: the consequences. The latest Massachusetts iteration of this is the push to pass H579/S263, frighteningly favorably reported out of the Joint Committee on Education on the last available day*. This would not only add another report to the pile of reports that districts have to submit to DESE--let me know when that fixes something, eh?--but would give the Department the authority to select curriculum, with districts selecting from a list generated. As the bill reads:

each local school committee shall use programs and curricula from the lists developed by the department or an approved alternative program

This honestly makes me both so angry and so frightened about the state of education in Massachusetts that words fail me. There are much more articulate words in the actual practitioners of the letter I shared earlier this month, of the position statement of the Massachusetts Association of School Superintendents, and the four fallacies of the reading wars.

One cannot, of course, separate this out from context: this is the same "my local elected officials are not doing exactly what I want, so I am taking it to the manager" that brought us mandates over everything from COVID to other nonsense. And someone, somewhere, is going to write a great book about the economic anxiety of the white middle class and its impact on public education and lines of governance.

But in the meantime, here we are. 

So, please go read all of Aukerman's series. Share it. Email it to your legislators.  

____________________________________________________________________

*and if you think the Globe "just happened" to choose the hometown of the Senate chair of that committee for one of their articles, I have a bridge to sell you.

Wednesday, November 16, 2022

A couple of notes on the WPS budget

It's been awhile since I posted anything on the WPS budget--we've been sailing along through quarter two here--but there are a few things that I haven't yet had a chance to note, so I thought I'd pull them together here.

"The School Committee has budgetary authority at the cost center level."
Of the many things I say about school finance frequently, this is one of the most frequent, in part because what "school committee authority" on budget actually means can be unclear if you haven't been around school budgeting.
If you have...please know that this level of authority has explicitly been the case since the early 90's.

What does this look like? The best illustration is the quarterly budget updates that the Worcester School Committee gets, at which we also exercise our transfer authority. 
Thus, in September, the Finance and Operations subcommittee received the first quarter budget report for FY23, which started in July. The fourth page gives the 30 cost centers the Worcester School Committee voted allocations to back in June and how much we adopted them for; how much of that has been spent; and what the projected balance for the end of the year is, based on what is known at this time. 
Based on that last, which is described in the prior pages (good budget reports don't just have numbers!), the subcommittee, and then the full School Committee, may make transfers as recommended by the administration between those cost centers. Administration can spend within and up to the allocated amount in every one of the thirty centers, and the School Committee does not have the authority to, for example, designate a salary (save for the small number of contracts which we negotiate directly). If a cost center is projected to be more than what is allocated, the money has to be transferred, by School Committee vote, from somewhere else. 
If you wonder over the course of the year where we're at on spending: this is the place to check. 
And if you wonder if folks know what they're talking about school spending, remember: cost center level. 

Back in September, the Worcester Regional Research Bureau published a look at the overall city budget, which I highly recommend to anyone trying to follow or figure out how it is that the city spends its money. It's also--from my corner of the universe--very useful in terms of showing what the cityside can and cannot do for schools, and does and does not do for schools.

Let me show you what I mean. 


This chart is page 7 of the report, and while it's a chart we see every year in the WPS budget (this year, it's page 30), there are some things that it is useful to have someone else call attention to, like exactly how little of the overall WPS budget is local contribution: it's less than a quarter. As a reminder, this is intentional: it's a function of the foundation budget and municipal wealth calculations working precisely the way that they should be.

I'd put this in concert with the note on the full rundown on allocations on page 6:


...something which you wouldn't often know in watching Council deliberate the WPS overall allocation.

Note that this voluntary contribution over required this year was originally projected to be just over $300,000; you can find that on page 432 of this year's WPS budget. Due to the changes in the state budget described in our September budget update, the district has a commitment from soon-to-be-no-longer-Acting City Manager Batista for the $1M that our budget otherwise would have dropped, which will push this contribution up. 
That's much appreciated, because otherwise we would have either had to cut the budget by $1M, or, sigh, use ESSER funds to fill the gap.

Going more deeply into the Worcester Public Schools budget, the Research Bureau not only issued a report, but gave us all a new toy to play with on the WPS budget. They've spent what's clearly a ton of time breaking down both revenue sources and spending in the Worcester Public Schools' budget from FY19 to the present. There not only are numbers: there are charts! And you can change them! You'll be able to see, for example, that the vast majority of our budget is spent on salaries, and most of that on teachers, just like every other district out there. Education is a people-centered system.
Give it a whirl, as it does a nice job of breaking down some of the internals of what exactly we're doing with our now over $500M a year.

And you don't get higher praise from WPS finance than a link straight from the budget page.

Now, all of the above is about the operating budget: that's the "running the system" part of the funding. For the major purchases--things like buildings?--you have to look at the capital budget, which the Research Bureau includes in the first report, on page 8. 
First, check out where the money is coming from: 

MSBA far and away is the most, and that's because, yes, we're building new schools. Much like chapter 70 state aid can't be spent on anything other than education, MSBA funding is coming to us to go straight into new buildings. The rest of the $90M is coming in through various local means.
Keep in mind, of course, that the MSBA is going JUST to those projects: it's building the new Doherty; it was building the new South; it's putting a new roof on Worcester Arts Magnet. It is project funding only, and it does not go to any other buildings.

Check out, though, what we spend our capital budget on:
The $60M at the top is the same $60M we saw coming in for those specific projects: it's a wash. 
Skim down, though, and look at where all of the other capital needs for 48 schools in buildings ranging in age from last year to before the Civil War. 

$4M. For everything. 

Now, I've been reading the city's capital budget for years, but I don't think that I've ever seen in presented in quite so stark of terms before. Something to consider as we head into the planning of another fiscal year. 

Please note that I am always, always happy to field questions on our budget, and if I don't know the answer, I'll find out!

Friday, November 11, 2022

and masking works

 ...go figure.

A new report released this week in the New England Journal of Medicine effectively used DESE's rescinding the mask requirement as an experiment: when that happened, many districts stopped requiring masks, but Boston and Chelsea didn't. The researchers looked at:

The final sample included 72 school districts, which comprised 294,084 students and 46,530 staff during the study period. The study period was defined as the 40 calendar weeks of the 2021–2022 school year, which ended on June 15, 2022 (the end of the last full reporting week in all districts).

And what did they find?

Before the statewide masking policy was rescinded, the trends in the incidence of Covid-19 observed in the Boston and Chelsea districts were similar to the trends in school districts that later lifted masking requirements. However, after the statewide masking policy was rescinded, the trends in the incidence of Covid-19 diverged, with a substantially higher incidence observed in school districts that lifted masking requirements than in school districts that sustained masking requirements. 

Specifically: 

 Overall, the lifting of masking requirements was associated with an additional 44.9 Covid-19 cases per 1000 students and staff (95% CI, 32.6 to 57.1) during the 15 weeks after the statewide masking policy was rescinded 

An editorial accompanying the research relates this to the massively inequitable costs of COVID. You can also find coverage on WGBH, The New York Times, The Washington Post.

And from one of the Twitter education satire accounts: