Posting these now as I couldn't get on Blogger in Medford High. All notes mine, and Prof. Schneider has the distinction of being one of the few people I've run into who talk faster than I can type. The below is my best:
Jack Schneider, presentation to MASC Division IX
Measuring School Quality Beyond Test Scores
“let’s start with what we have: state data and where most people get it” on the DESE website
limitations for policy and governance
clear quantifiable aim for, for example, raising SGP
don’t have the same information for, for example, educating the whole child
not same easy access to data
sends a message: if you don’t measure what you care about, you’ll start caring about what you measure
at most basic level, poorly designed interface
most people engage through Globe’s ranking ordering of MCAS performance
has more to do with family education and socio-economic
sends message of some schools are good, some are not
sends message of false precision: that we know what the number one and two schools are
which we don’t
“Dreamschool finder” reaction to Globe’s “Dreamtown finder”
“catagory called ‘hipster’ which if you ranked really high, you naturally have to live in Somerville”
if you ranked education really high, his hometown didn’t come up
“as long as schools were highly valued, you couldn’t get to Somerville”
if you dropped schools, you could get there
they were using SATs scores and another variable
whomever at the Globe was savvy about dealing with people, offloaded it all onto me
looked at the available data, have to work with that
some of the available data doesn’t stink: college aspiration data, survey of students about plans to attend college
can have a lot of low income students and still convince them that college is an aim
looked at available data: six categories
I wanted to use “the less bad use of available data” but became “Dreamschool finder”
tried to be a little fairer
did not rank order schools: had to input values
“good schools are about good fit as much as they are about a generic high-quality performance”
eventually talked about what other people should do in Somerville: put out RFP
was convinced that “it was going to be some huckster” who would do it
put together a team to propose starting with a blank slate
“What do people actually care about?”
a lot of polls used to build a draft framework to reflect what we believe a good school does
Somerville “pretty representative” of the US as a whole
began holding focus groups to amend the draft framework of what people care about in schools
made fewer and fewer changes over time until we got to a place where more and more people were looking at the framework and agreeing that those are their values in schools
“why we think the things we do about schools”
interesting to see teachers, administrators, parents, community members look at it and agreeing, common ground
Two large divisions: essential inputs and key outcomes
inputs: teachers and teaching environment, school culture, resources
outcomes: indicators of learning and character and well-being outcomes
for academic outcomes: using only growth and developing portfolio assessments
“there are ways to ask teachers questions to ensure an honest response: ensuring anonymity and collating at the school level”
School Culture: safety, relationships, academic orientation
every category is made up of subcategories and every subcategories has multiple measures
cannot presume precision
bring thirty measures to the task assuming that some will fall short, but eventually will get to answer
“don’t just want absence of bullying, want trust between students”
lots of questions to ask: absence rate, on time graduation rate, academic press questions of students: how much does this teacher encourage you to do your best? when you feel like giving up, how likely is it that your teacher will encourage you to keep trying?
have some measures of asking teachers: what do you see in the hallways? how much support do you have?
when asking questions of teachers and students, dozens of scales of measure
Data collection: all students grades 4-12
all teacher surveys (developed a sped survey specifically to be sure those students weren’t missed)
district data (teacher turnover)
state data (SGP)
data compilation and visualization:
web tool for School Committee and district leaders: secure portal just for them
able to attach measures to things schools and district care about
web tool for the public, as well: to be able to say that public can make more informed decisions and would strengthen the schools
many parents are desperate to advocate for their schools but don’t have the language: too often dismissed or that they themselves dismiss as it’s non-expert (in language) knowledge
imagine if they could say “this is what we care about”
“we’re hearing from parents that this is what they want to change at their school”
“the vision that parents have is the vision that matters: the schools belong to us”
urban schools are so badmouthed, so pilloried
wasn’t always that way, people clamored to get into urban schools, as those were the place to be
“very powerful stereotype of urban schools, that is then reinforced by that data we have” from MCAS rankings
“of course if you talk to the parents whose children have gone there, they seem pretty happy”
“if you’re in a school during off-hours and there are kids in the hallways and they’re working and they’re happy: that’s a good school”
currency of the realm is data: that’s the language people are talking now
question about equity: measurement of resources
enough measurements tell you where there are issues: and thus what can be fixed or needs to be fixed
“we’ve got a great principal, great teachers..the problem is resources...you can then activate” those who want to improve their schools and give them direction on energy
“public schools don’t market themselves” ad on T for charter schools or private schools
“exact opposite of what happens with public schools...slick version of reality” that’s being sold
“public schools aren’t even in control of the version of themselves that’s being presented to the world”
only a benefit that she was exposed to one more kind of thing
public schools aren’t even in control of the version of themselves that’s being presented to the world
“I believe a lot of our problems are solved if our kids go to school together, but our kids don’t go to school together, because parents have had the bejeezus scared out of them”
parents told this story that they can’t stay in cities, or, if they stay in cities, they have to send their kids to other schools
Q on ensuring all kids are paid attention: disaggregation of data, producing snapshots of all subgroups
“absolutely something that needs to be done”
school “fit is as important as whatever the generic good qualities of a school are”
parents need to figure out “who’s my kid, and how do make sure that” a school is right for my kid
Q measuring family background of kids in schools not accounted for within data
also levels of ELL and levels of special ed
charters taking least disadvantaged of disadvantaged: parental engagement...and “let’s look at the correlation of parent engagement and higher test scores”
Belsen: huge battle over what the purpose of a public schools: creating those for a job market. If you believe it’s more comprehensive than that, then you get a different picture.
Globe is pushing civics education: “well, who in the heck pushed it out?”
broader picture of what we want school to do
colleague refers to the “learnification” of schools: implies that everything in schools are something that can be measured
“the way to counter that is with the voices of the American people”
have evidence that this is what people want in their schools
“if all you do is measure two subject areas narrowly” then that’s all that’s going to get done
Q special education: if kids with particular needs don’t test well, and results bear that out
how to be more persuasive?
Schneider: focus group with special ed teachers: what Qs don’t apply to your students, what Qs aren’t here that should be on here to give us a picture of what your students are doing in the classroom
“every one of my kids has a personal goal, and when they meet that, it’s the best thing that happens to them all year”
gave students cell phones and texted them questions: have an 80% response rate on questions
building with real-time data: helping teachers use data to improve what they’re doing in the classroom
professional development site driven by what is known about (for example) around school climate
“and guess what? Suddenly you’re treating educators like professionals”
might make a difference and might also turnaround this slide on job satisfaction for teachers
“we’re better at our work when we are satisfied in our work” and are more likely to stay around
“and guess what? There’s data to show that” teachers who are in the classroom longer DO know more
Q on implementation: capacity? and how to continue? Much work is done and can be a starting point for other districts
thinking of starting a non-profit so “we could pay some people to come in and do the work” to create the framework specific to community
doesn’t take a lot in money: people have a lot of good systems and processes
creating a group so that this is self-sustaining with someone working quarter time to sustain it
in interest of urban superintendents, in particular, to ensure that measures are done in a more broad way
Q something that could be instead of the more corporate planning process: strengths, weaknesses, coming up with goals that are measurable...simplified process that could be done at the school council level for school improvement plans which are too driven by test scores; parents want to talk about school climate, rather
Schneider: easy first approach from broad categories of inputs and outcomes
want this to be an educational device: want parents and community members to learn more about each measure
something like “hallways are dark and dingy” when it’s backed up by the same data being used by system
“felt sense of things real and important” but set up against quantifiable; falsely set up against each other
Q found that student surveys are one of the most consistently reliable sources of information on teachers
questions put in words that are meaningful to students: test Qs with students, then take responses to teachers…”long iterative process”
Schneider: set up “so we can give everything away”
support schools as low cost as possible
student data currently being tested against other data across the country: going to be tested for strong correlation along with other student data
Q open sources would be useful, anything developed put back into open source; thinking of trying to have anyone who uses inputs data for aggregated data to start to understand across districts
interesting Q, not going to answer...don’t want to require entering data that requires student responses
at same time does empower communication and transparency
Q (this from a sped teacher) “comments about rigor, which is a word that I’ve really come to despise”
in grade 2, I give eight standardized tests a year
when people come in and give a snapshot look, not looking at everything that’s going on
students persevering in a task, sitting a chair “that’s important”
“we’re trying to teach in spite of everything they’re telling us to do”
Schneider: comment that parents support you
parents feel that standardized tests tell them the LEAST about teacher conferences of all the information available to them
tests don’t even aim among their aim of what a successful school does
Schneider: PARCC and MCAS are like a blood pressure test or a temperature
sometimes a high temperature is really bad, but some people’s temperatures run a little high
“when a good doctor says, ‘hey, you don’t look so good’ what is that even based on?”
Comment that this is a grassroots activism, start proving workable
Schneider: willing to work with districts in whatever way we can; want to be able to part of ongoing conversation
once we build it, we can give it to you
some basic texting thing, small costs involved
Comment (Belsen) sustainability of some of these gains on MCAS and so forth: don’t go on and complete college in same numbers, SATs, jobs long term
“inability that the gains that are leading the charge for this expansion are not sustainable and don’t really produce real gains long term”
fighting against a juggernaut that tells us what is
Sen. Jehlen: “you have more power than you realize”
bunch of bills in the Legislature: people will say ‘there’s nothing we can do; we need the information’ Now you have something to say.”
“Don’t underestimate how powerful you can be as the voices of your community.”
