May meeting of the Board of Ed: MCAS update

backup is here
Wulfson: continuing on promise to keep Board updated
2/3 of way through administration this spring
then standard-setting process that will then lead into the scoring
"extremely pleased with how this test is going"
"it's been an incredible year" for those doing the work



Associate Commissioner Michol Stapel
testing window coming to an end this Friday; bit of a longer window this year to accommodate computer testing
have completed 235,000 tests in each subject (on the computer)
daily average of about 40,000 testers per day
big question was around the technology: all sides point to a "very successful" session on technology
some local issues; one-off incidents
students have resumed testing the same day
observed a couple of days
"overall pleased with how technology" has worked
very early numbers on participation
"looks really good right now...98 to 99% participation rate in grades 3-8"
granted a very small number of waivers on online participation
"we're eager for Friday to arrive...pleased to have the end of the window approaching and have had a successful adminstration of the test"
Wulfson: development of 10th grade test happening; spring 2019, but field testing items next spring
"as more and more schools move to a one to one model, schools will be able to shrink that window (of testing)"
Stapel: not some minor tweaks that we were making; bringing it to the next level
"give meaning to those raw score points"
standard setting is how we give that meaning
"how many raw score points do you need to get into those achievement levels"
student expectations: content standards, grade-and subject-specific achievement level descriptors
descriptions of the knowledge, skills, and abilities associated with the top level of achievement at different levels
"continuum of performance that drills down into each standard in ELA and math"
in process of developing document that will guide them through standard setting
August 14-18 in Danvers: 6 committees
ELA grades 3-4, 5-6, 7-8
math grades 3-4, 5-6, 7-8
have combined the grades to have a more cohesive system across grades
looking for experienced teachers who are thoroughly knowledgeable about content, standards, and students in those grades
in response to Q from Peyser: looking for panelists who are "grade up" as well (for what comes next)
looking for a diverse group of people with a variety of experiences
Here's the plan of how they'll work:

Bob Lee: on achievement and cut scores
in 5 or 6 weeks, we'll have a bunch of kids who have demonstrated their achievement on a scale higher or lower
then divided them up into fourths, essentially
then we have to make cut scores: "I would love some experiment where we statistically set this"
there are other things that people are interested in
"want the content experts to come in and look carefully at what those levels mean"
find what those cuts really mean
so you don't get substantially different messages between elementary and middle school on student level of preparation

set number of points for each sort of answer, done individually

panelists have different opinions about what it means to partially meet or meet expectations

then show how many kids got it right

panelists negotiation around what they mean

monitoring around where panels are going

have to make sure the grades really hear each other
"have confidence about how many kids should be meeting expectations across the board...done on purpose and not by chance"
Lee: all things being equal, we'd like it to be equally difficult to meet expectations across grades
intended to give respect to the standards, because not all standards are equally difficult
Peyser: looking for coherence and consistency, but not necessarily a pattern across the grade levels
Noyce: MCAS performance at a certain level will suggest that you're on a trajectory for success at the next level
"we don't want to be misinforming the public by saying, 'Oh yeah, your kid is doing great,' and then" have that not be true the next year
things happen "kids may be goofing off in middle school"
"we could say that you're on the course and keep it up"
Peyser: another possibility is that expectations are changing "and it may take the field a while to bring...instruction up to that level"
Moriarty: have looked at this in other states
"if kids are magically a lot better in third grade reading at the end of this administration, is this objectively correct or are we allowing panels to lower our expectations?"
McKenna: hearing as a background "are we going to grade on a curve, and the answer is no"
Noyce: participated in first round of standard setting in first MCAS
looked at entire student response sets and sorted them into groups
"think there is a statistically problem with making decisions question by question"
wonder why that change was made
Lee: bookmark method, getting a little harder and a little harder
"item by item does generate more points"
in round two will be showing item difficulty
results then summarized by contractor and presented to DESE and to Commissioner for approval
each student receives a scaled score and achievement level
reports are printed at sent to parents/guardians by end of October
update to Board in September
Noyce asks if Board members could take an MCAS of their choice
Morton: think that some students will not do well on new test
think it important to communicate "this very thoughtful process to the public"
Sagan: would suggest that this doesn't matter
"this is about people who are going to be super alarmed, and not caring or even able to go back"
Peyser: reports not only explanatory, but informative
"what can I do to help my student improve"
opportunity to set off on the right foot "or reset the foot that we are on"
"needs to be framed in a way that allows them to understand what it means and see it as actually helpful"