Thursday, November 12, 2009
Public input
We're about to go to public input...I'm signing off. It will all be up online, though, so I'll post a link once it's up.
How specific should the application be?
how can these ideas be sustained?
interstate cooperation
"there's a lot more ways for this to go wrong then right"
what decision making and approval has to happen for this application to go forward (so if you get the money, who still has to say 'yes' to move forward)?
We're back to maybe this being 3-5 years
Need a leadership structure that sustains through leadership changes (aka: when we get a new governor or a new legislature)
What if you have a big consortium? "some leadership management" (oh, good...more administration) Yes:"an executive director model"..."executive committee, executive board" Boy, did this just get more expensive! And what sorts of models do we have in the US for multi-state executives that are vested with any real power? I'm having difficulty coming up with any.
What if a group of states have different standards? Can they be a group anyway? Sure, and it's a great research tool. What if they say, "Yeah, but we're going to keep doing what we're doing" even if it isn't measuring as well? (it sounds like no one has an answer to that.)
Assessments focused on age, stage, or grade? How are we sorting kids?
Literacy and numeracy are basics for college readiness. "if you're going to play, play" I'd like to see as many consortiums happen, but not spliting the subjects...Weiss interjects that states "would become vendors for each other" where states would develop an approach and offer it to other states...but it would quickly lapse into a vendor model, another member objects. In five or six years, there will be perfectly aligned tests and states will buy them, he says.
don't lose the capacity building at the local level...assessment takes on the role of the bad guy/good guy and capacity doesn't get built. Increase the capacity of the teacher, the principal...early models haven't done that.
interstate cooperation
"there's a lot more ways for this to go wrong then right"
what decision making and approval has to happen for this application to go forward (so if you get the money, who still has to say 'yes' to move forward)?
We're back to maybe this being 3-5 years
Need a leadership structure that sustains through leadership changes (aka: when we get a new governor or a new legislature)
What if you have a big consortium? "some leadership management" (oh, good...more administration) Yes:"an executive director model"..."executive committee, executive board" Boy, did this just get more expensive! And what sorts of models do we have in the US for multi-state executives that are vested with any real power? I'm having difficulty coming up with any.
What if a group of states have different standards? Can they be a group anyway? Sure, and it's a great research tool. What if they say, "Yeah, but we're going to keep doing what we're doing" even if it isn't measuring as well? (it sounds like no one has an answer to that.)
Assessments focused on age, stage, or grade? How are we sorting kids?
Literacy and numeracy are basics for college readiness. "if you're going to play, play" I'd like to see as many consortiums happen, but not spliting the subjects...Weiss interjects that states "would become vendors for each other" where states would develop an approach and offer it to other states...but it would quickly lapse into a vendor model, another member objects. In five or six years, there will be perfectly aligned tests and states will buy them, he says.
don't lose the capacity building at the local level...assessment takes on the role of the bad guy/good guy and capacity doesn't get built. Increase the capacity of the teacher, the principal...early models haven't done that.
The worm turns...and they said WHAT?
a panelist asserting that we're going good things in grades K-3 in assessment was greeted with loud (not entirely friendly) guffaws.
We're doing a sort of round-robin of the speakers now, with a lot of talk about metrics and other psychometric buzz words, and it's impossible to see who is saying what. Oh, and sometimes they're talking over each other. Sorry for the lapse here. Large portions of the audience have sort of zoned.
Someone says he wants to be able to tell "if it's the first grade or the second grade teacher that screwed that up" (in talking about a kid not reading at grade level by 2nd)
AH, a question on the role of the district (I'm sorry; I am not calling them LEA's)...could a group of LEA's form a consortium? someone asks. No, they are part of the state. However, the LEA's can opt in or out in developing the test "though once it's the state test, of course, all LEA's will have to take the test" replies Weiss.
Question on teachers developing assessments (which someone now calls "performance tasks"): "teachers have a very strong role" says Nellhaus (weird from the MCAS guy). Teachers, he says, "know just how to do that." Somebody is now talking about teachers not being able to do it "without really good expert guidance" but they can pilot them...(flabbergasting) "tremendous form in assessment building" ..."having teachers score is good, but getting feedback on how well the score is going" "tremendous value in training teachers" in how to develop test questions, oh, and you save money by not having to pay for question writers!
"We know that teachers mark papers differently if the student is known, versus if the student is not known" Bringing in marking that is absolutely blind...this from the Canadian guy.
We're doing a sort of round-robin of the speakers now, with a lot of talk about metrics and other psychometric buzz words, and it's impossible to see who is saying what. Oh, and sometimes they're talking over each other. Sorry for the lapse here. Large portions of the audience have sort of zoned.
Someone says he wants to be able to tell "if it's the first grade or the second grade teacher that screwed that up" (in talking about a kid not reading at grade level by 2nd)
AH, a question on the role of the district (I'm sorry; I am not calling them LEA's)...could a group of LEA's form a consortium? someone asks. No, they are part of the state. However, the LEA's can opt in or out in developing the test "though once it's the state test, of course, all LEA's will have to take the test" replies Weiss.
Question on teachers developing assessments (which someone now calls "performance tasks"): "teachers have a very strong role" says Nellhaus (weird from the MCAS guy). Teachers, he says, "know just how to do that." Somebody is now talking about teachers not being able to do it "without really good expert guidance" but they can pilot them...(flabbergasting) "tremendous form in assessment building" ..."having teachers score is good, but getting feedback on how well the score is going" "tremendous value in training teachers" in how to develop test questions, oh, and you save money by not having to pay for question writers!
"We know that teachers mark papers differently if the student is known, versus if the student is not known" Bringing in marking that is absolutely blind...this from the Canadian guy.
Henry Brown
Titled "Race (or Long, Slow, Slog) to the Top Assessment"
in principle, we know how to do it and could in 3 to 5 years; in reality, there are technical and lgoistical obstacles, capacity constraints, contractual issues, resistance and inertia
a "new develpmental pathway to jumpstart innovation"
adopt a systems approach:
in principle, we know how to do it and could in 3 to 5 years; in reality, there are technical and lgoistical obstacles, capacity constraints, contractual issues, resistance and inertia
a "new develpmental pathway to jumpstart innovation"
adopt a systems approach:
- new patterns of collaboration
- new measurement qualities
- new assessments
- and new platforms
need a comprehensive model of the domain, models of student learning, complete content standards, performance standards that are articulated, technology platform
four component system: diagnostic, extended projects, on demand test on previewed material, and on demand testing (forced choice and short answer responses)
focus on standard setting: mastery of material at grade level is not an end in itself; we want proficiency at the end of school. It's a milestone: you are this far along.
standards that you hope to measure should be an integral part of the test
technology is not an end in itself; it is a means to an end
lack of capacity of technology is a big issue, but the assistance of technology can be helpful: novel items, adaptive test designs, automated scoring
ESEA is undergoing reauthorization
change is difficult, and there are the constraints of contractual arrangements
identifying a viable state constortium
complaint that the assessment tail is wagging the instructional dog; complaint that you're involving teachers
(Interjection from Weiss that complaints from both sides "is what success looks like")
Money should be leveraged over time; alternative strategies supported; teachers involved at all levels; build in requirements; provide incentive to states; allow flexibility on timing
Scott Marion
starts by quoting an article from today's Times on the only thing holding us back "is a little thing called physics"
"Conceptually coherent comprehensive assessment system"
he wants the DoE to articulate a "clare and explicit theory of action"
how do we get from A to B
Congress will utimately reauthorize ESEA...but DoE must have a sense of the likely accountability uses before letting the funds
measuring a limited number of big ideas at deeper levels of understanding
measuring student learning, not cross-state comparisions
theorically sound design modles: student model, evidence model, task model...
design systems up front; reports need to be part of the system, not an add on
suggesting "curricular units" that focus on the "big idea" of the domain, used within the existing curriculum, basis for performance tasks, access for all students
why the obsession with instant results? He dismisses this as unnecessary as there have been other assessments throughout the year.
argues that we have much more of an instruction than an assessment problem when it comes to students with disabilities
new psychometric practices: traded validity for "pretty scales"
assessment should go along with core courses up to 10th grade
after 10th grade, more choices and more diversity
Points out that this is an ongoing cost, not a one time purchase
embrace differences
figure out what is absolutely essential
reconsider having every child tested on every item (sampling)
allow for multiple answers
be exceptionally clear on the goals
consider a phase-in over the next five years (not a one time jump)
recognize that there are lots of operational rules and regs to keep in mind
"Conceptually coherent comprehensive assessment system"
he wants the DoE to articulate a "clare and explicit theory of action"
how do we get from A to B
Congress will utimately reauthorize ESEA...but DoE must have a sense of the likely accountability uses before letting the funds
measuring a limited number of big ideas at deeper levels of understanding
measuring student learning, not cross-state comparisions
theorically sound design modles: student model, evidence model, task model...
design systems up front; reports need to be part of the system, not an add on
suggesting "curricular units" that focus on the "big idea" of the domain, used within the existing curriculum, basis for performance tasks, access for all students
why the obsession with instant results? He dismisses this as unnecessary as there have been other assessments throughout the year.
argues that we have much more of an instruction than an assessment problem when it comes to students with disabilities
new psychometric practices: traded validity for "pretty scales"
assessment should go along with core courses up to 10th grade
after 10th grade, more choices and more diversity
Points out that this is an ongoing cost, not a one time purchase
embrace differences
figure out what is absolutely essential
reconsider having every child tested on every item (sampling)
allow for multiple answers
be exceptionally clear on the goals
consider a phase-in over the next five years (not a one time jump)
recognize that there are lots of operational rules and regs to keep in mind
Gary Cook, University of Wisconsin
coming at it from "alignment standards" specifically in English Language Learners
Cycle test: test in April, results when school is out
disconnect between when test is administered and when it might be used: a year behind
He said, "let's do this in the fall" in Wisconsin...testing grade 3 content for 4th graders in December of 4th grade
multiple assessment cycle
he has a chart that has three assessments with three results a month later in each case
proposed assessment system delivered online (likens it to ATMs being universal)
"shared common core content"
loud laughs as he says that content setting is a "very exciting endeavor" (cites labor history as something that has to be taught in Wisconsin now which he didn't know was necessary)
common assessment across states, need to sort out what matters to each state (ah, so Texas history comes up, indirectly)
wants to see 100% of teachers involved
for a non-language skill, you have to have a language neutral assessment before you can have supported language assessment and finally move to a traditional assessment
(so we need to find out if someone knows math, even if they don't know English yet, by assessing in a way that doesn't require their English. English Language Learners have to fight their way through the question before they can even get to if they can answer the question.)
He's now scrolling through questions demonstrating how one might do this. Good stuff on the ELL end, using technology (oh, yeah...assuming we have the technology)
Things to think about:
beyond multiple choice
he's sure currentl assessment models will work
what do students (ELL or disabled) know? Using technology to get at that
how can we get people to use assessment data effectively?
Cycle test: test in April, results when school is out
disconnect between when test is administered and when it might be used: a year behind
He said, "let's do this in the fall" in Wisconsin...testing grade 3 content for 4th graders in December of 4th grade
multiple assessment cycle
he has a chart that has three assessments with three results a month later in each case
proposed assessment system delivered online (likens it to ATMs being universal)
"shared common core content"
loud laughs as he says that content setting is a "very exciting endeavor" (cites labor history as something that has to be taught in Wisconsin now which he didn't know was necessary)
common assessment across states, need to sort out what matters to each state (ah, so Texas history comes up, indirectly)
wants to see 100% of teachers involved
for a non-language skill, you have to have a language neutral assessment before you can have supported language assessment and finally move to a traditional assessment
(so we need to find out if someone knows math, even if they don't know English yet, by assessing in a way that doesn't require their English. English Language Learners have to fight their way through the question before they can even get to if they can answer the question.)
He's now scrolling through questions demonstrating how one might do this. Good stuff on the ELL end, using technology (oh, yeah...assuming we have the technology)
Things to think about:
beyond multiple choice
he's sure currentl assessment models will work
what do students (ELL or disabled) know? Using technology to get at that
how can we get people to use assessment data effectively?
Subscribe to:
Posts (Atom)