presentation is here
update on the MTEL alternative pilot
plus a set of licensure proposals including a vote to consider extending the pilot
the National Center for the Analysis of Longitudinal Data in Education Research (which they make CALDER, which isn't actually the acronym, people) panel: James Cowen, Dan Goldhaber, Roddy Theobald is here
Claire Abbott of DESE: in October 2020, Board voted for a three year pilot for alternatives to the MTEL
hope to establish more opportunities for educators to demonstrate necessary skills and so forth
written report in October of 2021, mostly on types of alternatives just then made available
today: most recent data coming out of the pilot
types of alternatives, who is accessing them, and preliminary rates
Cowan: The first two are the ones that will be focused on; they have the highest take-up rate
first is an alternative to the literacy skills test
MTEL Flex: after receiving a near-passing score, candidates submit a written analysis
third only has had thirty participants so far
the final one is newly coming online; hope to have data next year
first is an alternative to the literacy skills test
MTEL Flex: after receiving a near-passing score, candidates submit a written analysis
third only has had thirty participants so far
the final one is newly coming online; hope to have data next year
disproportionately candidates with emergency license and are working in the field
pass rates about the same on the CLST Alternative as on MTEL; about 9% higher on MTEL Flex
passage rates:
pass rates about the same on the CLST Alternative as on MTEL; about 9% higher on MTEL Flex
passage rates:
Final pass rate by assessment:
Middle bar re-weights the history of those involved
Bit more of a pass rate in the Flex group compared to traditional
West: understand the inference here: is 71.6% the rate you think would have passed if they had just taken the test again?
Yes, "the counterfactual test rate" taking the test again, rather than having this more flexible alternative
West: what we don't know is this is inducing people to be willing to do it again
And what's to come?
Cowan: will link to career milestone data, how effective are they, teacher diversity, what are candidate experiences
West: does a good job of explaining the rationale: "we know that MTEL scores are predictive of success in the classroom"
wait, we do?
Cowan: We also know that there are a lot of false positives and a lot of false negatives
and whether providing a pathway for some of those kept out will allow for
West: very excited that we're doing this
proposal pushes end date of pilot to June 2025; is that realistic to learn if it worked or not? Or should this go further?
Abbott: excellent question
feel confident that will have impact and efficacy data
feel that more teachers will participate over time
MTEL Flex is much less: $69 which is about half the cost
increase in access, particularly if cost has been a barrier
West: two years of data on the first cohort, and one year of data on the second cohort
Peyser: teacher effectiveness data
"I think it's more appropriate to call it teacher ineffectiveness analysis"
"scoring above or passing one of these assessments is not going to tell us if a teacher is going to be effective...and if you don't make it" doesn't show you won't be effective
"May be some easier ways of identifying" ineffectiveness
early indicators that MTEL Flex provides negative outcomes in teacher diversity in subject assessments
are they included into the communication and literacy data?
Cowan: MTEL Flex isn't offered for communication and literacy
proportion of white candidates taking MTEL Flex is higher, in any case
even small differences in pass rates could compensate
Peyser: looking at alternative standardized assessment; how do you know that they're equally rigorous?
"the reason that MTEL exists is an analysis that these other actually don't...assess the right content or at the right level"
"how do we know that these are apples to apples?"
Liz Losee: we've done it a little differently based on the options
We ask them to show which parts of the assessment are aligned to which test objectives
content experts at the Department also reviewed what was submitted to ensure alignment
done that at the front end, so that only assessments that meet that bar are being implemented
Peyser: if we think they are the same, it's probably cheaper to use one of the off the shelf options
if we look at the data and there is a difference in terms of the outcomes in race and ethnicity, what hypothesis would we have on what's different about these tests versus the MTEL?
Cowan: Think it's hard to say; may depend on earlier education, exposure to content in programs
theories around the format of the test: MTEL Flex is less of a standardized test, it's a written assessment
Peyser: more interested in the differences with the standardized test, as it feels as if the conversation has been that testing is the problem, and not the instrument
Cowan: not familiar with any quantitative research
people have raised this issue that it could be the format of the test, but the content and design varies between these two tests as well
Mohammed: in the proposed amendments for alternative
performance assessment will be removed for all administrator licenses?
West: understand the inference here: is 71.6% the rate you think would have passed if they had just taken the test again?
Yes, "the counterfactual test rate" taking the test again, rather than having this more flexible alternative
West: what we don't know is this is inducing people to be willing to do it again
And what's to come?
Cowan: will link to career milestone data, how effective are they, teacher diversity, what are candidate experiences
West: does a good job of explaining the rationale: "we know that MTEL scores are predictive of success in the classroom"
wait, we do?
Cowan: We also know that there are a lot of false positives and a lot of false negatives
and whether providing a pathway for some of those kept out will allow for
West: very excited that we're doing this
proposal pushes end date of pilot to June 2025; is that realistic to learn if it worked or not? Or should this go further?
Abbott: excellent question
feel confident that will have impact and efficacy data
feel that more teachers will participate over time
MTEL Flex is much less: $69 which is about half the cost
increase in access, particularly if cost has been a barrier
West: two years of data on the first cohort, and one year of data on the second cohort
Peyser: teacher effectiveness data
"I think it's more appropriate to call it teacher ineffectiveness analysis"
"scoring above or passing one of these assessments is not going to tell us if a teacher is going to be effective...and if you don't make it" doesn't show you won't be effective
"May be some easier ways of identifying" ineffectiveness
early indicators that MTEL Flex provides negative outcomes in teacher diversity in subject assessments
are they included into the communication and literacy data?
Cowan: MTEL Flex isn't offered for communication and literacy
proportion of white candidates taking MTEL Flex is higher, in any case
even small differences in pass rates could compensate
Peyser: looking at alternative standardized assessment; how do you know that they're equally rigorous?
"the reason that MTEL exists is an analysis that these other actually don't...assess the right content or at the right level"
"how do we know that these are apples to apples?"
Liz Losee: we've done it a little differently based on the options
We ask them to show which parts of the assessment are aligned to which test objectives
content experts at the Department also reviewed what was submitted to ensure alignment
done that at the front end, so that only assessments that meet that bar are being implemented
Peyser: if we think they are the same, it's probably cheaper to use one of the off the shelf options
if we look at the data and there is a difference in terms of the outcomes in race and ethnicity, what hypothesis would we have on what's different about these tests versus the MTEL?
Cowan: Think it's hard to say; may depend on earlier education, exposure to content in programs
theories around the format of the test: MTEL Flex is less of a standardized test, it's a written assessment
Peyser: more interested in the differences with the standardized test, as it feels as if the conversation has been that testing is the problem, and not the instrument
Cowan: not familiar with any quantitative research
people have raised this issue that it could be the format of the test, but the content and design varies between these two tests as well
Mohammed: in the proposed amendments for alternative
performance assessment will be removed for all administrator licenses?
Hills: roll call?
Abbott: this was just the presentation on the study; next presentation was on the proposed regulation amendment
the proposed amendments to the licensure regulations are here
This also includes a request for the extension of the alternative assessment pilot by one year, informed by the findings just presented
also two new licenses, an out of state SEI option, and other proposed technical revisions
several proposals "seek to mitigate challenges related to entering education and to better diversify our workforce"
working to reduce barriers to licensure over the past several years
have also extended flexibilities related to staffing for districts
to solicit public comment on proposed regulatory changes
Losee: looking to extend alternative assessment pilot by one year for additional data, allow more candidates to access
allow to accept assessments that were taken prior to approval date of this pilot
score varies according to the assessment
that number is relatively small (of those who have taken the test before the pilot was voted)
new provisional license for principals and assistant principals
constriction in pipeline for principals; hope to address some of these pipeline challenges
would align with similar pathways for teachers and superintendents (which have provision licenses)
performance assessment for leaders would then be required to initial licenses
more flexibility to create the four performance tasks while on the job
also would mean that need to adjust out of state candidates for experience; currently need one year of experience, would increase to three years
Stewart: in terms of constrictions
is it the out of state piece and no provisional?
Abbott: have seen a decline in the number of licenses being issued overall
number of out of state licenses is small
have heard that the performance tasks are easier to do if you're in the role; more challenging to complete if you're not doing the job
could now take the role, complete the tasks, get initial license
This also includes a request for the extension of the alternative assessment pilot by one year, informed by the findings just presented
also two new licenses, an out of state SEI option, and other proposed technical revisions
several proposals "seek to mitigate challenges related to entering education and to better diversify our workforce"
working to reduce barriers to licensure over the past several years
have also extended flexibilities related to staffing for districts
to solicit public comment on proposed regulatory changes
Losee: looking to extend alternative assessment pilot by one year for additional data, allow more candidates to access
allow to accept assessments that were taken prior to approval date of this pilot
score varies according to the assessment
that number is relatively small (of those who have taken the test before the pilot was voted)
new provisional license for principals and assistant principals
constriction in pipeline for principals; hope to address some of these pipeline challenges
would align with similar pathways for teachers and superintendents (which have provision licenses)
performance assessment for leaders would then be required to initial licenses
more flexibility to create the four performance tasks while on the job
also would mean that need to adjust out of state candidates for experience; currently need one year of experience, would increase to three years
Stewart: in terms of constrictions
is it the out of state piece and no provisional?
Abbott: have seen a decline in the number of licenses being issued overall
number of out of state licenses is small
have heard that the performance tasks are easier to do if you're in the role; more challenging to complete if you're not doing the job
could now take the role, complete the tasks, get initial license
five year provisional
Brian Devine: we're going to create a military spouse license: "why, you might ask? Well, we have to. There's a law!"
responds to ch. 154, sec. 10
need a bachelor's degree and a license from another state, as well as being a military spouse
it's valid for three years of employment
also revising:
new out-of-state option for SEI endorsement (SEI created in 2012); accept a comparable out of state credential; is not across the board, crosswalks of what different states require
Brian Devine: we're going to create a military spouse license: "why, you might ask? Well, we have to. There's a law!"
responds to ch. 154, sec. 10
need a bachelor's degree and a license from another state, as well as being a military spouse
it's valid for three years of employment
also revising:
new out-of-state option for SEI endorsement (SEI created in 2012); accept a comparable out of state credential; is not across the board, crosswalks of what different states require
performance assessment is only for principals, but the language has it for other languages; this would remove it
changing language to ensure alignment ("world languages")
history and social science
Lombos: question about rigor; all about removing barriers
a little concerned about rigor; I don't know how you're going to make us feel good about it
having language?
Devine: not sure what you mean about language
Lombos: makes us feel good that if we're going to take that out, ensures rigor
Devine: rigor already there
need to complete one of three approved routes to licensure
not the only measure to getting these licenses
Stewart: really interested in the bridges to diversity
Abbott: supports to emergency licensed educators; much more diverse
encouraged by their interest in staying in the field
grants to support pathways into further licensure
vast majority of them are committed to staying in the profession
Devine: leadership academy in diversifying the superintendent workforce
Canavan: recruiting at school level
provisional principal and assistant principal license
more frequent evaluation? additional support?
Abbott: interested in exploring how those provisionally licensed are doing
chance to support districts in embedding that induction mentoring program
Devine: required mentorship for first year
changing language to ensure alignment ("world languages")
history and social science
Lombos: question about rigor; all about removing barriers
a little concerned about rigor; I don't know how you're going to make us feel good about it
having language?
Devine: not sure what you mean about language
Lombos: makes us feel good that if we're going to take that out, ensures rigor
Devine: rigor already there
need to complete one of three approved routes to licensure
not the only measure to getting these licenses
Stewart: really interested in the bridges to diversity
Abbott: supports to emergency licensed educators; much more diverse
encouraged by their interest in staying in the field
grants to support pathways into further licensure
vast majority of them are committed to staying in the profession
Devine: leadership academy in diversifying the superintendent workforce
Canavan: recruiting at school level
provisional principal and assistant principal license
more frequent evaluation? additional support?
Abbott: interested in exploring how those provisionally licensed are doing
chance to support districts in embedding that induction mentoring program
Devine: required mentorship for first year
Mohammed: proportion on provisional licenses
can get that
Mohammed: support for school committees in working with superintendents on provisional licenses?
Abbott: new superintendent program run by MASS
MASC strong training for school committees around superintendent evaluation
can get that
Mohammed: support for school committees in working with superintendents on provisional licenses?
Abbott: new superintendent program run by MASS
MASC strong training for school committees around superintendent evaluation
to public comment passes
public comment through January; back to the Board for February meeting
public comment through January; back to the Board for February meeting
No comments:
Post a Comment
Note that comments on this blog are moderated.