On Monday night, the School Committee held a 1.5-hour Committee of the Whole on the District’s MCAS scores. Dr. Edouard-Vincent provided a summary presentation, which I will link when she makes it available online.
Overall the district is making substantial progress toward targets (61%). Sounds ok, right? Let’s talk about WHAT we are measuring. We are comparing our % of points earned across a set of ‘cumulative criterion-referenced’ targets. The targets include measurement of the following indicators:
Achievement
Growth
High School Completion
Progress toward attaining English Language Proficiency
Chronic Absenteeism
Advanced coursework completion
But it’s even more complicated than that. The weighting of these targets is broken down into two groups: high school grades (9-12) and non-high school grades (3-8). But wait! There’s more! Within each of those categories, there are subcategories for all students and the lowest-performing students, which then receive slightly different weights.
Here’s a quick summary of the weights in the subgroups:
The final step is that this year, 2018/2019, is weighted 60% vs. last year, 2017/2018, is weighted 40%.
It’s a complicated formula, so when we arrive at a “final” number we need to ask, “where do we go from here?” The natural inclination is to compare us. To other districts. To the best. To the worst. To each other. It would be extremely time consuming to do the research required to get to an apples to apples comparison. Short of that kind of analysis, it is very likely any comparative conclusions you draw will be flawed. If you want to get data crunchy, throw in student spending data, which is also not an apples to apples comparison, and you’ve got yourself a real research project. You can dive into detailed data on the Department of Elementary and Secondary Education website.
So back to the question at the top of this post. Sounds OK, right? As a whole, we are making progress. What’s happening in smaller subsets of our student population tells more detailed stories about their progress. And, as you might imagine, there are an endless number of things to talk about when you dive into something as complicated as this. On the topic of measurement, I love this quote “Not everything that can be counted counts, and not everything that counts can be counted” by Albert Einstein. Moreover, wherever we fall on the scale, if we are going to measure ourselves (or our students), we should be prepared to act.
I’m going to give you some of my big takeaways.
Quality of curriculum matters
The quality of the curriculum matters. A curriculum is the most foundational tool a teacher will use. It is also a tool that helps drive consistency when many teachers are expected to deliver similar content and results.
There are multiple places where we are using dated curricula across the district. Our middle school science curriculum is using textbooks from 2001, and Massachusetts released the Next Generation Science Standards in 2016. It is not surprising that we are not meeting our targets. The Science department is aware and is currently preparing to pilot two curricula that meet the current standards at the middle school level.
Our enVisions Math curriculum aligns to 2011 standards, and our students are evaluated against 2017 standards. I remember hearing from the 4th-grade community this past spring that the Math MCAS was dramatically harder than in years past and also that it was very long. I also recall my daughter’s teacher changing the sequencing of her teaching in the run-up to MCAS. These comments make some sense when you consider that the standards change in 2017 AND that we aren’t using modern materials.
Using outdated curricula puts the responsibility of content creation on the teachers to align with the standards. I see this often in my work, where people want to take a trainer (for adults) and have them also create content. It’s not impossible, but it does take time. It strikes me that we aren’t giving the teachers in our district more time to make this happen. We need to take a hard look at these curriculum deficiencies and close these gaps.
Root cause analysis has consequences
There was a lot of discussion about root causes. Why did the 4th grade McGlynn students struggle in Math? Why did the 5th-grade math scores decline from the previous year among students from Brooks? Why don’t more high school students take accelerated Science courses? For me, there are a million questions you have to ask when looking at root cause analysis and what to do about it. It’s not enough to address the root cause. We have to look at what consequences those choices will have. If we take a strong teacher out of the teaching ranks to act as a coach across more classes, that is likely a good thing. Unless, of course, we don’t address the vacuum we created by removing that same teacher from the original classroom. If we instead replace that seasoned, experienced teacher with a new teacher, what do we need to do to make sure that the new teacher is equipped and prepared to succeed? It will take MORE to fill the shoes of the original teacher, and we have to address that MORE factor.
The McGlynn middle school was explicitly mentioned as a cause for concern this year. There are multiple Math teachers out on medical leave for various reasons and lengths of time at present. Is it enough to put substitute teachers in place? Creative suggestions were offered in terms of tapping into other qualified teachers in the building. Can this can be done without impacting the students who rely on those teachers? Is that the right solution to fill the gap? How long will the gap be? And what becomes of the next gap we’ve created? How do we fill that gap? There are no right answers. There is always a desire to fix the problem when we are faced with situations like this, but the immediate fix can often have unintended consequences.
We are making forward progress
At the high school, our overall scores didn’t change dramatically (+2 percentage points), but we saw significant gains in our lowest-performing student cohort. We received 11 out of 12 points this year for high school completion vs. only 4 out of 12 points last year.
At the middle school level, results further reinforced the decision of the Superintendent to eliminate the middle school choice selection process in favor of a lottery system to make both schools equally diverse. We should expect to see results change with our population shifts next year.
At Andrews, students in both cohorts (all students and lowest-performing) made significant gains in the Achievement category. Achievement measures academic progress via the actual MCAS score (e.g., the student got a 480) and how it changes from year to year. As a result, the school posted significant gains in its overall target percentage (50% in 2017/2018 and 78% in 2018/2019). At McGlynn Middle, the story that the data tells is quite different. Here, the lowest-performing students went from a 34% overall score in 2017/2018 to an 88% overall score in 2018/2019. This change is significant. Paul Texeira, the English Language District Director, echoed this in his comments. He noted that comparison among our EL program as compared to others is challenging, but when you look at progress at the student level, it is clear that our students are making significant progress. Conversely, the all-student cohort performance at McGlynn Middle School declined from 49% in 2017/2018 to 21% in 2019.
Across the non-high school level, the gains district-wide were modest. Our overall score in 2018/2019 is 65% up from 61% in 2017/ 2018. The story under that shows some volatility across the four elementary buildings. Some posted significant gains in certain areas while others posted declines in that same area.
One of the most significant points of discussion at the elementary level included the full implementation of the FOSS science curriculum in the 2019/2020 school year.
The continued discussion around literacy was also front and center. The middle schools are better targeting the reading specialists to the students who need the most help this year, which wasn’t the case in years past. There was also mention of the various assessment pilots at the elementary schools. I do wonder whether we have enough reading support from K-8 based on all of the literacy discussions we are having. We will be assessing students sooner, and we will have the data to intervene sooner. But we will very likely also have an increase in the number of students identified as requiring support. I am worried that we are not prepared for this from a staffing and curriculum perspective. The good news in moving to a more proactive assessment approach is that we will be able to quantify the support required across our buildings and let data drive our staffing. Some buildings will require more support than others, and we need to be prepared to move away from the ‘everyone gets the same’ model and toward an ‘everyone gets what they need’ model. We talk a lot about equity in education, but this evolution in Medford will require us to think carefully about how we meet the needs of all students, and what that will take. Putting all the right pieces in place, assessment tools, curriculum, professional development, and staffing will require focus and commitment come budget season. Our students require a plan that considers the full picture to succeed.
Your call to action: If you have questions about performance at your child’s school or across the district, check out the data on the DESE website. There is a handy glossary of terms at the bottom of this page that you might find helpful.