Children Left Behind: Statistics and Abstractions

by Matthew Koslowski on September 23, 2009
in Essays

In This Essay

No Child Left Behind and the Spirit of Democratic Education”, Why School? by Mike Rose
Monday Metaphor: Growth, Learning with Impact by John Spencer
“Why Our Standards-Based Grading Sucks”, Learning with Impact by John Spencer
“MCAS scores fall shy of target”, Boston Globe, by James Vaznis
“Charter schools see more attrition”, Boston Globe by James Vaznis
“The next chapter on education reform”, Boston Globe by Gov. Deval Patrick
“Critical thinking? You need knowledge”, Boston Globe by Diane Ravitch
“These test-score jitters are a sign of high standards”, Boston Globe

Ideals and Realities

I had some great conversations about education and public policy with a friend. She would take the pragmatic side of the argument while I would take the idealistic side. While I would speak of sweeping visions of what education should be, she would want specific plans on implementation.

Our arguments usually ended with me saying that so much depended on implementation, that what I thought could really have a great impact, and her saying that no implementation would be perfect and I needed to get my head out of the clouds.

Implementing High Stakes Testing

Last week saw the publication of the test scores for the Massachusetts Comprehensive Assessment System, or MCAS. Part of the Massachusetts Education Reform Act of 1993, this standardized test fulfills the requirements of No Child Left Behind.

Even before the test results were released, I have been thinking about testing and No Child Left Behind as evidenced by my past posts. John Spencer’s video post, Monday Metaphor: Growth was one thing rolling around in my head a few days before the MCAS release.

His next post but one, “Why Our Standards-Based Grading Sucks”, published the day after the MCAS results, the same day I was in the midst of rereading “No Child Left Behind and the Spirit of Democratic Education” in Why School?, ramped up my thinking even more:

  • Do we have reasonable expectations for children’s achievement?
  • Where do our expectations of academic achievement come from?
  • Are the tests we are using sufficient to gauge academic achievement?
  • Are there better ways to gauge academic achievement?

Single Modalities, Multiple Modalities

The assessment system being implemented at Mr. Spencer’s school is going to be single modality: a multiple choice test. Those tests are easy to write and easy to administer and easy to analyze. But what kind of feedback do such tests give us about student achievement? These tests only provide an assessment of one type of intellectual and cognitive capacities.

The Massachusetts Comprehensive Assessment System is a grandiose name. So many years have passed since I took one of the proto-types that I had forgotten its structure. I had assumed it was a simple, straight-forward multiple choice test just like I remember the SATs. But comprehensive may not be ironic in the title. The MCAS uses the following forms of assessment:

  • multiple-choice questions
  • mathematical short answer questions
  • short open-response questions
  • long open-response compositions

Unlike straight objective tests, the MCAS in the ideal assesses recall, recognition and synthesis. In reality, the recall and recognition sections also assess the ability of students to form educated guesses.

More of a child’s abilities are measured on tests like these. Think of an analogy to the physical body. Just as you cannot get a complete picture of someone’s fitness and health by testing only their upper body strength, you cannot get a full picture of a child’s academic and intellectual fitness by testing only their ability to recall and recognize and guess.

Asking the Right Questions

“But there are, in fact, a host of procedural and technical problems in developing, administering, scoring, and interpreting such tests. (And there are also concerns about how schools and districts can manipulate them.) ‘In most cases,’ writes measurement specialist Robert Linn, ‘the instruments and technology have not been up to the demands placed on them by high-stakes accountability.’ No wonder, then, that there is a robust debate among testing experts about what, finally, can be deduced from the scores about a student’s or a school’s achievement.”
– Mike Rose from “No Child Left Behind and the Spirit of Democratic Education” in Why School?, pages 45-46.

Are we asking the right questions? I don’t think we are. Much like the recent Town Hall Meetings, if we can even call them that, there is a lot of shouting without a lot of discussion. Rather than delve into the deep, complex questions we are looking for simple solutions.

I applaud the editors of the Boston Globe for their recent editorial “These test-score jitters are a sign of high standards”. They argue that if the MCAS shows we fail to make the No Child Left Behind’s mandate for adequate yearly progress, it is because the test has integrity and has not been dumbed down to artificially inflate our success rate. I hope that this is true. By 2014, No Child Left Behind requires that 100% of students — including 100% of vulnerable populations, English language learners, and special needs students — will be expected to achieve “proficiency” the MCAS.

Is expecting every student to pass really a useful metric? Won’t some critics cry out that if every student passes the test was too easy?

Since we are going to continue to use high-stakes testing, I hope that our government can look at the scores, include the subsets of vulnerable populations, and allocate resources to help boost achievement. Use the MCAS as an assessment of the health of our education system, like a CAT scan so that we know where to focus in and where the illness is most severe.

But at the end of the day, a test score remains a statistical abstraction. If the MCAS is used to gather statistics, then it is an opportunity wasted.

Implementing Charter Schools

Talking about high stakes testing and the performance of our public schools often leads to a discussion of charter schools.

There is a lot of controversy around charter schools in this country. You cannot deny that they take resources away from public schools: even if they don’t take the cream of the crop, which many people including myself believe they do, tax money is taken from the public schools for each student enrolled in a charter school.

I am very confused by the claims about the success of charter schools. Neither side agrees: proponents say that charter schools are an undeniable success; opponents say that charter schools are no better than public schools. I heard of one study that said charter schools performed no better, and in some cases performed worse, than Boston’s public schools. I saw news articles about charter schools outperforming Boston’s public schools.

Gaming the System?

But are we using the correct metrics? Are we looking at only the students that graduate from the schools?

An article in the Boston Globe struck me. James Vaznis tells us that “Fewer than half of the students who enrolled in Boston charter high schools as freshmen over the past five years made it through to graduation, usually departing for other schools, according to a new study,” that was published on September 17.

Many students who left the charter schools re-enrolled in Boston public schools. Critics may claim that the students who left wanted to get an easier diploma. That is possible. That needs to be investigated.

Can a charter school like MATCH Charter Schools really judge itself successful when 25% of its seniors left during the academic year, with “some students [transferring to Boston public schools] just a few weeks before graduation”? What does it mean when a charter school publishes its graduating class’s college acceptance rate if 25% of its seniors dropped out?

Just as we need to be sure that we are using the correct metrics when we are evaluating a child’s achievement, we need to be using the correct metrics in our discussions of public schools versus charter schools. Especially when their is so much talk of expanding the role of charter schools. What if we were to eliminate public schools and look at the statistics of a world of charter schools? We might just find that they perform no better than the system we have now.

Comments are closed.