Sunday, April 15, 2012

The wrong statistics . . . and why the right ones matter in the Delaware education debate

Lots of sound and fury both from choice/charter supporters and detractors over the past few weeks in the Delaware blogosphere . . .

Lots of people raising interesting personal, ideologically based theories on both sides . . .

And an astounding absence of real data on important questions.

Here's one of the things we really need to know, in order to resolve the charter school/school choice controversy:

Has choice and charter created harm?

Harm would be hereby defined as a reduction in the educational outcomes of those students within the traditional public school system who have not had access to charter schools or have not been able to/chosen to utilize choice to find a more preferred school.

Harm would be determined by going back to the period, say 1985-1995, to determine how well the minority/low-income students were doing before choice/charter, and compare that to how well those students are doing now.

The problems?

1--DOE does not keep those stats currently (even if you look at NCLB cell stats if you could get them) in such fashion as to answer that question.

2--Since pre-1995 is also the pre-high-stakes assessment era, even if DOE kept those stats, you would be comparing apples to oranges, because there are not similar assessments.

The solution?

Somebody needs to design a research study that utilizes surrogates for testing (a combination of graduation rates, drop-out rates, SAT performance, and other items) in order to create a meaningful comparison of the outcomes for those students before/after choice/charter.

It could be done, and it would make a great dissertation.

I'm not certain it will ever get done (I damn sure don't have the time to do it), but what bothers me about the quality of the discussion is that nobody really seems to understand that this information is missing.

Everybody assumes that choice/charter has harmed these students because they are in increasingly concentrated low-income schools, and that may well be the case . . . .

But we don't actually know that.

And we need to find out--or at least acknowledge that information is missing from the conversation.

1 comment:

tom said...

It may be even worse than you think.

I doubt that it is even possible to design a useful study using surrogates. The SAT, for example, has been so watered down that test scores from the 80's are not even meaningfully convertible to scores from today's SAT. another problem is that many of the students you are trying to account for in the past had little hope of attending college, so they and/or their parents may have written off the SAT as a waste of time and money.

Likewise, changes in curriculum, demographics, regulatory and record keeping requirements have probably rendered comparisons of past & present grades, or graduation/drop-out rates pointless as well.