Tuesday, April 8, 2008

Separating Data from Ideology: Fact-checking the Texas Charter Schools Controversy

Perhaps it was too be expected that Dana, at Delaware Watch, would find a pro-government, anti-private enterprise lesson in the current Texas Charter Schools controversy.

Here's the gist of the story as Dana reported it:

Nearly One-Half of All Charter Schools in Texas Have Bilked Taxpayers for $26 Millions....

More news of success from the barely regulated, semi-private and "superior" education system....

"Nearly half" in Texas is no small number. It's "93 of the 211 charter operators," 20% of which, apparently, took the money and ran....

Why did these charter schools rip off the taxpayers?...

Perhaps a more fruitful question is why have so many of the Lone Star state's charter schools found themselves frequently distressed. Unsurprisingly, one explanation happens to be one of the reasons that many politically active misers find attractive about charter schools. Cheap labor = inexperienced labor....

It appears that charter schools in Texas have certainly accomplished the task of making public schools "better" if only by contrast. As long as 1/2 of the public schools don't commit fraud, pocket taxpayer dollars and, in some cases, close shop afterwards, public schools in Texas are doing a much better job than charter schools.

I challenged Dana on the statistics, pointing out that Texas has a horrible mechanism for calculating enrollment, and questioning both (a) the absence of comparative stats for traditional schools and (b) the distribution of the alleged fraud equally among 93 schools, a premise with which Dana disagreed vehemently:

"...the other 20% are being tarred with your broad brush based on discrepancies of 3-5 students per year."

I've been following this story for some time. Given that the Texas political environment is very pro school-choice, I can assure you it's not politically possible that your(and this is precisely what it is) wish that 20% of these schools are being niggled over an amount of students that would little more than a bookkeeping error.

We are talking about fraud, Steve.

Keep that comment in bold--Dana asserting it's not possible--clearly in your mind.

I responded:

My point about my "wish" that most of the violations come from a few schools is based on a known statistical principle for working with unknown data sets. When you know there is wide variation from the mean (your sources cited three examples out of 91 that were as much as 40 times above the mean), then statisticians usually employ the 80-20 estimation for data sets where individual points are not provided: 80% of the activity is expected to result from 20% of the reference points.

That would suggest that of 91 schools having a $26 million overrun, that 18 of the schools accounted for nearly 21 million of the problem, leaving the other 73 schools with about 5.2 million between them. This would mean that the mean for that 80% represented and over-count of roughly 13 students per school in a multi-year study. That's less than what NJ schools come out with on an annual basis.

Of course you'll argue that I have no basis for this, because the statistical breakdown for neither the charters nor the publics was included in your source material, even though I have merely used a standard statistical sampling method for dealing with incompletely described data sets. But my extrapolation of insufficient data is no less valid than your extrapolation of the same data set, where you used the term "bolked" (well, actually, you spelled it right and said "bilked") in blanket application to dozens of schools that may or may not have committed any malfeasant act.

And again, Dana, demurred:

"Of course you'll argue that I have no basis for this, because the statistical breakdown for neither the charters nor the publics was included in your source material...."

Precisely, which is why I'm not the least bit worried about this:

"So yes, you have overstepped your data in search of an ideological point. Don't worry, we all do it from time to time."

I'll just wait for the court cases and the settlements out of court. I'll post accounts of them here and enjoy your poignant silence when I do.

The problem here is that, for once, Dana has failed to do his homework.

The Dallas Morning News thoughtfully included the complete run-down of the debts owed by the charter schools, both those still in business and those defunct. If you bother to actually investigate the numbers, you'll find out the following:

The total debt alleged (we'll come back to that alleged) a little later) for the 93 charter schools is $26,092,850.

The worst 20% (18 schools) accounts for $21,454,657 (82.2%), which means each school averaging $1,191,925 in money owed to the State.

The remaining 80% (75 schools) accounts for $4,638,193 (17.8%), which means each school averaging $61,843 in money owed to the State.

Notice that these are almost exactly the figures I predicted by using the 80-20 rule, before I had ever seen the break-out data. (Why did that work? When you have a large group of data points an outliers that are between 10-40 above the mean--Dana's own article source cited three such cases--it is predictable that the data points will break down in an 80-20 pattern.

Moreover, the bottom 40% of the schools--that's 36 of the 93 schools--account for only $103,247 (.4%--realize that's Four-Tenths of One percent of the total $26 million, or that each of these schools averages just $2,868.

In fact, the bottom 54 of these school all owed less than $25,000, and eleven of them owed less than $1,000.

Now go back to Dana's firm prediction: it's not politically possible that your(and this is precisely what it is) wish that 20% of these schools are being niggled over an amount of students that would little more than a bookkeeping error.

Let's talk about book-keeping errors.

The Texas enrollment accounting system is so bizarrely complicated that screwing up a few days' worth of attendance can actually cost a school money. Here are the parts of the most recent Dallas Morning News story that somehow didn't make it into Delaware Watch:

The Texas Education Agency, which oversees public education in the state, is working to recover $17 million of the $26 million from nearly half of the charters now operating in Texas. TEA records show that 20 schools went out of business before the state could recover its money, leaving taxpayers holding a $9 million bag of debt.

These charters collected state funds either by inflating the number of students in their classrooms or by making accounting mistakes.

Lisa Dawn-Fisher, a top TEA official, said traditional school districts make attendance reporting errors, too. But charters make them to a greater extent.

Charters often don't have experienced staff or strong oversight and, unlike traditional public schools, they cannot generate revenue through property tax hikes or bond elections, according to TEA officials.

"There is a kind of perverse incentive for a charter school in financial distress to look at [attendance inflation] as a way to get more money," said Dr. Dawn-Fisher, deputy associate commissioner for school finance. "If they can't get the warm bodies in the building, they may feel an incentive to falsify records."

Notice that we're told here that traditional schools also make such accounting errors, but supposedly to a much less extent. Problem is, we're never given a figure that supports that contention.

Next we're told that the only two reasons for such problems are fraud or accounting errors.

Finally, Lisa Dawn-Fisher suggests that without tax revenues charter schools "may" feel the need to falsify records.

But further down in the article we find this:

Most Texas school districts receive per-student funding each year based on attendance projections from the previous year. The districts report the real data at the end of the year to settle any differences.

Students can choose to attend a charter school at any point during the year, creating more ups and downs in attendance than at regular schools.

Therefore, TEA makes charters report attendance figures every six weeks. So, theoretically, the state's monthly payments are based on real numbers -- not projections.

Problems with charter attendance reports run the gamut. Some don't report absences throughout the year. So, the state gives them money as if every student showed up each day. Others count students long after they dropped out. And some schools report too many students in special education and other programs that generate extra state funding.

Some schools don't file paperwork on time. So, TEA pays them based on estimates in earlier reports that showed too many students. TEA has the authority to stop payment if a school files paperwork late, according to TEA spokeswoman DeEtta Culbertson.

TEA officials say they can't know how much of the current $26 million debt results from attendance inflation vs. other reporting errors until final reports come out in May. They stress that some errors come from factors beyond a charter's control.

Some charters could get money back from TEA at the end of the year. Recent data shows that roughly 100 charters stand to get back a total of $4.4 million because of reporting problems.

So what do we know now? We know that half of the charter schools are so careful about reporting enrollments that the State owes them $4.4 million in additional money.

We know that the TEA created a mechanism for supervision of charter school enrollment that by its own admission it doesn't administer effectively.

We also know that this is not the first time the TEA has been found to be either lax in collecting and reporting data. In 2000 a detailed study presented to the Texas State School Board Association found that the TEA was not only incredibly inefficient at collecting data about statewide drop-out rates, but may well have cooked the data it did collect.

Now let's go back to the figures I presented above. Given that Texas provides roughly $5,400 per student per annum for the charter schools (which is about $1,800 less than students in traditional schools get), it turns out that the overwhelming majority of the charter schools in the Lone Star State either reported their attendance correctly, reported it so conservatively that they're going to get money back, or made errors that would have involved between 1-5 students over the course of several years. In fact, based on the actual figures rather than an ideologically driven assumption, only 60 of 211 charter schools (28.4%) were alleged by the State to owe more than $25,000. Only 37 of 211 charter schools (17.5%) were alleged to owe more than $100,000.

The reality is that most of the charter schools on this list fall into one of three categories:

1) Outright criminal fraud that needs to be prosecuted. Arguably, any school above $100,000 falls into the category of needing to be closely examined. If the evidence of fraud is found, there need to be prosecutions. But we also need to be certain (ala Delaware's own Georgetown charter school) that they don't really fall into the next category....

2) Major book-keeping and accounting errors, based on inexperience, insufficient resources, and a lack of support by the state; or....

3) Schools whose errors are undoubtedly clerical, minor, and consistent with those being made by traditional schools.

The evidence suggests that about 37 of 211 schools fell between categories 1 and 2, and that 174 of 211 charter schools fall into category 3.

To suggest that nearly half of the Texas charters have committed fraud or attempted to bilk the State is not only hyperbole, it is simply irresponsible--especially in the face of the statistical evidence to the contrary.

So here's the bottom line, brethren and cistern:

1. When the State doles out public money, the State has an inherent responsibility to create and operate a system that audits the use of that money. Texas did not do this. That the Gulf Shores Academy could walk off with $8.5 million (the equivalent of claiming 1,574 students that did not exist, represents not only criminal fraud but also bureaucratic negligence on the part of the TEA.

2. When the recipients of public money commit fraud with those funds, they need to be prosecuted and put in jail.

3. When you experiment with private/public collaborations in education, the free market guarantees that some of these collaborations will fail. That's the cost of experimenting. Unfortunately, government-run schools are never allowed to go out of existence even when they are demonstrably failing. Students keep getting sent through their doors year after year; at least when the charters go belly up, the students leave.

4. The fact that some charter school entrepreneurs committed fraud, and that most Texas education bureaucrats did not catch it until it reached a figure as large as $26 million (out of a $37 BILLION Texas public education budget--about 7/100s of one percent) is not a legitimate argument for or against charter schools as a concept--unless you have already made up your mind for ideological reasons.

We also know, by the way, that Charter Schools in Texas are making some significant progress in providing meaningful educational alternatives. According to the National Center for Policy Analysis, in Texas:

Charters enroll more the twice as many African-American students (37.2 percent) as traditional public schools (14.2 percent), a slightly smaller percentage of Hispanic students (43.2 percent versus 44.7 percent), and less than half the proportion of white non-Hispanics students (17.7 percent versus 37.7 percent).

They enroll more low-income students who are eligible to participate in the federal free/reduced-price lunch (FRL) program (68.2 percent) than do traditional public schools (54.6 percent).

However, charters and traditional public schools serve about the same percentage of special education students (11.3 percent and 11.6 percent, respectively)....

Open-enrollment charter students out-performed traditional public school students in grades 6 through 9 in reading/ELA (English Language Arts) and mathematics....

In reading/ELA, students at alternative charter campuses performed above traditional alternative education students in grades 8 through 10.

In math, students at alternative charter campuses performed above traditional alternative education students in grades 5 and 7 through 10....

So let's not be so quick, next time, to accuse a lot of people working hard in Texas education of defrauding the people of the Lone Star state. The facts do matter.

No comments: