Kilroy has up a post on the DSTP; he and I have exchanged emails on this topic in the past, and his post convinces me that it is time to tell (most) of what I know and witnessed.
I say "most" because I am going to explain what happened in some detail, but I am also going to leave out the names--at least 90% of them.
From 1992-1995 I co-chaired the Delaware Social Studies Curriculum Frameworks Commission, which was charged with developing new K-12 Social Studies Standards. I also chaired the writing team, and did the bulk of the actual writing and editing of the final document. We produced standards in four different areas: History, Geography, Economics, and Civics.
It is important to note that all the curriculum frameworks commissions (SS, English/language arts, Science, and Math) were charged to create standards that were to be tested through a process called authentic assessment. This was one of the big buzzwords in the early 1990s, and it essentially meant that students would not take a test, but would produce a product or perhaps even a portfolio of work to demonstrate mastery of subject matter. The standards to be evaluated for such assessments differ markedly from conventional pencil-and-paper tests. Here's an example:
In History we developed four standards that would play out with increasing sophistication as students reached higher grades:
History Standard One--chronology: how history locates events in time; including time lines, cause-and-effect, and tracing trends and ideas through history
History Standard Two--analysis: how historians examine and analyze documents and artifacts; difference between primary and secondary sources;
History Standard Three--interpretation: how historians build different narratives depending on the evidence they examine, the questions that they ask, and the perspectives from which they work
History Standard Four--content: what you should actually know about US and World History (who was George Washington, when was the New Deal, yada, yada).
The point of these standards was that History Standards 1-3 were considered skills or process standards, that students would use to engage the content from History Standard 4, and that any assessment would expect students to demonstrate through a project the ability to take an event, person, or phenomena from HS 4, and then locate it in time (HS1); analyze data about it (HS2), and then develop an interpretive narrative (HS3). Essentially, we wrote standards not for students to learn history, but to practice history.
And that's how we were told that the eventual DSTP would test them.
[We were also assured (and I went to several meetings with concerned parents to pass on these assurances) that special needs students would be assessed at their functional/developmental level, not their grade level. That later turned out to be such a bald-faced lie on the part of the Department of Education that I have since been deeply ashamed for helping to promulgate it.]
So we wrote that kind of standards. The process was as political as academic, as you might imagine. The UD Geography Department and the Delaware Geography Alliance (which is essentially a UD subsidy operation) got to dictate the Geography standards which, not unreasonably, reflected the then-current National Geographic standards.
The UD Center for Economic Education received virtual carte blanche to saddle Delaware students with a set of laisse faire ultra-conservative standards in economics that could actually offend even a libertarian. For example, it is not intellectually possible even to explain the concept of neo-colonial economics in a Delaware classroom any more, because the DE Economics Standards are so tightly written from a Western Industrial Capitalism model that there isn't even room in the abstract to consider other theoretical economic models.
Ah--if I only knew then what I know now.
But we wrote good standards, overall.
And then the General Assembly decided it was cost prohibitive to create the kind of assessment those standards (again, in ELA, SS, Math, and Science) required. So they hired people to come in and write old-fashioned paper tests for standards that were not designed to support them. The Math people did a pretty good job because a lot of the math standards had been cribbed from the nationally normed NCTM [National Council for Teachers of Mathematics], but for everybody else....
The experience was a nightmare.
Let me explain what happened to the History standards. DOE hired an individual from Maryland whose primary background was English/Language Arts to supervise the Social Studies test, giving that individual complete autonomy from the only Social Studies expert on staff at DOE. That individual came from a school of thought that believes that only skills are important; that content is merely the vehicle to teach skills. So that person then reached a decision [which was in turn rubber-stamped by the General Assembly] only to test History Standards 1 (chronology), 2 (analysis), and 3 (interpretation), and not to test history content knowledge at all. Ever.
The DSTP History test to this day does not test History content.
As a result, groups of teachers sat down and tried for months to come up with test items that would somehow assess student performance on chronology, analysis, and interpretation without being based on historical content. Are you screaming yet? Then, when they found something they thought might work, they simply threw it into the test without ever vetting their historical material with actual historians.
So on the original DSTP 8th Grade examination we found some of the following:
--An extended answer question on trade between the "thirteen British North American colonies" and the Caribbean/England during the period 1783-1790. yeah, that's right, after Yorktown
--An extended answer question that required 8th graders to draw conclusions about a picture of young Black men conducting a sit-in at a Woolworth's lunch counter in the late 1950s--when the curriculum guide said 8th Grade American History should only go as far forward as 1877....
There was more, but it is too ugly to tell.
During this period, the individual spearheading the creation of all the DSTP materials stayed around for about 18 months--long enough to engineer the award of the final work and the grading of the assessments to the company he then promptly went to work for.
Meanwhile, successive Secretaries and Associate Secretaries of Education came and went, and each one stirred the pot. In about 1998 we added "Performance Indicators" that were supposed to break down the standards into manageable pieces and give teachers a sequence of instruction. Only problem: State law says the DSTP is to be created from the Standards, not the Performance Indicators, so the branch of DOE doing the Assessments refused to acknowledge or use the new PIs even as the Curriculum branch was passing them out to all districts and demanding they be used to revise curriculum.
Kilroy is exactly right: successive generations of Delaware school children have been abused by a system of testing that--except in Math and some parts of ELA--has actually no educational reliability or validity (and I am using those terms in their technical senses) whatever.
So of course we made those tests make or break for student progress and school assessments under No Child Left Behind.
We didn't have to do it: we simply did it because they were already in place and neither the legislators nor the educrats could ever admit the dirty little secret to the public: our tests did not align with our standards, and our test results in most academic areas were actually damn near meaningless.
If I had a child today who was mandated to go to summer school on the basis of the DSTP, especially for failing either Social Studies or Science, I would refuse and sue the State, based on the fact that the State would not be able to prove in a court of law that they were actually testing the standards they had adopted.
I'd win.
The era (I am tempted to say "error") of high stakes testing and the concept that assessment should drive instruction is hopefully passing us by, but it has irreparably harmed a full generation of students just as palpably as the infamous New Math of the early 1960s left us with a generation of school children who could not even do simple calculations.
The problem is that in Delaware we got in at the very start of the trend, and have ridden it out to the end, warping it even further by bringing in pseudo-experts to keep it going and then refusing to actually pay to have real tests or aligned standards developed.
The irony? Our new effort to develop or purchase an adaptive, computerized test is, frankly, heading down the same stupid road, because in making the DSTP a political football we have neglected as a State to ask the right question.
The question we're asking: Can't we have a better test?
The question we should be asking: Why did we let wild-eyed educational researchers ever convince us that high-stakes testing was either good for kids or for school systems?
Until we get somebody in power (and, unfortunately, it is not Jack Markell, Matt Denn, or Lillian Lowery--as much respect as I have for all of them) who is willing to examine the roots of our testing and achievement problem, rather than simply whore out our children's futures in a Race to the Top for more Federal money, public education in Delaware will only continue to provide a mediocre payback for a massive investment.
I say "most" because I am going to explain what happened in some detail, but I am also going to leave out the names--at least 90% of them.
From 1992-1995 I co-chaired the Delaware Social Studies Curriculum Frameworks Commission, which was charged with developing new K-12 Social Studies Standards. I also chaired the writing team, and did the bulk of the actual writing and editing of the final document. We produced standards in four different areas: History, Geography, Economics, and Civics.
It is important to note that all the curriculum frameworks commissions (SS, English/language arts, Science, and Math) were charged to create standards that were to be tested through a process called authentic assessment. This was one of the big buzzwords in the early 1990s, and it essentially meant that students would not take a test, but would produce a product or perhaps even a portfolio of work to demonstrate mastery of subject matter. The standards to be evaluated for such assessments differ markedly from conventional pencil-and-paper tests. Here's an example:
In History we developed four standards that would play out with increasing sophistication as students reached higher grades:
History Standard One--chronology: how history locates events in time; including time lines, cause-and-effect, and tracing trends and ideas through history
History Standard Two--analysis: how historians examine and analyze documents and artifacts; difference between primary and secondary sources;
History Standard Three--interpretation: how historians build different narratives depending on the evidence they examine, the questions that they ask, and the perspectives from which they work
History Standard Four--content: what you should actually know about US and World History (who was George Washington, when was the New Deal, yada, yada).
The point of these standards was that History Standards 1-3 were considered skills or process standards, that students would use to engage the content from History Standard 4, and that any assessment would expect students to demonstrate through a project the ability to take an event, person, or phenomena from HS 4, and then locate it in time (HS1); analyze data about it (HS2), and then develop an interpretive narrative (HS3). Essentially, we wrote standards not for students to learn history, but to practice history.
And that's how we were told that the eventual DSTP would test them.
[We were also assured (and I went to several meetings with concerned parents to pass on these assurances) that special needs students would be assessed at their functional/developmental level, not their grade level. That later turned out to be such a bald-faced lie on the part of the Department of Education that I have since been deeply ashamed for helping to promulgate it.]
So we wrote that kind of standards. The process was as political as academic, as you might imagine. The UD Geography Department and the Delaware Geography Alliance (which is essentially a UD subsidy operation) got to dictate the Geography standards which, not unreasonably, reflected the then-current National Geographic standards.
The UD Center for Economic Education received virtual carte blanche to saddle Delaware students with a set of laisse faire ultra-conservative standards in economics that could actually offend even a libertarian. For example, it is not intellectually possible even to explain the concept of neo-colonial economics in a Delaware classroom any more, because the DE Economics Standards are so tightly written from a Western Industrial Capitalism model that there isn't even room in the abstract to consider other theoretical economic models.
Ah--if I only knew then what I know now.
But we wrote good standards, overall.
And then the General Assembly decided it was cost prohibitive to create the kind of assessment those standards (again, in ELA, SS, Math, and Science) required. So they hired people to come in and write old-fashioned paper tests for standards that were not designed to support them. The Math people did a pretty good job because a lot of the math standards had been cribbed from the nationally normed NCTM [National Council for Teachers of Mathematics], but for everybody else....
The experience was a nightmare.
Let me explain what happened to the History standards. DOE hired an individual from Maryland whose primary background was English/Language Arts to supervise the Social Studies test, giving that individual complete autonomy from the only Social Studies expert on staff at DOE. That individual came from a school of thought that believes that only skills are important; that content is merely the vehicle to teach skills. So that person then reached a decision [which was in turn rubber-stamped by the General Assembly] only to test History Standards 1 (chronology), 2 (analysis), and 3 (interpretation), and not to test history content knowledge at all. Ever.
The DSTP History test to this day does not test History content.
As a result, groups of teachers sat down and tried for months to come up with test items that would somehow assess student performance on chronology, analysis, and interpretation without being based on historical content. Are you screaming yet? Then, when they found something they thought might work, they simply threw it into the test without ever vetting their historical material with actual historians.
So on the original DSTP 8th Grade examination we found some of the following:
--An extended answer question on trade between the "thirteen British North American colonies" and the Caribbean/England during the period 1783-1790. yeah, that's right, after Yorktown
--An extended answer question that required 8th graders to draw conclusions about a picture of young Black men conducting a sit-in at a Woolworth's lunch counter in the late 1950s--when the curriculum guide said 8th Grade American History should only go as far forward as 1877....
There was more, but it is too ugly to tell.
During this period, the individual spearheading the creation of all the DSTP materials stayed around for about 18 months--long enough to engineer the award of the final work and the grading of the assessments to the company he then promptly went to work for.
Meanwhile, successive Secretaries and Associate Secretaries of Education came and went, and each one stirred the pot. In about 1998 we added "Performance Indicators" that were supposed to break down the standards into manageable pieces and give teachers a sequence of instruction. Only problem: State law says the DSTP is to be created from the Standards, not the Performance Indicators, so the branch of DOE doing the Assessments refused to acknowledge or use the new PIs even as the Curriculum branch was passing them out to all districts and demanding they be used to revise curriculum.
Kilroy is exactly right: successive generations of Delaware school children have been abused by a system of testing that--except in Math and some parts of ELA--has actually no educational reliability or validity (and I am using those terms in their technical senses) whatever.
So of course we made those tests make or break for student progress and school assessments under No Child Left Behind.
We didn't have to do it: we simply did it because they were already in place and neither the legislators nor the educrats could ever admit the dirty little secret to the public: our tests did not align with our standards, and our test results in most academic areas were actually damn near meaningless.
If I had a child today who was mandated to go to summer school on the basis of the DSTP, especially for failing either Social Studies or Science, I would refuse and sue the State, based on the fact that the State would not be able to prove in a court of law that they were actually testing the standards they had adopted.
I'd win.
The era (I am tempted to say "error") of high stakes testing and the concept that assessment should drive instruction is hopefully passing us by, but it has irreparably harmed a full generation of students just as palpably as the infamous New Math of the early 1960s left us with a generation of school children who could not even do simple calculations.
The problem is that in Delaware we got in at the very start of the trend, and have ridden it out to the end, warping it even further by bringing in pseudo-experts to keep it going and then refusing to actually pay to have real tests or aligned standards developed.
The irony? Our new effort to develop or purchase an adaptive, computerized test is, frankly, heading down the same stupid road, because in making the DSTP a political football we have neglected as a State to ask the right question.
The question we're asking: Can't we have a better test?
The question we should be asking: Why did we let wild-eyed educational researchers ever convince us that high-stakes testing was either good for kids or for school systems?
Until we get somebody in power (and, unfortunately, it is not Jack Markell, Matt Denn, or Lillian Lowery--as much respect as I have for all of them) who is willing to examine the roots of our testing and achievement problem, rather than simply whore out our children's futures in a Race to the Top for more Federal money, public education in Delaware will only continue to provide a mediocre payback for a massive investment.
Comments
As an IT professional, I've interviewed many job applicants that had the right "credentials" and passed the various certifications, but the last time we interviewed for a Network Administrator, I lost 20 out of 25 on a simple networking trouble shooting question. All of these people who blindly chase certifications never understand why or how something is done.
I see HS testing isn't any better
What can be done about this? Have you thought about sending this analysis to every member of the state legislature? Ask for a sit down w/ the Governor, perhaps? Maybe contact the News Journal?
Also, our BYU boy Tanner bailed and went to Harcourt Brace.
The sad part is DOE has tried their best to skew the dropout rate and the lower on the cut scores particularly at the 8th grade level in 2006.
The legislators compounded the problem by increasing the high school credit requirements. Raising the bar on high school credit requirements at a time many students couldn’t even met the basic DSTP standard really put undue emotion pressure on struggle students. This also negatively impacts the graduation rate.
One reason we see slight improvement in the 10 grade DSTP is because the high school dropout rate is the greatest at the 9th grade level mainly due to DSTP retention requirements. Weed out the dead wood before the 10th grade DSTP. Honestly, if there were even such a thing a malpractice in education, “managers” not teachers DSTP would go on trial.
The legislators aren’t grasping the real breakdown in education which is related to student discipline. Teachers can’t teach and students willing to learn can learn due to classroom disruptions.
Public schools have become mental health day treatment centers for those children social services and family court can’t manage or rather effectively serve. The odds are behind every dysfunctional students is a dysfunctional parent.
Charter schools and magnet schools are the lifeboats for those smart enough to jump ship however; the system is required to educate all students. There will be no effective reform of public schools until legislators are willing to address the real “social” problems associated with public schools.
The system can change the standards and curriculum all they want but achievement levels particularly of those students below the achievement gap will not respond until real intervention reaches the parents and community.
Educators walk on egg shell not to offend dysfunctional parents to avoid lawsuits and not to cause accusations of discrimination. I say a little yell therapy is in order for many parents.
This gets my vote for best post of the week...possibly of the year.
bbl
DE should scrap this boondoggle ASAP. Markell said he would; I hope he keeps his word. The NAEP (Nat'l Assessment of Ed. Progess, I believe it means) is an online assessment that our students take each year anyway. Not only would the state save a bundle in printing costs each year, we'd be "killing two birds w/one stone," so to speak by using NAEP. No Child Left Behind does NOT stipulate WHAT test a state must use for its annual assessments. Some states use simple basic skills tests. As Steve noted, the DSTP is in part the legacy of committed educationists (like Pat Forgione, who quickly bolted for the Clinton admin. once he got his "way" here in DE) that, if they didn't really matter (as they do w/NCLB) wouldn't be that big a deal. But now that they do (and have for years), it's a travesty.
Let's use a simple online assessment that is custom made made for each student (like NAEP). These tests modify themselves based on the student's progress throughout the test! You get a lot of correct answers, the test gets tougher. Demonstrate weak skills in an areas? The test continues to put up questions in that area in different ways. Etc.
I often wondered why the NAEP wasn’t the means to measure standards nationally. Our new U.S. Secretary of Education Arne Duncan (jury is still out on him) is pushing for national uniformity in standards. NCLB permitted each state to adopt their own assessment test perhaps because the feds didn’t want to intrude on local control. What suck for students, when their families relocate to another state and they are put at a disadvantage?
I think there is more reason to celebrate when being praised by positive NAEP results than positive DSTP results. The intent of NCLB was well meaning however 50states will 50 varying standardized test has caused great harm to public education in America. Let’s just hope Arne Duncan doesn’t allow Wendy Kopp of Teach for American at the big table when reforming NCLB.
Thank you for speaking the truth.
PS I spent countless hours formating units/lesson plans to standards ... then to Performance Indicators ... then back to Standards... because I was told to.