Educators in the Coalition of Essential Schools share a commitment to
the idea that they need to assess students’ progress to help students keep learning and to help teachers keep getting smarter about how to teach. There is also a large degree of consensus among Coalition educators that standardized tests cannot be the most important element in an assessment system-because assessments should be closely tied to the actual work of individual students and classrooms and because even excellent performance on standardized tests cannot adequately capture the development of the habits of mind that are the goal of a rich and rigorous education.
Within this widespread agreement, there is considerable diversity of opinion and action regarding standardized tests. Some believe that the tests are a valid component of a more comprehensive scheme; others maintain that they cause much more harm than good. Some think that the tests should be boycotted; others say that Coalition schools should accept the tests and show the world that our students-engaged in active, authentic learning experiences-do just fine on these kinds of assessments.
The diversity of the tests makes this range of opinions much more complex. Some tests are norm-based; some are criterion-referenced; some tests include performance tasks, such as writing an essay; others are strictly multiple choice. Some tests are “high stakes”-that is, they determine students’ eligibility for promotion or graduation, teachers’ pay level or a school’s funding level-and some tests are used simply to give feedback to the school and parents. Depending on the nature and purpose of the test, CES educators may feel more or less inclined to support its use.
The rapidly increasing employment of standardized tests to determine grade promotion, graduation, and status on all levels-classroom, school, district, state and national-competes with the personalized forms of assessment that are at the core of Coalition schools and increases the pressure on Coalition educators to define where they stand. Not only do most states now mandate some form of high-stakes public school testing, but the newly reauthorized Elementary and Secondary Education Act compels standardized assessments with federally imposed consequences for sub-par performance.
This issue of Horace takes a look inside Coalition schools in Massachusetts, California, Maine and New York to examine what some Coalition educators, parents, and students are doing in some of the very different testing climates across the country.
Maine: State Supervised Local Control
Coalition schools in Maine have had the chance to take advantage of a positive political climate to create performance-based assessments that are the central demonstrations of learning at all levels: student, community, district and state. Since 1996, state law has required Maine school districts to develop “a combination of state and local assessments to measure progress and ensure accountability” toward student achievement of the Maine Learning Results, Maine’s state learning standards. This combination is heavily weighted toward locally developed assessment measures, called Local Comprehensive Assessment Systems; schools develop ninety percent of the criteria that define student success. The legislation also requires that the Maine Educational Assessment, the statewide component that constitutes the remaining ten percent of the assessment equation, be aligned with the Learning Results. Schools are required to implement the entire system, including their locally developed assessments, by 2004. In 2007, high school graduation will depend on successfully completing the Local Comprehensive Assessments.
Maine Coalition schools, already working toward creating authentic measures of learning, needed to align their assessment processes with the Maine Learning Results. They received support for that demanding effort from the Southern Maine Partnership (the CES regional center in Maine) which worked with teachers to develop the Learner-Centered Accountability Project (LCA). The LCA provides schools with a four-level assessment model that allows maximum local flexibility while guiding alignment toward the Maine Learning Results; along the way, the LCA also encourages schools to examine their locally developed assessments for validity (do the assessments measure what they’re supposed to?) and reliability (do people agree on the quality of the work?).
Allyn Hutton, principal of Sacopee Valley High School in Hiram, Maine,
has embraced the opportunity to develop a Local Comprehensive Assessment System. Hutton observes, “The difference in Maine is that we have been given the opportunity to develop locally. I have a real local control staff that says, ‘We’re going to do it ourselves.’ Not all schools are as proactive as we are, and they assume that the state will hand them something after three years. We don’t want to be in that position. There’s enough of our staff that says that the state test isn’t good. Our biggest impetus is not wanting the state to come in and say, ‘This is what you have to do.’ Along with that we’ve had to fight for and have been given professional development time and support time. There have been bumps in the road-sometimes we feel like we’re in the middle of a jungle and don’t know how to get out, but the result is that
we have created some wonderful assessments.”
For example, Sacopee Valley’s Allison Werlock, ninth grade English teacher, created “Portrait of a Reader,” a Level II assessment that asks students to reflect on their experiences as readers. (For details about Level II and other assessments, see “Southern Maine Partnership’s Local Comprehensive Assessment System” on page 4.) Students respond to various questions about how their reading skills have been assessed through the academic year, their reading strengths and weaknesses, their favorite book, their attitudes toward reading, and their reading genre preferences. This assessment, which is added to students’ portfolios, also asks students to look at the benchmarks for ninth grade English reading skills and for each “identify an activity we have done this year and explain how it demonstrates that you have mastered that skill.” Students benefit from the chance to reflect on their reading achievement and to understand how the work they’ve done corresponds to the standards for success that they’re expected to meet.
Hutton believes that consistent professional development and conversations about assessment are the keys to mapping curriculum and evaluation to the Maine Learning Results. “I found university courses for the faculty to work on assessment. If I am going to ask teachers to do something, I have to provide support.” Donna Dionne, chemistry teacher
at Topsham, Maine’s Mt. Ararat High School, simultaneously praises her school’s commitment to the local comprehensive assessment endeavor and cautions against the resulting challenge. In a presentation at an Association for Supervision and Curriculum Development conference, Dionne and colleagues David Hammond and Gerrie Netko offered other teachers working to create comprehensive, standards-aligned, performance-based assessments “advice for the overwhelmed: ask for help from those who have been there; keep your sense of humor; if you don’t have administrative support, money and willing teachers, don’t even try.”
David Ruff, Director of School Reform at the Southern Maine Partnership, says, “Accountability means measuring not only the kids but also ourselves-it also helps us figure out where to make change.” Ruff and others working on developing local assessment systems that will soon carry high stakes for students and schools believe that the entire educational environment needs to be accountable for success and that locally designed performance-based assessments are the best way to get there.
New York: Decades of Performance-Based Work Now Must Make a Case in Court
Similar to their Maine counterparts, schools that belong to the New York Performance Standards Consortium (the CES regional center in New York) are collaborating to engineer a locally controlled assessment system, though within extremely challenging policy conditions. The Coalition schools belonging to the Consortium are working to preserve their cultures and longstanding practices of performance-based assessment while battling New York state’s Regents examinations. New York has been employing the Regents exams for decades as a gauge to student achievement and, for those students who did well, a mark of distinction. During the past several years, the State Department of Education has transformed them into a high stakes test, requiring that students pass the Regents tests to graduate from high school. Cecilia Cunningham, principal of Middle College High School in Long Island City, considers the Regents exams impossible to disregard: “The Regents aren’t a school account- ability system. They’re a student accountability system. No test, no graduation. So we have to deal with these.”
For years, prior to the elevation of the Regents as an exit test, many CES schools in New York obtained waivers that in large part exempted their students from taking the Regents, allowing teachers and students to focus on more personalized forms of assessment. However, during recent years, with the advent of political and personnel changes, the schools lost their waivers. In response, they turned to litigation, currently ongoing, to pursue their ability to preserve school control over decisions about who is ready for graduation and what constitutes adequate demonstration of learning and achievement.
In January 2000, Consortium educators made their case to a state education department panel on assessment. The Consortium educators tied their performance-based assessments (PBAs), to the New York State Standards for Learning-demonstrating alignment among the curriculum, the state learning standards and the school-based assessments that demonstrated students’ mastery. Consortium schools made the case that, according to Ann Cook, Co-Director of Urban Academy in Manhattan, “Performance-based assessment tasks needed ongoing systems so these weren’t stand-alone at the end of the year, but arose from the work of the school.” Marion Mogulescu, Co-Director of Vanguard High School in Manhattan, explains, “We had to submit descriptions of our curriculum and show how it mapped onto the state standards. And we had to show how we evaluate students to assess their learning. Our curriculum shows how we meet and exceed those standards.” The panel denied much of the Consortium schools’ request to substitute the PBAs for the Regents, recommending that the mathematics and English Regents be required of their high school students. Commissioner
of Education Richard C. Mills recommended that a panel further evaluate the Consortium’s assessment system, and in April 2001, Mills denied the request for exemptions from the Regents exams.
To an extent, this denial pushed the schools further down the path toward developing an alternate system. Cece Cunningham explains, “We met with the Commissioner to show him the work, and he said, ‘I can see that you have a great deal of validity-what you’re doing matches New York state standards. Now aim for reliability-that has to be one system that you’re developing.’ We decided to work at this. We decided on a common set of performance assessments across all of our schools and we agreed to use the same set of rubrics that we developed as a collective over the course of a year. We developed inter-rater reliability over the course of a summer, looking at assessments from our schools in language arts. We then requested that we be allowed to continue to use our assessment system, codified across our schools. We were able to say, ‘This is what assessments look like; this is how they align to standards.’ “As a result of this work, the Consortium schools now share scoring rubrics used for evaluating PBAs in all subject areas-this ensures reliability across classrooms and schools. An example of one course’s learning goals, demonstration of mastery and corresponding physical science state standards appears in “Space Science at Middle College High School” on page 8. Every class in every New York Performance Standards Consortium school makes public this correspondence of curriculum, assessments and standards in order to demonstrate their preparedness to meet and exceed the state’s expectations.
While legal challenges-the Consortium filed suit against Commissioner Mills and the New York State Department of Education in August 2001-make their way through the courts, Consortium schools are required to give the Regents tests. This creates a dilemma for teachers, who now must “teach to the test” in a way that they believe undermines their ability to coach students for the more multidimensional learning tasks they have developed. Cece Cunningham sees the Regents threat as life or death for Coalition schools: “Unless we in small schools come to grips with this, the work that Coalition has done will become undone. We’re at a moment here when we have to face this-the testing in New York state has the power to destroy fifteen to twenty years of work.”
Eric Nadelstern, recently principal of International High School in Long Island City and now Deputy Superintendent for New and Small Bronx High Schools, feels that it’s worth holding on to performance-based assessment systems as criteria for academic success and that there’s a brighter future beyond the high-stakes Regents testing. In December 2000, The New Century High Schools Consortium was established with grants totaling thirty million dollars from the Bill & Melinda Gates Foundation, the Carnegie Corporation of New York and the Open Society Institute. Nadelstern says, “We received this grant to create new, small, autonomous schools, a premise that is very much in sync with school systems that permit choice and a variety of instructional approaches and assessment. These foundations understand the value of that. I think they’ve already seen beyond what happens when you implement a test and curriculum with uniform standards and realize that’s not the solution. What they have embraced allows democratic communities of scholars to form around needs of kids to create assessment systems that support anomalous schools with strong ties to the community.” In the meantime, in CES schools in New York, students straddle two worlds-they cram for the Regents while they prepare multidisciplinary, individualized portfolios of best work.
California: Trying to Find the Best Test
Moyra Contreras, principal of Melrose Elementary School in Oakland, California, describes her school’s journey toward authentic assessment in a Spanish-English bilingual setting. Staff at Melrose battle on two fronts. They strive to hold onto their performance-based assessment practices in the face of increasing demands on the district and state levels for standardized testing-testing that has major implications, at the elementary school level, for school funding and autonomy. And they struggle to maintain their bilingual education program, involving 75% of the school’s students, in a hostile environment. In 1998, California voters approved Proposition 227, which removed state support for bilingual education. But the Melrose faculty is committed to maintaining the program, committed to their convictions that bilingual classes help students both transition more effectively into English language learning and preserve crucial ties to their cultures and families.
The issues of testing and bilingual education are linked by the persistent underperformance of English language learners on English-language
standardized testing. Stanford University’s Kenji Hakuta found that
“the SAT-9 is a poor excuse for a measure of English development and academic achievement for LEP [limited English proficiency] students.
The test was developed to give normative data in reading and math for native English speakers. The test measures things that are qualitatively different from what would be expected of students learning English.”
Disappointed by her students’ performance on the SAT-9-knowing that it did not demonstrate the literacy and critical thinking skills she and her staff knew existed-Contreras sought out additional assessments that would better demonstrate her students’ abilities while finding ways to minimize the impact of the SAT-9. Contreras and her staff believe that their students’ capabilities cannot be adequately demonstrated by the SAT-9, not only because of its inadequacies as an assessment of English language learners and its inherent limitations as a one-shot, one-dimensional standardized test, but also due to its bias against socioeconomically disadvantaged populations-a characterization that applies to most of Melrose’s student body. As UCLA’s W. James Popham explains, “One of the chief reasons that children’s socioeconomic status is so highly correlated with standardized test scores is that many items on standardized achievement tests really focus on assessing knowledge and/or skills learned outside of school-knowledge and/or skills more likely to be learned in some socioeconomic settings than in others.”
Contreras explains how this plays out at Melrose. “We looked at the SAT-9 and tried to figure out why middle class kids do well and poor kids don’t. Five percent of our students are middle class and when we disaggregated their scores we noticed a real difference: they do better. This is due to factors not happening in the school; it’s attributable to the background that they have. Our kindergarten teachers are responding to this by working with parents to instill home literacy practices. Issues of poverty make this a hard community to work in. There’s not a lot of support for kids. Demoralizing external demands aren’t helpful; they’re destabilizing. And we get negative feedback from the state, and our students and community sense that.”
In order to have some external measure of students’ abilities, Melrose requested that all of its bilingual education students take the SABE/2, a basic skills test in Spanish required by California’s Standardized Testing and Reporting Program for English language learners enrolled in district schools for twelve months or fewer. “We also wanted to show that we should be held accountable in our bilingual program. It’s frustrating to be held accountable to the SAT-9, which we know doesn’t show what our kids know-but we do want to be held accountable.” Though she had to fight the Oakland Unified School District for permission to test all English language learners, not just recently enrolled students, Contreras says, “The results were positive, which is a demonstration of our commitment to bilingual education. The SABE combats our SAT-9 results.” The group of Melrose students that took both tests demonstrated better math proficiency on the SABE/2 test than on the SAT-9, a validating result for Contreras and the Melrose staff.
Contreras and the Melrose staff also use political energy to challenge the SAT-9, though not without potential cost. “The staff has done a lot of work with parents over the last five years about language acquisition, bilingual learning and maintenance of culture. As a result, more than 40% of our parents requested that their children not take the SAT-9s. Still, 60% did take it, which is a huge impact on the school, with two weeks of test administration. We need to keep track of waivers and place those students not taking the test elsewhere for two hours a day.” In addition to the impact at the time of testing, Melrose faces skewed SAT-9 results as a result of such a large percentage of its student body opting out of testing. The California Department of Education has questioned the validity of other districts’ SAT-9 scores in similar situations, producing additional (often unwelcome) state scrutiny.
Contreras says, “Part of the school culture is having local school control and choice about how to assess students,” and she knows well that the school benefits from allies. “We have external support from BayCES (the Bay Area Coalition of Equitable Schools, the CES regional center in the Bay Area) and people in the district office, and we have a tradition as a site-based decision making school. It’s not in the district’s interest to pick a fight with us.” As Tom Malarkey, Coordinator of the Teacher Inquiry Project at BayCES explains, “Melrose embodies what the district would like to see,” pointing to its success in retaining staff, involving parents and community, demonstrating respect for students’ civil rights-all of this in addition to raising test scores over the years. But Contreras adds, “They want the end result to be what we have but they want to mandate our curriculum and assessment-they want to mandate how we get there.”
Massachusetts: Testing Success-is it Worth a Celebration?
Among other tactics, Melrose Elementary School chose for its students to take a more appropriate standardized test to combat the effects of the SAT-9. But does focusing on a standardized test take away from teaching and learning deeply and meaningfully? This is the argument put forth by Linda McNeil in Contradictions of School Reform, where she analyzes the detrimental effects of preparation for the TAAS in Texas. Mary Toomey, principal of South Lawrence East School, a Coalition elementary school in Lawrence, Massachusetts, has a very different notion of how to use the tests in service of her school and her students. She held a public celebration, MCAS Magic, to honor individual students’ successes on the Massachusetts Comprehensive Assessment System (MCAS), the state standardized test. “We recognized student achievement, lifted spirits, gave a clear message of positive value and had kids think critically about their own learning. I had a hundred and thirty kids who scored in the highest ranges of the MCAS (proficient and advanced) and if we left it to the media, they would continue to get the message that if you are living in this district, then your schools are not doing the job, the scores are too low, learning is not happening. We are changing that self-fulfilling prophecy. A correlation exists between the use and practice of authentic assessment and improved standardized scores. Do one, do it well, the other will follow. We still need the Deborah Meiers of the world to fight the inequities of the MCAS system, but for most of us, finding some middle road where students thrive, teachers are collaborating, and achievement is improving, is a win-win situation. At MCAS Magic, we sent a clear message that the work is more important than the test.”
Toomey had good reason to celebrate her students’ success. Elementary schools in Massachusetts are evaluated by the state Department of Education based on MCAS results (among other factors) and South Lawrence East was facing a grave situation. “The real issue is that we’ve been at the bottom of the barrel and the state pointed us out at as underperforming. The parents, teachers and kids here were demoralized.” In addition to the success of its individual students, South Lawrence East’s aggregate scores rose, gaining it more breathing room from state scrutiny and, Toomey hopes, eventual autonomy.
Like Moyra Contreras in Oakland, Toomey recognizes the inequities and limitations of the state-mandated tests: “Problems exist when unfair
comparisons are made. Even within the district, no two schools are alike in demographics. When you look at the state, our district has the largest percentage of students considered second language learners of any city or town in the state. Can we be fairly compared?” But Toomey also believes that work at essential skills and habits of mind will positively influence MCAS scores while political savvy and well-timed promotional events are likely to positively influence student and community attitudes.
The significance of the MCAS shifts when students move on to high school. Students are required to pass the MCAS to graduate, and these raised stakes increase the anxiety among students and the conflict within schools dedicated to personalized learning and evaluation. Linda Nathan, principal of Boston Arts Academy, decided to create conditions for MCAS success while holding firm to authentic assessment. “We decided that we would help every kid pass this test. I am not going to have one test be the reason why people can’t pursue their futures and graduate. So we decided to focus on the minimal things we could do to make success happen.” Success did happen at Boston Arts Academy. Staff and students carved out time for focused test preparation conducted by the Princeton Review and students scored well, better than at most Boston high schools.
However, Nathan takes a dim view of celebrating her students’ MCAS success. “Kids will pass, but will they love literature? Are they reading real books? Are they doing long-term projects? That’s what worries me-we can get kids to pass, but that’s not what I want for my own children. I want them doing research. I want what the best private schools have, students engaged in literature and science. High stakes testing won’t get us there-or why aren’t the best private schools doing this?”
At the same time, Nathan feels that there’s room for the MCAS in a more limited capacity: “I am not against testing for diagnostic purposes and dipsticking-we don’t have to test all kids for that. I would love to have MCAS as part of my repertoire but not for high school graduation. It’s a decent reading test, but we knew a lot of this data before they took it. Yet we spent a month in testing.” To Nathan, spending so much time administering a test that matched the school’s own evaluation of students strengths and weaknesses-and spending yet more time retesting those who failed initially- represented a severe drain on Boston Arts Academy’s real purpose. That the test verified what faculty-who know students well-already knew is relatively cold comfort compared to its cost in terms of time and energy.
Lessons
Linda Mabry, Associate Professor of Education at Washington State University, Vancouver, writes, “A body of evidence of achievement, rather than just a score, takes some time to understand. This is both a blessing and a curse-a promotion of more valid inferences at the expense of efficiency.” Developing personalized, performance-based assessments depends on deep commitment, faculty conversation, planning time and professional deve-lopment, district and outside support-and these require time and money. If you deprive schools of these things and also demand accountability, then impersonal, incomplete standardized testing fills the void. This dynamic – with a dangerous denial of the complexities of good assessment and a vision of education that does not comprehend the importance of inquiry, curiosity, and individualization-has fueled testing’s growth.
Nevertheless, Coalition schools have found ways to maintain and strengthen their commitments to authentic assessment in a growing collection of ways. There’s clear evidence of success in New York and Maine in the development of alternate systems-rigorous, aligned to state learning goals, and based on performance assessment. Students, school staff and parents have resisted tests, as documented here in the choices of Melrose Elementary School parents and, for example, in last year’s widely reported story of Regents boycotts in Scarsdale, New York. Schools like South Lawrence East are also finding ways to create explicit, positive links between multidimensional assessment and learning and standardized test performance. And to some extent, external calls for accountability have helped educators see that their assessment systems would benefit from added rigor and have forced them to create better systems. Authentic learning and authentic assessments go hand in hand-and so working for more rigorous assessments has strengthened the schools’ instruction as well. Cece Cunningham says, “All of the Coalition principles have to be in place: student as worker, small class size-there’s no question about that. This can’t happen as an add-on or in isolation, which is the way assessment is so often done.”
Schools committed to finding alternatives to external testing must create networks and find means of outside support to sustain their efforts. The Southern Maine Partnership, for example, offers its schools crucial planning and professional development resources. Maine Coalition schools agree that without SMP’s support, and additional local and national resources, they would not be able to take on the daunting work of creating Local Comprehensive Assessment Systems. The New York Performance Standards Consortium makes it possible for its member schools to mount not only legal but practical, pedagogical defenses. Without the shared work on inter-rater reliability, advocacy for student-centered learning and assessment, and more, the schools would not be able to sustain the energy to evaluate students personally, respectfully and holistically.
In this miasma, the concept of accountability has the potential to cut in all directions-just as the states use tests in an effort to make schools accountable, schools (along with advocacy organizations, parent and student groups, and the like) have the opportunity to hold test publishers and the national, state and district agencies that use them accountable for choosing and using assessments fairly. Ann Cook says, “We have to be more rigorous ourselves and force the system to be accountable. There’s this complacency about testing -as if it’s a higher art,” and the New York Performance Standards Consortium has invested heavily in this strategy, publicly convening a Regents Assessment panel in conjunction with the Rockefeller Foundation. The panel participants actually took the English Language Arts exam and analyzed it, finding, among other flaws, that the exam did not require students to demonstrate mastery of the standards it was supposed to assess. As the panel’s report notes, “Research methods and skills are not applicable to the required tasks…the Regents tells the students that most of the learning and intellectual work mandated by the standards are simply not important.”
Nearly all Coalition schools find themselves dealing with the notion that standardized tests are the answer to school accountability, and despite the clear threat, moments of success and triumph have resulted. For some schools, it’s an opportunity to fine tune, organize, improve and strengthen their own assessment systems; it’s a wake up call. External standardized assessments, a powerful force, compel schools to find allies and use politics to awaken parents, communities, courts and legislators to the threat. Some, like South Lawrence East School, take control of the statistics, touting student success and making the point that their performance-based assessment practices created those results.
It seems likely that most Coalition schools will need to continue to find ways for students to learn to do authentic, meaningful work while making some room for external assessment. The stakes are too high to say the tests don’t matter. Dan Hoffman, Director of the Ohio Coalition of Essential Schools, articulates a realistic perspective: “The tests in Ohio are not going away soon, and although we continue to need champions who argue the perils of heavy reliance on standardized tests, we also need champions to help coach teachers and schools to prepare the children better.”
While many argue, with Hoffman, that schools have to do it all, others warn that the aims of standardized tests actually compete with CES’ vision of powerful education for a democracy, that the two visions are ultimately incompatible. There may be other ways of looking at the notion of accountability, however. There are indicators of achievement which all would agree have at least as much validity as tests scores-and which are relatively easy to quantify. Marion Mogulescu suggests that schools should rely on a constellation of measures of success-college acceptance and persistence, attendance, and parental involvement. Linda Nathan agrees: “We had remarkable college acceptance rates. Why isn’t anyone at the policy level asking what we’ve done?” In the current climate of competing paradigms, in her hardest moments, Cece Cunningham relies on her students’ transformations to remind her that it’s worth fighting for performance-base assessment at Middle College High School and beyond. “When you see what happens to a teenager when she talks about content and her own learning-that’s the best work that takes place inside of schools.”
References Cited (see Horace’s Where to Go for More, page 19, for additional resources):
Hakuta, Kenji, What Can We Learn About the Impact of Proposition 227 from SAT-9 Scores? www.stanford.edu/~hakuta/SAT9/SAT9_2000/bullets.htm
Mabry, Linda, Portfolios Plus: A Critical Guide to Alternative Assessment
(Corwin Press, Inc., 1999).
MacNeil, Linda, Contradictions of School Reform: The Educational Costs of Standardized Testing (Routledge, 2000).
Popham, W. James, Why Standardized Tests Don’t Measure Educational Quality, Educational Leadership, March, 1999 (Volume 56, Number 6).