nominally high performing, part 3

I still haven't gotten around to posting excerpts from Richard Elmore's article on nominally high performing schools. I will, soon. 

A nominally high performing school is a school that looks good on paper thanks to high-SES students. But when you take value added measurements, you find that while the students are high performing, the school is not. Instead, it falls into a category William Sanders, inventor of value added measurement for schools, calls slide and glide:  

I've caught the most political heat from some of the schools in affluent areas, where we've exposed what I call "slide and glide." One of the top-dollar districts in the state had always bragged about its test scores, but our measurements showed that their average second-grader was in the 72nd percentile. By the time those children were sixth-graders, they were in the 44th percentile. Under our value-added scheme, the district was profiled in the bottom 10 percent of districts in state. 

Helping Teachers Raise Achievement: an interview with William Sanders 

Here is Ted Hershberg on nominally high performing schools:

The problems with AYP ["adequate yearly progress," the NCLB measurement of school quality] are clearly evident in ... schools whose students are meeting their AYP goals, but where little growth is occurring. Most often found in affluent communities where high-test scores go hand-in-hand with family income, these schools can be referred to as "slide and glide" because they appear to be resting on the laurels of their students. It is important to understand that NCLB does nothing to hold these schools accountable for providing their students with the annual growth to which they are entitled. In a global economy characterized by fierce competition for demanding jobs that pay high salaries and benefits, this is a highly significant shortcoming. 

"Value-added Assessment and Systemic Reform: A Response to America's Human Capital Development Challenge"

Ted Hershberg

Professor, Public Policy and History

Executive Director, Operations Public Education

University of Pennsylvania

In the past week the Times has carried two stories on P.S. 8, a "hot" school in Brooklyn that has just received a grade of "F" on its NYC report card. Interestingly, while the first story, starting with the title (In Brooklyn, Low Grades for a School of Successes), falls squarely within the tests don't measure quality genre, the second is the single hardest-hitting news story about an affluent and popular school I have ever seen, certainly in the New York Times.

From the first article: 

A respected Brooklyn Heights elementary school so popular in its gentrifying neighborhood that it has doubled enrollment since 2002 is set to get an F in the second year of the Bloomberg administration's heavily contested system of grading individual schools, renewing questions about the methodology behind the grades.

Mind you, that is the lede. The point of this story is not that P.S. 8's grade of "F" renews or perhaps raises questions about the quality of the school. 

No. 

P.S. 8's "F" can only renew and/or raise questions about the quality of the grading. Which was already questionable. 

The school, Public School 8, was once avoided by the well-off residents of neighboring brownstones but has been the paragon of a turnaround tale in recent years, leading Mayor Michael R. Bloomberg to declare in 2006 that if the rest of New York’s schools made similar strides, “the future of this city would be assured.”

This year, at a July 29 news conference announcing plans for an annex to accommodate the flood of students wanting to attend P.S. 8, a parade of public officials praised the school and its principal.

Schools Chancellor Joel I. Klein said, “You have built a very successful school here,” and Deputy Mayor Dennis M. Walcott added, “You have done a monumental job in both recruiting as well as maintaining a school where parents want to send their children.”

then comes the piling on:

When the first round of report cards was unveiled last fall, there were some counterintuitive results and many complaints, but now, P.S. 8 could be the most highly regarded and popular school to receive an F.

Last school year, 67 percent of its students passed standardized tests in English and 83 percent in math, and many who know the school said such a grade would be misleading and preposterous.

“It’s a real indictment of the grading system if it takes a school that is improving rapidly and is already doing pretty well and brands it with an F,” said City Councilman David Yassky, whose district includes P.S. 8.

from there the story moves into a terrific section on the tests and how they work: 

In New York City’s formula, the largest factor in determining a school’s grade, 60 percent (up from 55 percent last year), is based on how, say, this year’s fourth graders did compared to their third-grade scores, an analysis known as a “growth model.” Student performance — median scores and how many passed the test — counts for 25 percent, but is calculated not in absolute terms but by judging each school against schools citywide as well as against 40 other schools with similar demographics. In addition, 5 percent is based on attendance, and 10 percent on the surveys.

the principal speaks:

Last year, P.S. 8 received a C — which Mr. Phillips, the principal, described as disappointing in a letter to parents that took issue with the grading system and schools that emphasize test-taking over enrichment activities.

“It is my professional and personal opinion that this grade does not reflect our school and the work we do,” wrote Mr. Phillips, 45. “This grade will give us food for thought, but it will not change the way we move forward into the future or alter our core values.”

note: the principal is a graduate of "a prestigious fellowship program ... for outstanding principals at Columbia University’s Teachers College." 

the co-president of the PTSA weighs in:

Several P.S. 8 parents suggested that the F said more about the grading system than the school. They cited events like the annual read-a-thon fund-raiser, an art program that culminated in student work’s being showcased at the Guggenheim, and the school’s recent selection as Brooklyn’s Rising Star Public Elementary School for 2008 by Manhattan Media, a publisher of weekly newspapers.

“To me this kind of grading system seems inadequate to express the education that’s occurring within an entire school, and in fact becomes quite detrimental to the education process,” said Joanne Singleton, co-president of the Parent Teacher Association.

“When you walk into P.S. 8, the children are smiling, they are happy to be there,” she added. “They are comfortable in the school environment, they are comfortable interacting with their faculty, and it is truly a community of learning.”

If you were wondering why parents don't have a union, which you probably were not, this is it. 

So that was Thursday. 

Today's article appears to have been ghostwritten by Andrew Wolf:

How could a red-hot school in Brooklyn Heights — with surging enrollment from middle-class and wealthy families, with test scores that are above average, and with extras paid for by parents’ fund-raisers — be declared a failure?

For some people, news that Public School 8 on Hicks Street will be getting an F on its upcoming report card cinches the case that education officials have lost their test-taking marbles.

And yet there is a strong argument that the F grade is just the sort of blunt truth-telling needed for schools that are highly regarded in the vaporous, unchallenged esteem of conventional wisdom.

More than 80 percent of the kids at P.S. 8 passed a standardized math test. Two-thirds passed the language arts test. In 2006, the mayor said the school should be imitated. In July, the schools chancellor announced that an annex would be built to accommodate the demand in what he said was a “very successful school.”

In reality, children who start the year at P.S. 8 with decent or good scores in math and English actually have gone backward, said James S. Liebman, the chief accountability officer for the city’s Department of Education.

“You drop them off at the beginning of the year, and on average, by the end of the year, your child lost ground in proficiency,” Mr. Liebman said.

Children on the lower end of the scale — the ones who had the most room for improvement — made only the slightest gains compared with those at similar schools, Mr. Liebman said, while at most schools across the city, there were big improvements.

“Where was the child last year, and where is the child this year?” he asked. “You’re comparing them to themselves.”

For many people, the F grade for P.S. 8 ratifies their skepticism about standardized tests. If all the children, like those in Lake Wobegon, are above average, how could the school be failing?

decline at the top

“If you use high-stakes tests and nothing else, you’re measuring ZIP code, race, socioeconomic status,” Mr. Liebman said. “Most importantly, you need to measure how much kids improve after a year at their school.”

[snip]

On average, Mr. Liebman said, the higher-performing students at P.S. 8 lost a little more than one-tenth of 1 percent in proficiency in English and math; the lower performing students gained about a tenth of a percent. Why should anyone care about such numbers?

They make a major difference by the time the students are 18, Mr. Liebman said. Students get scores between 1 and 4. Of those who finish eighth grade with a 3.0 proficiency in math and English, just 55 percent graduate from high school four years later. For those with 3.5 scores, the graduation rate is 75 percent.

“I know it’s troubling to people in the neighborhood, and it should be troubling,” he said of the F grade. “The point is, compared to any other school in the city, this school is off the charts on the low end.

“We’re trying to move away from a school that gets by on its reputation.”

At P.S. 8, Image Didn't Match Performance
Jim Dwyer

This is amazing. I have no idea -- no guess, even -- how such a turnaround could have occurred. 

For the time being I'm going to assume that I need to be reading anything and everything Jim Dwyer writes. 

bonus points:

The kids at the top are showing the least progress. Perhaps enrichment isn't the best education to offer kids who are already being enriched at home? Which is not to say that I would wish the JASON Project on disadvantaged kids, either. 

Interesting that the principal was ordered not to speak until the scores were "officially released."

The 2nd story is not accompanied by a photograph of students sitting on the floor with nary a book, pencil, or desk in view.

I believe this is the first time I've seen a newspaper or magazine allow a testing expert to make the point that education is cumulative. Over time a small loss here, a missed opportunity there, the kind of thing routinely shrugged off by administrators in wealthy schools, add up. It's the miracle of compound interest in reverse. 

The author of the first story appears not to grasp the concept of value added: 

Also contributing to the F grade was P.S. 8’s rapid change in population. A quarter of the students now qualify for free lunch, compared with 98 percent in 2002, and more than half the students are white or Asian-American, up from 11 percent in 2002. Most of these changes are happening among the youngest children, before tests begin in the third grade.

Eighty-nine percent of last year’s prekindergarteners at P.S. 8 were white, for example, as were 60 percent of kindergarteners, 49 percent of first graders and 54 percent of second graders. The test-taking grades — 3, 4 and 5 — were 27 percent, 31 percent and 19 percent white, respectively. Throughout the country, there is an achievement gap between black and Hispanic students and their white and Asian counterparts.

As the demographics changed, test scores shot up, but most of the change came between 2002, when 32 percent of the students met state standards in English and 26 percent in math, and 2005. For the last three years, scores have improved more slowly, with about two-thirds passing in English, and 71 percent, 81 percent and 83 percent in math.

She seems here to be assuming that the rapid increase in test scores caused by an influx of high-SES students is the basis on which the school was graded in the past, or would have been. 

It is not.

What is value-added assessment?

value added assessment at Education Consumers Foundation