"Since a number of posters had questions about the school report card, I asked a friend of mine who is Director of Research at the New York Performance Standards Consortium if she would put together some info that parents could easily understand about how to interpret the report cards. I have posted her response below."
Demystifying the School Report Cards
Basics on the Construction of the Elementary School Report Card:
- 85% of the report card grade is derived from the state ELA and math test scores: 55% from changes in the test scores from year to year (progress) and 30% from the actual scores (performance).
- 15% of the report card grade is derived from attendance (5%) and parent and teacher surveys (10%).
- Each category progress, performance, attendance, parent surveys, and teacher surveys is weighted so that two-thirds of each sub-score is derived from a comparison with 40 peer schools and one-third from a comparison with all city schools.
- A school's peer group is a collection of 40 schools that have a similar population. The way in which similar schools are determined is 40% based on the percentage of African-American and Latino students in a school; 40% based on the percentage of students who qualify for free or reduced lunch; 10% based on percentage of students who receive special education services; and 10% based on the percentage of students who are English Language Learners (ELLs).
There are MANY problems with the construction of the school report cards. As a result, some excellent schools got lousy grades. DO NOT TAKE YOUR SCHOOL'S GRADE AT FACE VALUE! The school report card over-emphasized state test scores and changes in test scores at the expense of meaningful measures. Many high-performing schools were at a disadvantage because their students were already getting top scores and had little wiggle room to show improvement.
Consider These Problems:
- THE SCHOOL REPORT CARD OVERLY RELIES ON TEST SCORES (85%). Even test companies agree that test scores should always be used with multiple measures of performance to get a more accurate, in-depth assessment.
- THE SCHOOL REPORT CARD IGNORES THE MANY FACTORS THAT MAKE A GREAT SCHOOL: whether a school has music, art,dance and language classes or a library; whether it encourages childrento learn to work cooperatively; whether children are making progressbased on measures other than test scores.
- The DOE DEFINES PROGRESS VERY NARROWLY. A student who got one wrong on their ELA test in 4th grade and two wrong on their ELA test in 5th grade is not considered to have made progress, even though the number wrong falls within the standard error of measurement and the student is obviously succeeding at the 5th grade level. Remember - progress counts for 55% of the grade.
- THE GRADES ARE BASED ON AN ARTIFICIAL CURVE. The DOE predetermined that 15% of schools would score an A; 40% a B; 30% a C; 10% a D; and 5% an F. (Aside from the percentage of schools with A¹s, which the DOE changed to 22%, the curve remains faithful to the DOE¹s predetermination.) There is a very complex formula for determining the grade, and there are minute differences statistically between some schools with different grades.
PSP Member Responses:
"Wow, this analysis is so helpful! It articulates my intuitive but less informed response to this whole concept. It also confirms my fears about how these grades will be used. I will copy this and bring it with me to discuss with parents at our school (PS 8 - which got a C, which is utterly laughable).
Also - I am currently learning a lot about performance measures in general, for work I'm doing in the health care field. One of most important things I've garnered is that, for performance measures to be truly meaningful, they should be developed in conjunction with all "stakeholders" - meaning, in this case, teachers, principals and parents. I see there was some data from parent and teacher surveys, but I have no idea how those surveys were designed or who designed them - and in any case don't think that goes far enough.
The report cards were designed after consultation with superintendents, principals, parents, and even students. I think people are raising some valid concerns about them (although some of the complaints are a bit overblown), but I would suggest to all parents the following:
These reports give you *really important* information that you have never had access to before. If you wish to ignore the overall grade, go ahead. But do not ignore the data that you see that talks about student progress. For *way too long*, schools have been evaluated based only on how many kids are at or above level on standardized tests.
The problem with that is that most of student performance is (statistically speaking) derived from parental demographics. What these progress reports offer is useful information on how much the students are progressing once they are at the school, which, in
effect, "controls" for parental demographics.
Therefore, better than any other statistical measure -- it allows you to see what the school is contributing to student learning. No one believes that it provides all of the answers about a school, but the new information is certainly something that you want to know about your school. And if the students aren't progressing on the tests, don't you at least want your principal to explain why, and whether it matters to him or her?"
"All of this information was provided for many years in the previous version of the school report card. It included standardized test results for every grade tested (3rd and up), with breakdowns for gender, ethnic group, English language learners, special education
students, and many other categories, compared to the previous year and the year before that, and compared with "similar schools." Plus there was detailed information on the faculty (how many years of experience? How many years in this school? certified in the area in which they were teaching?), police incidents, lots of data.
When we get report cards for our kids, do we want a single grade, a giant A, B, C, D or F to categorize them? Do we want this letter grade for our kids' schools, determined by an extremely arcane formula which compares it to a mysterious group of "similar schools"?
"It's true that much of the information in the reports (absolute scores on tests, broken down by sub-groups, attendance) has been available before. But the parent and teacher engagement results are new.
The other big piece of new information is the student progress measures. This is the first time that the DOE has on a systemic basis tracked the progress of individual kids over time, and then aggregated that at the school level. I think savvy parents will want to consider this information as they evaluate their kids' schools.
My big thing about these cards is that I don't want the fact that they are not a "perfect" measure of schools (I don't know if there is such a thing) to overshadow the fact that they provide a new and insightful tool to understand school and teacher performance.
Absolutely, people should make their own judgments about their kids' schools based on their own observations and their kids' experience.
But these new measures are way better than any statistical measure that has yet been developed. So my hope is that parents use this new information to better understand what is going on in their kids' schools."
"I think part of the problem with this discussion - for me at least - is that there seems to be a lack of consensus about what a good education actually looks like today. It's a subject I approach with trepidation. I'm not even sure how to frame this discussion or where
we stand now. But i have had a scare or two. I'm haunted by my own experiences with graduates I've worked with, intelligent people who sometimes lack basic skills like spelling or composition or who, more poignantly, are very passive in their approach to work and life, wanting to be instructed, motivated, organized beyond what seems reasonable. These have included students from Ivy League schools. I find myself wondering "how could this happen?"
I *wish* these report cards would spark a discussion about what we really want for our children. Personally, I'd like my children to acquire a love of knowledge for its own sake along with the tools to equip them to fully engage with life, to have big dreams and to be
able to pursue them where they lead, whether that's to math or science or art, or music or literature. I don't mind whether they write books, run a business or wash windows. I hope they will appreciate the people who came before them and make a tomorrow beyond
our best imaginings. To me that doesn't sound controversial but I know that there are people who measure success in a slightly different way.
How does any of this translate to the kindergarten when I drop my son off every day? I'm not sure but I can't help but think we'd be better placed to help teachers create a culture of learning if we took it upon ourselves to figure out where we stand on some of these points.
Ideas are welcome!"
"Exactly!
And I think this is why so many of us are genuinely disturbed by the DOE's overreliance on standardized testing. These various "accountability measures" seem to take into account only a very narrow set of values when so much theory seems to point to *multiple* intelligences and *varied* ways of making one's way in the world.
To me, "good education" might look different than it would to you, and it may--in fact, should--look different for my first grader than for your third grader. At a parent meeting earlier this year, my child's teacher said that she hoped school would be a "place of joy
and social connection" and that is EXACTLY what I want for my child now, though I may be more concerned about other things down the road. In part, I may think the way I do because I WAS a very accomplished test taker as a child, but I was also awfully competitive and not particularly happy. Like the young people N. cites, my perfect scores may have gotten me into elite schools, but their
presence masks other crucial (nonacademic) gaps in my schooling.
As far as school report cards per se, I too see the letter grade as totally reductive--even on a symbolic level (for those who would say just ignore them and read the details in the reports); they are the opposite of the expansiveness educators should encourage. I would agree that having *some* comparative measure is probably useful--and that all schools need to be constantly striving to better themselves--but the report card is completely based on a FALLACY: that the tests can capture student [academic] growth from year to year.
Consider:
+If a school chooses a slightly different path in its math curriculum from what is tested in a given year, the test will not accurately record student growth.
+In 6th grade students do not correct a grammar passage, in 7th grade they do.
+The quality of the actual tests are of huge import. How reliable a measure are
they?
Then there's the student; these scores are derived from his performance on ONE day (or possibly two). What if his parents are divorcing? Her pet is sick? She just got her first period or stayed up late watching TV? The resident bully told him he would fail?
The danger is that by doling out grades that are 85% (!!!) based on test scores, we will only become further entrapped in a test-focused system. The pressure on schools and teachers and students increases. What is the fallout when children begin to think of themselves as "2"s or "3"s or "4"s or their schools and teachers as "B"s or "C"s, or God forbid, "F"s?
Tons of dollars have been spent on this initiative. Imagine if that money had been funneled into more expertise in the classroom or smaller class size. Overall, the city does not know how to address the needs of students who are not on grade level. Why not focus
all efforts on that?
"I believe that savvy parents will do what they've always done - see for themselves, visit the school and read insideschools.org. My dd's wonderful, incredible school got a "C", which we thought would be an "A" after thoroughly reading through its written availuation.
A few things to consider:
1. The metrics were changed 3 times throughout the evaluation process making it hard for principals to keep up.
2. My dd's school is an inclusion school and no points were give to it for that - over 15% of the student body are disabled or have other health impairments that lead to frequent absences. You won't see this on the report.
I read yesterday that the "F" schools will not receive more funding, but less, and the "A" schools will receive more funding. How is this fair? It would appear that this works to weaken the system which is already leaving mostly schools in low-income areas out to dry.
And finally - 80 MILLION dollars went into this - our tax dollars at work. I could see 80 MILLION dollars working in many other ways.
"Thanks for this response. You've pointed out some things that didn't hit me with the same impact before, and which I agree we should look at seriously. My chief problem is that a letter grade is not nearly as nuanced as each of the points you make below. Because we are presented with one grade, and have to dig for the details, it seems impenetrably reductive. But that doesn't mean that what's behind the grade doesn't involve some useful, if flawed, data."
"I have a question about the progress measure - I understand (and agree) that it is good to take progress from baseline into account rather than giving all the credit to schools with students in high achieving neighborhoods. What I'm curious about is how they assess progress. If a student is on grade level and remains there, that is completely appropriate. Is the school expected to bring every student up every year? This is of course not possible and would penalize schools serving students who are already on target. Now, Stuyvesant HS got an A, so apparently they did have some provision for starting high. Does anyone know?"
"From what I understand the "progress" is based on the underlying scale scores for the grades 3-8. In other words, they do expect kids to make a year's worth of progress every year. Everyone knows that all kids don't actually make that goal every year, but the idea is that all kids should be encouraged to make a year's worth of progress.
I don't know how they are doing it for high schools, but I presume they are using some similar system.
As for Stuyvesant et al, they are compared to their horizon schools, just like everyone else -- and for Stuy, their horizon is presumably the other 9 specialized schools."
" There is a great article about the report cards in the Metro section of the NY Times today. It illustrates just how off the mark they can be. If the link below doesn't work, you can just go to www.nytimes.com
LINK: http://www.nytimes.com/2007/11/09/education/09school.html?_r=1&ref=nyregion&oref\=slogin."
