Highly Publicized Test Scores and AYP

[Anonymous]

Soooo.... after much research, thought and deliberation, we decided to place our DS in a Public Charter School for Pre-K. He will start in September. A highly conisdered and much sought after PCS that we were lucky enough to get a space in the initial lottery.

Only this past Friday to learn that the school did not make AYP this year in Math or English. The school did make AYP last year and made progress this year but not enough to make AYP. I also notice that many of the sought after PCS did not make AYP and many Publics didnt either - of those with a diverse socio-economic and racial population.

What does this mean? It is intersting because we all really have no idea what the equivalent test scores would be in the most sought after private schools. I explored them all and loved the school and now wonder how much the test scores should matter. What are your thoughts?

[Anonymous]

OP here. I read my post again...should read "considered". I explored so many schools, both public and private in DC. The interesting aspect about the privates is that the qualifications of the teachers varies widely. I understand that a certification does not translate into an excellent teacher, but it does mean that the teacher has been educationally prepared to teach. Private Schools test scores are just that "private". I am just interested in having this conversation.

[Anonymous]

Was the school starting from a high baseline on the tests? I believe that AYP is also dependent on change from year to year, which could be more difficult in a school that starts at a high level. Somebody else can correct me if I've got this wrong, however.

[Anonymous]

I'm interested in learning the answer to some of these questions too. When our kids did private they took the ERB exams, and my understanding was that it was part of getting NAYEC (sp?) certification. The kids were coached before the ERBs. Some of it, like teaching vocabulary, I would have wanted to see anyway. But it is an interesting point of comparison between public and private.

[Anonymous]

IMHO we should:

• ignore the NCLB-required standardized test data

• make our decisions about where to send our children to school based on the quality of the educational opportunities offered and whether the educational philosophy is a good fit with our child

• let our elected officials know that NCLB does not improve the quality of public or public charter schools

[Anonymous]

Anonymous wrote:IMHO we should:

• ignore the NCLB-required standardized test data

• make our decisions about where to send our children to school based on the quality of the educational opportunities offered and whether the educational philosophy is a good fit with our child

• let our elected officials know that NCLB does not improve the quality of public or public charter schools

I second this!

[Anonymous]

I disagree with this. The one sort of positive part of NCLB is that it tracks data by race and economic status - this had not been done previously. The achievement gap is really real and NCLB also holds schools accountable for trying to close that gap.

This may also be why some charters did not make AYP - because they may have lost ground with closing the acheivement gap. That said everyone has to identify their own standards, which may have nothing to do with test scores. I am AA and personally look at how well a school is doing with minority students. We are fortunate to attend a DCPS school where the gap isn't as wide.

I think it's a losing proposition to try and find a way to use testing to compare public with private though.

[Anonymous]

Anonymous wrote:

Anonymous wrote:IMHO we should:

• ignore the NCLB-required standardized test data

• make our decisions about where to send our children to school based on the quality of the educational opportunities offered and whether the educational philosophy is a good fit with our child

• let our elected officials know that NCLB does not improve the quality of public or public charter schools

I second this!

I third. BUT The problem is, how as an outsider do you evaluate the "educational opportunities offered"? How can you tell the quality of what is being offered, and the results?

[Anonymous]

You evaluate the school the same way you would evaluate a private school:

You observe classes taught by various teachers.
You examine the curriculum, the textbooks and the supplementary materials.
You ask questions about how teachers assess progress.
You ask questions about how administrators assess teachers.
Then you try it out and see if it works for your child.
Once you find the right school, the results will be determined by your child.

[Anonymous]

In evaluating test scores for a charter, I look behind them to see why the school did not make AYP, and base my evaluation on that. Let's take for an example, Capital City Charter, which may or may not be the school the OP's child is headed for, but it meets the description, highly sought after, did not meet AYP in Math or Reading.

This long post steps you through how to access data in on the OSSE website, intersperses my commentary about intrepretation of the Cap City specific data and my viewpoint on flaws in the law. I think you'll be able to tell which is which. . . .so here goes. . .

Go to the test score site: http://www.nclb.osse.dc.gov/

Select all the way on the right, top banner “Summary AYP Reports”. In the right box, labeled “LEA Reports” select Capital City PCS from the drop down list, then fill in the bubble for “Historical AYP Report”, click yellow box to get report.

This summary report shows red “no” and blue “yes” for whether the school made AYP or not, by subgroup. For Capital City, for 4 consecutive years, the school has not made AYP in Reading. The only year it made AYP in Math was 2007. This is seen in the right most column.

In order for a school to make AYP in a given year, every subgroup (that is large enough to be counted) must also make AYP. The nice thing about this particular report is that you can easily see which subgroups failed to make AYP. Starting in the second column from the right “Total” you see all blue “yes.” That means that the whole school, with all test scores counted in aggregate, made AYP. Where are the Nos? In reading, Hispanics did not meet AYP in all years except 2008. In 2008, Hispanics meet by “safe harbor” criterion, which means the group did not meet the target but did improve performance from the previous year by more than 10 %. Economically disabled kids, in most years failed to make AYP in reading.

You can also use this summary chart to see who does make AYP. White non-hispanic, a column of straight “yes” Black, non-hispanic “yes” in reading, “no” in math for 2 years, “yes” in math for 2 years. Disabled, “yes” in both subjects in 2008, “no” in both for 2009.

Assuming the OP had looked at test scores from 2008, and decided this school was okay for her child, what’s different with 2009 results? There’s a backslide in math, with blacks going from yes to no, a backslide in reading with Hispanics from yes to no and Econ Disab from yes to no. A change in the special ed kids status, which I would strongly discount as meaning anything, because if the children in this group are more or less competent in a given year, it really quickly skews the data. There is simply a much greater range of possible abilities, and very dependent on the individuals. There have been highly regarded NW DCPS schools that did not make AYP based on the performance of "disabled" students, something I think is truly meaningless.

So, having looked at the Summary Report, now, in analyzing a school, I dig down into the specifics. Return to “home” on the blue banner, left most selection. In the left drop down box, Select year: 2009, Report type: AYP Report, Group: Charters, Level: Elementary, School: capital City, Subject: Math.

Up pops a report that shows % proficient in 2008 by subgroup, along the rows, and a bit more data about 2009. On this report, start on the last row. In 2008, 72.97% of Cap city kids were proficient in Math, in 2009, 62.94% proficient. That’s a big drop. Looking at the subgroups, Black dropped by over 10% (66.04% to 54.72%), Hispanic by over 20% (62.79% to 41.86%) and White stayed essentially the same (90% to 91.11%). It might concern me to ask what’s going on with Math instruction at Cap City that 2 of 3 ethnic groups have such a dramatic drop in % proficient?

Let’s go to a more detailed level yet with the data. Back to the left drop down box. (website is quirky, you might need to hit refresh with your browser a couple of times). Year: 2009, etc. down to Report type: AYP Report.

This report shows Math and Reading, for 2008 and 2009, by the 4 possible score ranges, Below Basic, Basic, Proficient, Advanced. AYP targets are made or lost on the number of kids in Proficient and Advanced, but you get no more credit for an Advanced kid than a Proficient one. One criticism of the NCLB is that this highly motivates schools to identify kids high in the Basic group, and work intensively to get them into Proficient, while abandoning kids who are down in Below Basic and not working to get Proficient kids into Advanced. This is a general criticism, I have not ever observed it in action, and certainly am not saying that Capital City does this.

But looking at the Math data on the AYP Report card, down at “School Total”, first shaded blue total rows, you can see that there are fewer advanced kids in 2009 vs. 2008 (13.89% vs. 25%), about the same in the Proficient bin (27.97% in 08 vs 28.61% in 09) more Basic kids in ’09 (28.47% vs. 24.32%) and more Below Basic (9.03% vs. 2.74%). So, this chart reveals that the drop in overall Math proficiency of 10% fewer kids, came mainly from kids dropping out of the Advanced bin. You can look up and see the distribution by ethnic group, English proficiency, and Economic status.

There’s something else I find very interesting about Cap City data. The school has almost equal number of Black (54 tested), Hispanic (43 tested) and White (45 tested). As I’ve looked at public and charter school options in DC, this is a very rare mix.

So, my answer to the OP question, should she rethink the school decision?

Probably not.

These scores are for kids 3rd to 6th grade. Her child has 4 years (preK to 2nd) before even being included in the testing group. Assuming the OP is white, white kids do fine at Capital City. If you value diversity, this is a school that has it. Thing to watch: math instruction (although I didn’t step through the Reading data, there isn’t nearly the slippage between the two years).

Of course, the OP might also have been talking about Two Rivers, another large lottery school that didn’t make AYP, and hopefully then I’ve spent enough time that she can do her own analysis for that school.

Good luck!

[Anonymous]

OP here...dont mind sharing...AA and the school is 2 Rivers. Thanks for the analysis.

[Anonymous]

OP here again....Your analysis was very helpful. It really helped to alleviate my immediate concerns about my decision. I was able to transfer your analysis to Two Rivers and other schools. Thank you for taking the time to post your lengthy but informative response.

[Anonymous]

Anonymous wrote:In evaluating test scores for a charter, I look behind them to see why the school did not make AYP, and base my evaluation on that.

So, in other words, the data reflects who is taking the test, not the quality of the teaching and learning?

If that's the case, wouldn't it be more helpful to simply judge the school by what it offers every day rather than by the results of a one-time standardized test?

[Anonymous]

22.23 PP who did the analysis of Cap City. See the DC Public Schools forum. Someone else (not me) posted the scores comparing Eaton, Key, Lafayette, by subgroup. When you get down to the subgroup level, for those 3 schools, the white and AA performance are very similar.

My guess is the most important predictor of a school's performance is the makeup of the class. The next important is how much test cramming the school does. Last year my kids were in a DCPS school that crammed for the test. Scores are up 10 points. But 5 weeks of the year, the whole school had "specials" cancelled while the 3th thru 6th graders took practice tests. My K student got unnecessary homework in "constructed response", test prep 3 years in advance. I disliked it. Headed to a charter school next year.

But, the schools scores did go up by 10%.

[Anonymous]

Anonymous wrote:

Anonymous wrote:In evaluating test scores for a charter, I look behind them to see why the school did not make AYP, and base my evaluation on that.

So, in other words, the data reflects who is taking the test, not the quality of the teaching and learning?

If that's the case, wouldn't it be more helpful to simply judge the school by what it offers every day rather than by the results of a one-time standardized test?

In all honesty, it's really a combination of all of the above factors. However, you can't teach students who are chronically absent, which is the case for many on the lower socioeconomic scale. My students who were absent were watching younger siblings while parents worked 2-3 jobs, were translating for family members, stepping in to help out with family businesses, or being recruited by gang members.

So although the collection and analysis of data has its strengths, this data is summative and does not get to the root causes of why children are failing. For example, if in a high school class of 30, 60% of Hispanics receive Ds (which is not unheard of depending upon class make up), WHY aren't they passing? Is it attendance? And if it is, what's keeping these students from coming to school? Is it a literacy issue? If so, is English a second language for some? Are they reading on a 6th grade level, for example? Are they coming to us hungry? Are they tired from working? There are all sorts of factors that bring down AYP, and unfortunately, the public is eager to place blame upon the teachers.

NCLB is simply a way to collect and monitor data, but it doesn't solve the problems BEHIND the data. There are plenty of well run schools with dedicated teachers who are gifted in the classroom, but unfortunately their test scores are low b/c of factors discussed above. So take test scores with a grain of salt (and a shot of vodka!).