Data Analysts - Where are you? (CAPE)

[Anonymous]

I'm genuinely curious about possible reasons why some charter schools (e.g., Lee, ITS, Stokes) have relatively 'good' CAPE scores for all students but scores for economically disadvantaged kids that are so much worse. Compare these schools to some DCPS schools like Burroughs, Whittier, or Garrison--all of which have good scores for all students (similar to Lee/ITS/Stokes) and nearly as good scores for disadvantaged kids. The obvious difference between the charters above and the DCPS schools is that the charters have lower percentages of at-risk kids (like 15% vs. 50%). Is it that the DCPS schools, because they have more at-risk kids, focus more efforts and resources at those kids than ITS/Lee/Stokes?

[Anonymous]

Anonymous wrote:Can anyone help me figure out why several schools have data suppressed for 3+ when data was not suppressed for 4+?

For example, for Whittier and DC Bilingual, 4th grade math scores are "DS" for 3+ but you can see how many 4th graders scored 4+. Obviously the sample size would be greater for the 3+ sheet so it seems confusing and hard to figure out what disclosure risk would be introduced?

Because something about that data is too individually identifying. Maybe there is only one kid who got a 3 or none? Alternatively, are 1 & 2 suppressed as well? It may be that that data would allow you to back out other suppressed data.

Without identifying my school, our 4+ data for Black students in math is suppressed overall, but provided for 2 grades. When you back out all of the data and compare it against the totals, it seems like it's likely because <5% of that single grade got 4+ (which is horrifying and very out of line with the other grades).

[Anonymous]

Anonymous wrote:When I select "for non-economically disadvantaged students," I see no data. Am I missing something? Why can the data be shown for economically disadvantaged students but not the inverse?

Because there are not very many students who are "non-economically disadvantaged" at that school.

[Anonymous]

Anonymous wrote:I'm genuinely curious about possible reasons why some charter schools (e.g., Lee, ITS, Stokes) have relatively 'good' CAPE scores for all students but scores for economically disadvantaged kids that are so much worse. Compare these schools to some DCPS schools like Burroughs, Whittier, or Garrison--all of which have good scores for all students (similar to Lee/ITS/Stokes) and nearly as good scores for disadvantaged kids. The obvious difference between the charters above and the DCPS schools is that the charters have lower percentages of at-risk kids (like 15% vs. 50%). Is it that the DCPS schools, because they have more at-risk kids, focus more efforts and resources at those kids than ITS/Lee/Stokes?

One answer is that non at-risk parents are supplementing/supporting in a way that at-risk parents are not, at those schools. (Whereas at DCPS they are teaching well across the board.) Another possibility is that there is something about the culture at those schools that is keeping at-risk kids from being fully engaged and set up to succeed.

[Anonymous]

Anonymous wrote:I'm genuinely curious about possible reasons why some charter schools (e.g., Lee, ITS, Stokes) have relatively 'good' CAPE scores for all students but scores for economically disadvantaged kids that are so much worse. Compare these schools to some DCPS schools like Burroughs, Whittier, or Garrison--all of which have good scores for all students (similar to Lee/ITS/Stokes) and nearly as good scores for disadvantaged kids. The obvious difference between the charters above and the DCPS schools is that the charters have lower percentages of at-risk kids (like 15% vs. 50%). Is it that the DCPS schools, because they have more at-risk kids, focus more efforts and resources at those kids than ITS/Lee/Stokes?

My observation as an ITDS family is that, in the past, at-risk kids came to the school in older grades and left quickly. They arrived at the school because something about their in-boundary school didn't work for their family. There was no at-risk preference thus no preferential on-ramp for joining the school in earlier grades. Then after having a poor educational experience at another school, they joined ITDS in an older grade (thus their academic experience has more to do with previous schools) and were perhaps too far behind to catch up.

Now, more nearby families are joining the school earlier and hopefully will be able to learn along with their same-age peer and perform as well as them as they age into the CAPE or other test.

[Anonymous]

Anonymous wrote:

Anonymous wrote:I'm genuinely curious about possible reasons why some charter schools (e.g., Lee, ITS, Stokes) have relatively 'good' CAPE scores for all students but scores for economically disadvantaged kids that are so much worse. Compare these schools to some DCPS schools like Burroughs, Whittier, or Garrison--all of which have good scores for all students (similar to Lee/ITS/Stokes) and nearly as good scores for disadvantaged kids. The obvious difference between the charters above and the DCPS schools is that the charters have lower percentages of at-risk kids (like 15% vs. 50%). Is it that the DCPS schools, because they have more at-risk kids, focus more efforts and resources at those kids than ITS/Lee/Stokes?

My observation as an ITDS family is that, in the past, at-risk kids came to the school in older grades and left quickly. They arrived at the school because something about their in-boundary school didn't work for their family. There was no at-risk preference thus no preferential on-ramp for joining the school in earlier grades. Then after having a poor educational experience at another school, they joined ITDS in an older grade (thus their academic experience has more to do with previous schools) and were perhaps too far behind to catch up.

Now, more nearby families are joining the school earlier and hopefully will be able to learn along with their same-age peer and perform as well as them as they age into the CAPE or other test.

I think for ITDS you need to compare the middle school grades to nearby middle schools such as Brookland and McKinley. If you're averaging all of ITDS's grades and comparing to an elementary school, it isn't apples to apples.

ITDS tends to lose about 25-30% of rising 5th graders, primary to Latin. This is due to sibling preference and lottery luck. They are replaced by new kids, some of whom are on or above grade, others below, so on average the performance of the cohort drops. ITDS, being a small school, can be appealing to families whose kids struggle academically or have needs better served in a small setting, and I think that has an impact as well.

[Anonymous]

Anonymous wrote:

Anonymous wrote:I'm genuinely curious about possible reasons why some charter schools (e.g., Lee, ITS, Stokes) have relatively 'good' CAPE scores for all students but scores for economically disadvantaged kids that are so much worse. Compare these schools to some DCPS schools like Burroughs, Whittier, or Garrison--all of which have good scores for all students (similar to Lee/ITS/Stokes) and nearly as good scores for disadvantaged kids. The obvious difference between the charters above and the DCPS schools is that the charters have lower percentages of at-risk kids (like 15% vs. 50%). Is it that the DCPS schools, because they have more at-risk kids, focus more efforts and resources at those kids than ITS/Lee/Stokes?

One answer is that non at-risk parents are supplementing/supporting in a way that at-risk parents are not, at those schools. (Whereas at DCPS they are teaching well across the board.) Another possibility is that there is something about the culture at those schools that is keeping at-risk kids from being fully engaged and set up to succeed.

I think these are reasonable explanations and also that it's going to vary a lot by school. I think if you are going to look at CAPE scores for immersion or Montessori programs you should compare them to other immersion and Montessori programs. Comparing Lee to Burroughs doesn't make a ton of sense but you could compare it to CHML. Comparing Stokes to Garrison makes less sense than comparing it to DCB or Tyler or Yu Ying. And then when you make those comparisons you also have to bake in the differences in demographics. But the DCPS schools you mention (Burroughs, Whittier, or Garrison) can be easily compared to each other because they have similar profiles -- DCPS neighborhood schools with similar demographics. If you want to compare those schools to charters you need to look at charters with more similarities -- places like Two Rivers or Cap City and Kipp (varying similarities but all closer matches than Lee or Stokes).

[Anonymous]

Anonymous wrote:I'm genuinely curious about possible reasons why some charter schools (e.g., Lee, ITS, Stokes) have relatively 'good' CAPE scores for all students but scores for economically disadvantaged kids that are so much worse. Compare these schools to some DCPS schools like Burroughs, Whittier, or Garrison--all of which have good scores for all students (similar to Lee/ITS/Stokes) and nearly as good scores for disadvantaged kids. The obvious difference between the charters above and the DCPS schools is that the charters have lower percentages of at-risk kids (like 15% vs. 50%). Is it that the DCPS schools, because they have more at-risk kids, focus more efforts and resources at those kids than ITS/Lee/Stokes?

Selection effects and network effects? Not all at risk kids are equally at risk, and not all schools are well known among different segments of at risk kids? Good luck ever getting the power to specify an effect, but it would be interesting.

[Anonymous]

It's interesting, but also these schools are so small in the testing grades that random variation and people's lottery luck plays a big role. It's hard to untangle it all.

[Anonymous]

Anonymous wrote:It's interesting, but also these schools are so small in the testing grades that random variation and people's lottery luck plays a big role. It's hard to untangle it all.

That's true but also when you see a consistent trend year after year I do think you cab draw some conclusions about a school. You can't always specify the reason behind what is happening but you can conclude that a school is not successful with economically disadvantaged kids.

[Anonymous]

Anonymous wrote:

Anonymous wrote:Can anyone help me figure out why several schools have data suppressed for 3+ when data was not suppressed for 4+?

For example, for Whittier and DC Bilingual, 4th grade math scores are "DS" for 3+ but you can see how many 4th graders scored 4+. Obviously the sample size would be greater for the 3+ sheet so it seems confusing and hard to figure out what disclosure risk would be introduced?

Because something about that data is too individually identifying. Maybe there is only one kid who got a 3 or none? Alternatively, are 1 & 2 suppressed as well? It may be that that data would allow you to back out other suppressed data.

Without identifying my school, our 4+ data for Black students in math is suppressed overall, but provided for 2 grades. When you back out all of the data and compare it against the totals, it seems like it's likely because <5% of that single grade got 4+ (which is horrifying and very out of line with the other grades).

Thanks. I understood the DS suggested there was a disclosure concern. I just could not think of what that concern might be given 4th grade math scores for 4+ were fine to present but not 3+. But, maybe you are right and the issue is that the 4+ counts would be very similar to the 3+ counts and show only a handful of students got 3s? (To me, that's still not a clear disclosure concern but I can see a business rule that gets you there.)

[Anonymous]

Anonymous wrote:

Anonymous wrote:

Anonymous wrote:Can anyone help me figure out why several schools have data suppressed for 3+ when data was not suppressed for 4+?

For example, for Whittier and DC Bilingual, 4th grade math scores are "DS" for 3+ but you can see how many 4th graders scored 4+. Obviously the sample size would be greater for the 3+ sheet so it seems confusing and hard to figure out what disclosure risk would be introduced?

Because something about that data is too individually identifying. Maybe there is only one kid who got a 3 or none? Alternatively, are 1 & 2 suppressed as well? It may be that that data would allow you to back out other suppressed data.

Without identifying my school, our 4+ data for Black students in math is suppressed overall, but provided for 2 grades. When you back out all of the data and compare it against the totals, it seems like it's likely because <5% of that single grade got 4+ (which is horrifying and very out of line with the other grades).

Thanks. I understood the DS suggested there was a disclosure concern. I just could not think of what that concern might be given 4th grade math scores for 4+ were fine to present but not 3+. But, maybe you are right and the issue is that the 4+ counts would be very similar to the 3+ counts and show only a handful of students got 3s? (To me, that's still not a clear disclosure concern but I can see a business rule that gets you there.)

Yes. that is the concern. Its actually a huge debate now; lots of data providers are using this as an excuse to tamper or hide data. The census now does 'noise' infusion. So most census data now has random data inserted in it. OSSE simply doesn't show anything that would reveal a count under some cutoff. Its all silly and not really 'privacy' in the standard sense.

[Anonymous]

Anonymous wrote:

Anonymous wrote:

Anonymous wrote:

Anonymous wrote:Can anyone help me figure out why several schools have data suppressed for 3+ when data was not suppressed for 4+?

For example, for Whittier and DC Bilingual, 4th grade math scores are "DS" for 3+ but you can see how many 4th graders scored 4+. Obviously the sample size would be greater for the 3+ sheet so it seems confusing and hard to figure out what disclosure risk would be introduced?

Because something about that data is too individually identifying. Maybe there is only one kid who got a 3 or none? Alternatively, are 1 & 2 suppressed as well? It may be that that data would allow you to back out other suppressed data.

Without identifying my school, our 4+ data for Black students in math is suppressed overall, but provided for 2 grades. When you back out all of the data and compare it against the totals, it seems like it's likely because <5% of that single grade got 4+ (which is horrifying and very out of line with the other grades).

Thanks. I understood the DS suggested there was a disclosure concern. I just could not think of what that concern might be given 4th grade math scores for 4+ were fine to present but not 3+. But, maybe you are right and the issue is that the 4+ counts would be very similar to the 3+ counts and show only a handful of students got 3s? (To me, that's still not a clear disclosure concern but I can see a business rule that gets you there.)

Yes. that is the concern. It’s actually a huge debate now; lots of data providers are using this as an excuse to tamper or hide data. The census now does 'noise' infusion. So most census data now has random data inserted in it. OSSE simply doesn't show anything that would reveal a count under some cutoff. It’s all silly and not really 'privacy' in the standard sense.

Differential privacy is the literal devil, and of course people are using it to disguise inconvenient results

[Anonymous]

Anonymous wrote:I'm genuinely curious about possible reasons why some charter schools (e.g., Lee, ITS, Stokes) have relatively 'good' CAPE scores for all students but scores for economically disadvantaged kids that are so much worse. Compare these schools to some DCPS schools like Burroughs, Whittier, or Garrison--all of which have good scores for all students (similar to Lee/ITS/Stokes) and nearly as good scores for disadvantaged kids. The obvious difference between the charters above and the DCPS schools is that the charters have lower percentages of at-risk kids (like 15% vs. 50%). Is it that the DCPS schools, because they have more at-risk kids, focus more efforts and resources at those kids than ITS/Lee/Stokes?

Some schools (inc. some examples you named) just have stronger instructional culture and teaching than others, which was developed over time through consistent leadership and sustained effort. It's not always a demographics puzzle.

[Anonymous]

Some of the schools up for charter review this cycle have really terrible scores.

I wonder if the disaster of Eagle Academy will make this a more in-depth review cycle. Probably not. Yaaaay, autonomy!