Help me Edit: Response to Brookings Report

[Anonymous]

I'd note up front that in many ways I agree with the core of what these Brookings authors are flailing at. But no, I don't think it's great scholarship. They're trying to do quantitatively something that requires journalistic, anthropological study. And it's worth it too! DCUM and this forum is a clear subculture with its own in-lingo, euphemisms, ways of talking about some things anonymously, ways of thinking around others that some people don't even want to address or consider.

[Anonymous]

been a decent amount of ad hominem in this thread!

[Anonymous]

Haven’t seen Jeff in awhile, so I imagine him at a desk typing madly away with purpose. His response could become even longer than the Brookings report. Don’t do it, Jeff! Save yourself!!

Hopefully though he’s stepping away a bit to process things.

[jsteele]

Anonymous wrote:Haven’t seen Jeff in awhile, so I imagine him at a desk typing madly away with purpose. His response could become even longer than the Brookings report. Don’t do it, Jeff! Save yourself!!

Hopefully though he’s stepping away a bit to process things.

Nope. First, I never work at a desk. Second, I just spent an hour split between my rower and exercise bike. I'm back, I'm ready, and my arms and legs are tired.

[Anonymous]

Anonymous wrote:Out of curiosity, is there anyone who believes the actual academic scholarship done here (word frequency analysis without contextual controls) qualifies as good scholarship? I'm not sure I have seen a Brookings-level report before where the discussion about the publication is so unanimous in agreement that the underlying methodology is significantly flawed. Maybe I am missing something, though.

It's a really bad methodology.

For example, here's a paper from 2012 that covers something that was current then:

https://dl.acm.org/doi/abs/10.5555/2390500.2390505

This tutorial aims to cover the basic motivation, ideas, models and learning algorithms in deep learning for natural language processing.
Recently, these methods have been shown to perform very well on various NLP tasks such as language modeling, POS tagging, named entity recognition, sentiment analysis and paraphrase detection, among others.

"Named entity recognition" is something like figuring out whether IB means "inbounds" or "International Baccalaureate".
"Sentiment analysis" is the core of what Vanessa Williamson is trying to do with the data: figure out how people feel about different schools.

This is a tutorial. From 2012. It was old stuff then.

[Anonymous]

Also here is a tutorial that shows how to do sentiment analysis.

With open-source tools (i.e. free), and in realtime (i.e. much much harder than analyzing a big pile of static data you scraped and writing a report.)

https://towardsdatascience.com/real-time-sentiment-analysis-on-social-media-with-open-source-tools-f864ca239afe

The word embedding algorithm takes as its input from a large corpus of text and produces these vector spaces, typically of several hundred dimensions. A neural language model is trained on a large corpus (body of text) and the output of the network is used to each unique word to be assigned to a corresponding vector. The most popular word embedding algorithms are Google ‘s Word2Vec, Stanford ‘s GloVe or Facebook ‘s FastText.

Word embeddings represent one of the most successful AI applications of unsupervised learning.

This is well-trodden ground by now.

[Anonymous]

Anonymous wrote:Also here is a tutorial that shows how to do sentiment analysis.

With open-source tools (i.e. free), and in realtime (i.e. much much harder than analyzing a big pile of static data you scraped and writing a report.)

https://towardsdatascience.com/real-time-sentiment-analysis-on-social-media-with-open-source-tools-f864ca239afe

The word embedding algorithm takes as its input from a large corpus of text and produces these vector spaces, typically of several hundred dimensions. A neural language model is trained on a large corpus (body of text) and the output of the network is used to each unique word to be assigned to a corresponding vector. The most popular word embedding algorithms are Google ‘s Word2Vec, Stanford ‘s GloVe or Facebook ‘s FastText.

Word embeddings represent one of the most successful AI applications of unsupervised learning.

This is well-trodden ground by now.

I do this kind of stuff and the problem isn't the method, it's that the conclusions aren't supported by the method. Regression analysis is well-trodden ground, but it's still sometimes the correct way to solve something. 'Scrape a website, do some analysis' is the correct way to go at all kinds of questions. And if all you wanted to answer was 'what schools are people talking about', you could. It's the judgements they made about why that are the problem. But that's not like, oh, use more cutting-edge text analysis.

[Anonymous]

jsteele wrote:

Anonymous wrote:Haven’t seen Jeff in awhile, so I imagine him at a desk typing madly away with purpose. His response could become even longer than the Brookings report. Don’t do it, Jeff! Save yourself!!

Hopefully though he’s stepping away a bit to process things.

Nope. First, I never work at a desk. Second, I just spent an hour split between my rower and exercise bike. I'm back, I'm ready, and my arms and legs are tired.

Even better! Good for you.

[Anonymous]

Anonymous wrote: 'Scrape a website, do some analysis' is the correct way to go at all kinds of questions.

Oh, I agree.

Specifically, though: 'Scrape a website, count words' is NOT the correct way to get to this question or anything related to it. Seriously, Claude Shannon was building (not that useful) Markov models of word context in 1948, and those went beyond counting words. In this case some more cutting-edge text analysis might have actually been helpful.
But also, the questions are bad. It's both.

[Anonymous]

Anonymous wrote:

Anonymous wrote: 'Scrape a website, do some analysis' is the correct way to go at all kinds of questions.

Oh, I agree.

Specifically, though: 'Scrape a website, count words' is NOT the correct way to get to this question or anything related to it. Seriously, Claude Shannon was building (not that useful) Markov models of word context in 1948, and those went beyond counting words. In this case some more cutting-edge text analysis might have actually been helpful.
But also, the questions are bad. It's both.

It’s because clearly all the researchers wanted was more fuel to pour on the dumpster fire of racism and education in DC. Looks like they’ve accomplished that.

Honestly, the report makes me want to defend dcum in a way that I never would have dreamed. I think Jeff’s points are valid. DC parents are stuck between a rock and a hard place. The responsibility never lies with individuals for systemic policy issues like this; good policy helps people make decisions that need to be made collectively. It’s kind of like recycling; I can’t save the environment alone I need a citywide program to recycle. We don’t have good leadership at the citywide level. We’ve pushed things into this bizarre “choice” aka lottery and charter system, and shocker, people are using it. Each individual making a rational non-racist choice causes a whole system to lean toward segregation. But we knew that.

Let me see a proper social scientist - not these jokers, not some podcaster - try to find out what is going on and offer some policy ideas. We can’t really on individual parents to solve centuries of our checkered past.

[Anonymous]

Anonymous wrote:

jsteele wrote:

Anonymous wrote:

jsteele wrote:

Anonymous wrote:yes, exactly. A lot of the books that have come out this year (How to Be Anti-racist, Caste) are about exactly this. It's really understandable that people get defensive when they think they are being "called a segregationist." this year has been a long journey of trying to get people to face instead that they are participating in a racist system. it's a subtle difference but maybe one that can relieve some of that defensiveness.

To an extent I agree with you and I acknowledge that I probably should have viewed things more in this light. However, with regard to this report, I think its research is extremely shoddy, doesn't support the conclusions, and both ignores and reveals the obvious. Because the research is so light and flawed, what stands out are the allegations that are repeated throughout the report about supporting segregation. Perhaps the authors could have made their point without using such a loaded term? Is there really any justification for using such a term toward people who have chosen to remain in DC public schools rather than fleeing for private or the suburbs? Why antagonize the very folks with whom you must partner to find a solution?

Jeff, this reaction is white fragility in action. You can do better.

You may be correct that it is white fragility, but it is also reality. If people are interested in hard truths, it is a simple fact that this sort of language alienates your most likely allies. Why accuse people who didn't choose private schools and who didn't flee to the suburbs of supporting segregation? What solution does that help achieve?

This sounds like whataboutism, with respect to those who moved to the suburbs or choose private.

Should the authors not even bother to do this sort of research, for fear of how it will land with some defensive people? Hopefully for those that react less defensively, or move from initial defensiveness to actually thinking about whether they can do anything better, there will be a positive impact.

What sort of research did they do? None. Had they done real research and looked at actual data, they might have come to a different conclusion about parents in DCPS.

[Anonymous]

Now Perry Stein at the Post has picked this up.

https://www.washingtonpost.com/local/education/dc-urban-moms-school-segregation-study/2021/03/31/8320b6e4-9160-11eb-a74e-1f4cf89fd948_story.html

And, great... she's quoting DCUM posts. Sigh.

I wish she had covered some of the actual substantive criticism here of the report.

[Anonymous]

Also, Post article has no mention of wealthy families going to suburban schools, which seems like a pretty big relevant point.

[Anonymous]

Brookings has never been a source of serious research. In general, it has a bunch of somewhat flashy names and then a bunch of research assistants. Research-informed opinion pieces, whether identified out right or disguised as policy briefs, is its style.

[jsteele]

Anonymous wrote:Now Perry Stein at the Post has picked this up.

https://www.washingtonpost.com/local/education/dc-urban-moms-school-segregation-study/2021/03/31/8320b6e4-9160-11eb-a74e-1f4cf89fd948_story.html

And, great... she's quoting DCUM posts. Sigh.

I wish she had covered some of the actual substantive criticism here of the report.

Perry spent a long time listening to me rant so blame me for not doing a better job making my points. She couldn't include everything I said but I think she did a pretty good job overall. The report was always going to get the better part of the coverage.