By Will Payne

This is a pretty rough model of why I think culture is something group leaders should pay a lot of attention to, particularly how people enter the group and how much we should worry about the effects of this

The unpolished scam

Famously, some scam emails have terrible spelling and grammar, you might have seen something like this email before and wondered how anyone could possibly follow up on that email.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/51949eeb-a4b6-4625-88e8-d34a24493606/Untitled.png

The answer is that the world is big, the market of all people reached by the email is massive, and there is someone vulnerable enough - or foolish enough - to fall for this first email. If someone falls for the first email chances are they're gullible enough to fall for the whole scam. Scammers want their first email to be the least believable so that all the non believers and skeptics of the world ignore them leaving anyone they could plausibly take money from easier to pick out of the replies. It's better to be ignored by most so long as the people who pay attention are the people you care about.

(Here is a better explanation of this scamming tactic)

The Evaporative Filter

A while ago I read a rationalist article called Every Cause Wants to Be a Cult and an associated article Evaporative Cooling of Group Beliefs, the analogy plays to my Physics background but the phenomenon isn't that different from the scammers and their unpolished emails. The model is as follows: imagine a group of people who agree on an idea, most are quite average, some are very skeptical, and some are radical. Whenever something in this group happens to shake the foundations of the group - some evidence against the ideology that they support, a social scandal within the group, or the death of a thought leader - people start to lose faith. Who are the first people to leave when that happens? It's not the radicals, and not the average people (not straight away at least), it's the skeptics. This leads to a group which is slightly more radical than it was before, slightly more prone to crazy ideas which aren't inline with the initial group consensus. This scares off a few more of the most skeptical, which makes the group more extreme, and so on, and so forth until you have a unrecognisable group of the most radical people in the original society - plus a bunch of new more zealous people they picked up.

This implication scares me. If the model is correct groups tend towards their most off-putting beliefs, because anyone who might oppose them gets driven away by how off-putting they are. The culture of a group isn't static, it has momentum. Culture attracts people who agree with it and drives away people who don't, and the product is motion towards the extremes. The most scary part is that this wouldn't be a one off problem, this could happen on every axis (maybe something happens which makes the radicals start to leave first and you get an overly skeptical group which never generates new ideas), it seems like an incredibly natural thing, every cause wants to be a cult you need to work to make sure it isn't. There is no static culture and you need to keep correcting for shifts like the one described above in order to avoid cultishness.

Tying in to EA Groups

As a scammer your first email does most of the work in selecting the people you'll spend time trying to fool. The first email might reach 1000 times as many people as actually respond. As a EA group member the way you describe EA for the first time selects for the people who will join your community. If you describe it in the way that persuaded you then you'll only get people like you. If you misrepresent EA then you'll get people who don't fully agree with the core claims of the movement. Maybe they'll leave later on and you'll have wasted their time and your own. Maybe they'll stick around and start shifting the culture of the movement.

Culture has momentum, extreme ideas attract extremists. As an EA group leader misrepresenting EA not only attracts people who don't agree with the claims, it might shift the culture in a way that attracts even more of them. Maybe it scares off those people who do agree with the core claims of the movement. Culture can change in other ways, events that attract you might only attract people like you, and put off people not like you, this is especially bad because then you get a community which only wants to put on this kind of event. New ideas might scare off the most skeptical EAs and put us in an echo chamber. Culture has momentum, small changes in one direction can be harder to reverse than they were to create.

The general claim I'm making here is something like: representing an idea is a lot harder, and a lot more important than it might seem. Small errors can have large effect, and one person misunderstanding can affect a lot more than just their own beliefs. As EA community builders we need to work hard to keep the culture in a good place, (both by working hard to get there and by working hard to hold it there). We also affect the culture by being responsible for welcoming people to the community, this means (consciously or unconsciously) we are filtering the people who join the community. We should make sure to filter for people who agree with the core ideas - but not zealously - and actively make sure not to filter only for people who think like us.

A few examples