This seems like an awfully uncharitable and cherry-picked treatment of a community of well-intentioned and occasionally eccentric disagreeable online nerds. You're effectively just noticing that a community that actively aims to overcome bias is not free from bias itself, but no one within that community would ever deny that.
It's not the bias the problem, it's the conviction of being better and "less wrong" than the rest of us.
That, along with constantly calling yourself rational and being well-intentioned, easily leads to cultish beliefs to be pursued despite how wacky they seem at their face, because it was your superior brains that led you to them.
If we add the often good (or enormous) financial resources, it makes for a seriously dangerous cult.
For the record, I’m not saying this particular loser on twitter will ever have the power to institute a dictatorship.
But it is insane to say that EA and rationalism* are not about believing you are superior and more rational than your peers, or that most EA/rats don’t make an identity out of it (look at any of their online presences). Anthropic literally exists to funnel money and influence to its EA/rat cofounders and early investors, who sincerely believe that they and they alone are smart and rational enough to shepherd our society through its “technological adolescence”. I wish people who agree with this would at least be honest about it.
Ok, less wrong every day; we have some rationalists here, whether you think yourself as rational or not let's get an idea of how rational you actually are: do you agree with the views of @tenobrus mentioned below (https://xcancel.com/tenobrus/status/2023473440364392850), specifically that humanity should be placed under a dictatorship?
I know he's EA, but let's not pretend that EA is so distinct from the rationalist community.
I'm not a rationalist and I don't know who these people are. As far as I can tell they're a bunch of online weirdos who don't matter. But as for that statement, "humanity will be far far better off if they're subjected to the values of effective altruists," that depends on the values in question, which haven't been explained in this thread. But there is definitely a set of values that exists that humanity would be better off subjected to than pure democracy. This is different from supporting an undefined alignment dictatorship in the abstract.
Everyone implicitly believes the above when they believe the concept of a "backwards society" can even exist, or when they think their own society would be better if their lost candidate had power. In both cases you think that there exist a set of values that are better to live under than what the democratic process resulted in.
I think actually most people believe this about their own values. I could find you infinite posts online that say "humanity would be better off if the other political party was gone [and my political party had sole power]"
There might (might, see later) be a set of values, but there definitely isn't anyone or anything that, given dictatorial powers, can be trusted in applying such values forever fairly without abusing or misusing their power.
I'm far from sure that even the rationalists' god-like AI could play that role, even in the extremely unlikely hypothesis that it will materialize at some point.
Not to mention that the best set of values could change with the circumstances, and most of all values are a vague concept, WAY too vague to make a dictator behave properly in all circumstances, even if he were somehow compelled to always follow them.
> humanity would be better off if the other political party was gone [and my political party had sole power]"
Even people wishing that don't necessarily long for a dictatorship, far from it; you could wish for the party you loathe to not exist, but still want everything else of your democratic system to stay unchanged (the possible parties are infinite, removing one doesn't necessarily turn your system into a single-party system).
Well you're arguing that dictatorship must be perfect ("applying such values forever fairly without abusing") to be better than democracy. It doesn't need to be infallibly perfect, it just needs to be better than democracy, which is also not perfect.
I have trouble understanding the problem with a benevolent dictatorship, yes. Obviously in real life there are all kinds of problems that arise due to human nature. In an abstract sense, a benevolent philosopher king is better than democracy.
I don't think I read about this, it just seems obvious that if a truly great intelligent leader leads a country it's better than the stupid masses voting about things.
It's not just that they aren't free of bias. They appear to incubate truly wacky stuff - Leverage Research [1], the Zizians, all the AI/x-risk stuff. I am not the only person to have noticed this [2]. There's something about their approach that appears to actually make people more susceptible to nonsense, not less.
The problem is the whole idea of working from first principles to try to discover the actual truth, rather than just going along with other ideas that other people have created through history. The benefit of going along with other peoples' ideas is that they've been tested out for hundreds or thousands of years and been found to not be that destructive.
If you start reasoning from first principles you can very easily make an incorrect assumption somewhere down the line that snowballs until you're talking about shrimp welfare, or making decisions based on the potential sextillion humans that exist in 1,000,000 AD, or planning to convert the entire universe into unthinking organic matter that can only experience bliss.
>There's something about their approach that appears to actually make people more susceptible to nonsense, not less.
It would be highly surprising if an open-access online community with the kind of slogans that might be particularly appealing to somewhat spectrumy individuals did not occasionally attract some eccentrics. The point I am trying to make here is that these are pretty fringe offshots and are far from the representative views of the community as a whole. Appeals to reason don't make any community immune to human nature. After all, their main place of gathering is named "less wrong", not "correct".
I gave an example elsewhere in the post and I was accused of dishonesty and strawmanning — even though it's obviously something that everyone in that community is expected to believe, because it's true (in the same way that 2 + 2 = 4 is true) and because it's specifically highlighted as a fundamental truth in the Sequences.
There was also someone else ITT pointing out that the community are not at all uniform, and that got a pithy response about it being largely "childless white male Americans" as if that is supposed to matter (and disregarding that fully 5% of survey respondents consider themselves not cis, and 19% not straight, which are both way above baseline).
So pardon me if I don't consider this sort of question to be in good faith.
>appears to actually make people more susceptible to nonsense
I think it is more that teenagers and very young adults and people high in the trait of "openness to new experiences" are more susceptible to nonsense and more attracted to new intellectual movements, of which the rationalist community is one.
Also, I expect a rationalist to be lower in conscientiousness than average (because the conscientious people are too busy to devote a lot of time to reading long intellectual texts) and low conscientiousness makes violence and other transgressive behavior more likely.
I guess people who are high in both openness and conscientiousness and who like to read long texts prefer to read them through books over online forums and social media.
Regarding the Zizians, I recommend the Behind the Bastards series on them and the rationalist community. People involved with the rationalists will want to distance themselves from it of course, but the series makes a decent argument that the Zizians were simply the result of taking the ideology to its rational conclusion, that there is a definite throughline from one to the other in terms of philosophy and ideals.
The author here clearly is extremely superficially familiar with the writings of rationalist people and views them from a preconceived standpoint. Comparing beliefs of Yudkowsky to dianetics is nothing short of misguided, whether you like him or not. This is a textbook study of motivated reasoning.
A libertarian socialist podcaster tries to discuss an online culture that is adjacent to Silicon Valley tech scene. He then proceeds to take movement’s worst manifestations as its essence. One would have to be naive to miss the source of his bias.
My feeling is that he understands your "online culture" much better than you, is very balanced and careful in the video, and you're probably a lot younger, inexperienced with internet groups and impressionable than him.
Take his advice to talk with (and listen to!) people outside your cadre, and to consider the possibility that you're entirely in the wrong. And maybe to touch grass.
taking what ideology to the conclusions?
The zizians throught some truly bizzare things, most of which are in no way representative of the rationilst community.
I’ve interacted with a number of rationalists that are just as crazy as Ziz but managed to not implode with the help of money (either lucking into it via family/crypto or a tech job). My personal hot take is that if she’d been cis a number of people would still be alive.
Asking "why are there so many rationalist cults?" (as the URL does) in this context is kind of like asking "why are there so many mathematical cranks?", and then holding forth that this somehow demonstrates a fundamental problem with mathematics.
I think the issue with the community is they take their eccentric beliefs as being gospel truths which can lead to truly unhinged and irrational beliefs and actions (see: the Zizians).
That's what can make the rationalist movement feel more like a religious cult and less like a bunch of eccentrics. As an outsider looking in, it seems like the entire movement is filled with just accepting priors with very little evidence for why those priors should be accepted.
Consider a community filled with 100 people; 95 of them who don't take their eccentric beliefs as gospel truths and 5 who do. Those 5 will inevitably become the public face of the community when they make the news. However there is very little that the 95 could have done to stop them.
> However there is very little that the 95 could have done to stop them.
How about the 95% stop having gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?
How is the 5% supposed to know that the 95% is actually larping belief? And how is an outsider who observes the 95% espousing their whacky beliefs supposed to know who is a member of the 95% and who is a member of the 5% (and if those percentages are accurate).
You can observe the same dynamic in any church. Everyone in the congregation will have different levels of belief, but when it comes to sharing a testimony you'll be hard pressed to distinguish between who the true believers are and who's just "faking it until you make it".
Rationalist have groups meetup and conferences. For example [1]
> What beliefs?
It's pretty diverse and not uniformly held. Some of the more strange ones that seem to have pretty wide appeal are around AI safety. For example, Roko's basilisk.
> What proclamations? What evidence of insincerity?
I was responding to the comment above me that suggested that 95% don't actually hold rationalist beliefs and are instead simply having fun with it. Only a small 5% are actually sincere.
Okay, but those pieces don't establish your claim. You would need a Roko's basilisk-like concept that only 5% of the community agree with, and a "group meetup" or "conference" where "they all" (i.e.: also the 95%) "proclaim that their beliefs (i.e.: including the basilisk thing) are true".
If I observed such a gathering knowing that 5% of the people there have sincerely come to a particular conclusion based on the available evidence and the other 95% think the evidence doesn't support that conclusion, I would be extremely surprised to see everyone going around "larping belief". And I don't think you can actually show such a thing occurring.
Which claim exactly did I make which you are refuting? Please actually quote it.
> I would be extremely surprised to see everyone going around "larping belief". And I don't think you can actually show such a thing occurring.
I never claimed I could or that I observed such a thing. The 95/5% didn't come from me, it came from the person I responded to.
My point was that
- It's impossible to distinguish who's a true believer and who isn't.
- That the meetups of the rationalists re-enforce beliefs, particularly in true believers. Also that these beliefs of the rationalists can be pretty whacky and based on a jenga tower of priors.
- That there isn't "nothing" a rational person could do to dissuade someone from a whacky belief.
> Which claim exactly did I make which you are refuting? Please actually quote it.
When you said:
> How about the 95% stop having gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?
you implicitly claimed that "the 95%" indeed have such gatherings.
As far as I can tell, the gatherings that they have are not of that nature.
> My point was that It's impossible to distinguish who's a true believer and who isn't.
And my claim is that it's actually trivial to tell who's a true believer and who isn't, because the concept of "true believer" does not pertain to "rationality" (by construction!) but to fringe beliefs, and people who are doing rationality correctly (which I expect to be the large majority) simply don't profess the fringe beliefs (because doing so is inherently antithetical to rationality). Therefore, the fringe 5% are the 5% of people actually saying the fringe thing, and the 95% don't actually say the fringe thing. Trivial.
For example, you won't catch Scott Alexander demanding "donate to MIRI or the basilisk might reach back in time and torture you".
> That the meetups of the rationalists re-enforce beliefs, particularly in true believers. Also that these beliefs of the rationalists can be pretty whacky and based on a jenga tower of priors.
And you have done nothing to establish this, and I believe you're incorrect.
> you implicitly claimed that "the 95%" indeed have such gatherings.
> As far as I can tell, the gatherings that they have are not of that nature.
I literally cited you an example of such a gathering. Do you refute it or just deny that it exists?
> And my claim is that it's actually trivial to tell who's a true believer and who isn't, because the concept of "true believer" does not pertain to "rationality" (by construction!)
That's your mistake. The rationalists are irrational. Roko's basilisk, which I cited earlier, is an example of the sorts of ideas and beliefs they get up to which are irrational.
Your claim "This group is rational because they call themselves such" is unfounded. Trivially so. Just like the Christian Scientists aren't scientists just because it's part of their name.
> For example, you won't catch Scott Alexander demanding "donate to MIRI or the basilisk might reach back in time and torture you".
I never claimed universality. In fact, in response to you I explicitly said that the beliefs are diverse. Being able to cherry pick someone that's more rational than others and a part of the group doesn't prove that there aren't very influential members of the group who are generating a lot of fringe ideas. Specifically Eliezer Yudkowsky, the founder of LessWrong which is a core part of the rationality movement.
> And you have done nothing to establish this, and I believe you're incorrect.
I've established the meetups exist. I've established that a particularly nasty group came from these meetups, the Zizians. Specifically the founder got their start from the LessWrong forums and the "Center for Applied Rationality" [1] workshops (meetups).
You seem to be banging on a stereotype. Yudkowsky doesn't speak for everyone (not even everyone on lesswrong) and AI safety/paranoia is an incredibly polarizing issue.
There's a loosely structured community of people who self-identify as "rationalist". There are some general themes of belief but no coherent platform, per-se. Some of them are nutjobs. Maybe at an elevated rate from general society, but not dramatically so.
you two have fundamentally different readings of the statement "95 of them who don't take their eccentric beliefs as gospel truths and 5 who do."
One reading is the views are eccentric within the community (e.g. roko's basilisk). The other reading is eccentric view which come out of the community but are not held.
You introduced the claim that the 95% are larping along with the 5% and sharing the same proclaimations.
If you think that, it seems fair to ask you why. Most of the weird stuff (e.g. zizian) isnt accepted by the rationaist majority. A few unusual things are, like AI X-risk.
> I literally cited you an example of such a gathering. Do you refute it or just deny that it exists?
You did nothing to attempt to establish that the gathering was of the nature you ascribe to it, and I see nothing in the Reddit post to support the claim.
> Roko's basilisk, which I cited earlier, is an example of the sorts of ideas and beliefs they get up to which are irrational.
> Your claim "This group is rational because they call themselves such"
No; my claim is based on reading their work and talking to them.
> I never claimed universality.
For your argument to work, you require evidence of a specific belief that is commonly asserted in the community despite a recognition that it is fringe and likely untrue.
> I've established that a particularly nasty group came from these meetups, the Zizians.
You did not in fact previously establish that the Zizians "came from those meetups". You merely mentioned them upthread. But even taking for granted that they "came from" the meetups (I haven't checked your current evidence but I'm sure it's adequate), this does nothing to establish that the meetups are "gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them".
I would never disagree that the rationality community contains a higher than normal share of lunatics. Whether it produces or attracts them is an open question.
>the issue with the community is they take their eccentric beliefs as being gospel truths
I am genuinely curious where you get that impression.
The central element to the whole thing is that people should be willing to question their own beliefs and change their mind based on the data.
If there is anything that turns rationalists into eccentrics, I it is being overly open to new ideas or beliefs.
