I still think this episode _looks bad for him_ and that he should have been _embarrassed_, because there should have been generalizable policy changes that could have prevented this class of error—perhaps, thinking more carefully about whether a paraphrase that makes a subject look unreasonable is truly a fair paraphrase before going to print with it. Potential sources deciding whether they should talk to the journalist are mostly worried about the risk that their views will end up being miscommunicated, not whose fault a miscommunication would be if it happened. Unlucky accidents do not impinge upon the innocence of one's soul under the moral law, but the moral law is unmercifully permissive about everyone else's right to _avoid unlucky people_.
-### Higher Innocence
+## Higher Innocence
And that's why accidentally misleading people should be embarrassing: because the desire to avoid such embarrassment motivates a counterfactual reduction in such incidents of people being misled, notwithstanding that it was a totally unintentional accident and the speaker is perfectly innocent in their soul.
In the same way, people don't need to know what it feels like to be me in order to speak sensibly about rejecting my own understanding of my beliefs and motives. They could claim that my stated beliefs and motives are inconsistent with my behavior (which they can see) while remaining agnostic about how it feels to be me (which they don't have access to). Such a claim [probably wouldn't be based on nothing at all](/blog/2022/May/plea-bargaining/); indeed, the very existence of persistent disagreements (when you'd expect honest rational agents to simply update on each other's evidence and converge on the correct probability) [suggests that hidden motives are common](/blog/2023/Aug/assume-bad-faith/).
-If I buy cookies for the [salon](https://en.wikipedia.org/wiki/Salon_(gathering)) from a bakery owned by my girlfriend in Australia (at a substantial markup), it kind of makes sense for other salon members to wonder if my choice might have been biased by nepotism rather than reflecting true optimization towards the goal of the salon having the best desserts. At the same time, my belief that I wasn't swayed by nepotism is something I can provide reasoning and evidence for: for example, I could show the other salon members the notes and spreadsheets I compiled while comparison shopping, which justify my belief that my girlfriend just happens to be the optimal dessert vendor for the salon.
+If I buy cookies for the [salon](https://en.wikipedia.org/wiki/Salon_(gathering)) from a bakery owned by my girlfriend in Australia (at a substantial markup for international shipping), it kind of makes sense for other salon members to wonder if my choice might have been biased by nepotism rather than reflecting true optimization towards the goal of the salon having the best desserts. At the same time, my belief that I wasn't swayed by nepotism is something I can provide reasoning and evidence for: for example, I could show the other salon members the notes and spreadsheets I compiled while comparison shopping, which justify my belief that my girlfriend just happens to be the optimal dessert vendor for the salon.
-Now, maybe the other salon members don't find my spreadsheets convincing, but it should be clear that the details of the spreadsheets at least give us _something to talk about_. It's not just a deadlock between an unfalsifiable accusation of self-deception and an unfalsifiable denial, because my catering decision (and therefore the beliefs and motives that led to it) was tangled up with empirical realities of the world. If my comparison shopping was thorough and my girlfriend's bakery is just the best, that makes my story about only wanting the best cookies for the salon more credible; if I didn't really shop around much, that makes my story less credible. The correctness of my understanding of my beliefs and motives is something I can rationally defend to skeptics in front of third parties; it's not something that needs to be assumed as a prerequisite for even talking at all.
+Now, maybe the other salon members don't find my spreadsheets convincing, but it should be clear that the details of the spreadsheets at least give us _something to talk about_. It's not a deadlock between an unfalsifiable accusation of self-deception and an unfalsifiable denial, because my catering decision (and therefore the beliefs and motives that led to it) was tangled up with empirical realities of the world. If my comparison shopping was thorough and documents the many ways in which my girlfriend's bakery is just the best, that makes my story about only wanting the best cookies for the salon more credible; if I didn't really shop around much, that makes my story less credible. The correctness of my understanding of my beliefs and motives is something I can rationally defend to skeptics in front of third parties; it's not something that needs to be assumed as a prerequisite for even talking at all.
## Why the Self-Understanding of Those Who Think It's Impossible Should Be Rejected
In light of these observations, I think Piper's apparently well-meaning advice is better understood as a negotiating threat: don't criticize us in this way, or we'll walk. Even if you privately think us self-decieved, you must not reveal that you think that, or we'll destroy the surplus of you being able to interact with us.
-If Piper doesn't think of herself as making a negotiating threat, I guess that implies that I'm rejecting her self-understanding? But, you know, I don't think I'm doing it maliciously, as an attack. I'm just applying my knowledge of game theory, human psychology, _&c._ to the words I see before me and observing, hey, this looks to me like a veiled negotiating threat intended to suppress a class of criticism; that's an interesting phenomenon to analyze in a blog post.
+If Piper doesn't think of herself as making a negotiating threat, I guess that implies that I'm rejecting her self-understanding? But, you know, I don't think I'm doing it maliciously. I'm just applying my knowledge of game theory, human psychology, _&c._ to the words I see before me and observing, hey, this looks to me like a veiled negotiating threat intended to suppress a class of criticism; that's an interesting phenomenon to analyze in a blog post.
But if you think I'm kidding myself about that, then I'm happy to talk about it.
Although I'm not any kind of superhuman stoic, I think I have developed a certain tolerance and even taste for criticism. The email-avoidance behavior only kicks in when I'm facing particularly heightened social risk when fighting with people whose opinions I have reason to care about. In contrast, being criticized by randos elsewhere on the internet is more amusing than anything else. It's flattering to be hated: it means that someone thinks you're important enough to be worth hating, rather than ignored by default (as almost everything must be in our information-rich, attention-poor world).
-The people I'm complaining about often point to /r/SneerClub as the epitome of "bad" speech that one shouldn't be so much as tempted to look at, but my main complaint about Sneer Club is that they're just not that smart and rarely come up with any really penetrating criticism. In theory, one might have hoped that a dedicated anti-fan club might be an indispensible source of valuable criticism, as people who hate you are motivated to search hard for all the things you're doing wrong that your friends like you too much to tell you about or even notice. Unfortunately, in practice they mostly just go for the cheap dunks. Thus, /r/SneerClub ends up being mostly pretty boring to me, although the novelty of having dedicated haters is worth enough passing interest that I did make sure to [submit one of my particularly sneerable-seemingly pieces there](https://old.reddit.com/r/SneerClub/comments/n44drl/sexual_dimorphism_in_yudkowskys_sequences_in/). (My submission got locked, but [someone else's got 36 karma](https://old.reddit.com/r/SneerClub/comments/nt37j1/the_rationalist_subcultures_resident_blanchardian/) before the mods decided to delist it because "it feels for once like an actual bullying thing to post".)
+The people I'm complaining about often point to /r/SneerClub (a group of "haters" of our local intellectual scene) as the epitome of "bad" speech that one shouldn't be so much as tempted to look at, but my main complaint about Sneer Club is that they're just not that smart and almost never come up with any really penetrating criticisms. In theory, one might have hoped that a dedicated anti-fan club might be an indispensible source of valuable criticism, as people who hate you are motivated to search hard for all the things you're doing wrong that your friends like you too much to tell you about or even notice. Unfortunately, in practice, they just go for the cheap dunks. Thus, /r/SneerClub ends up being mostly pretty boring to me, although the novelty of having dedicated haters is worth enough passing interest that I did make sure to [submit one of my particularly sneerable-seeming pieces there](https://old.reddit.com/r/SneerClub/comments/n44drl/sexual_dimorphism_in_yudkowskys_sequences_in/). (My submission got locked, but [someone else's got 36 karma](https://old.reddit.com/r/SneerClub/comments/nt37j1/the_rationalist_subcultures_resident_blanchardian/) before the mods decided to delist it because "it feels for once like an actual bullying thing to post".)
[TODO: told I'm a masochist, but Bayesomasochism is an important rationality practice]
Because what else could he do? Even if it stings to be so distrusted, it wouldn't make sense for him to angrily snap, "so you don't trust me?" She doesn't—but he knows that the reason she doesn't isn't an indictment of his soul, but rather has to do with the distribution of souls that look the same as him from the outside. In this world (and in the environment of evolutionary adaptedness), there are a lot of men who are happy to act however they need to in order to convince a woman to sleep with them, with no intention of being faithful to her. A man who does intend to be faithful can't ask for feedback because of the adverse selection problem where the fact of asking implies that it's less safe for her to answer in expectation (because there are a greater number of unfaithful men who could also use the answer).
-That is, unfaithful men face a greater fraction of possible worlds in which scrutiny turns up a dealbreaker, so scrutiny decreases the expected value of the relationship more for unfaithful men than faithful men. As a corollary, faithful men are willing to accept more scrutiny rather than insisting on being trusted right away. It would be better for both the woman and the faithful man if no scrutiny had to be applied, but the woman can't do that, because then an unfaithful man would take advantage of her.
+That is, unfaithful men face a greater fraction of possible worlds in which scrutiny turns up a dealbreaker, so scrutiny decreases the expected value of the relationship more for unfaithful men than faithful men. As a corollary, faithful men are willing to accept more scrutiny rather than insisting on being trusted right away. (Merely refraining from snapping would also be cheap for the unfaithful man; the expensive part is the opportunity cost of sticking around while being scrutinized and distrusted.) It would be better for both the woman and the faithful man if no scrutiny had to be applied, but the woman can't do that, because then an unfaithful man would take advantage of her.
----
-And that's why honest people are never touchy about the matter of being trusted. They know that in order to distinguish themselves from dishonest people, they have to pay a price that's too expensive for dishonest people to pay. Sometimes the price might be very expensive, but it's the only way things could be: the hidden Bayesian structure of the universe is not kind. Those who try to wriggle out of paying the price should understand what signal they're sending.
+And that's why honest people are never touchy about the matter of being trusted. They know that in order to distinguish themselves from dishonest people, they have to pay a price that's too expensive for dishonest people to pay. Sometimes the price might be very expensive, but it's the only way things could be. (The hidden Bayesian structure of the universe is not kind.) Those who try to wriggle out of paying the price should understand what signal they're sending.
projects / An_Algorithmic_Lucidity.git / commitdiff