drafting "Hazards of Selection Effects"

diff --git a/hazards_of_selection_effects_on_approved_information.md b/hazards_of_selection_effects_on_approved_information.md
index 6d8736cfda3dd9e81a5437a85ecc8566b7f11e82..c250d23446c36d3b10c09bdb20a75cc802c8d0b4 100644 (file)
--- a/hazards_of_selection_effects_on_approved_information.md
+++ b/hazards_of_selection_effects_on_approved_information.md
@@ -20,26 +20,25 @@ But what if there were subtler manifestations of the problem, that _weren't_ obv
 
 > **Goofusia**: I don't see why you tolerate that distrustful witch Goody Osborne at your salon. Of course I understand the importance of criticism, which is an essential nutrient for any truthseeker. But you can acquire the nutrient without the downside of putting up with unpleasant people like her. At least, I can. I've already got plenty of perceptive critics in my life among my friends who want the truth, and know that I want the truth.
 >
-> **Gallantina**: But aren't your friends who know you want the truth selected for agreeing with you? If there _were_ some crushing counterargument to your beliefs that would only be found by someone who _didn't_ know that you want the truth, how would you ever hear about it?
+> **Gallantina**: But aren't your friends who know you want the truth selected for agreeing with you, over and above their being selected for being correct? If there _were_ some crushing counterargument to your beliefs that would only be found by someone who _didn't_ know that you want the truth, how would you ever hear about it?
 
-This one is subtle. You can't invite the whole world to your salon; you can't _not_ do some sort of filtering.
+This one is subtle. Goofusia isn't throwing a crying fit every time a member of the salon criticizes her ideas. And indeed, you can't invite the whole world to your salon. You can't _not_ do some sort of filtering. The question is whether salon invitations are being extended or withheld for "good" reasons (that promote the salon processing true and useful information) or "bad" reasons (that promote false or useless information).
 
-Goofusia isn't throwing a crying fit
+The problem is that being friends with Goofusia and "know[ing] that [she and other salon members] want the truth" is a bad membership criterion, not a good one, because people who aren't friends with Goofusia and don't know that she wants the truth are likely to have different things to say. Even if Goofusia can answer all the critiques her friends can think of, that shouldn't give her confidence that her ideas are solid if there are likely to be possible serious critiques that wouldn't be independently generated by the kinds of people who become Goofusia's friends.
 
+The "nutrient" metaphor is a tell. Goofusia seems to be thinking of criticism as if it were a homogenous ingredient necessary for a healthy epistemic environment, but that it doesn't particularly matter where it comes from. In analogy, it doesn't matter whether you get your allowance of potassium from bananas or potatoes or artificial supplements. If you find bananas and potatoes unpleasant, you can still take supplements and get the potassium that way.
 
+But unlike chemically uniform nutrients, criticism isn't homogenous: different critics are differently equipped to notice different flaws in a piece of work.
 
-The "nutrient" metaphor is a tell. Goofusia seems to be thinking of criticism as if it were a homogenous ingredient 
 
-
-
-[TODO: "know I want the truth" is a bad filter, Parliment debating the Duke's monopoly]
+[TODO: "know I want the truth" is a bad filter, salon bakery]
 
 
 ## Filtering Information Sources
 
-> **Goofusia**: You shouldn't have to be the sort of person who follows discourse in crappy filter-bubbles in order to understand what's happening. Samuel Parris's news summaries are the sort of thing that lets me do that. Our salon should work like that if it's going to talk about the atheist threat and the witchcraft crisis. I don't want to have to read the awful corners of the internet where this is discussed all day. They do truthseeking far worse there.
+> **Goofusia**: One shouldn't have to be the sort of person who follows discourse in crappy filter-bubbles in order to understand what's happening. The Rev. Samuel Parris's news summaries are the sort of thing that lets me do that. Our salon should work like that if it's going to talk about the atheist threat and the witchcraft crisis. I don't want to have to read the awful corners of the internet where this is discussed all day. They do truthseeking far worse there.
 >
-> **Gallantina**: But then you're turning your salon into a Samuel Parris filter bubble. Don't you want your salon members to be well-read?
+> **Gallantina**: But then you're turning your salon into a Rev. Parris filter bubble. Don't you want your salon members to be well-read? [TODO: followup question alluding to contamination frame]
 
 This one is similar.
 
@@ -50,7 +49,7 @@ The "awful", "crappy", and "do[ing] truthseeking far worse there" characterizati
 
 ## Suppressing Information Sources
 
-> **Goofusia**: I caught Goody Osborne distributing pamphlets quoting the honest and candid and vulnerable reflections of Samuel Parris on guiding his flock, in pretty obviously reasonable ways, and just trying to somehow twist that into maximum anger and hatred. It seems quite clear to me what's going on in both of the relevant pamphlets, and I think signal boosting both of them is a pretty clear norm violation in my culture.
+> **Goofusia**: I caught Goody Osborne distributing pamphlets quoting the honest and candid and vulnerable reflections of Rev. Samuel Parris on guiding his flock, in pretty obviously reasonable ways, and just trying to somehow twist that into maximum anger and hatred. It seems quite clear to me what's going on in both of the relevant pamphlets, and I think signal boosting both of them is a pretty clear norm violation in my culture.
 >
 > **Gallantina**: I read those pamphlets. They seemed like intellectually substantive satire of a public figure. If it's a norm violation to signal-boost satire of public figures, 
 
@@ -62,4 +61,4 @@ The conjunction of "somehow" and "it seems quite clear to me" is a tell.
 ## An Analogy to Reinforcement Learning From Human Feedback
 
 [TODO: you can't "just not" do RLHF and still have a competitive AI in the current paradigm, but you can try to be aware of problems with supervision; it's better if the CEO says, "Yes, we thought of that, too; we think it's fine because of these-and-such mitigations." If the CEO says, "Well, _I_ think our raters are great. Are you insulting our raters?", that does not inspire confidence
-]
\ No newline at end of file
+]