Harry Potter and the Methods of Rationality
⛰ What It's About
It's a Harry Potter fanfiction book written by Eliezer Yudkowsky, an AI researcher and decision theorist. In Yudkowsky's parallel universe, Petunia is married to a famous biochemistry professor at Oxford university and Harry is a nerdy genius kid who was brought up as a science-minded child. Harry goes on to explore the magic world from a rationalist's point of view; trying to apply scientific method to the study of magic. HPMOR adapts the story of Harry Potter to explain complex concepts in cognitive science, philosophy, and the scientific method.
🧠 Thoughts
The bystander effect
"The Dark Lord had raged upon wizarding Britain like a wilding wolf, tearing and rending at the fabric of their everyday lives. Other countries had wrung their hands but hesitated to intervene, whether out of apathetic selfishness or simple fear, for whichever was first among them to oppose the Dark Lord, their peace would be the next target of his terror." said professor Mcgonegal. (The bystander effect, thought Harry, thinking of Latane and Darley’s experiment which had shown that you were more likely to get help if you had an epileptic fit in front of one person than in front of three. Diffusion of responsibility, everyone hoping that someone else would go first.)
Preventing the bystander effect
-
If you're the observer
Simply being aware of this tendency is perhaps the greatest way to break the cycle. When faced with a situation that requires action, understand how the bystander effect might be holding you back and consciously take steps to overcome it. However, this does not mean you should place yourself in danger.
-
If you're the victim
One often recommended tactic is to single out one person from the crowd. Make eye contact and ask that individual specifically for help. By personalizing and individualizing your request, it becomes much harder for people to turn you down.
Fundamental attribution error
“Suppose you come into work and see your coworker kicking his desk. You think, ‘what an angry person he must be’. Your coworker is thinking about how someone pushed him into a wall on the way to work and then shouted at him. Anyone would be angry at that, he thinks. When we look at others we see personality traits that explain their behavior, but when we look at ourselves we see circumstances that explain our behavior. People’s stories make internal sense to them, from the inside, but we don’t see people’s histories trailing behind them in the air. We only see them in one situation, and we don’t see what they would be like in a different situation. So the fundamental attribution error is that we explain by permanent, enduring traits what would be better explained by circumstance and context.” McGonagall’s eyebrows drew up. “I think I understand...” she said slowly. “But what does that have to do with you?” “People think that I saved them from You-Know-Who because I’m some kind of great warrior of the Light. like I destroyed the Dark Lord because I have some kind of permanent, enduring destroy-the-Dark-Lord trait. I was fifteen months old at the time! I don’t know what happened, but I would guess it had something to do with, as the saying goes, contingent environmental circumstances. And certainly nothing to do with my personality. People don’t care about me, they aren’t even paying attention to me, they want to shake hands with a bad explanation.”
FAE explains why we often judge others harshly (by describing them as having bad traits) while letting ourselves off the hook at the same time by rationalizing our own unethical behavior.
As a simple example of the behavior which attribution error theory seeks to explain: consider the situation where Alice, a driver, is cut off in traffic by Bob. Alice attributes Bob's behavior to his fundamental personality, e.g., he thinks only of himself, he is selfish, he is a jerk, he is an unskilled driver; she does not think it is situational, e.g., he is going to miss his flight, his wife is giving birth at the hospital, his daughter is convulsing at school. On the flip side, if Alice cut Bob off in traffic, Alice will tend to convince herself that she had to do so. She'd focus on situational factors, like being late to a meeting, I am late for my job interview, I must pick up my son for his dental appointment and ignore what her behavior might say about her own character.
Planning Fallacy
“About why you’re convinced that you always have to be on your guard against terrible things happening to you.” “Well...” Harry said slowly. He tried to organize his thoughts. How could he explain himself to McGonagall, when she didn’t even know the basics? “Muggle researchers have found that people are always very optimistic, like they say something will take two days and it takes ten, or they say it’ll take two months and it takes over thirty-five years. Like, they asked students for times by which they were 50% sure, 75% sure, and 99% sure they’d complete their homework, and only 13%, 19%, and 45% of the students finished by those times. And they found that the reason was that when they asked people for their best-case estimates if everything went as well as possible, and their average-case estimates if everything went as normal, they got back answers that were statistically indistinguishable. See, if you ask someone what they expect in the normal case, they visualize what looks like the line of maximum probability at each step along the way—namely, everything going according to plan, without any mistakes or surprises. But actually, since more than half the students didn’t finish by the time they were 99% sure they’d be done, reality usually delivers results a little worse than the ‘worst-case scenario’. It’s called the planning fallacy, and the best way to fix it is to ask how long things took the last time you tried them. That’s called using the outside view instead of the inside view. But when you’re doing something new and can’t do that, you just have to be really, really, really pessimistic. Like, so pessimistic that reality actually comes out better than you expected around as often and as much as it comes out worse. It’s actually really hard to be so pessimistic that you stand a decent chance of undershooting real life. Like I make this big effort to be gloomy and I imagine one of my classmates getting bitten, but what actually happens is that the surviving Death Eaters attack the whole school to get at me. But on a happier note—” “Stop,” McGonagall said
Reciprocation Pressure
The same could be said of Draco’s clever use of reciprocation pressure for an unsolicited gift, a technique which Harry had read about in his social psychology books (one experiment had shown that an unconditional gift of $5 was twice as effective as a conditional offer of $50 in getting people to fill out surveys). Draco had made an unsolicited gift of a confidence, and now invited Harry to offer a confidence in return...
Positive Bias
“Now the way this game works,” said the boy, “is that you give me a triplet of three numbers, and I’ll tell you ‘Yes’ if the three numbers are an instance of the rule, and ‘No’ if they’re not. I am Nature, the rule is one of my laws, and you are investigating me. You already know that 2–4–6 gets a ‘Yes’. When you’ve performed all the further experimental tests you want—asked me as many triplets as you feel necessary—you stop and guess the rule, and then you can unfold the sheet of paper and see how you did. Do you understand the game? .......................................................................................................“Very well,” said the boy, “take the paper out and see how you did.” Hermione took the paper out of her pocket and unfolded it. Three real numbers in increasing order, lowest to highest. “What you’ve just discovered is called ‘positive bias’,” said the boy. “You had a rule in your mind, and you kept on thinking of triplets that should make the rule say ‘Yes’. But you didn’t try to test as many triplets as possible that should make the rule say ‘No’. In fact you didn’t get a single ‘No’, so ‘any three numbers’ could have just as easily been the rule. It’s sort of like how people imagine experiments that could confirm their hypotheses instead of trying to imagine experiments that could falsify them—that’s not quite exactly the same mistake but it’s close. You have to learn to look on the negative side of things, stare into the darkness. When this experiment is performed, only 20% of grownups get the answer right. And many of the others invent fantastically complicated hypotheses and put great confidence in their wrong answers since they’ve done so many experiments and everything came out like they expected.”
What Do I Know and How Do I Know It?
You know, Draco, just as the fundamental question of rationality is ‘What do I think I know and how do I think I know it?’, there’s also a cardinal sin, a way of thinking that’s the opposite of that. Like the ancient Greek philosophers. fThey had no clue what was going on, so they’d go around saying things like ‘All is water’ or ‘All is fire’, and they never asked themselves, ‘Wait a minute, even if everything is water, how could I possibly know that?’ They didn’t ask themselves if they had evidence which discriminated that possibility from all the other possibilities you could imagine, evidence they’d be very unlikely to encounter if the theory wasn’t true—”
Does the Perfect Crime Exist?
What do you think you know, and how do you think you know it? If you did commit the perfect crime, nobody would ever find out—so how could anyone possibly know that there weren’t perfect crimes? And as soon as you looked at it that way, you realized that perfect crimes probably got committed all the time, and the coroner marked it down as death by natural causes, or the newspaper reported that the shop had never been very profitable and had finally gone out of business…
Desensitization Therapy
“Oh, well, what happened was that Fred and George and I saw this poor small boy at the train station—the woman next to him had gone away for a bit, and he was looking really frightened, like he was sure he was about to be attacked by Death Eaters or something. Now, there’s a saying that the fear is often worse than the thing itself, so it occurred to me that this was a lad who could actually benefit from seeing his worst nightmare come true and that it wasn’t as bad as he feared—” Hermione sat there with her mouth wide open. “—and after we were done giving him all the candy I’d bought, we were like, ‘Let’s give him some money! Ha ha ha! Have some Knuts, boy! Have a silver Sickle!’ and dancing around him and laughing evilly and so on. I think there were some people in the crowd who wanted to interfere at first, but bystander apathy held them off at least until they saw what we were doing, and then I think they were all too confused to do anything. Finally he said in this tiny little whisper ‘go away’ so the three of us all screamed and ran off, shrieking something about the light burning us. Hopefully he won’t be as scared of being bullied in the future. That’s called desensitization therapy, by the way.”
Systematic desensitization is used in the field of clinical psychology to help many people effectively overcome phobias and other anxiety disorders. Examples include dog phobia and social anxiety.
Consequentialism
What you did was mean!” “I think the word you’re looking for is enjoyable, and in any case you’re asking the wrong question. The question is, did it do more good than harm, or more harm than good? If you have any arguments to contribute to that question I’m glad to hear them, but I won’t entertain any other criticisms until that one is settled. I certainly agree that what I did looks all terrible and bullying and mean, since it involves a scared little boy and so on, but that’s hardly the key issue now is it? That’s called consequentialism, by the way, it means that whether an act is right or wrong isn’t determined by whether it looks bad, or mean, or anything like that, the only question is how it will turn out in the end—what are the consequences.”
Consequentialism is an ethical theory that judges whether or not something is right by what its consequences are. For instance, most people would agree that lying is wrong. But if telling a lie would help save a person’s life, consequentialism says it’s the right thing to do.
Why Students Bully
“I have always wondered at how students bully each other,” sighed the voice. “How children make life difficult for themselves, how they turn their schools into prisons even with their own hands. Why do human beings make their own lives so unpleasant? I can give you a part of the answer, Padma Patil. It is because people do not stop and think before causing pain, if they do not imagine that they themselves could also be hurt, that they might also suffer from their own misdeeds. But suffer you will, oh, yes, Padma Patil, suffer you will, if you stay on this road."
Blinding (Masking)
“Law of science, Draco. First I tell you the theory and the prediction. Then you show me the data. That way you know I’m not just making up a theory to fit; you know that the theory actually predicted the data in advance. I have to explain this to you anyway, so I have to explain it before you show me the data.”
Blinding is a research strategy which aims to limit bias. In a blind or blinded experiment, information which may influence the participants of the experiment is withheld (blinded) until after the experiment is complete. Good blinding can reduce or eliminate experimental biases that arise from a participants' expectations. A blind can be imposed on any participant of an experiment, including subjects, researchers, technicians, data analysts, and evaluators. Blinding researchers involves presenting the theory first, then interpreting data to see whether the theory checks out or not. Also, they are forbidden to know which group receives placebos. Blinding test subjects involves not telling them if they are receiving a placebo. In this way, subjects in the control and treatment groups experience the placebo effect equally. When both researchers and subjects are blinded, it is called a double-blind study.
Asch Conformity Experiments
The Asch conformity experiments were a series of psychological experiments conducted by Solomon Asch during the 1950s. The experiments revealed the degree to which a person's own opinions are influenced by those of groups. Asch found that people were willing to ignore reality and give an incorrect answer in order to conform to the rest of the group.
Seated in a room with the other participants, you are shown a line segment and then asked to choose the matching line from a group of three segments of different lengths. The experimenter asks each participant individually to select the matching line segment. On some occasions, everyone in the group chooses the correct line, but occasionally, the other participants unanimously declare that a different line is actually the correct match. So what do you do when the experimenter asks you which line is the right match? Do you go with your initial response, or do you choose to conform to the rest of the group?
Asch's experiments involved having people who were "in" on the experiment pretend to be regular participants alongside those who were actual, unaware subjects of the study. Those that were in on the experiment would behave in certain ways to see if their actions had an influence on the actual experimental participants.
In each experiment, a naive student participant was placed in a room with several other confederates who were "in on" the experiment. The naive subjects were told that they were taking part in a "vision test." All told, a total of 50 students were part of Asch’s experimental condition.
The confederates were all told what their responses would be when the line task was presented. The naive participant, however, had no inkling that the other students were not real participants. After the line task was presented, each student was verbally announced which line (either 1, 2, or 3) matched the target line.
There were 18 different trials in the experimental condition, and the confederates gave incorrect responses in 12 of them, which Asch referred to as the "critical trials." The purpose of these critical trials was to see if the participants would change their answer in order to conform to how the others in the group responded.
During the first part of the procedure, the confederates answered the questions correctly. However, they eventually began providing incorrect answers based on how they had been instructed by the experimenters.
The study also included 37 participants in a control condition. In order to ensure that the average person could accurately gauge the length of the lines, the control group was asked to individually write down the correct match. According to these results, participants were very accurate in their line judgments, choosing the correct answer 99% of the time.
After combining the trials, the results indicated that participants conformed to the incorrect group answer approximately one-third of the time. These results suggest that conformity can be influenced both by a need to fit in and a belief that other people are smarter or better informed.
At the conclusion of the experiments, participants were asked why they had gone along with the rest of the group. In most cases, the students stated that while they knew the rest of the group was wrong, they did not want to risk facing ridicule. A few of the participants suggested that they actually believed the other members of the group were correct in their answers.
Factors Affecting Conformity
- Conformity tends to increase when more people are present. However, there is little change once the group size goes beyond four or five people.
- Conformity also increases when the task becomes more difficult. In the face of uncertainty, people turn to others for information about how to respond.
- Conformity increases when other members of the group are of a higher social status. When people view the others in the group as more powerful, influential, or knowledgeable than themselves, they are more likely to go along with the group.
- Conformity tends to decrease, however, when people are able to respond privately. Research has also shown that conformity decreases if they have support from at least one other individual in a group.
Learn How to Lose
“The Dark Lord came to that school openly, without disguise, glowing red eyes and all. The students tried to bar his way and he simply Apparated through. There was terror there, but discipline, and the Master came forth. And the Dark Lord demanded—not asked, but demanded— to be taught.” Professor Quirrell’s face was very hard. “Perhaps the Master had read too many books telling the lie that a true martial artist could defeat even demons. For whatever reason, the Master refused. The Dark Lord asked why he could not be a student. The Master told him he had no patience, and that was when the Dark Lord ripped his tongue out.” There was a collective gasp. “You can guess what happened next. The students tried to rush the Dark Lord and fell over, stunned where they stood. And then…” “There is an Unforgiveable Curse, the Cruciatus Curse, which produces unbearable pain. If the Cruciatus is extended for longer than a few minutes it produces permanent insanity. One by one, the Dark Lord Crucioed the Master’s students into insanity, and then finished them off with the Killing Curse, while the Master was forced to watch. When all his students had died in this way, the Master followed. I learned this from the single surviving student, whom the Dark Lord had left alive to tell the tale, and who had been a friend of mine…” “Dark Wizards cannot keep their tempers,” Professor Quirrell saidquietly. “It is a nearly universal flaw of the species, and anyone who makes a habit of fighting them soon learns to rely on it. Understand that the Dark Lord did not win that day. His goal was to learn martial arts, and yet he left without a single lesson. The Dark Lord was foolish to wish that story retold. It did not show his strength, but rather an exploitable weakness.” Professor Quirrell’s gaze focused on a single child in the classroom. “Harry Potter,” Professor Quirrell said. “Yes,” Harry said, his voice hoarse. “What precisely did you do wrong today, Mr. Potter?” Harry felt like he was going to throw up. “I lost my temper.” “That is not precise,” said Professor Quirrell. “I will describe it more exactly. There are many animals which have what are called dominance contests. They rush at each other with horns—trying to knock each other down, not gore each other. They fight with their paws—with claws sheathed. But why with their claws sheathed? Surely, if they used their claws, they would stand a better chance of winning? But then their enemy might unsheathe their claws as well, and instead of resolving the dominance contest with a winner and a loser, both of them might be severely hurt.” Professor Quirrell gaze seemed to come straight out at Harry from the repeater screen. “What you demonstrated today, Mr. Potter, is that— unlike those animals who keep their claws sheathed and accept the results—you do not know how to lose a dominance contest. When a Hogwarts professor challenged you, you did not back down. When it looked like you might lose, you unsheathed your claws, heedless of the danger. You escalated, and then you escalated again. It started with a slap at you from Professor Snape, who was obviously dominant over you. Instead of losing, you slapped back and lost ten points from Ravenclaw. Soon you were talking about leaving Hogwarts. The fact that you escalated even further in some unknown direction, and somehow won at the end, does not change the fact that you are an idiot.” “The next time, Mr. Potter, that you choose to escalate a contest rather than lose, you may lose all the stakes you place on the table. I cannot guess what they were today. I can guess that they were far, far too high for the loss of ten House points.” “I would have taken the slap, waited, and picked the best possible time to make my move,” Harry said, his voice hoarse. “But that would have meant losing. Letting him be dominant over me. It was what the Dark Lord couldn’t do with the Master he wanted to learn from.” Professor Quirrell nodded. “I see that you have understood perfectly.”
The Robbers Cave Experiment
There was a legendary episode in social psychology called the Robbers Cave experiment. It had been set up in the bewildered aftermath of World War II, with the intent of investigating the causes and remedies of conflicts between groups. The scientists had set up a summer camp for 22 boys from 22 different schools, selecting them to all be from stable middle-class families. The first phase of the experiment had been intended to investigate what it took to start a conflict between groups. The 22 boys had been divided into two groups of 11——and this had been quite sufficient. The hostility had started from the moment the two groups had become aware of each others’ existences in the state park, insults being hurled on the first meeting. They’d named themselves the Eagles and the Rattlers (they hadn’t needed names for themselves when they thought they were the only ones in the park) and had proceeded to develop contrasting group stereotypes, the Rattlers thinking of themselves as roughand-tough and swearing heavily, the Eagles correspondingly deciding to think of themselves as upright-and-proper. The other part of the experiment had been testing how to resolve group conflicts. Bringing the boys together to watch fireworks hadn’t worked at all. They’d just shouted at each other and stayed apart. What had worked was warning them that there might be vandals in the park, and the two groups needing to work together to solve a failure of the park’s water system. A common task, a common enemy. Harry had a strong suspicion Professor Quirrell had understood this principle very well indeed when he had chosen to create three armies per year. Three armies. Not four. And definitely not segregated by House… except that no Slytherins had been assigned to Draco besides Mr. Crabbe and Mr. Goyle.
The Prisoner's Dilemma
The Prisoner’s Dilemma, according to Harry’s teachings, ran thus: Two prisoners had been locked in separate cells. There was evidence against each prisoner, but only minor evidence, enough for a prison sentence of two years apiece. Each prisoner could opt to defect and betray the other; and this would take one year off their own prison sentence but add two years to the other’s. Or a prisoner could cooperate, staying silent. So if both prisoners defected, each testifying against the other, they would serve three years apiece; if both cooperated, or stayed silent, they would serve two years each; but if one defected and the other cooperated, the defector would serve a single year, and the cooperator would serve four. And both prisoners had to make their decision without knowing the other one’s choice, and neither would be given a chance to change their decision afterward. Draco had observed that if the two prisoners had been Death Eaters during the Wizarding War, the Dark Lord would have killed any traitors. Harry had nodded and said that was one way to resolve the Prisoner’s Dilemma—and in fact both Death Eaters would want there to be a Dark Lord for exactly that reason. (Draco had asked Harry to stop and let him to think about this for a while before they continued. It had explained a lot about why Father and his friends had agreed to serve under a Dark Lord who often wasn’t nice to them…) In fact, Harry had said, this was pretty much the reason why people had governments—you might be better off if you stole from someone else, just like each prisoner would be individually better off if they defected in the Prisoner’s Dilemma. But if everyone thought like that, the country would fall into chaos and everyone would be worse off, like what would happen if both prisoners defected. So people let themselves be ruled by governments, just like the Death Eaters had let themselves be ruled by the Dark Lord. Harry had continued afterward, the fear of a third party punishing you was not the only possible reason to cooperate in the Prisoner’s Dilemma. Suppose, Harry had said, you were playing the game against a magically produced identical copy of yourself. Harry had nodded again, and said that this was yet another solution to the Prisoner’s Dilemma—people might cooperate because they cared about each other, or because they had senses of honor, or because they wanted to preserve their reputation. Indeed, Harry had said, it was rather difficult to construct a true Prisoner’s Dilemma—in real life, people usually cared about the other person, or their honor or their reputation or a Dark Lord’s punishment or something besides the prison sentences. But suppose the copy had been of someone completely selfish— (Pansy Parkinson had been the example they’d used) —so each Pansy only cared what happened to her and not to the other Pansy. Given that this was all Pansy cared about… and that there was no Dark Lord… and Pansy wasn’t worried about her reputation… and Pansy either had no sense of honor or didn’t consider herself obligated to the other prisoner… then would the rational thing be for Pansy to cooperate, or defect? Some people, Harry said, claimed that the rational thing to do was for Pansy to defect against her copy, but Harry, plus someone named Douglas Hofstadter, thought these people were wrong. Because, Harry had said, if Pansy defected—not at random, but for what seemed to her like rational reasons—then the other Pansy would think exactly the same way. Two identical copies wouldn’t decide different things. So Pansy had to choose between a world in which both Pansies cooperated, or a world in which both Pansies defected, and she was better off if both copies cooperated. And if Harry had thought ‘rational’ people did defect in the Prisoner’s Dilemma, then he wouldn’t have done anything to spread that kind of ‘rationality’, because a country or a conspiracy full of ‘rational’ people would dissolve into chaos. You would tell your enemies about ‘rationality’.
Wisdom VS Intelligence
Well, sounding wise wasn’t difficult. It was a lot easier than being intelligent, actually, since you didn’t have to say anything surprising or come up with any new insights. You just let your brain’s patternmatching software complete the cliche, using whatever Deep Wisdom you’d stored previously.
Cognitive Dissonance
The old wizard’s face was peaceful. “I am not perfect, Harry, but I think I have accepted my death as part of myself.”“ Uh huh,” Harry said. “See, there’s this little thing called cognitive dissonance, or in plainer English, sour grapes. If people were hit on the heads with truncheons once a month, and no one could do anything about it, pretty soon there’d be all sorts of philosophers, pretending to be wise as you put it, who found all sorts of amazing benefits to being hit on the head with a truncheon once a month. Like, it makes you tougher, or it makes you happier on the days when you’re not getting hit with a truncheon. But if you went up to someone who wasn’t getting hit, and you asked them if they wanted to start, in exchange for those amazing benefits, they’d say no. And if you didn’t have to die, if you came from somewhere that no one had ever even heard of death, and I suggested to you that it would be an amazing wonderful great idea for people to get wrinkled and old and eventually cease to exist, why, you’d have me hauled right off to a lunatic asylum! So why would anyone possibly think any thought so silly as that death is a good thing? Because you’re afraid of it, because you don’t really want to die, and that thought hurts so much inside you that you have to rationalize it away, do something to numb the pain, so you won’t have to think about it—” “No, Harry,” the old wizard said. His face was gentle, his hand trailed through a lighted pool of water that made small musical chimes as his fingers stirred it. “Though I can understand how you must think so.” “Do you want to understand the Dark Wizard?” Harry said, his voice now hard and grim. “Then look within the part of yourself that flees not from death but from the fear of death, that finds that fear so unbearable that it will embrace Death as a friend and cozen up to it, try to become one with the night so that it can think itself master of the abyss. You have taken the most terrible of all evils and called it good! With only a slight twist that same part of yourself would murder innocents, and call it friendship. If you can call death better than life then you can twist your moral compass to point anywhere—”
Denial of Loss
There comes a point in every plot where the victim starts to suspect; and looks back, and sees a trail of events all pointing in a single direction. And when that point comes, Father had explained, the prospect of the loss may seem so unbearable, and admitting themselves tricked may seem so humiliating, that the victim will yet deny the plot, and the game may continue long after.
Scope Insensitivity
Context: Harry discusses with Hermione whether plants are sentient or not. “Look, it’s a question of multiplication, okay? There’s a lot of plants in the world, if they’re not sentient then they’re not important, but if plants are people then they’ve got more moral weight than all the human beings in the world put together. Now, of course your brain doesn’t realize that on an intuitive level, but that’s because the brain can’t multiply. Like if you ask three separate groups of Canadian households how much they’ll pay to save two thousand, twenty thousand, or two hundred thousand birds from dying in oil ponds, the three groups will respectively state that they’re willing to pay seventy-eight, eighty-eight, and eighty dollars. No difference, in other words. It’s called scope insensitivity. Your brain imagines a single bird struggling in an oil pond, and that image creates some amount of emotion that determines your willingness to pay. But no one can visualize even two thousand of anything, so the quantity just gets thrown straight out the window. Now try to correct that bias with respect to a hundred trillion sentient blades of grass, and you’ll realize that this could be thousands of times more important than we used to think the whole human species was."
Nate Soares on Scope Insensitivity:
I'm not very good at feeling the size of large numbers. Once you start tossing around numbers larger than 1000 (or maybe even 100), the numbers just seem "big". Consider Sirius, the brightest star in the night sky. If you told me that Sirius is as big as a million earths, I would feel like that's a lot of Earths. If, instead, you told me that you could fit a billion Earths inside Sirius… I would still just feel like that's a lot of Earths. The feelings are almost identical. In context, my brain grudgingly admits that a billion is a lot larger than a million, and puts forth a token effort to feel like a billion-Earth-sized star is bigger than a million-Earth-sized star. But out of context — if I wasn't anchored at "a million" when I heard "a billion" — both these numbers just feel vaguely large.
Seeking Validation
“The more you try to justify yourself to people like that, the more it acknowledges that they have the right to question you. It shows you think they get to be your inquisitor, and once you grant someone that sort of power over you, they just push more and more.” This was one of Draco Malfoy’s lessons which Harry had thought was actually pretty smart: people who tried to defend themselves got questioned over every little point and could never satisfy their interrogators; but if you made it clear from the start that you were a celebrity and above social conventions, people’s minds wouldn’t bother tracking most violations. “That’s why when Ron came over to me as I was sitting down at the Ravenclaw table, and told me to stay away from you, I held my hand out over the floor and said, ‘You see how high I’m holding my hand? Your intelligence has to be at least this high to talk to me.’
How to Keep a Secret 101
It could do a lot of harm if that secret got out! Haven’t you heard the saying, three can keep a secret if two are dead? That telling just your closest friends is the same as telling everyone, because you’re not just trusting them, you’re trusting everyone they trust.
Selfishness
“It is very simple, Mr. Potter, to understand how Azkaban was built, and how it continues to be. Men care for what they, themselves, expect to suffer or gain; and so long as they do not expect it to redound upon themselves, their cruelty and carelessness is without limit. All the other wizards of this country are no different within than he who sought to rule over them, You-Know-Who; they only lack his power and his… frankness.”
The One VS the Many
Those metal doors were not the doors of individual cells, Professor Quirrell had said, each one opened into a corridor in which there would be a group of cells. Somehow that helped a little, not thinking that each door corresponded directly to a prisoner who was waiting right behind it. Instead there might be more than one prisoner, which diminished the emotional impact; just like the study showing that people contributed more when they were told that a given amount of money was required to save one child’s life, than when told the same total amount was needed to save eight children…
Befriend Ambiguity
“On our first day of class, you tried to convince my classmates I was a killer.” “You are.” said professor Quirrell. “But if your question is why I told them that, Mr. Potter, the answer is that you will find ambiguity a great ally on your road to power. Give a sign of Slytherin on one day, and contradict it with a sign of Gryffindor the next; and the Slytherins will be enabled to believe what they wish, while the Gryffindors argue themselves into supporting you as well. So long as there is uncertainty, people can believe whatever seems to be to their own advantage. And so long as you appear strong, so long as you appear to be winning, their instincts will tell them that their advantage lies with you. Walk always in the shadow, and light and darkness both will follow.”
Standard Scarcity Effect
If Harry hadn’t been told that he couldn’t leave Hogwarts, he probably would’ve jumped at the chance to spend more time in Hogwarts, he would’ve plotted and connived to get it. Hogwarts was literally optimal, not in all the realms of possibility maybe, but certainly on the real planet Earth, it was the Maximum Fun Location. How could the castle and its grounds seem so much smaller, so much more confining, how could the rest of the world become so much more interesting and important, the instant Harry had been told that he wasn’t allowed to leave? He’d spent months here and hadn’t felt claustrophobic then. You know the research on this, observed some part of himself, it’s just standard scarcity effects, like that time where as soon as a county outlawed phosphate detergents, people who’d never cared before drove to the next county in order to buy huge loads of phosphate detergent, and surveys showed that they rated phosphate detergents as gentler and more effective and even easier-pouring… and if you give two-year-olds a choice between a toy in the open and one protected by a barrier they can go around, they’ll ignore the toy in the open and go for the one behind the barrier… salespeople know that they can sell things just by telling the customer it might not be available… it was all in Cialdini’s book Influence, everything you’re feeling right now, the grass is always greener on the side that’s not allowed. If Harry hadn’t been told that he couldn’t leave, he probably would’ve jumped at the chance to stay at Hogwarts over the summer……but not the rest of his life.
Snowball Effect?
You know, said the part of him that refined his skills, didn’t you sort of ponder, once, how every different profession has a different way to be excellent, how an excellent teacher isn’t like an excellent plumber; but they all have in common certain methods of not being stupid; and that one of the most important such techniques is to face up to your little mistakes before they turn into BIG mistakes? …although this already seemed to qualify as a BIG mistake, actually… The point being, said his inner monitor, it’s getting worse literally by the minute. The way spies turn people is, they get them to commit a little sin, and then they use the little sin to blackmail them into a bigger sin, and then they use THAT sin to make them do even bigger things and then the blackmailer owns their soul. Didn’t you once think about how the person being blackmailed, if they could foresee the whole path, would just decide to take the punch on the first step, take the hit of exposing that first sin? Didn’t you decide that you would do that, if anyone ever tried to blackmail you into doing something major in order to conceal something little? Do you see the similarity here, Harry James Potter-Evans-Verres?
The Milligram Experiment
Stanley Milgram had done the Milligram experiment to investigate the causes of World War II, to try to understand why the citizens of Germany had obeyed Hitler. So he had designed an experiment to investigate obedience, to see if Germans were, for some reason, more liable to obey harmful orders from authority figures. First he’d run a pilot version of his experiment on American subjects, as a control. And afterward he hadn’t bothered trying it in Germany. Experimental apparatus: A series of 30 switches set in a horizontal line, with labels starting at ‘15 volts’ and going up to ‘450 volts’, with labels for each group of four switches. The first group of four labeled ‘Slight Shock’, the sixth group labeled ‘Extreme Intensity Shock’, the seventh group labeled ‘Danger: Severe Shock’, and the two last switches left over labeled just ‘XXX’. And an actor, a confederate of the experimenter, who had appeared to the true subjects to be someone just like them: someone who had answered the same ad for participants in an experiment on learning, and who had lost a (rigged) lottery and been strapped into a chair, along with the electrodes. The true experimental subjects had been given a slight shock from the electrodes, just so that they could see that it worked. The true subject had been told that the experiment was on the effects of punishment on learning and memory, and that part of the test was to see if it made a difference what sort of person administered the punishment; and that the person strapped to the chair would try to memorize sets of word pairs, and that each time the ‘learner’ got one wrong, the ‘teacher’ was to administer a successively stronger shock. At the 300-volt level, the actor would stop trying to call out answers and begin kicking at the wall, after which the experimenter would instruct the subjects to treat non-answers as wrong answers and continue. At the 315-volt level the pounding on the wall would be repeated. After that nothing would be heard. If the subject objected or refused to press a switch, the experimenter, maintaining an impassive demeanor and dressed in a gray lab coat, would say ‘Please continue’, then ‘The experiment requires that you continue’, then ‘It is absolutely essential that you continue’, then ‘You have no other choice, you must go on’. If the fourth prod still didn’t work, the experiment halted there. Before running the experiment, Milgram had described the experimental setup, and then asked fourteen psychology seniors what percentage of subjects they thought would go all the way up to the 450-volt level, what percentage of subjects would press the last of the two switches marked XXX, after the victim had stopped responding. The most pessimistic answer had been 3%. The actual number had been 26 out of 40. The subjects had sweated, groaned, stuttered, laughed nervously, bitten their lips, dug their fingernails into their flesh. But at the experimenter’s prompting, they had, most of them, gone on administering what they believed to be painful, dangerous, possibly lethal electrical shocks. All the way to the end. Harry could hear Professor Quirrell laughing, in his mind; the Defense Professor’s voice saying something along the lines of: Why, Mr. Potter, even I had not been so cynical; I knew men would betray their most cherished principles for money and power, but I did not realize that a stern look also sufficed. It was dangerous, to try and guess at evolutionary psychology if you weren’t a professional evolutionary psychologist; but when Harry had read about the Milgram experiment, the thought had occurred to him that situations like this had probably arisen many times in the ancestral environment, and that most potential ancestors who’d tried to disobey Authority were dead. Or that they had, at least, done less well for themselves than the obedient. People thought themselves good and moral, but when push came to shove, some switch flipped in their brain, and it was suddenly a lot harder to heroically defy Authority than they thought. Even if you could do it, it wouldn’t be easy, it wouldn’t be some effortless display of heroism. You would tremble, your voice would break, you would be afraid; would you be able to defy Authority even then? Harry blinked, then; because his brain had just made the connection between Milgram’s experiment and what Hermione had done on her firs day of Defense class, she’d refused to shoot a fellow student, even when Authority had told her that she must, she had trembled and been afraid but she had still refused. Harry had seen that happen right in front of his own eyes and he still hadn’t made the connection until now… Milgram had tried certain other variations on his test. In the eighteenth experiment, the experimental subject had only needed to call out the test words to the victim strapped into the chair, and record the answers, while someone else pressed the switches. It was the same apparent suffering, the same frantic pounding followed by silence; but it wasn’t you pressing the switch. You just watched it happen and read the questions to the person being tortured. 37 of 40 subjects had continued their participation in that experiment to the end, the 450-volt end marked ‘XXX’. And if you were Professor Quirrell, you might have decided to feel cynical about that. But 3 out of 40 subjects had refused to participate all the way to the end. The Hermiones. They did exist, in the world, the people who wouldn’t fire a Simple Strike Hex at a fellow student even if the Defense Professor ordered them to do it. The ones who had sheltered Gypsies and Jews and homosexuals in their attics during the Holocaust, and sometimes lost their lives for it. And were those people from some other species than humanity? Did they have some extra gear in their heads, some additional chunk of neural circuitry, which lesser mortals did not possess? But that was not likely, given the logic of sexual reproduction which said that the genes for complex machinery would be scrambled beyond repair, if they were not universal. Whatever parts Hermione was made from, everyone had those same parts inside them somewhere……well, that was a nice thought but it wasn’t strictly true, there was such a thing as literal brain damage, people could lose genes and the complex machine could stop working, there were sociopaths and psychopaths, people who lacked the gear to care. Maybe Lord Voldemort had been born like that, or maybe he had known good and yet still chosen evil; at this point it didn’t matter in the slightest. But a supermajority of the population ought to be capable of learning to do what Hermione and Holocaust resisters did. The people who had been run through the Milgram experiment, who had trembled and sweated and nervously laughed as they went all the way to pressing the switches marked ‘XXX’, many of them had written to thank Milgram, afterward, for what they had learned about themselves.
Contagious Lies
Harry ate another bite of his cereal, his eyes going distant now, no longer meeting her own. “Think of it this way: You skip school one day, and you lie and tell your teacher you were sick. The teacher tells you to bring a doctor’s note, so you forge one. The teacher says she’s going to call the doctor to check, so you have to give her a fake number for the doctor, and get a friend to pretend to be the doctor when she calls—” “You did what?” Harry looked up from his cereal then, and now he was smiling. “I’m not saying I really did that, Hermione…” Then his eyes abruptly dropped back down to his cereal. “No. Just an example. Lies propagate, that’s what I’m saying. You’ve got to tell more lies to cover them up, lie about every fact that’s connected to the first lie. And if you kept on lying, and you kept on trying to cover it up, sooner or later you’d even have to start lying about the general laws of thought. Like, someone is selling you some kind of alternative medicine that doesn’t work, and any double-blind experimental study will confirm that it doesn’t work. So if someone wants to go on defending the lie, they’ve got to get you to disbelieve in the experimental method. Like, the experimental method is just for merely scientific kinds of medicine, not amazing alternative medicine like theirs. Or a good and virtuous person should believe as strongly as they can, no matter what the evidence says. Or truth doesn’t exist and there’s no such thing as objective reality. A lot of common wisdom like that isn’t just mistaken, it’s anti-epistemology, it’s systematically wrong. Every rule of rationality that tells you how to find the truth, there’s someone out there who needs you to believe the opposite. If you once tell a lie, the truth is ever after your enemy; and there’s a lot of people out there telling lies—” Harry’s voice stopped.
Hesitation
Hessitation is always easy, rarely useful. So the Defense Professor had told him; and while you could quibble about the details of the proverb, Harry understood the weaknesses of Ravenclaws well enough to know that you had to try answering your own quibbles. Did some plans call for waiting? Yes, many plans called for delayed action; but that was not the same as hesitating to choose. Not delaying because you knew the right moment to do what was necessary, but delaying because you couldn’t make up your mind—there was no cunning plan which called for that. Did you sometimes need more information to choose? Yes, but that could also turn into an excuse for delaying; and it would be tempting to delay, when you were faced with a choice between two painful alternatives, and not choosing would avoid the mental pain for a time. So you would pick a piece of information you couldn’t easily obtain, and claim that you couldn’t possibly decide without it; that would be your excuse. Although if you knew what information you needed, knew when and how you would obtain that information, and knew what you would do depending on each possible observation, then that was less suspicious as an excuse for hesitating. If you weren’t just hesitating, you ought to be able to choose in advance what you would do, once you had the extra information you claimed you needed.
In a moral dilemma where you lost something either way, making the choice would feel bad either way, so you could temporarily save yourself a little mental pain by refusing to decide. At the cost of not being able to plan anything in advance, and at the cost of incurring a huge bias toward inaction or waiting until too late…
Seize the Opportunity!
Fighting bullies might not be the best way to become a heroine. But Father had once told her that the trouble with passing up opportunities was that it was habit-forming. If you told yourself you were waiting for a better opportunity next time, why, next time you’d probably tell yourself the same thing. Father had said that most people spent their whole lives waiting for an opportunity that was good enough, and then they died. Father had said that while seizing opportunities would mean that all sorts of things went wrong, it wasn’t nearly as bad as being a hopeless lump. Father had said that after she got into the habit of seizing opportunities, then it was time to start being picky about them.
Is It Too Good to be True?
“I’ve read more references to Nicholas Flamel,” Hermione said. “The Rise and Fall of the Dark Arts says he secretly trained Dumbledore to stand up to Grindelwald. There’s a lot of books that take the story seriously, not just this one… you think it’s too good to be true?” “No, of course not,” said Harry “The idea of ‘too good to be true’ isn’t causal reasoning, the universe doesn’t check if the output of the equations is ‘too good’ or ‘too bad’ before allowing it. People used to think that airplanes and smallpox vaccines were too good to be true.”
No Evidence: Extremism
“Problem one is that there’s no logical reason why the same artifact would be able to transmute lead to gold and produce an elixir that kept someone young. I wonder if there’s an official name for that in the literature? Like the ‘turned up to eleven effect’, maybe? If everyone can see a flower, you can’t get away with saying flowers are the size of houses. But if you’re in a flying saucer cult, since nobody can see the alien mothership anyway, you can say it’s the size of a city, or the size of the Moon. Observable things have to be constrained by evidence, but when somebody makes up a story, they can make the story as extreme as they want. So the Philosopher’s Stone gives you unlimited gold and eternal life, not because there’s a single magical discovery that would produce both of those effects, but because someone made up a story about a super happy thingy.”
Are All Children Innocent?
“I read once,” Harry said, his voice a bit unsteady as he tried to match deep-sounding words to deep-sounding words, “that it’s wrong to think of little children as innocent, because not knowing isn’t the same as not choosing. That children do little harms to each other with schoolyard fights, because they don’t have the power to do great harm. And some adults do great harm. But the adults who don’t, aren’t they more innocent than children, not less?”
