Lecture 5: Case study

[Disclaimer: These informal lecture notes are not intended to be comprehensive - there are some additional ideas in the lectures and lecture slides, textbook, tutorial materials etc. As always, the lectures themselves are the best guide for what is and is not examinable content. However, I hope they are useful in picking out the core content in each lecture.]


The lecture series up to this point has followed a pretty typical structure. After the introductory lecture, each of the three main "topic" lectures picked out a single aspect of cognition and discussed some of the key ideas relating to each of them:

Throughout, I've tended to avoid going into detail about the specific methodology used in each experiment, preferring to focus on core ideas and theories.

Obviously, these are only a few topics in cognitive psychology. Every textbook will cover many more topics, and later on in this class other lecturers will extend the topic list beyond these.

In any given one of these lectures, I've tried to highlight how it relates to many different tasks. For example, in the reasoning lecture I talked about the Wason selection task, property induction (inductive arguments), etc.

In today's lecture, I want to "reverse" this structure, picking a single research paper and talking about how it ties together different thoughts about attention, similarity and reasoning. In the process, we'll bring in some ideas about social cognition, learning and so on. In a somewhat self-serving move, I've selected one of my own papers that came out last year:

You aren't expected to read the original paper for this class, but if you're interested, I've included the link.


In our discussion of inductive reasoning we discussed the idea of premise monotonicity, which roughly corresponds to the very intuitive idea that "adding more facts should strengthen an argument"! As an example, this argument

Dolphin cells contain TH4 hormone
Therefore cow cells contain TH4 hormone

feels much weaker than this one:

Dolphin cells contain TH4 hormone
Mouse cells contain TH4 hormone
Bat cells contain TH4 hormone
Therefore cow cells contain TH4 hormone?

And as usual, we should note that this phenomenon isn't restricted to "arguments that are explicitly written as syllogisms", you can see something pretty similar in everyday arguments and conversations too. As a general rule, more evidence is better.

Except... not always. Consider this pair of arguments. As before, our "baseline" argument is this one:

Dolphin cells contain TH4 hormone
Therefore cow cells contain TH4 hormone

But this time around, the two "new" facts that we will introduce happen to pertain to other marine mammals:

Dolphin cells contain TH4 hormone
Whale cells contain TH4 hormone
Seal cells contain TH4 hormone
Therefore cow cells contain TH4 hormone?

Intuitively, this argument seems to feel much weaker than the original one. This phenomenon is referred to as premise non-monotonicity (adding more examples weakens the strength of the argument), and it tends to appear in situations where all of the premise items are all similar to another in a very particular way.

A theoretical perspective

Premise non-monotonicity is a rather curious thing. It appears to be a phenomenon in which all three of our lectures seem relevant: when a particular pattern of similarity appears (dolphin, whale, seal), it seems to call our attention to a particular category (marine mammals), and this in turn drives the way we reason about the situation. Here's what I mean. From the lecture slides, we have the following observations:

(1) Similarity shapes reasoning:

(2) Similarity directs attention:

Which leads to the question...

Why does this happen?

There are several possibilities for why this happens, but the one that we focused on in the Ransom et al (2016) paper is the idea that in everyday life arguments are made by other people, and other people have a vested interest in communicating their ideas to you. Unlike nature (which is dumb and unhelpful), humans are intelligent agents with complex goals and a rich language. We “transmit” information to each other via a complicated mechanism... we aim to persuade each other of the truth or falsity of different ideas. This has an effect on the evidentiary value of a "human uttered statement", when compared to the evidentiary value of a "fact sampled randomly from the world.

Here's how it works...

  1. Suppose I want to persuade you to believe that dugongs produce TH4 hormone.
  2. I might think well, if I choose these similar animals (dolphins & whales), I can assume that you are not stupid, and you will notice this similarity.
  3. I can also assume that you will notice that the common feature between them is "marine mammals".
  4. On your end of the "communication channel", you know that I know this. You can assume that I wanted you to think of marine mammals, so you can assume that this similarity is relevant, and so...
  5. You can take a hint.

... all of which highlights the importance of theory of mind. We We have intuitive theories about the workings of each other’s minds, so we can select relevant information that drives attention to the right answer.

It relies on an assumption that we are being helpful to one another. That is, it is perfectly reasonable human behaviour to tell as story like this....

“I’ve studied TH4 hormone for many years… and I have discovered it in the cells of whales, seals and dolphins. I want you to believe that dugongs will produce TH4 hormone"

It is also perfectly reasonable human behaviour to tell a story like this...

"I’ve studied TH4 hormone for many years… and I have discovered it in the cells of mice, bats and dolphins. I want you to believe that cows will produce TH4 hormone"

However, it violates the assumption of helpfulness if I were to try and tell a story about TH4 hormone that went like this...

"I’ve studied TH4 hormone for many years… and I have discovered it in the cells of whales, seals and dolphins. I want you to believe that kittens will produce TH4 hormone"

If I were to do this in real life and expected you to take me seriously, you would (quite rightly) conclude that I am a jerk. This idea has a lot of analogs. Here are a few examples:

So ultimately, the claim is that a lot of what's going on when we "let" the salient similarities that appear in someone's argument drive our attention to a particular category, and then go on to endorse that category is... we assume someone is trying to help us learn about the world.

An experimental test

Reseach goals

To generate an experimental test, we considered two different ways of constructing an argument, and we wanted to see what impact these might have on the premise monotonicity/non-monotonicity effect. Our two methods were:

Key procedural detail, and the dependent variable

We did this within the context of a reasoning task that used the following procedure:

Our hypotheses

More procedural information, and independent variables

One potential issue that we have in designing this study is the fact that people know it is a psychological experiment, and people know that psychological experiments are designed by psychologists... who aren't always trustworthy people! So it's not going to be easy to convince people that they're seeing "truly random" information.

To get around that, we designed a pretty "over the top" experiment to really drive home the idea. We did this by manipulating two independent variables.

  1. The cover story manipulation. When introducing the reasoning problem, we tried the obvious trick of just telling people about how the argument was (supposedly) generated. In the relevant cover story condition we told people that the facts were selected by a previous participant who wanted to help them make good guesses; in the neutral cover story we didn't say anything in particular about the origins of the argument;and in the random cover story condition we told people that the premise items were chosen at random from a data base of facts.

  2. The experience manipulation. To help ensure that people really did believe that the items were generated in a helpful way (relevant) or a random way, we also included a series of "filler" trials that looked exactly like the ones we actually cared about, but were designed to seem useful or useless. In the relevant filler condition, the arguments used in the filler items tended to be very convincing and consisted of highly relevant evidence (e.g., eagles + hawks -> doves seems to include useful evidence); whereas in the random filler condition the filler trials included some very unhelpful evidence (e.g., eagles + NOT tortoises -> doves)

The order of arguments was designed so that people tended to see the filler items before seeing the target items (see slide 61-62)

An incomplete 2x3 design

Taken together, these two manipulations yield four possible conditions:

Notice that there are two "logically possible" conditions that we did not include:

In most situations, it's considered good scientific practice to include all possible combinations of your independent variables, yet for this study we didn't. Why?

Hopefully the answer is obvious: those two conditions introduce a very nasty confound... namely that the cover story and the filler items contradict each other. We would end up lying to participants if we included these conditions, and participants might rightly conclude that we were being jerks.

We used two different "target" arguments that would be expected to produce premise non-monotonicity under normal circumstances...

Target argument #1

Grizzly bears produce TH-L2
Therefore, lions produce TH-L2


Grizzly bears produce TH-L2
Black bears produce TH-L2

Therefore, lions produce TH-L2

Target argument #2

The second target argument had roughly the same structure and the same logic to it. The single-premise version of the argument asked people to generalise from tigers -> ferrets and the two-premise version asked people to generalise from (tigers + lions) -> ferrets

Control argument

Finally, in order to check that the effect doesn't happen all the time, we included a control condition where it really shouldn't make much of a difference, because the second premise should always strengthen the argument. We did this with an argument where people first rated orangutan -> gorilla and then rated (orangutan + chimpanzee -> gorilla


Slides 70-78 provide a graphical illustration.

A digression

What do you think happened? How confident are you that I'm about to tell you that our predictions came true? You're probably very confident, right? Because... if our predictions didn't come true, I wouldn't be writing these lecture notes about a cool study... and you wouldn't have read all these preliminaries. The fact that I have set up the story like this - assuming that I am a helpful human trying to persuade you of something - strongly hints at what the answer is going to be, doesn't it? This whole set up is very clearly a Chekhov's gun and it's about to go off in the results section that will inevitable reveal that I was right in my predictions.

In contrast, how confident do you think I was when I ran the study originally? Nature doesn't supply us with Chekhov's guns, and my predictions don't always come true (unfortunately). So when we ran the study, I had no such guarantee that the results would tell a nice story... and I was not at all confident that it would work.

Oh, hey... that's actually our hypothesis right? Human reasoning when a knowledgeable and helpful informant (teacher!) presents you with an argument is qualitatively different from when an uncaring and indifferent universe generates observations (science!).

... the inevitable pattern of results

As shown on slides 80-83, we got exactly the pattern of results we predicted. Obviously we did, otherwise I wouldn't have written this lecture!

Sadly, not all my experiments work out that nicely.

... another digression ...