Bob Long: If you're old enough to remember the King of Late Night, Johnny Carson - you may recall his skit Carnac the Magnificent. He pretended he could hold up a letter and tell everybody what it said without even opening it. While Johnny, of course, was using that skit as a spoof on the whole issue of Extra Sensory Perception but seriously there are a lot of people who've been doing studies on that issue of whether there are certain people who do have special psychic abilities. I’m Bob Long. We welcome you to another edition of Stats and Stories... It's a program where we look at the statistics behind the stories, and the stories behind the statistics. Our focus today is on the whole issue of ESP - Extra Sensory Perception. Stats and Stories Reporter Bethany Miller tells us ESP is one field of study where skepticism abounds.

Bethany Miller: There are some scientific subjects where clinical studies give us confidence in their findings. We've talked in previous Stats and Stories episodes about forensic scientists making great progress studying DNA. We've also discussed scientists developing medicines for specific types of diseases or learning that certain workplace chemicals cause cancer. One area that is difficult to study is whether certain people have Extra Sensory Perception - an ability that goes beyond their normal 5 senses. Miami University Professor Joe Johnson is a cognitive psychologist. He says parapsychology is a difficult field because the burden of proof is higher, given people's natural skepticism.

Joe Johnson: People who are doing visual perception and trying to determine how people perceive light or how they perceive features of a stimulus or different things in the visual world, you can ask people to report pretty faithfully what they're seeing. And those sorts of things are easier to investigate, I think. Something like parapsychology I think is a little more fuzzy. It's not as well defined exactly what the phenomenonmaybe or how we would know if we're successful in discovering them.

Miller: Rose Marie Ward is the Director of the Center for Enhancement of Learning and teaching at Miami. She agrees with Joe Johnson that parapsychology is outside the norm. She says there are people who think it would be nice if some individuals had that ability to see the future.

Rose Marie Ward: And since the vast majority of people have had some kind of sensory experience where they like felt like something was going to happen or knew something was going to happen, it feels a little bit believable. And so they hope that it's true. But there's not a lot of concrete evidence to show that those kinds of powers exist.

Miller: Ward says there are scientific methods for studying if someone has extra sensory powers, but the key is eliminating biases that could lead to the results you hope to find.

Ward: For example, I had a friend who thought every time he walked down a street, the streetlights would go off. And so every time it would happen, he's like 'look the light just went off. We walked under it and it went off.' That's a confirmation bias. Instead you should look for all of the times they don't go off and examine that.

Miller: Rose Marie Ward says there definitely are people who believe in ESP…and she points to the number of TV shows that deal with that topic. Joe Johnson says it's hard to say whether media representations are causing any damage to the field of parapsychology

Johnson: I think it can dilute the message. And I think it's harder not just for the scientist but especially for the average person to separate and to discriminate between is this something that was done in a scientific way, which is a more credible source of information, or is it something that's a little bit softer or a little bit more for entertainment if anything else.

Miller: For example, Johnson says how people perceive the color red isn't as contentious as studies trying to show that someone can see the future. Joe Johnson says a true scientific study requires good design and good control to rule out biases. For Stats and Stories, I'm Bethany Miller.

Long: Joining me on Stats and Stories for our discussion are Miami University Statistics Department Chair John Bailer, Media, Journalism and Film Chair Richard Campbell. And our special guest today: Jessica Utts - professor and chair of the Department of Statistics at Cal Irvine, and she has spent many, many years investigating data from parapsychology, which is the scientific study of possible psychic abilities. And Jessica we welcome you to the show today.

Jessica Utts: Thank you; it’s a pleasure to be here.

Long: Well for those who don’t know, we’ll start off with what we mean by Extra Sensory Perception. Also I’m just kind of interested, along with that, with how you got interested in this whole area of study.

Utts: Extra Sensory Perception generally, first of all, it’s a misnomer because we don’t know that there’s an extra sense and there probably is not. But it basically refers to the ability to have access to information that doesn’t come through or normal five known senses, however that might happen.

Long: How did you get interested in this?

Utts: I actually was- I kind of had a life-long interest in it anyway. I was an undergraduate math and psychology major, quite interested in all aspects of psychology and then I had a fortuitous circumstance when I was on my first sabbatical at Stanford University; I ended up meeting these people that were doing this research for the Federal government, research on ESP for the Federal government and they needed statistical help and I became their statistical consultant and eventually got more and more involved in the work they were doing and the rest is history, as they say.

Long: John Bailer.

John Bailer: So how was the government using, what kind of studies was the government doing with respect to this?

Utts: The government actually had a scientific program and then a classified program where they were using people to psychically spy on our enemies actually. And so that was very interesting. I got involved in the scientific part of it where they were trying to figure things like, is it real, first of all, obviously. And if so, how does it work and can it be taught and can our enemies be using it against us? Is there any way to block it if somebody else is trying to use it to get information from us?

Richard Campbell: What is psychic spying? And I know this was an actual government program for like fifteen years, right?

Utts: Yeah, our government, for a long time, they had a group of people who they would have basically try to use their own psychic abilities to gain information about various things that were of interest to the government. So it might be we have a satellite image of a facility in the Soviet Union and we want to know what’s going on inside. Or we have somebody who’s been taken hostage and we want to know where he is; those kinds of things.

Long: John Bailer.

Bailer: So when you looked at these types of studies, you want to then gather data and try to evaluate this. So to formally evaluate and assess whether these abilities exist, could you talk about what would be a gold standard of the type of experiment that might be conducted to investigate this phenomena?

Utts: Absolutely, so the kinds of things I talked about where we were trying to gain information about our enemies and so on, those are not scientific experiments. Obviously, anecdotes are not scientific experiments. And the main reason that anecdotes can’t count as scientific experiments is that we don’t have any way to assess what just happens in a daily basis; coincidences are quite common and just by chance and so on. So there are two things you need to do a good scientific study. One is you need to close all the loopholes for how people might have access to information in other ways through cheating or texting or whatever, subtle cues even. If you’re doing an experiment, you could never have somebody in the same room who knows the answer to the question you’re trying to get a psychic to answer because they could provide subtle clues. So the first thing is you need this kind of ruling out any means of ordinary communication. And the second criterion for a statistical evaluation is that you need to be able to assess what should happen by chance alone because if you can’t do that, you can’t do a statistical assessment.

Long: It seems to me that this field is a little different than a lot of other scientific fields. You look at forensic science and they’re always looking for the best answers on fingerprints and DNA and things like this, but there’s always this doubt. Some people, as John pointed out, are strong believers in this and others are really skeptical, so does that make it more difficult sometimes when you’re trying to experiments with people?

Utts: It makes it very difficult for the science to get a fair hearing because most people frankly, are not interested in what the science has to say because they’ve already made up their mind. Almost everybody already has an opinion on whether or not this stuff is real and the people who are convinced it’s real are just as problematic as the people who are convinced there’s no way it can be real because neither side really wants to look at data; they think they already know the answer.

Long: It struck me, and you just confirmed that it’s a place where people have very strong opinions one way or the other.

Utts: Right.

Long: Richard.

Campbell: So in your studies, can you talk about, so what did you find? I mean do people have psychic abilities? Were you able to show that from the research that you did?

Utts: My bottom line is if we were to treat this area like we treat other areas of science, we would be totally convinced that psychic abilities are real. There’s so much evidence out there, the statistical evidence, we look at things that are called p-values, the p-values are so tiny here that if it’s only chance…suppose that there really is no such thing, the probability of getting the results that we’ve seen so far just by chance are astronomically small. So there really is a lot of strong statistical evidence. On the other hand, it’s obviously not a strong effect otherwise we would all know it existed, so it’s a weak effect, just like things like the effect of aspirin on preventing heart attacks is a weak effect, you don’t see that in everyday life, you can’t just see that by people walking around on the street. So it’s a weak effect, but it’s consistently statistically sound.

Long: You’re listening to Stats and Stories, where we always talk about the statistics behind the stories, and the stories behind the statistics...and today we are talking about this whole field of parapsychology. I’m Bob Long. Our regular panelists today: Miami University Statistics Department Chair John Bailer and Media, Journalism and Film Chair Richard Campbell... and our special guest today: Jessica Utts - professor and chair of the Department of Statistics at Cal Irvine. And as we said, she has spent a number of years investigating this whole field. We also wanted to find out what people on the street know about our topic...so one of the things we asked them was what exactly is Extra Sensory Perception?

Woman on the street #1: I think it's when people can sense others' feelings or people sense things that aren't within the five normal senses, like to the extreme - sensing like a presence that you wouldn't see with the human eye.

Man on the street #1: Maybe like if you were like blindfolded and had like three different drinks and you were able to - based on taste - determine what those drinks were.

Man on the street #2: I just thought of it as reading the future or anything besides like the 5 senses. I'm serious.

Woman on the street #2: I think it's a lot linked to spirituality and what you believe, and if you believe in ghosts and you see something...a shadow.

Woman on the street #3: Honestly, I have no idea what that means, but I feel like it would have to do with an extra sense.

Man on the street #3: If someone is more than perceptive than average and that their like sensory threshold is lower than that of a normal person.

Woman on the street #4: Something that doesn't use like the five basic senses but it's still something that's in your head. I don't know how to explain it but that's like as close as I can get.

Long: John Bailer, we’ll go to you for the next question.

Bailer: You know, one of the things in an earlier conversation that we’d had, you mentioned a variety of other ways that this phenomena is described, some of the other terms that are used. Could you go through some of that list and help us see what makes sense to you when you describe it and how you characterize the phenomena?

Utts: Sure, so a sort of neutral one was supposed to be just the Greek letter Psi, P-s-i, that’s used, but one of the ones they used in the government work was anomalous cognition with the idea that there’s something going on that we can’t explain about cognition. More recently people having been using the term non-local consciousness to indicate that there is something going on here that has to do with our consciousness, but not something where you’re just sort of locally getting the information.

Long: Richard.

Campbell: Could you talk about kind of walk us through what one of your experiments is like?

Utts: So a very simple experiment would be somebody- so first of all, let me start, before the experiment is done, you have to prepare what’s called a target pool, a pool of possibilities for what you’re going to try to have somebody guess. So let’s say you have a collection of 200 photographs, that’s what you’re going to use as your target pool and to make it more concrete, let’s say you have 200 photographs taken from a magazine like National Geographic. So you have these 200 hundred photographs, what the experiment, the way it proceeds is that you randomly select one of the photographs and that’s going to be the so-called target and the person who’s going to try to use their abilities to do this is simply told, there’s a photograph, I’m going to show it to you later and I want you to describe it now. So that’s the experiment, the person describes it however they can, they maybe draw a picture of what they think it might be, they talk, and so on. At the end of all of that, you then take the actual photograph that was the target and three others that also were randomly selected, or collected from the pool, not randomly selected, so you start out with a set of four photographs that are as different as possible, you randomly select one to be the target, then when you’re done with the experiment, you show the person the four possibilities and ask them, which one of these do you think you were trying to describe? And so by chance, they should get it right one out of four times because if the target’s really randomly selected from the four choices, no matter what they say, the one that’s going to match best with a probability of one-fourth. So then you can do a statistical assessment of that.

Long: John Bailer.

Bailer: So this touched on the two areas that you had talked about earlier, like closing the loopholes or what information people might have and also having some sense of what you would expect to occur by chance alone, so it sounds like that does describe well that experiment, in terms of capturing it. You’ve also mentioned in previous conversations the importance of prior belief in an analysis. So if somebody is approaching a data analysis with a very strong skeptical view, they might be looking at data in a different way than somebody that has a more sympathetic or open-minded view to this. Can you talk a little bit about some of the strategies that you use when you’re involved in analyzing these data?

Utts: First of all, there’s a famous quote, and we’re not quite sure where it originated, “Extraordinary claims require extraordinary evidence.” So even if we use standard, frequentist, statistical methods, which are things like what we call p-values or confidence intervals, people who are strong deniers while require extraordinary evidence and that actually would require a long time to collect when you have something that has a relatively small effect size. More recently, people have been using what’s called Bayesian analysis. Bayesian analysis is a method of statistics that allows you to build in your prior beliefs and so that’s kind of cool because you can actually look at the data and combine it with your prior beliefs and then come out with what you’re left with after you combine those two things. And ideally, the data will move people’s beliefs in one direction or the other, I’m not sure that always happens because some people may start with a zero prior belief, but if they don’t and if they’re honest about it, that’s what should happen.

Bailer: Let me follow up then, just to clarify. So let’s think about our skeptic, you said in the set-up of the experiment that you just described, you said well if they’re just guessing the match, it’d be about a one in four chance of guessing the match, so what would a skeptic’s prior belief, how would that be expressed with respect to that?

Utts: Okay, well with that, first of all, let me differentiate between two types of people. A skeptic and what I’ll call a denier. A denier essentially has zero prior belief probability that this could be real, so let’s ignore them for now. Let’s just go with a true skeptic who has an open mind and will allow some possibility for this to be real. So for that person, they probably would start out with the idea that the probability of a match is truly one-fourth, but that’s not the only value they’ll consider, so they may consider a range of possibilities around one-fourth, centered on one-fourth, but they’re open to it being something higher than that.

Long: Richard.

Campbell: You talked about Bayesian statistics. So are there statisticians who push back on this, incorporating prior belief systems in this kind of research.

Utts: So Bayesian statistics is kind of a tricky area of statistics. It gets a little complicated because there are various ways to incorporate your prior belief.

Campbell: Yes.

Utts: One is just to come up with a quantitative, do you believe this or not, and if so, what’s your probability that it’s real? That part everybody can understand, but then there’s a second question that has to be asked and that is if it is real, how large is the effect? And that’s where you get into some controversy and if you do the analysis wrong, then you, well I think one way of doing the analysis wrong is for somebody to say if it’s real then it’s a large effect. We all know that’s not true or we wouldn’t be having a debate, right, if it was real and it was a large effect we would all know that is true. If we do a Bayesian analysis where you compare the data, you ask the data to decide between two things: it’s real and it’s a large effect or it’s not real, of course the data is going to support that it’s not real. If more realistically you ask the data, is it real and a small effect or is it not real, it turns out that the accumulated data actually supports that it’s real but a small effect.

Long: You’ve been involved, I know because of the years in your research, you’ve been in a lot of media related shows, correct? And talk to me a little bit about some of the issues there, what people like Larry King and others want to know about this whole field and what you’re really trying to prove, what the differences may be and how the media perceives it and how you perceive it.

Utts: I’ll tell you what they don’t want, and that is statistics. They want gee-whiz stories, anecdotes, they want large effects, that sort of thing. But the fun part is that I think if I can just get across the message that there is something science needs to take a look at here, there is something very important going on. Maybe it’s that we really don’t use statistics appropriately in general because we’re using the same methods that are used in statistics in general and we’re finding these effects, so maybe it’s that. Or more realistically, I think, is that we don’t understand physics and consciousness and there’s a little crack in the cosmic egg, as there was a title of a book a long time ago, telling us that there’s something going on here that we really need to take seriously, beyond just the statistical stuff.

Long: Richard, I know you had a question.

Campbell: So, I want to follow up on that because I know one of the things, given your involvement in media, you see stories that journalists do that are about data, about statistics. Talk a little bit about common errors that you see journalists make, I mean this is part of, I think what we’re trying to do here is figure out how to help journalists tell stories that get the data right, or explain it better to a general public.

Utts: Right, so nothing to do with the ESP research because this is not an issue there, in general, one of the things that journalists get wrong is the whole idea of causation versus correlation. If you do what’s called an observational study, where you simply observe what’s going on, you cannot make a cause and effect conclusion. I’ll give you an example that was in the news within the past few years. A study had showed that coffee drinkers actually seemed to have longer life spans than non-coffee drinkers, and some of the headlines that reported that study said things like, “Drink coffee, live longer.” So if you want to live longer, start drinking coffee. The idea being that you could alter this with a cause and effect relationship, whereas in fact, obviously they did not randomly assign people to drink coffee or not and so there are all kinds of what we call confounding variables that could have been part of that that perhaps for instance, people who are already not as healthy stop drinking coffee and so therefore they died younger and so forth. So that’s one, correlation versus causation, that’s probably the biggest one.

Long: You’re listening to Stats and Stories, and today we are talking about parapsychology, the topic of ESP - Extra Sensory Perception. I’m Bob Long. Miami University Statistics Department Chair John Bailer and Media, Journalism and Film Chair Richard Campbell are with us as always. Our special guest today is Jessica Utts - professor and chair of the Department of Statistics at Cal Irvine. We wanted to find out if people know someone who seems to have an ability to predict events before they happen.

Woman on the street #5: People think they can if they have a situation that's happened to them before and they - I don't know - assume that it might happen again. But I don't think that people actually have ESP.

Man on the street #4: I don't know anyone who may have that, and I honestly don't know what to think if that could exist.

Woman on the street #6: I don't know. I know some of my friends are into the coffee grind thing to predict the future. And to me it's a bunch of hocus pocus. I don't really believe in that.

Man on the street #5: Not really. I mean I kind of feel like it's a carnival trick. But I've never, I feel like it's mostly coincidences and not that people are actually psychic or anything like that.

Man on the street #6: I don't know if people have ESP, but I would say some people have greater intuition than others where they have a feeling that something is going to happen. But I wouldn't say that it's like a psychic thing.

Long: Okay, so I’ve got this question. Suppose you have someone out there who thinks that they do have some kind of psychic ability to see things that other people, or sense things before other people would and if they’re curious about that, is there any way they can find out whether there’s something different about them compared with others?

Utts: That’s a good question. There are actually some online websites and people actually are developing apps now to do this. One website people might check out is gotpsi.org, it’s g-o-t-p-s-i.org and you can go on there and you can actually try some tests for yourself.

Bailer: Have you done it?

Utts: I actually have not because you have to register and give your email address and I don’t want my colleagues to find out that I can’t do this stuff.

Long: I’m also curious, when people think about this field, there’s so many things out there. They think of the traditional psychics and then they also think about a lot of the unsolved mysteries, unsolved crimes where a family member or a detective calls in somebody who supposedly has special psychic powers, and I’m just kind of curious your feeling about all of that.

Utts: I think that, of course that gets a lot of media attention and there probably is something there, if psychic abilities are real, there’s no reason to suspect they wouldn’t work in those kinds of setting, but where I do sort of discouraged, I guess is the right word, is some of the really unscientific kinds of things that this gets connected with. You know like, UFOs and those kinds of things. People somehow lump all this stuff together and that’s not what this is about.

Long: That’s kind of my point. It seems like they throw in a lot of things that really are not at all what you’re trying to do. John Bailer.

Bailer: Well I think that when you talk about the UFO phenomena versus this, it seems like this issue of can you experimentally evaluate it.

Utts: Right!

Bailer: So that’s one aspect that fundamentally differentiates this, it seems, from the other phenomena that you just described, so it’s testable. The other things that aren’t testable, what are you going to do?

Utts: Exactly right, in fact, I said one of the things we need in order to do statistics at all is we know what should happen by chance so if you can set up an experiment where you know what should happen by chance, then you can use statistics, but I don’t know how you would do that with a UFO sort of hypothesis.

Campbell: So say there is an unsolved mystery and somebody approaches the police department and says, “I have psychic ability; I can find this person.” Could the police come to you and say, can you test and see whether this person has psychic ability? And have you ever been asked to do that?

Utts: I have not been asked to do that and it’s kind of tricky because in solving police crimes, again, you’re not really setting up something where you know what should happen by chance. So there have been cases where people have actually been accused of the crime because they seemed to have information so that’s a very tricky business to get into. There is a guy who worked for the Federal government named Joe McMoneagle who actually does this now for a living and he works with companies and so on and he has been part of a Japanese television show where he has been brought over there several times to find missing people. So they’ll give him some information and he’ll actually psychically find these people and he seems to be pretty good at it, but it’s not a kind of a formal scientific test.

Long: John Bailer.

Bailer: Are there certain groups where experimentally have been demonstrated to have greater psychic abilities than others, and how was this demonstrated, if there are?

Utts: Yes, indeed. So some of the experiments have been done on people who you would consider to be more creative than the rest of us mere mortals. One of them was done at the Julliard School of Music and they actually did much better than the general population. The experiments I described you would expect people to get right about a fourth of the time, they got them right about a half of the time and the general population of volunteers seems to get it right about a third of the time. So that was quite strong and it was actually replicated by a study in Scotland at the University of Edinburgh on students there who were in creative kinds of majors.

Long: I’m curious, too because of the science that somebody like you does a study like this, I think I read somewhere that a lot of times, the naysayers immediately come out and try to debunk what you’re saying. Does that happen more in this field than in others?

Utts: Absolutely, and that is, to me, one of the real scandals of science here because this, as I said, needs to be investigated scientifically and the deniers, mainly, not the skeptics, but the true deniers really do rely on ad hominem attacks and they try to ruin the reputations of people that get involved in this. There’s a Nobel Laureate in the UK who strongly believes this is real and has tried to do research on it and has run into all kinds of problems with his reputation being attacked and he’s a Nobel Laureate. So what young scientist is going to want to risk their reputation if that’s what’s happening to a Nobel Laureate, so to me that’s a huge disservice to science in general to do that kind of attack? We really ought to be allowed to investigate this without that kind of a stigma placed on it.

Long: John.

Bailer: You talked a little bit about effect, about small effect, weak effects, big effects, can you formalize the sense of what effect means? You’ve kind of alluded to it some in your responses, particularly in your experiment where you said chance was a quarter, what would be a small effect, weak effect, and large effect in that context?

Utts: Right, so the way these experiments are set up, they are set up so just by chance someone should get the answer a fourth of the time, so we don’t start at zero. Psychologists define small, medium, and large effects in the way could understand as follows: a small effect can only be seen statistically, as though you couldn’t notice it even if you noticed a lot and you were very observant, you wouldn’t really see it going on. A medium effect would be observable to the naked eye of an expert. So let’s say for example, you want to know if there’s a difference in average body temperature between 18 year olds and 90 year olds. Well a nurse or someone who takes temperatures all the time would probably be able to tell you that yes, there is. That’s about a medium size effect. A large effect anybody would be able to see with the naked eye. So for example, the difference in average heights between males and females; that’s a large effect, we all can see it. We don’t have to see many people to recognize that is there. So what we’re talking about here is a small effect. It’s similar to again, the effect of aspirin in preventing heart attacks, that’s the level of it.

Long: We have time for maybe one more question; John we’ll go back to you.

Bailer: If there is one thing that you would like us to be doing a better job teaching our students, whether they are in statistics or students just in general that would make them more statistically literate, what might that be?

Utts: I think the most important thing would be to think critically and in order to do that, you need to know what that means and what kinds of questions you need to ask. So in statistics, I would love it if people understand not the formulas, but the basic ideas of why statistics is important in daily life and how you can improve your life by understanding the ideas of statistics, not the formulas.

Bailer: Oh boy, I like that answer.

Long: Well Jessica Utts, we want to thank you very much for travelling here and sharing your insights with us today. It’s been a pleasure.

Utts: Thank you, my pleasure.

Bailer: Thank you Jessica.

Campbell: Thank you.

Long: And if you’d like to share your thoughts with us on our program, please send your emails to us at StatsandStories@MiamiOH.edu. Be sure to listen for future editions of Stats and Stories where we’re discussing the statistics behind the stories and the stories behind the statistics.

Click to close the script.