Episode 26: Moving beyond the 'just so' when reporting science
Release Date: 2/13/2017
David Spiegelhalter is Winton Professor for the Public Understanding of Risk in the Statistical Laboratory at the University of Cambridge, Chair of the Winton Centre for Risk and Evidence Communication , and President of the Royal Statistical Society . He is passionate about helping the public understand uncertainty and risk . His recent book, Sex by Numbers , describes scientific research that provides a view of the world of sex.
Rosemary Pennington: With February comes what feel like the coldest and darkest days of the year, but right in the middle of the month sits a shining spot of light; at least, if you’re the romantic type. Americans will spend an average of $146 buying gifts for their sweethearts, or people they hope will become their sweethearts this Valentine’s Day. At the same time, journalists and bloggers will be furiously writing stories trying to make sense of love, attraction, and sex. Those things are the focus of this episode of Stats and Stories, where we look at the statistics behind the stories and the stories behind the statistics. I’m Rosemary Pennington. Stats and Stories is a partnership between Miami University’s Departments of Statistics and Media, Journalism, and Film, as well as the American Statistical Association. Our regular panelists are Department of Statistics Chair John Bailer and Department of Media, Journalism, and Film Richard Campbell. Today’s special guest is David Spiegelhalter, Winton Professor for the Public Understanding of Risk in the Statistical Laboratory, Center for Mathematical Sciences, University of Cambridge. He’s also the author of the book, Sex by Numbers: What Scientists Can Tell Us About Sexual Behavior. Thank you for being here this afternoon, David.
David Spiegelhalter: Thank you for asking me.
Pennington: Just to start off the conversation, how did someone who seems to be focused on communicating risk get interested in studying the numbers behind sex?
Spiegelhalter: Good question. I ask myself that question sometimes, as well. It’s my publisher, and I did a book risk with my publisher and then he said, “Oh, I’d like you to do something else.” And they work with the Wellcome Collection which is this exhibition space, well-funded in London, that puts on medical exhibitions and they were going to put on an exhibition about sex, about history of sex research with Kinsey and surveys and so on and so on and they wanted a book to accompany it. So essentially this is a book to accompany an exhibition.
John Bailer: What’s been the reaction to this book?
Spiegelhalter: It got incredibly good reviews; I was amazed. And it was serialized for a week in a popular newspaper, double-page spread every week. But then it didn’t actually sell fantastically well. Let’s say it’s one of those critical but not popular successes, which is a bit disappointing. But I think I can understand why. It sort of falls between two stools. It was sort of marketed as a popular book, Sex by Numbers, with that title and it was from a popular publisher and yet actually, it’s a book about statistics. It’s got graphs in it and I tried to keep the numbers down, but actually it ends up not really being a sort of fluid read. And I think it’s quite good, but in retrospect it probably would have been better marketed as a slightly more technical book for students in social science because that’s actually what it is. It’s a serious book about the difficulties of doing research, serious research, in this complex area.
Richard Campbell: David, this is Richard Campbell, and my job is to ask about the journalism side of this. What did journalists get right and wrong when they’re writing about you and covering the work you’ve done? Not just in this book, but in general, possibly.
Spiegelhalter: Yeah, I’ve got a very good relationship with journalists on the whole. I work closely with them. When this book was serialized in The Daily Mail, a very big circulation newspaper, I worked well with them. The actual journalists, I’ve always worked well with. It’s the editors putting in the headlines that are always the problem and so the inappropriate headline, just for example, in The Daily Mail we were talking about the frequency of sex and sexual behaviors among all the women and then the headline writer put something about promiscuity. I wouldn’t dream of using a word like that; it’s so value laden. And actually I thought it was a word that hadn’t been used since about 1973, but it appeared in the newspaper. I was furious, I was so angry. On the whole, it was covered well, very good reviews, very good interviews, however you may have heard of a particular disaster that happened when I was talking about this at Hay Literary Festival, the major literary festival in this country, and I was giving a popular talk and when I talk about this stuff, I fill it full of jokes. And one of the aspects I was talking about was the finding that the frequency of sex has declined among same sex couples between 16 and 44. They were reporting in 1990, having sex five times a month, then in 2000, about four times a month, and then a median of three times a month in 2010. So I made some fatuous joke by saying, “At this rate, nobody will be having any sex at all by 2040.” And then I said, “Well why not?” And then quite seriously, I said, “Actually the people who did this research are suggesting the decline of sex could very much be because of increased use of electronic media, people have got their phones on all the time. People are so engaged with communication with everybody that that time for intimacy is being shrunk.” And then I made the joke, “Well I think it has to do with boxsets, people are saying, ‘No dear, I’m not coming to bed now, I’m watching the latest series of Game of Thrones.’” And it got a laugh, but a journalist in the audience for a major newspaper didn’t quite get the joke and the next day there was a story saying, “Cambridge Professor Says There Will Be No Sex by 2030 Because of Game of Thrones.” And I was a bit cross and complained and they changed the article online, but then it was too late because the way journalism works now, somebody writes an article and every other outlet then picks it up. So this story went around the world. So many headlines in all countries and different languages, all to do with this Cambridge professor who says there’s going to be no more sex in the future because of Game of Thrones. And you put together “sex,” “Cambridge professor,” and “Game of Thrones,” and it gives them a chance to put some salacious photograph up, as well. I thought, there’s forty years of reputation gone down the drain in one go, in fact, nobody took any notice and I’ve used it to get a huge amount of laughs whenever I give the talk about this.
Bailer: That’s a wonderful story. What makes doing research about sex so difficult?
Spiegelhalter: I think it’s a tremendously difficult idea to do rigorous statistical research on. You want to find out about something that essentially is private. You can’t just go out and see it. You can’t put cameras in bedrooms, if you did, it would probably change behavior rather a lot. But yet you want to know what people are up to, so the only way really is to ask them and although there are some indirect signs you might use as well. You can’t just walk up to somebody and ask them, “How many times do you have sex in a month?” Except some people do, some people do street interviews with this stuff, which I don’t think would be that reliable. But if you want to do it seriously, it’s expensive because the people who do the big surveys don’t even want to do online surveys, they don’t want to use standard panels, they don’t want to use telephone surveys, so you have to send someone to randomly chosen houses and they have to arrange for an interview with somebody, spend time explaining to them why they’re doing this and why this is valuable, small reward, about £30, about $50-60. Then do an interview, which the person speaking has to be guaranteed that it’s going to be anonymous and real serious secrets aren’t going to go any further. This is all done, both in the US and the UK, these surveys will be done with computer assisted interviewing where the person giving the responses is doing it onto a laptop and the interviewer cannot see either the questions or the answers that are being put in. And then the laptop’s closed down and so nobody should be able to identify the answers with the individual. So these are very difficult surveys to do.
Pennington: You’re listening to Stats and Stories where we discuss the statistics behind the stories and the stories behind the statistics. The topic today: Sex and Numbers. I’m Rosemary Pennington. Joining me are panelists, Miami University Statistics Department Chair John Bailer and Media, Journalism, and Film Department Chair Richard Campbell. Our special guest is David Spiegelhalter, author of Sex by Numbers and stats professor at the University of Cambridge.
Campbell: I had a question to follow-up on reliability. One of the things I read was that women admit to having more frequent sex if wired up to a lie detector, in one of your studies. So since people lie about reporting sex all the time, how do you deal with that?
Spiegelhalter: It is a problem, of course. There are various checks you can do, first is an internal check, by asking the same question in two different ways at different ends of the interview. You can get some check over time by seeing whether, essentially whole cohorts of people are answering reliably. It’s an interesting design, what you do essentially in 2000 they were asking 25-year-old women, “When did you first have sex?” And then in 2010, you ask 35-year-old women, “When did you first have sex?” and they should give similar answers. So you can do it sort of consistently, they’re not the same women, but they’re the same group of people so if you’re doing a good survey and people are giving good answers, you should get reasonable answers. That’s in a way a global check. You can do a global check by asking people whether they’ve had abortions or not and checking that with abortion rates. So there are certain external checks you can do. Actually, just checking whether someone is telling the truth or not is extremely difficult at the time. You try to engender trust and so on and the interviewers, I’ve talked to the interviewers, really believe they are getting reliable answers. However we know that some areas where there clearly is a lack of reliability and the classic one is asking people how many sexual partners they’ve had in their lifetime. And the point is that, mathematically, if you’ve got a closed population of men and women, then the average, in terms of the mean number of sexual partners, that men have had and women have had should be the same. It must be, logically, because there’s a single number of partnerships and the average number of partners should be the same, and it isn’t. In previous surveys, earlier surveys, men will often report, on average, having had twice as many partners as women have. That gap is lower now, and it’s certainly lower if you ask about recent sexual partners, the gap gets a lot lower. And the fact that the gap gets lower when you ask about recent sexual partners suggests this may be something as much to do with memory or recall and reporting, as it is to, what we might call social desirability biases. Generally, women not wanting to admit they’ve had as many sexual partners as men. And there’s all sorts of other reasons people are suggesting such as that we’re not including female prostitutes in the survey and yet that may comprise a lot of the partnerships of men. So all sorts of reasons can be included in this. I think it’s a big mix of reasons. There’s some evidence, this is a lovely study done at American University, a randomized trial in which students were randomized into three groups and then asked, “How many sexual partners have you had?” And one group was guaranteed anonymity, another group had sort of the threat of revelation because one of their peers just came and picked up the paper and took it away and could see it, and another group was wired up to a lie detector. Now the lie detector was fake, it was just some machine with wires on it, it wasn’t a lie detector, but they were told it was a lie detector. And the evidence in this randomized study was that women who were wired up to the lie detector did admit to having slightly more sexual partners, there was a significant difference. It was an interaction for men, it wasn’t an effect of the lie detector. So it suggests that some element of social desirability bias, also perhaps just effort made. There’s a sort of suggestion, and I don’t think there’s strong evidence for this, that if you ask women to recall their sexual partners, they will start counting and remembering names and people. There may be some they don’t want to remember and they might just rule it out and not use it. People who have had many sexual partners, if you ask men, they might be more liable to make a rough estimate. And this is shown in the raw data, fascinating graphs in the raw data because you see severe rounding. Once you get above ten or fifteen, ten people you might clearly be remembering names and faces. I should say for men and women between 35 and 44, the most common response is that they’ve had one sexual partner. That’s the most common, about a sixth of people say that. But after about ten or so, people might seem vague; faces start blurring into each other and they start seeing fifteen, twenty, thirty, fifty, five hundred or whatever. There’s one really notable man who said forty-seven, so I think he’s a statistician.
Campbell: Very good, thank you.
Pennington: One thing you bring up in your book is you talk a little bit about Alfred Kinsey’s research. And I did my Ph.D. at IU; I worked pretty closely with the Kinsey institute at times on some stuff. And so I was sort of interested in your take on the way his work stood up because it’s been controversial over time and sort of, what do we know? What does Kinsey tell us about sex that is still something that scientists are sort of using and working with today?
Spiegelhalter: Kinsey was extraordinary. It’s just fascinating because then I started reading biographies and everything like that. He was extraordinary. Compared with how surveys are done now, well he invented them really, but he broke every rule. Things have changed completely. He would make friends with people off of a cigarette, use ordinary language rather than more medical terms. He would go out of his way to find extreme cases. He would visit gay bars to interview those people. A lot of people were from prison that he interviewed to get extremes of behavior. He was a biologist. He was really interested in the range of behavior and so he enlisted some extraordinary statistics. Which are quite surprising now, let alone what they must have been like in the late 1940s. It was quite shocking at the time and the other thing he did, of course, was interviews. No coding scheme, he just filled up a sheet of paper full of this sort of hieroglyphics. This secret scheme that wasn’t written down about how the questions, what the responses, what the little scribbles meant in terms of the responses to the questions. So he was extraordinary, and a lot of his stats are treated rather skeptically because of his sampling methods. He wasn’t completely against random sampling or something like that. What has incredibly stood up to the test of time is the Kinsey scale of sexuality. Essentially he was one of the first people to stop considering people as being either heterosexual or homosexual gay or straight, what we might call them now. And he invented a whole range from naught to six in terms of the degree of same sex attraction and behavior, with complete heterosexuality at one end and complete homosexuality at the other and a whole range in between. And that was an enormous insight that he had.
Bailer: I’m just curious, what was the most surprising thing you learned in the course of doing the research for this book?
Spiegelhalter: I can’t resist talking about the sex ratio. This is just how many boys are born for every girl. And in Western countries now, it’s about twenty-one boys for every twenty girls. So a sex ratio of 105, that’s said to be 105 boys for 100 girls. It’s higher in countries which practice some selective abortion, but about 105 is the current rate. You may think, first of all, what’s that got to do with sexual behavior and why is that interesting? In the book, I show a plot that I had never seen done before, which is taking UK data, which records the sex ratio, the number of boys and girls born back to 1837. And I had never seen that plotted before, and when you plot it, you get some incredibly distinct patterns. And what you do is you get a declining sex ratio from about 1870 to about 1910. Really, it comes steadily downwards and it reaches an actual bottom around 1900. Then it starts going up again. And then you get these massive these massive spikes, and these massive spikes are in 1919 and 1944 and a bit of one in 1973. So what’s that? That’s the fact that more boys are born at the end of wars. And I hadn’t heard this and when you start researching it, you realize this has been discussed for quite a long time. I didn’t know anything about that; I thought it was riveting. And there’s all sorts of explanations about why this might happen, suggestion that there’s some evolutionary reason. That animals, that the gender of their offspring is slightly affected by the status that that gender might have in the society. So somehow, this is the Trivers-Willard hypothesis, it’s known as some ability to influence the gender essentially by the needs of society. I don’t believe that’s operating in this case and there is an interesting reason that someone suggested which I kind of like so I’m a supporter of it. And there is some evidence that actual sexual frequency influences, to a very small extent, the gender. So when people have more sex, younger people having more sex tend slightly to have more boys. And why might this be the case? There’s some evidence again, that if you conceive earlier in the cycle, there’s some tendency, a little excess, of it being a boy. Before the time of peak fertility, just before ovulation. So if you conceive earlier in the cycle, some tendency to be a boy. Now why do these things fit together? Well if you have lots of sex, it’s more likely that you’ll conceive earlier because there’s more of a chance that you actually conceived by the time of peak fertility. When do people have lots of sex? Why should that be associated with the end of wars? Coming home on leave, coming home after being deployed, people having lots of sex. The enormous amount of children being born at the end of wars, 1919 in the UK, more children were born than any other year, before or since. So the suggestion that some people have strongly made, and I really believe this, that you have more boys at the end of war just because people are having more sex, frantic sex. Maybe not that frantic, but frequent. The support that I see for this also, the decline in sex ratio in the UK between 1870 and 1900 why might that be declining? Well there is simultaneous historical evidence that people were having less sex at the end of the Victorian period. There’s a big decline in fertility in the UK in the second half of the 19 th century. It’s when we had our fertility transition from kind of natural fertility, five or six children, down to two, happened between about 1870 and 1910. An extraordinary change in society because women were controlling their own fertility, but they were not, on the whole, using artificial contraception and historians have suggested this essentially was abstinence. The continence theory, there was this big emphasis on continence, essentially not having sex. So there is some indirect support for the fact there, the decline in sex ratio, I think is associated with people just having less sex, but you shouldn’t use this as a way to try to fix to have a boy or a girl. You need about 300,000 births to spot these differences. These are tiny differences.
Pennington: You’re listening to Stats and Stories and our discussion considers how scientists study sex. David, what is the most frustrating thing that you have seen in the news when it relates to the reporting of statistics? Either related to this work or maybe other work you’ve done.
Spiegelhalter: Oh, I get so angry so often, I can’t remember. When did I last get angry? I start shouting at the radio and things like that. I think the fact that on the whole, statistics are used as arguments. No matter what people claim, they use to persuade rather than to inform, and I just challenge anyone. Anytime you hear a number in the news or the newspaper, people are pretending to use it in order to make something look big and dangerous and this really annoys me, numbers just being used as rhetorical arguments rather than to inform people. And of course it happens all the time. It happens in our case, in the Brexit discussions. Last year, people for the Referendum, a huge misuse of statistics there. And frankly, an inability of, for example, interviewers on the radio to challenge those adequately at the time. I just think they’re not well enough trained to do that. And once a number has got out, once a number is in circulation, it’s very difficult to do anything about it, it takes on a life of its own.
Bailer: So how do we combat that misuse?
Spiegelhalter: It’s very difficult. I think you have to, in a way, be ready for it. You have to challenge more at the time. Being a suggestion that is quite nice, on major radio programs in the UK, if someone is using numbers in an argument, that the fact checking team of the BBC or something goes at it hammer and tongs, and they should, within twenty minutes or so, be able to come back. Even if the interview is finished and onto something else, within the same program, you should be able to criticize that number, that argument. If it’s a live program, obviously if it’s recorded, it’s easier, but if it’s live, that you should be able to fact check and get back there immediately. There’s no good doing it a day later, it’s too late.
Campbell: You had an earlier book called The Norm Chronicles, and in your subtitle, it was “Stories and Numbers About Danger and Death.” So you think a lot about the relationship between stories, you’ve told us a few here today, and numbers. Can you talk about how you think of those as compatible?
Spiegelhalter: This is a thing where obviously I should be talking to you, as a journalist. I think that telling stories with numbers is just the most interesting topic now. It’s the topic of the future. And it’s more than just data journalism, I think. It’s when it’s data driving the story, but it’s choosing a narrative and a framing that makes a story engaging or carry people through, make them interested in, I would say the facts, but at least in some quantification. And by using good storytelling, using imagery, using individual stories, anecdotes perhaps, but they aren’t served to reinforce what the numbers are telling you rather than being an opposite. Too often, storytelling is based on extreme cases that don’t represent actually, the body of evidence. And so I just wish I was better at it. That is something I’m interested, both in terms of visualizations, sort of dynamic infographics that can tell a story almost like a storyboard style, but also of course in verbal narrative, to be able to include numbers, even graphs, into a story without putting people off. Even when I’m just reading, if I start finding numbers, it breaks it up, it means, in Kahneman’s language, you have to start thinking slowly instead of thinking fast. We want to make people think slowly, but we want to keep them engaged enough so they don’t just give up. And I think this is the real challenge of the future in a society where factual, we all know that factual narratives are under challenge.
Bailer: You do a brilliant job of the story and the narrative, David. There’s lots of evidence for the effectiveness of what you’ve done. Kudos on that. I’m curious. One aspect of the storytelling and the narrative, one device that we often see is the two sides of the story, even if those sides don’t have equal weight. So Richard and I have talked a lot about this and trying to combat that. And the other part I’ve like in your book, talking about the mean and the median and differentiating that. And I’ll even add more complexity to the question so it’ll make it impossible to track what I’m asking, just the idea of conveying the uncertainty and distribution of responses to an audience that seems most enamored with having some point value that they want to really invest in.
Spiegelhalter: That’s why I think number of sexual partners is such a good, I love showing that. I could give a whole lecture on one graph because it’s got so many interesting features. If you look at the number of sexual partners, the most common value is one, the mode. So you say, is that representative? If you look at the average, it’s really quite high because it’s so influenced by, the graph, I can’t even put it on a slide because it goes out to thousands, some people are saying, and that’s going to be enormously influential, drag the average up. Just like Bill Gates wandering into a room rather changes the average income, but he doesn’t change the median income at all. And so the median is really the most useful, I think, communication of what’s going on. The thing is, I’m terribly pedantic, as all statisticians are, about the use of language and so I would talk about the average number of sexual partners of people in society, but then that’s different from the number of sexual partners for the average person. By the first I mean the mean; by the second I mean the median, the average person being the one that’s halfway along and how many sexual partners have they got? Anybody hearing those things might not even distinguish the two. The average income and the income of the average person are completely different and because I’m very careful with my language, I can distinguish those and I notice when someone else distinguishes those. But it’s very easy for somebody who doesn’t realize what’s going on to start thinking they’re the same thing, which the decidedly aren’t.
Pennington: That’s all the time we have for this episode of Stats and Stories. David, thank you so much for such an interesting conversation, today.
Spiegelhalter: No, thanks very much for having me.
Pennington: Stats and Stories is a partnership between Miami University’s Departments of Statistics and Media, Journalism, and Film and the American Statistical Association. Stay tuned and keep following us on Twitter or iTunes. If you’d like to share your thoughts on the program, send your email to StatsandStories@MiamiOH.edu and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.s.
Click to close the script.