039 – How to Lie with Smoking Statistics

[00:00:00] Lily: Hello and welcome to the IDEMS Responsible AI podcast, our special series of the IDEMS podcast. I’m Lily Clements, a data scientist, and I’m here with David Stern, a founding director of IDEMS. Hi David.

[00:00:18] David: Hi Lily. If I’m correct, today’s topic actually came from my co-director Danny.

[00:00:25] Lily: Yes. Yeah. After he listened to our podcast on how to lie with statistics and we referenced a book by Daryl Huff. From 1950 something, 1954, maybe? We definitely say it in a podcast, I remember that. But there was a book by Daryl Huff called How to Lie with Statistics, and we referenced that, and how it’s still relevant today. And then Danny the co-director of IDEMS looked into it and came across another book, which was started by Daryl Huff called how to lie with smoking statistics.

[00:00:59] David: This is a really interesting piece of history. And it’s one of those things where the tobacco industry has been at the forefront of the misinformation wars, which we’re right in the middle of today. But almost all of the tactics and the sort of elements about misinformation which are so prevalent in our society as we speak, a lot of them can be traced back to the tobacco industry and their gradual fight as more and more evidence came out that smoking wasn’t good for you. In fact, it was pretty bad for you.

[00:01:42] Lily: It’s easy now, like it’s fun to read now because today it’s so obvious that smoking’s bad for you and it causes lung cancer. And so it’s interesting to read these books by people. So Dowell Huff, his book, it was never published but there are bits of the manuscript online that I’ve been reading. And his book was commissioned by the tobacco industry in the 1960s after the Surgeon General’s report that smoking causes lung cancer.

And the aim of the book was to kind of say, or my understanding was the aim of the book was to go through and say okay, but this is how the statistics can be misconstrued to show that there’s a link between smoking and lung cancer, but actually there might not be.

[00:02:23] David: Exactly, because of course you’ve got to remember, it wasn’t that much earlier in history when smoking was promoted, advertised, you know, thought to have health benefits. This is the crazy thing, it was promoted for the health benefit. And so to be suddenly at this position where you have this huge industry, which has gone from being the healthy lifestyle choice and promoting itself as such to suddenly, the situation now where every pack of cigarettes you buy has very graphical images of showing exactly what smoking does to you.

[00:03:03] Lily: Well, I was going to say, in the UK, anyway, and I think EU, every single pack looks exactly the same. They aren’t advertised on TV, they’re now hidden so that you can’t see them when you get to the supermarket, you have to specifically ask for them. It’s kind of completely turned 360, or 180, on how they’re they’re not advertised now.

[00:03:24] David: Yes, and this is something where you have to be absolutely certain this is what you want to do to yourself before, and it’s made very clear that you have to be informed. But that’s because the evidence is now so strong.

Go back to the 60s and the evidence was in its infancy. I’m not trying to defend anyone for being paid by the tobacco industry to try and discredit the research. Absolutely, on the contrary, this is something where the implications of that, in general, on our society have been huge. This was a really, really negative long term impact on society from the efforts that came out of this.

However, I do want to put into perspective that this is something where people’s belief, people’s understanding at that point, there was genuine doubt, as there is doubt now about other things where in 10, 20 years time, we will have much more certainty because the evidence will be stronger.

What I think we need to learn from this, is this element that when you look at the sort of how to lie with statistics approach and smoking statistics, this ability to be able to use sound statistical scientific argument approaches to both promote and or discredit is extremely powerful. And I think what’s so important is to recognize that the fundamental distinction between science and pseudoscience is not what analytic methods you use. It’s the standpoint you start from.

If your standpoint is, I have this belief and I’m looking for evidence to support the belief, you are taking a pseudoscience approach. I don’t care whether you’re right or wrong, you’re not being scientific. If, on the other hand, you start from the standpoint of having a belief and trying to find evidence to disprove your belief, you are taking a scientific approach. A good scientist is always trying to show how they are wrong, not how they are right.

[00:06:11] Lily: That’s really interesting, yeah, okay. I’ve not heard it phrased like that, but I really like that.

[00:06:18] David: It’s not taught like this. I mean, most of our scientists nowadays probably wouldn’t agree with that statement. But good scientists should always be looking to challenge their beliefs, not to find evidence to support them. You’re looking for ways in which you’re wrong.

[00:06:39] Lily: Sure. And well, going back to the book Daryl Huff, who, by the way, Danny corrected, is a journalist, not a statistician.

[00:06:47] David: Absolutely.

[00:06:48] Lily: But the manuscript actually is, there’s statistics in it, again, really interesting, really relevant bits of it, which I’m thinking are very relevant for different courses we’re making that we can reference back to it. And then going into, I don’t know, say correlation and causation, saying that just because there’s a correlation between smoking and lung cancer doesn’t mean that smoking causes lung cancer, and at the time I guess they didn’t have that evidence, but it could be due to something else, and he goes into genotypes.

It could be that you’ve got a personality which is more likely to get addicted to things like smoking, and your genotypes are such as well that you are more likely to develop lung cancer. But I suppose through that, is he doing pseudo science? Is he using that to challenge his view, or is he using that to…

[00:07:40] David: Well, this is one of the interesting things, and this is one of the big problems of our society right now. Almost by definition, he is doing pseudoscience because he’s paid by the tobacco industry. If you’re paid by a vested interest, they wouldn’t publish it if it didn’t correspond to the views that they hold. And so therefore you are by definition doing pseudoscience. You’re looking to prove what you’re trying to do.

And this is one of the big problems in our society because almost all our science is now funded by people with vested interests. Very little government or neutral funding exists to just pursue scientific approaches where the results aren’t sought after by people with a vested interest. And that’s a really interesting element that we aren’t learning the lessons of the past.

I suppose one of the questions that people could say is this is supposed to be a Responsible AI podcast. What does this have to do with Responsible AI? But it has everything to do with Responsible AI. Because if your tech industry is the one developing the algorithms, then the algorithms are likely to be biased. They have vested interest, they’re not trying to develop independent ethical, scientifically valid algorithms. They are trying to develop algorithms which support their profitability.

And this is something where it might not be immediately obvious how to do this differently. I certainly don’t have the answers, but I do recognize that within our society, we are handing over to pseudoscience, many of the processes where we should be supporting science. And that worries me. I’ve often mentioned I’m generally an optimist. I am not optimistic about this. I do not see in our societies, at the moment, how we are going to move towards more sound scientific evidence. At the moment, the tendency I see is towards pseudoscience. And that scares me. Because pseudoscience is not actual knowledge, it’s just people protecting vested interests.

And it’s, I don’t have answers as to how to reverse that trend, but it is a trend which I think can be traced back to groups like the tobacco industry, funding books like how to lie with smoking statistics. And it’s something where it’s an approach which has so much… I mean, it’s so obvious that if you can delay for a few years by casting doubt and keep profits coming in for a few more years, well, that far outweighs the cost of funding someone to publish a small book. So obviously that’s good for business, but it’s not good for society. And it’s certainly not scientific.

[00:11:09] Lily: Well, and I believe we don’t know why the book wasn’t published. But the book wasn’t published.

[00:11:16] David: My understanding, and I don’t fully understand this, is that there were further negotiations happening between the author and the publishers, and it was at a time when gradually the evidence was becoming clear. It’s not clear to me who withdrew it and why, exactly. But my understanding is that it was not, not published at the desires of the tobacco industry, which would have been a very different perspective. My understanding is that the not published was a decision which was at different levels in different ways.

[00:11:51] Lily: Sure.

[00:11:51] David: And I don’t know whether it was the author who finally pulled the plug or the publishers themselves.

[00:11:57] Lily: Sure. Well, okay. Maybe someone did the science, not the pseudo, but anyway.

[00:12:03] David: Probably, I don’t know. But as you say, there’s the content of the book from a learning perspective is absolutely sensible. It’s just the fact that at that point in time it was applied as pseudoscience rather than as science. You could have had the same book commissioned, and this is where arguably, if the same book had been commissioned by a scientific body, it would have been perfectly appropriate and you could have had a whole range of examples and a whole range of instances.

And a lot of that means is where is the money coming from and what are the interests behind that money?

[00:12:48] Lily: I see.

[00:12:49] David: And that is unfortunately at the heart of pretty much all the scientific processes that are happening at the moment. They come back to tracing back the money, where’s it actually coming from? There’s so much money being poured into misinformation and pseudoscience at the moment, it scares me.

[00:13:09] Lily: I mean, this is a book from 1966 or thereabouts, and, and if we have then the kind of, not this pseudoscience approach, or not that the money comes from people that have a kind of agenda, then we could have actually had a really great book that was published.

[00:13:27] David: Exactly. Let’s take exactly the parallel of what’s happened with the smoking for pseudoscience. So let’s say you now had such a piece of research where the funding of that research meant that it wasn’t available for people for promotions within the governmental institutions and so on, because of the sources of the funding in this way or that way. I have no idea how this would work, but you know, let’s say your equivalent of your big scary image of lung cancer on the package when you’re consuming it was there and was needed legally. And you actually couldn’t access it openly. You couldn’t share it openly. You had to have it behind your sheet. You had to actually ask for it. You know, this sort of element would transform actually scientific communities. And it’s totally unrealistic. I can’t even imagine a society which would take the approach to smoking that we’ve currently got in the UK to pseudoscience.

But then again, in the 1960s, nobody could have imagined the approach to smoking we have now. So maybe in 50, 60 years time, the world will be a different place, which I find unimaginable at the moment, where science is valued over pseudoscience, and pseudoscience comes with a health warning. Because it is. The negative health consequences of pseudoscience are devastating. That is a world which I would love to see in my lifetime, but I don’t think it’s going to happen.

[00:15:07] Lily: Sure, sure. Going back a bit, you said before that you’re generally an optimist, as we know you are.

[00:15:13] David: Yeah.

[00:15:14] Lily: This is something you feel a little less optimistic about. But as well as being an optimist, you are also someone that likes to kind of empower people. But you’re saying here that the power here, I’m hearing, comes from people who have an agenda and have the money or the resources to do it. What can I do as a statistician, as a data scientist, as a regular person that’s kind of wanting to know?

[00:15:39] David: For this problem? For this problem, I really wish I knew. I, honestly, the problem of science and pseudoscience, this is a problem where I would love to put intellectual effort in. I would love to contribute to this, but it feels insurmountable to me. I do not know how to fight the misinformation wars.

And that’s essentially what this is. And I know there are people, I know people trying to work in this space, trying to fight some of these battles. I wish I knew how to help to support myself. I wish I knew how to protect myself. I don’t. I don’t know how not to get caught up in pseudoscience. You know, believe things which may not be true, because I, I honestly don’t know.

Let me put it this way. Again, I’m going to come back to being the optimist eventually, but leave me for a minute on my sort of realism approach, because this is what it is right now. The reality is a lot of faith was placed in science 50 and more years ago, and science underperformed. And this is the simple truth, that the societal expectation from scientists was more than scientists could deliver. I’ll just take the very simple example of AI as we’re in a responsible AI podcast.

[00:17:08] Lily: Yeah.

[00:17:09] David: The ideas behind AI are old. The concept of AI is old. And the progress that was made was slow. For 30 years, in the early days of AI, basically nothing happened societally, all the nice discussions, there’s wonderful science fiction about what could be, you know, all these sorts of things. A lot of the intellectual work went in. All these ideas about, I’m afraid I’m going to quote Douglas Adams in a sense, this amazing computer, which was built to do these amazing calculations, which took him this immense amount of time. And eventually the answer just pointed out that you’d asked the wrong question.

I mean, this was fantastic insight about AI and about everything we’re dealing with right now. In Douglas Adams, The Hitchhiker’s Guide to the Galaxy, these ideas were popularized way before the technology could catch up. In the 90s, there were advances in the technologies of AI, and actually to the extent that AI became part of everyday life.

And it is now everyday life, but it wasn’t in people’s consciousness, because it was behind the scenes. It hit people’s consciousness when suddenly an AI program was able to beat a chess grandmaster. Now that took people by surprise and it hit them. But then again, it wasn’t able to deliver on to the things that people expected it to.

And it’s only just very recently, in the last few years, that we’ve got to this idea that, actually, the things that we thought AI could do 50 years ago and more, which it couldn’t do, it can now do. It’s this element that we’ve been in this period of transition for over 100 years now, where technology has been really rapidly developing, but science has not been keeping up with or ahead of societal expectation, it has been behind it. And that has led to this situation I would argue we’re in now, where science is undervalued and pseudoscience is starting to really take in prominence.

I do not have the answer of how to resolve this, how to change this. At all. But I am an optimist and I do believe that actually that trajectory, you know, this has been rapid growth for a long time. At some point things will slow down. Once things have slowed down, I would expect science to win out. While things are moving so rapidly, I don’t see how it’s possible. So right now, I don’t see how to help science win.

But I do believe that the incredible growth trajectory society has been on for the last hundred plus years will plateau at some point, in some way. Of course, if you believe some leaders of big tech, that would not be until we’ve actually conquered space. But that I’m not sure about. So, if it involves conquering space, that might be longer before it plateaus than if it’s just limited to Earth. But at some point I would expect there to be a plateau of this growth.

And at that point of the plateau, I would expect science to win out. If it survived that long. So in the interim, I just don’t see how it can. I don’t see how it can compete. The financial pressures and so on. I at the moment cannot see a way forward.

No, sorry, that’s not the right way. I see ways forward and I think the development is great. We don’t want to slow down development because there’s so many positives come out of it. But I can’t see a way that proper science can out compete pseudoscience in this rapidly changing society. And I just don’t have an answer to that.

[00:21:22] Lily: Yeah, of course, very interesting as always. I just want to pick you up on something. So you said, we don’t want to slow the development.

[00:21:31] David: Oh, that’s a whole nother episode, isn’t it?

[00:21:35] Lily: I’m just thinking of conversations we’ve had on regulation and we won’t go into this here.

[00:21:40] David: Yes, there’s a whole other episode into that. You’re right. There are questions about what could be slowed down or should be slowed down. But in general, we don’t want to slow down societal development.

[00:21:52] Lily: Okay.

[00:21:52] David: I think is a correct statement. We might want to support societal development. We might want to slow down certain aspects of technological development. That is possible. You know, do we want to slow down scientific development? I’d argue probably not. Scientific development, we want to encourage and speed up, but there’s all sorts of different types of development. So you’re really right to pick me up on this.

I think societal development, towards the right aims. And I would argue that the Sustainable Development Goals are sensible aims, whether they are the right ones or not, they are good enough for me as a set of aims. If we were, as societies, heading towards the Sustainable Development Goals, I would argue we don’t want to slow down societal development.

If we find that technological development is actually moving us away from the Sustainable Development Goals, then we might want to question that. And that is the sort of questions that need to be done. And if the Sustainable Development Goals aren’t the right goals, what are the right goals for society? I think those are really important questions to ask and answer. And this is not the right episode for this. You’re absolutely right to pick me up on that. That is a hard element to discuss and good other discussions could come out of that.

[00:23:19] Lily: No, it’s been a very interesting discussion. I mean, from what I can take from this and a really interesting part that you did say as well is so what can I do about it? Well, as a scientist, when you’re using statistics, you want to try and disprove yourself. But otherwise…

[00:23:37] David: Let me reframe that slightly. You want to try and challenge your beliefs. So, a good scientist, if you have beliefs, you want to use scientific methods to challenge your beliefs and find the limits of your knowledge and your understanding and what you believe, rather than confirm it.

[00:24:00] Lily: So I could now do a book that says how to lie with smoking statistics all over again. But on reverse.

[00:24:15] David: Yeah. [Laughs].

[00:24:17] Lily: Not quite. Maybe not quite like that, but don’t use it just to say yes, that’s exactly what I thought.

[00:24:24] David: A good scientist, when they’re doing their work, their most interesting work will be the work where they find something different to what they expected. So good scientists actually embrace the cases where what they expected doesn’t play out in the work they do and they dig in to understand more and better and actually to try and draw out and improve what they change, what they think and what they understand based on what they find out from their work.

If as a scientist you are never challenged by your results, your results always just confirm what you’re doing. You’re probably either not asking the right questions or you’re not doing science. You might well just be doing pseudoscience where you’re just looking for that confirmatory bias, confirmation of whatever biases you might have.

I’ll give you a really concrete example of this to finish. One of the big topics of our day is climate change.

[00:25:32] Lily: Sure.

[00:25:33] David: In climate change, one of the things which is most well established is that on average across the world temperatures are rising. There are fantastic graphs related to this. There’s really solid data behind this. This is really substantial. Let’s say, as a scientist, you take a particular context and you actually find that in that context temperatures aren’t rising.

Now you don’t want to be a climate change denier, but the fact that the weather patterns in that context might be leading to not only not temperature rising, but declining temperatures. This is really, really interesting and it’s worth further study. It is not a contradiction of the fact that there is climate change, which is caused by factors which we have understood could be leading to temperature rising on average, such as CO2 emissions and so on, where there are scientific processes which are understood which could be explaining the rising temperatures.

But that does not mean that everywhere you will always find rising temperatures in every context. And so let’s say you find a context where you’re not having rising temperatures but you’re having something else. Wonderful. That is something not to be brought forward as evidence against climate change or global temperature increase, but to be brought forward as a surprising anomaly, which warrants further study.

And this, is a key example of how a good scientific process should be taken. Now, if you take that out of context and you use that to say, look, climate change isn’t real because there’s somewhere where the temperature isn’t rising, then that’s not science. But as a scientist, find that there are things which contradict your general belief, that is good science. And when you find that, you don’t move on, you dig into it to understand and that’s worthy of further study, even though it goes against your general beliefs and your general areas of interest.

You might be interested in supporting that people make changes to avoid continued climate change. But if you find something which goes against what you’ve been finding elsewhere, you should have further study of it. So that to me is a really simple explanation of the scientific process in its best form.

And it’s not difficult to see why that’s a good process. Because that leads to deeper knowledge, it might mean that wind patterns have changed in the Northern Hemisphere, you’re getting more winds coming from the north, bringing cold air rather than winds coming from the south bringing hot air. It might just be that, that you’ve found that the changes have led to you being in a slightly different scenario. Fantastic. Understanding that is really important.

But I guess this is exactly where, as scientists, we need to recognize how much we don’t know. We know a lot of things, but medical science, despite all its advances, the power of the placebo effect is a real sign of how little we understand. We know the placebo effect is so important and so real, and yet it is scientifically so little understood. There’s so much left that we don’t know. I wonder what society will be like if we are able to use scientific method to understand some of these things better over the next 50, 100 years.

[00:29:33] Lily: And the placebo effect being relevant here, do you mean that using the placebo effect just as an example in the medical field, or are you equating the placebo effect to kind of confirmation bias, which is this bias that you have where you go, okay, yes, I thought that that would happen, therefore, it happens.

[00:29:53] David: Okay, no, I was not equating it in that way. I was simply using the placebo effect. So, and maybe we should actually explain this. So the placebo effect is when in, let’s say medical treatment, when you get a pill, which shouldn’t actually do anything, but you get better anyway. The placebo effect is essentially in a trial where you’re taking, let’s say, medicine pills, you get given a sugar pill which has no active ingredients, nothing which should make you better, and yet it has been found in study after study that just getting given that pill and being told this pill will make you better, makes you better.

And it’s been shown that actually if the doctor who’s telling you it doesn’t know that the pill does nothing, that makes you even better. This is why you have to have double blind studies to do good medical research. So just that belief that you’re going to get better is known to have a really significant impact on health outcomes.

[00:31:00] Lily: As a child, if I injured myself, say I got stinging nettles, my mum would be like, Oh yeah, just rub this, she’ll give you a random leaf, rub this on it, it’ll make it better. And you’re like, okay. And, and it does. It makes you feel better because you’re told by someone who you believe knows everything.

[00:31:19] David: Yeah.

[00:31:20] Lily: It’s gonna work.

[00:31:22] David: Yeah. And some things pass anyway. But there was much more to it than that. It is a very, very, very poorly understood as I, you know, issue. The implications of actually taking this into our healthcare systems, you know, would be huge. Really bringing in the placebo effect properly and actually using that for healthcare outcomes could be amazing. But we don’t understand it. And so we don’t actually know how to use it in sensible ways because it’s not understood.

Anyway, I think probably we’ve got off track a bit and it might be a sensible time to stop.

[00:32:03] Lily: Yes, absolutely. But thank you very much. It’s been a very interesting conversation.

[00:32:08] David: Thank you.