paleoaerie

Home » Posts tagged 'nature of science'

Tag Archives: nature of science

6 Ways to Completely Fail at Scientific Thinking, Part III

All of the mistakes discussed so far are universal among humans to a greater or lesser degree. These last two are also universal and extremely common, leading to a world filled with pain and suffering, bigotry, and misunderstandings on a grand scale. I should warn you that this discussion will make many people uncomfortable because it cuts into the core of how people view themselves. People define themselves through the memories of their experiences and we tend to remember sound bites better than the complexities of reality, which makes for a dangerous combination.

5. We tend to oversimplify our thinking. 

Of all of the mistakes, this one has most likely caused the most problems. When we were still living as hunter-gatherers in small bands, this was a benefit and can still be in some areas. When you live in an environment filled with potentially life-ending threats, you need to be able to recognize and react to them quickly. When that rustle in the bush may be a Smilodon about to attack, you can’t afford to think about all the different options because if you do, you are dead. But most of us no longer live in that sort of environment. We can take the time to think. We just have to fight our natural instincts that are hardwired into our brains. It’s tough, I realize that. It’s impossible to do all the time. But I hope you will see why it is so important that we try.

kirkmeme

It is at the core of stereotypes and the “us vs. them” mentality that drives everyone to some extent. Any time you hear someone say, “Blacks are…,” or “Muslims are…,” or insert any group you want, that person is oversimplifying their thinking. It does not matter what you say after that first phrase, it will not accurately describe all members of that group. All “Blacks” are not actually black, nor do they share the same heritage, culture, language, or anything else. All those people that are thought of as Muslims by those using that stereotype are not in fact Muslim. I say this because almost invariably when non-Muslims refer to Muslims in a stereotypic fashion, they are confusing Arab (or anyone from the Middle East) and Muslim. Muslims and Arabs, like any large group, do not all share the same beliefs and culture.

In the first post in this series, I mentioned the anti-vaccine movement. It all started from ONE paper (since thoroughly discredited and debunked) that only referred to ONE specific vaccine. The whole point of the paper was to discredit that specific vaccine so the author could sell his own version. But no one in the anti-vaccine seems to remember that and they have simplified the topic to ALL vaccines.

In science, this sort of thinking causes people to read a single set of experiments (or even one experiment) on a specific target and then try to apply the result to everyone. This mistake is rampant in the medical field. A study will be published saying that a series of rats showed a result and instantly the media says that all humans will have the same result. Fortunately, scientists are well aware of the differences between rodents and humans. A result in rats and mice often does not carry over into humans. This is why all drugs have to go through human trials after they pass animal trials.

Even if a drug works in the small sample of humans, that sample is not truly representative of all humans. You may have heard that science has proven that vitamins are pointless and may even be harmful? The studies that indicated vitamins had no benefit were all done on healthy volunteers that mostly had good diets. So yes, if you are healthy and are getting everything you need from your diet, you don’t need vitamins and the excess can actually hurt you. Unfortunately, most people do not fall into this category, so for them, taking vitamins can indeed help. (This is just another example that eating right and having a healthy lifestyle will avoid many of the health problems most people have and will save you money in the long run. Exercise is almost always preferable to pills and is free.) Even healthy humans are incredibly variable and have different metabolisms. The same drug will not work the same on everyone.

All those internet memes you get with a picture of someone with a saying on it? Fabulous examples of oversimplification. The internet is full of examples of overly quick and thoughtless thinking. Here is a tip, if anyone can boil down the essence of a social problem with one pithy statement, it is almost guaranteed to be WRONG. I have heard more than one person say that because Muslims flew planes into the World Trade Centers, all Muslims were evil and should therefore all be killed, because “they all want to kill us anyway.” To any rational person, this statement is clearly, insanely, wrong. You may wonder why I have mentioned Muslims a few times. That is because right now, it is the most prevalent and dangerous stereotype I know and one which is very familiar to everyone. They either hold that view or know many who do.

I could go on and on about how people oversimplify for the rest of my life, but it gets seriously depressing rather quickly, so I will stop here. But I hope you get the point: Oversimplification, overgeneralizing, has led to the wrongful deaths of hundreds of millions of people and is the source of much of the hatred in the world. Be aware of just how common this mistake is and STOP DOING IT.

You how do you avoid this problem? Never take one study or one source as truth. It is ok to keep an open mind about something, but don’t put your faith into it unless you can verify it through other reliable sources. Wait for other studies that confirm the results because it may be that the first study was wrong. Avoid overgeneralizing. Just because something worked once, do not think it will work every time. Always, always, always keep the parameters of a study in mind, respect the limitations of any study. A result on one mouse in one situation has little to do with results from many people in all sorts of variable conditions. Do not extrapolate beyond the data without clearly understanding that the extrapolation is purely speculative guesswork and may not hold up in reality.

6. We have faulty memories.

One way that our memories are faulty is in that confirmatory bias discussed in the previous post. You can see this problem in everyone who gambles, be it at a casino or the stock market. Most people remember their successes far more commonly than their losses. People can lose fortunes this way. Casinos are masters at exploiting this mistake. If a gambler wins early, they tend to continue playing long after they have lost their winnings and more. Every time they win, they remember that one win and forget all the loses before that. Some people do the opposite, focusing on their failures and minimizing their successes, which leads to problems therapists deal with every day.

Science, particularly medical science, has a form of institutionalized faulty memory. It is much easier to publish positive results than negative ones. Therefore, experiments that didn’t work tend to be glossed over and forgotten, focusing on the ones that succeed. Of course, if those successes are due to chance or faulty experimental design, ignoring the negative results leads the whole field astray. How serious is the problem? A paper in 2012 found that only 6 out of 53 “landmark” papers in haemotology (study of blood) and oncology (cancer research) could be replicated. This sort of publication bias on the positive can have profound problems. It may sound like this means that science can’t be trusted, but what it really means is that it is critically important to never jump on the bandwagon and follow the advice of a new study. Wait until it can be confirmed by other research. Science is all about throwing hypotheses out there and testing them to see if they really work. One test doesn’t do it. Multiple tests are needed and you cannot forget the failures.

Where faulty memory really comes into play is in just how easy it is to change our memories. Simply hearing another person’s experiences can change our own. My favorite study showing this interviewed people about their experiences at Disney World. The participants watched an ad showing people interacting with Bugs Bunny at Disney World.  The fact that this event is impossible (Bugs Bunny was not owned by Disney and so could not make an appearance at Disney World) did not keep many of the people from saying that they had fond memories of seeing Bugs Bunny at Disney World.

Kida discusses the results from some researchers in which they asked students where they were when they first heard of the space shuttle Challenger exploding. They asked shortly after the event and then again two and a half years later. Despite claiming that their memories were accurate, none of them were entirely accurate. Some of them were wildly off. Yet the students insisted that they were correct and disavowed the record of their earlier remembrance. There are several studies like this that say the same thing: our brains do not faithfully record our experiences and those memories change both over time and through suggestion by others. 

So, what does this mean for us? It means that we have a bad habit of misattributing things, combining memories or making them up whole cloth. Criminal psychologists are deeply aware that eyewitness testimony is the least reliable evidence that can be brought into court, despite the fact that it is considered the most reliable by most people. People commonly say “I’ll believe it when I see it,” and “I saw it happen with my own two eyes!” We put a lot of stock into our perceptions and our memories. But, as is quite clear from decades of research, neither our perceptions or our memories are at all reliable.

So what are we to do if we can’t rely on our own experiences? Make records, take pictures, write it down. Compare experiences with other people. There is some truth to the now common statement; “Pics or it didn’t happen.”

 Final thoughts

And so we conclude the introduction to the six basic errors in thinking we all make to a greater or lesser extent. These mistakes are universal, they happen repeatedly daily basis. Yet they have grave consequences. In science, we have ways to try to avoid them. We record data. We share it with others and let them try to poke holes in it. We do not trust only one example and demand verification. Scientists make these mistakes all the time. But by being aware of the mistakes and having procedures in place to deal with them, we can minimize the problems.

Thinking mistakes

6 Ways to Completely FAIL at Scientific Thinking, Part 1

dontbelieveWhen I was a kid, I was always taught the scientific method is a matter of developing hypotheses, testing them, and using the observations from the tests to revise the hypotheses. Very straightforward, but overly simplistic. My teachers rarely, if ever, talked about the crucial strategy of multiple working hypotheses, coming up with every imaginable way that could explain our observations before we started trying to test them. But the most important thing that was never taught was how to think explicitly and clearly. Logical and clear thinking is the heart and soul of science. In fact, there is no decision that cannot be improved by clearly thinking about the question and the available data. We just celebrated Independence Day in the United States. It is time we celebrate our independence from fuzzy, ill-defined, and confused thinking. In the last post, I discussed the critical importance of clearly defining a problem in terms of actionable questions. The first step is to understand a problem well enough that it can be clearly articulated and defined. Then all factors that contribute to the problem can be clarified. But you can’t stop there. Once you have a list of known factors, you have to decide which ones you can actually do something about and not waste time arguing about those you can’t change. Focusing on the definable and workable factors produces results. Wasting time on things you can do nothing about is counterproductive. In this post, I am going to briefly discuss the first three of six general mistakes that EVERYONE makes from time to time. You can never be completely rid of them, but you can be aware of them and try to reduce their influence in your life. If you do this, I promise you will make better decisions. You will improve your life and the lives of those you touch. These six mistakes are outlined and fully discussed in the book Don’t Believe Everything You Think: The 6 Basic Mistakes We Make in Thinking, by Thomas Kida. I highly recommend you get this book and read it.

1. We prefer stories to statistics. People are terrible at statistics, even people who really should know better, so bad in fact, that they make them up to sound smart. You can easily find numerous variations of the statement, “80% of all statistics are made up on the spot, including this one.” Or, as often (likely incorrectly) attributed to Mark Twain, “there are three kinds of lies: lies, damn lies, and statistics.” So it’s no wonder that people suck at them and prefer stories. There are abundant studies illustrating how our brains are wired to listen to stories, how personal stories influence our behavior more than statistics, such as this one, or this one. Statistics happen to abstract groups, stories happen to identifiable people. We even prefer to dress up our information to make it more personal, more interesting, but the very act of storifying information makes that information less likely to be true. It is much more likely that Bill robbed Peter than it is that Bill robbed Peter AND paid Paul. The more complicated things get, the less likely. But this mental shortcut can cause serious problems. This is well illustrated by the anti-vaccination scaremongering going about. The whole anti-vax movement can really be traced to one report by Dr. Andrew Wakefield in 1998 that found a correlation between vaccines and autism, research that has been completely discredited and proven fraudulent. Since then numerous studies have linked into the alleged link and found nothing, such as this one. But no matter how many studies find no link, many people hear Jenny McCarthy talk about her autistic son, they hear others talk about their autistic children, and come to the conclusion that all the studies must be wrong, because the stories carry more weight with them. Disregarding the millions of children who get vaccines that never develop autism, people focus only on the stories of people that claim otherwise. Thus, thousands of children are getting sick and dying because of a belief in stories over statistics.

2. We seek to confirm, not to question. Have you ever read something that you disagreed with and instantly dismissed it or conversely, have you ever accepted evidence simply because it agreed with what you thought? If so (and you have, everyone does), you are guilty of confirmation bias. Confirmation bias causes people to seek out and weigh information that already agrees with their point of view and disregard evidence that disagrees with them without ever really analyzing the data. If you get all your news from either FOXNews or MSNBC, you are likely to rarely, if ever, hear contrary points of view and are thereby limiting input to only that which you already agree. Thus, people who do so will weigh that evidence in favor of their preconceptions and will assume that their view is more prevalent than it really is. If one gets all their information about evolution from the Institute for Creation Research, they will never get accurate information about the theory as the ICR is based on the belief that evolution is false, so they seek only information that discounts it. The only way to avoid this is to seek out diverse news outlets. While you read them, remind yourself that you will suffer from confirmation bias, so you may (hopefully) be able to give evidence from all sides a thorough critique.

Lest you think that only untrained laymen fall into this trap, confirmatory bias is rampant in science as well and it is a serious problem. Even the ivied halls of Harvard do not protect one from poor thinking and confirmatory bias as this article by Neuroskeptic clearly illustrates wherein he takes a fellow neuroscientist to task for not recognizing the fallacy of only looking for confirmatory results. There is a publication bias in the scientific literature towards positive results, the negative results get mentioned much less often. While this is true in all fields, it is particularly important in medical research, and psychology research has been hit particularly hard lately.

dilbert-confirmation-bias

Next post, I will cover the next two common mistakes. Stay tuned.

The Gender Gap in STEM fields and the Nature of Science

If ink was blood, the discussion of the lack of women in STEM fields would exsanguinate the whole of the human species. Yet for all the talk, the problem remains. Why? Let me answer that by bringing to your attention a post by Dr. Janet Stemwedel, a professor of philosophy at San José State University who writes on this and other topics concerning the thinking underlying how science is done. The post is not long, so I will post most of it here, you may find the original here. To clarify the post in case it doesn’t come across clearly: she quotes a block of text discussing the gender gap in the sciences and then provides her response to it.

However, there are times when people seem to lose the thread when they spin their causal stories. For example:

“The point of focusing on innate psychological differences is not to draw attention away from anti-female discrimination. The research clearly shows that such discrimination exists—among other things, women seem to be paid less for equal work. Nor does it imply that the sexes have nothing in common. Quite frankly, the opposite is true. Nor does it imply that women—or men—are blameworthy for their attributes.

Rather, the point is that anti-female discrimination isn’t the only cause of the gender gap. As we learn more about sex differences, we’ve built better theories to explain the non-identical distribution of the sexes among the sciences. Science is always tentative, but the latest research suggests that discrimination has a weaker impact than people might think, and that innate sex differences explain quite a lot.”

What I’m seeing here is a claim that amounts to “there would still be a gender gap in the sciences even if we eliminated anti-female discrimination” — in other words, that the causal powers of innate sex differences would be enough to create a gender gap.

To this claim, I would like to suggest:

1. that there is absolutely no reason not to work to eliminate anti-female discrimination; whether or not there are other causes that are harder to change, such discrimination seems like something we can change, and it has negative effects on those subject to it;

2. that is is an empirical question whether, in the absence of anti-female discrimination, there would still be a gender gap in the sciences; given the complexity of humans and their social structures, controlled studies in psychology are models of real life that abstract away lots of details*, and when the rubber hits the road in the real phenomena we are modeling, things may play out differently.

Let’s settle the question of how much anti-female discrimination matters by getting rid of it.

Dr. Stemwedel hits directly on a error that is seen throughout every facet of social interactions, to say nothing of public understanding of science. The discussion had strayed into unproductive areas and away from any attempt to resolve the problem. Dr. Stemwedel attempted to bring the focus back to the problem needing to be solved.

In this instance, going back to the original post to which Dr. Stemwedel was responding, the initial question put to Neil DeGrasse Tyson was “Why are there fewer women in science?” The true answer to that question is that there are multiple reasons for the gender gap, one of which happens to be that there is a clear discriminatory bias against women in science (this fact is so well documented that linking to only one or two articles seems pointless as a quick Google search will immediately turn up a very long list of sources). Tyson did not deny there were other factors, but he focused on the sociological aspect as that problem is highly prevalent and something that can be changed.

Chris Martin, the author of the post Dr. Stemwedel was responding to, goes to great lengths to talk about all the other issues involved in the gender gap, ignoring the reason Tyson focused on the sociologic aspect. Contrary to the claims of Mr. Martin, the point was indeed to draw attention away from anti-female discrimination. He clearly states in his post that he believes evolutionary pressures explain the gender gap better than sociological ones and that the sociological factors are therefore unimportant. His whole post is a response to the assertion that this bias prevents many women from entering the field.

Faculty ratings of applications that were exactly the same except for sex of applicant.  www.pnas.org/content/early/2012/09/14/1211286109#aff-1

Faculty ratings of applications that were exactly the same except for sex of applicant. http://www.pnas.org/content/early/2012/09/14/1211286109#aff-1

Unfortunately for Mr. Martin, the evolutionary factors Mr. Martin espoused are not as well understood and nowhere near as well documented in humans as is the discrimination against women. But here is the important thing: We can’t do anything about past evolutionary pressures. We CAN do something about the discrimination.  No one is arguing there aren’t other reasons. But if you want to solve the problem, you focus on the aspects of the problem you can fix.

This problem in thinking is so widespread, one could easily argue that everyone does it. Certainly, unless taught to do otherwise, everyone does indeed make this mistake. In all of science and engineering, there is one basic rule that is critically important you must do: clearly define the problem in terms that can be dealt with. You have to be able to understand the problem in clear enough terms that you can parse out what things are relevant from those that are not. In this case, claiming there are evolutionary pressures is irrelevant to the problem of reducing the gender gap. We know that anti-female discrimination occurs. We know it is widespread. Of all the problems that Mr. Martin mentioned, this is the one that most obviously lends itself to resolution because it involves behaviors that the vast majority of people can agree on that needs to end. Unless one can reasonably expect to fix evolutionary pressures, distracting attention from the problem that can be solved is counterproductive. The good news is that when people do focus on the problems, things can and have gotten better. We still have a long ways to go, but it is improving.

resized_the-most-interesting-man-in-the-world-meme-generator-i-don-t-always-state-the-problem-clearly-but-when-i-do-it-is-half-solved-already-75c417The point of this post is not really to argue about anti-female discrimination in science, although that is a hugely important topic and something that every teacher needs to be aware of and help to stop. The real point of this post is to highlight the importance of knowing your question well enough and clearly enough that you can make progress on answering it, solving the problem. This particular topic just happens to allow me to highlight two really important topics at the same time. The next post will cover some more vitally important mistakes in thinking that anyone trying to teach the nature of science, and really, any other subject, needs to understand so you can work hard on helping your students overcome them.