Is science broken?

With the rebirth of Cosmos on TV, Neil DeGrasse Tyson and the team have brought science back into the mainstream. No longer is science confined to the latest puff piece on cancer research that is only in the media because a) cancer and b) the researchers are pressuring the funding bodies to give them money. The terms geek and nerd have stopped being quite the derogatory terms they once were. We even have science memes becoming as popular as Sean Bean “brace yourself” memes.

Sean dies

This attention has also cast a light on the scientific process itself with many non-scientists and scientists passing comment on the reliability of science. Nature has recently published several articles discussing the reliability of study’s findings. One article shows why the hard sciences laugh at the soft sciences, with the article talking about statistical errors. I mean, have these “scientists” never heard of selection and sample bias? Yes, there is a nerd pecking order, and it is maintained through pure snobbishness, complicated looking equations, and how clean the lab-coat remains.

purity

As a science nerd, I feel the need to weigh in on this attack on science. So I’m going to tear apart, limb by limb, a heavy hitting article: Cracked.com’s 6 Shocking Studies That Prove Science Is Totally Broken.

To say that science is broken or somehow unreliable is nonsense. To say that peer review or statistical analysis is unreliable is also nonsense. There are exceptions to this: sometimes entire fields of study are utter crap, sometimes entire journals are just crap, sometimes scientists and reviewers suck at maths/stats. But in most instances these things are not-science, just stuff pretending to be science. Which is why I’m going to discuss this article.

A Shocking Amount of Medical Research Is Complete Bullshit
#6 – Kinda true. There are two problems here: media reporting of medical science and actual medical science. The biggest issue is the media reporting of medical science, hell, science in general. Just look at how the media have messed up the reporting of climate science for the past 40 years.

Of course, most of what is reported as medical studies are often preliminary studies. You know: “we’ve found a cure for cancer, in a petri dish, just need another 20 years of research and development, and a boatload of money, and we might have something worth getting excited about.” The other kind that gets attention isn’t proper medical studies but are spurious claims by someone trying to pedal a new supplement. So this issue is more about the media being scientifically illiterate than anything.

Another issue is the part of medical science that Ben Goldacre has addressed in his books Bad Science and Bad Pharma. Essentially you have a bias toward positive results being reported. This isn’t good enough. Ben goes into more detail on this topic and it is worth reading his books on this topic and the Nature articles I previously referred to.

Many Scientists Still Don’t Understand Math
#5 – Kinda true. Math is hard. It has all of those funny symbols and not nearly enough pie charts. Mmmm, pie! If a reviewer in the peer review process doesn’t understand maths, they will often reject papers, calling the results blackbox. Other times the reviewers will fail to pick up the mistakes made, usually because they aren’t getting paid and that funding application won’t write itself. And that’s just the reviewers. Many researchers don’t do proper trial design and often pass off analysis to specialists who have to try and make the data work despite massive failings. And the harsh reality is that experiments are always a compromise: there is no such thing as the perfect experiment.

Essentially, scientists are fallible human beings like everyone else. Which is why science itself is iterative and includes a methods section so that results are independently confirmed before being accepted.

And They Don’t Understand Statistics, Either
#4 – Kinda true, but misleading. How many people understand the difference between statistically significant and significance? Here’s a quick example:

This illustrates that when you test for something at the 95% confidence interval you still have a 1 in 20 chance of a false positive or natural variability arising in the test. Some “science” has been published that uses this false positive by doing a statistical fishing trip (e.g. anti-GM paper). But there is another aspect, if you get enough samples, and enough data, you can actually get a statistically significant result but not have a significant result. An example would be testing new fertiliser X and finding that there is a p-value of 0.05 (i.e. significant) that the grain yield is 50kg higher in a 3 tonne per hectare crop. Wow, statistically significant, but at 50kg/ha, who cares?!

But these results will be reported, published, and talked about. It is easy for people who haven’t read and understood the work to get over-excited by these results. It is also easy for researchers to get over excited too, they are only human. But this is why we have the methods and results sections in science papers so that calmer, more rational heads prevail. Usually after wine. Wine really helps.

Scientists Have Nearly Unlimited Room to Manipulate Data
#3 – True but misleading. Any scientist *could* make up anything that they wanted. They could generate a bunch of numbers to prove that, for an example of bullshit science, the world is only 6000 years old. But because scientists are a skeptical bunch, they’d want some confirming evidence. They’d want that iterative scientific process to come into play. And the bigger that claim, the more evidence they’d want. Hence why scientists generally ignore creationists, or just pat them on the head when they show up at events: aren’t they cute, they’re trying to science!

But there is a serious issue here. The Nature article I referred to was a social sciences study, a field that is rife with sampling and selection bias. Ever wonder why you hear “scientists say X is bad for you” then a year later it is, “scientists say X is good for you”? Well, that is because two groups were sampled and correlated for X, and as much as we’d like it, correlation doesn’t equal causation. I wish someone would tell the media this little fact, especially since organic food causes autism.

Other fields have other issues. Take a look at health and fitness studies and spot who the participants were: generally, they are university students who need the money to buy tinned beans and beer. Not the most representative group of people and often they are mates with one of the researchers, all 4 of them. Not enough participants and a biased sample: not the way to do science. The harder sciences are better, but that isn’t to say that there aren’t limitations. Again, *this is why we have the methods section so that we can figure out the limitations of the study.*

The Science Community Still Won’t Listen to Women – Update
#2 – When I first wrote this I disagreed, but now I agree, see video below. As someone with a penis, my mileage on this issue is far too limited. That is why it was only when a few prominent people spoke out about this issue that I realised science is no better than the rest of society. It hurts me to say that.

There is still a heavy bias toward men in senior positions at universities and research institutes, women get paid less, women are guessed to be less competent scientists, and apparently, it is okay to ogle female scientists’ boobs… Any of these sound familiar to the rest of society? This is gradually changing, but you have to remember what age those senior people are and what that generation required of women (quit when they got married, etc). That old guard may have influence but they’ll all be dead or retired soon where their influence will be confined to the letters to the editor in the newspaper. After seeing the video below, especially the way the question was asked, I think it is clear that the expectations for women create barriers into and through careers in science (the racism is similar and is one I see as a big issue). So it starts long before people get into science, then it continues through attrition.


Fast forward to 1:01:31 for the question and NDGT’s* answer (sorry, embed doesn’t allow time codes).

Recently there has been a spate of very public sexist science moments. Whether it be telling female scientists they should find a male co-author to improve their science, or Nobel Laureates who don’t want to be distracted by women in the lab, it is clear that women in science don’t get treated like scientists. Which is why I find the Twitter response to the Tim Hunt debacle, #distractinglysexy, to be exactly the sort of ridicule required. Recent events seem to imply at least repercussions are occurring now.

Scientists are meant to be thinkers, they are meant to be smart, they are meant to follow the evidence. They aren’t meant to behave like some cretin who hangs out on the men’s rights movement subreddit discussion. Speaking of which, watch science communicator Emily Graslie discuss the comments section of Youtube.

Here’s another from Thought Cafe and Dr. Renée Hložek.

Update: After the first photo of a black hole was published, women in STEM were back in the headlines, with people wanting to again marginalise women in STEM – not to mention how the media love to promote the “lone genius” when science is a team thing. Vox had a great article on it which included some great graphs from Pew Research.

ps_2018.01.09_stem_0-01
Source
ps_2018.01.09_stem_0-04
Source

It’s All About the Money
#1 – D’uh and misleading. Research costs money. *This is why we have the methods section, so that we can figure out the limitations of the study.* Money may bring in bias, but it doesn’t have to, nor does that bias have to be bad or wrong. Remember how I said above that science is an iterative process? Well, there is only so big a house of cards that can be built under a pile of bullshit before it falls down in a stinky mess. Money might fool a few people for a while (e.g. climate change denial) but science will ultimately win.

Ultimately, science is the best tool we have for finding out about our reality, making cool stuff, and blowing things up. Without it we wouldn’t be, this article wouldn’t be possible, we wouldn’t know what a Bill Nye smack down looks like. Sure, there is room for improvement, especially in the peer review process and funding arrangements, and science is flawed because it is done by humans, but science is bringing the awesome every day: we have to remember that fact.

Other rebuttals:

http://theness.com/neurologicablog/index.php/is-science-broken/

*Wow, who’d have thought including Neil DeGrasse Tyson in this context would age quite so badly!?

Advertisements

Will individuals respond differently to homeopathic remedies prepared with unboiled vs. boiled water?

I recently watched a debate between Ben Goldacre and Peter Fischer on homeopathy. During the course of the debate, an audience member asked, “If water has a memory, how come you’re not sick every time you drink water out of the tap?.

A homeopathic practitioner answered (paraphrased) that boiling the water resets the memory and that homeopathic remedies are only effective when using boiled water. He makes another comment implying that if a remedy were prepared with tap water, it wouldn’t be effective.

I realize the above related question (“Does water have a memory…?”) is nearly identical. I’m trying to ask it another way as it’s possible to persist with the water memory concept despite the other question’s answer. One could simply say (my hypothetical response),

“Well, we don’t know how it works and perhaps it isn’t by the known mechanism of how water behaves… but trials indicate that it works, nonetheless and that’s all I need.”

Since the audience member in the video indicated tangible predictions, I’m interested if they’ve ever been put to the test. Thus, my question is:

Has a trial ever been conducted in which homeopathic remedies prepared from both unboiled and boiled water were compared against one another in terms of patient response?

If there is another way to answer this question please go for it.

Answer:

It really doesn’t make any difference if the water is boiled or not, homeopathy doesn’t work.

The Minimum Dose and Avagadro’s Number The second and most controversial tenet in homeopathy is that remedies retain biological activity if they are diluted in a series (usually in a 1:10 or 1:100 diluent–volume ratio) and agitated or shaken between each dilution. Hahnemann began this process to reduce toxicity, but later he claimed that this “potenization” process extracted the “vital” or “spirit-like” nature of these substances (2). The limit of molecular dilution (Avagadro’s number) was not discovered until the later part of Hahnemann’s life; by then homeopaths all over the world were reporting that even very high potencies (dilutions lower than Avagadro’s number) produced clinical effects. The implausibility of such claims has led many to dismiss any evidence of homeopathy’s effectiveness as artifact or delusion (3). http://www.annals.org/content/138/5/393.full

But lets pretend for a moment that water does have memory. The aspect of boiling has not been researched. A search of Google Scholar nets no results for boiling and homeopathy. When referring to “how-to” guides of preparations it becomes obvious that homeopaths are merely after clean or unpolluted water to make their preparations in.

Ingredients … 1/2 or 1 litre of boiled water (distilled water may be bought at pharmacies in some countries, if you want that, and bottled, rinsed water is commonly sold in groceries too)

Another example:

Preparing your own bottle: Boil the glass bottle and dropper in filtered water for 15 min. and let it cool completely. Fill it just to the neck with filtered or distilled water.

So clearly the idea that boiling is the only way to reset the water is not backed up by the practices employed by homeopaths themselves. This combined with the fraudulent claim that water retains memory shows that this is another misdirection to allow justification.

Tyson Adams’ 2013 Book Awards: The Awesomes

This is the third year of The Awesomes™, the award I give to books that had me staying up late to finish them, the books that had me rapt until the end, and sometimes past the end. I’ve read a few books this year (+70) so here are my favourites of 2013 and this year’s Awesome™.

As you will have noticed, my reviews of books are more about my impressions of the book and talking about how much I liked the book, rather than a recap of the plot, etc. My reasoning behind this is simple, I want to say “read this book” to people rather than fall into my bad habit of spoiling the ending, or being a bitch about books I didn’t enjoy. My list is based upon what I have read this year, so obviously some great books have missed out due to lack of reading hours in the year (blame the rugrat). Also my read list does include some books that were published prior to 2013. There were also some categories that were sadly under-represented, whilst others had some very intense competition. Also, the fact that I finished a book shows that it was worth reading. I have my reading rules that stop me wasting valuable reading time on books I’m not enjoying. This means that any books on my read list are entertaining (well, unless I was particularly disgusted with the crappiness of the book in question).

Awesome™ of 2013
Luther: The Calling – Neil Cross
Killer Instinct – Zoe Sharp
Gone Girl – Gillian Flynn
Never Go Back – Lee Child
Without Fail – Lee Child
Altar of Eden – James Rollins
The Secret of Excalibur – Andy McDermott

Zero at the Bone – David Whish-Wilson

Awesome Literary Fiction
There were no nominees in this category this year. Better luck next year.

Awesome Mystery & Thriller
Killer Instinct – Zoe Sharp
Without Fail – Lee Child
Altar of Eden – James Rollins
The Secret of Excalibur – Andy McDermott

Never Go Back – Lee Child

Awesome Crime
Luther: The Calling – Neil Cross
Gone Girl – Gillian Flynn

Zero at the Bone – David Whish-Wilson

Awesome Fantasy
There were no nominees in this category this year. Better luck next year.

Awesome Paranormal Fantasy
There were no nominees in this category this year. Better luck next year.

Awesome Science Fiction
There were no nominees in this category this year. Better luck next year.

Awesome Horror

The Strain trilogy – Guillmero Del Toro and Chuck Hogan

NB: cheating here as it was only 4 stars, but deserves the nod as the TV series is now in development and looks like they might have a winner.

Awesome Romance
There were no nominees in this category this year. Better luck next year.

Awesome Humor
There were no nominees in this category this year. Better luck next year.

Awesome Nonfiction

Bad Science – Ben Goldacre

Awesome Graphic Novels & Comics
Midnighter – Garth Ennis
Batman: The Black Mirror – Scott Snyder

Luthur Strode – Justin Jordan

Awesome Indie
No 5 star indies this year, although several 4 star and a few non-mentionables.

Awesome Poetry
There were no nominees in this category this year. Better luck next year.

Awesome Shorts/E-zines
I’m putting this category in just so that I can pimp:

Thrills, Kills and Chaos

Still Awesomes
I re-read – well in some cases I listened to the audiobook – several books this year. They deserve a mention for still being awesome. Sometimes books are better on their second outing, sometimes they are worse, sometimes you wonder why you didn’t throw the book out the first time (I’m looking at you Holden Caulfield).

Dirk Gentley’s Holistic Detective Agency – Douglas Adams (better than I remember)
Long Dark Tea-time of the Soul – Douglas Adams (similar to how I remember)
Life, the Universe, and Everything – Douglas Adams (similar)
Fight Club – Chuck Palahniuk (slightly better)
Game Keeper – Guy Ritchie and Andy Diggle (better)

Congratulations to all the nominees and winners. I hope that I have a chance to read more fantastic books from these authors again in 2014 and that everyone else does too.

Book Reviews: Bad Science by Ben Goldacre

Bad ScienceBad Science by Ben Goldacre
My rating: 5 of 5 stars

It has been a while since I’ve blogged one of my book reviews. I guess that is part of the parenting manual that I didn’t read: hobbies are no longer priorities. There have been plenty of good books pass before my eyes since my last review, but I felt content just to let a short sentence, a star rating and an update to my Twitter feed to promote a good read.

This is slightly different. Ben’s book made me annoyed.

I’m a science nerd. I prefer to read the original research papers rather than the media coverage of them, as it is always terrible, usually based on a half-arsed press release and never links to the actual research. I am constantly amazed that in our modern age of computers, internet, vaccines, satellites and zero calorie drinks that people still believe in stuff that wasn’t plausible 200 years ago. And that’s why Ben’s book annoyed me. He made it painfully obvious how deliberate some of the misinformation campaigns have been.

I knew homoeopathy was rubbish (magic water droplets on sugar pills, or as I prefer to call it, a placebo), I knew that complimentary medicine is the term for stuff that hasn’t ever been proven to work, I knew anti-vaccine campaigners clearly didn’t remember that my dad’s generation had polio victims everywhere, I knew that “Big Pharma” are a mixed bag of good and bad science. So if I knew all of this already, why am I annoyed? Because I didn’t realise just how culpable the news media were and how media liaison and PR companies are straight up lying to people.

You see, I always thought, and this is still mostly true, that most media get science wrong because they don’t understand it and it isn’t easy to do the background research to check a press release on a study. I see this as not having the specialist science reporters doing the science journalism (imagine if a science journalist reported on climate change from the beginning, we’d have emissions at zero by now). But with Ben’s section on the MMR “controversy” and the “nutritionists” in the media, he paints a very clear picture of culpability that the media needs to address, or as Ben points out, people will just go to science blogs written by actual scientists in that field.

Excellent book and a must read for anyone who still reads newspapers or watches the TV news.

View all my reviews

PS: Yes, I’m also guilty of not publishing results, like most scientists I know. I make no excuses, I’m a terrible person.