Steve Kass

The popular press is beginning to report on an article that appeared in PLoS Pathogens today, and you can bet public alarm will spread incomparably faster than the “highly virulent fungus” discussed in the article.

The Los Angeles Times was one of the first to pick up on the scare. They interviewed the article’s lead author, Edmond J. Byrnes, III for their report. In PLoS Pathogens, Byrnes and his coauthors described an “increasingly fatal fungal outbreak” in their discussion, without providing statistical support or an inline reference. The Times knows what readers will glom onto, however, and they devoted a couple of paragraphs to this “more lethal” angle.

The spread is also a concern because the strain of the fungus that moved into the United States in 2004 has mutated to become more lethal than the original strain that invaded British Columbia in 1999.

Five of the 21 people who contracted the fungus in the United States have died (about 25%), compared with 8.7% of the 218 infected people in Canada. The fungus has also infected many different species of mammals.

These details aren’t in today’s PLoS Pathogens article. Today’s article focuses on the molecular biology of C. gattii, not its epidemiology. My admittedly cursory search for the source of the Times’s numbers turned up one mention of an “over 25%” U.S. fatality rate. That was in a previous article of Byrnes’s, where he referenced the figure as the “unpublished observations” of two other scientists. But let’s give Byrnes and the Times the benefit of the doubt and suppose the numbers are reliable.

Math time. If you know only a handful of small numbers about a disease, drawing any epidemiological conclusions — especially alarmist ones that might be misconstrued — is risky, but mathematics can still help us evaluate the numbers. There will be lots of ifs, but that’s to be expected when you have limited data. The standard way to proceed is to ask the following question: If in fact nothing scary is happening (meaning that the infections in the U.S. are not more deadly than they are in Canada), might we really see as many as five deaths in 21 cases?

Under the probabilistic assumptions of this kind of analysis, getting infected with C. gattii is treated like a crap shoot as to whether you die or not; you die with probability 8.7%, the Canadian fatality rate. If 21 people get infected and for each one God rolls the dice to see who dies, what’s the likelihood at least five of them will succumb? Well, it’s around 3%. To put that in context, imagine that tomorrow’s traffic is terrible, like once-a-month terrible. Would you chalk it up to bad luck or a real change in traffic congestion? I’d chalk it up to bad luck and assume a real change only if traffic was similarly terrible again tomorrow or maybe next week. (Even then I would probably look for other explanations, like an announced construction project or a visiting dignitary.)

Other factors should be considered when comparing fatality rates to test the hypothesis of increasing deadliness, particularly with such small numbers of cases. Were the U.S. persons infected by C. gattii diagnosed as promptly (or not) as the Canadians and given identical treatments? If not, the higher U.S. fatality rate could be due to late diagnosis or ineffective treatment. Were the U.S. persons infected or killed similar to the Canadians in age, general health, and other factors known to be independent predictors of mortality from disease? If not, the higher U.S. mortality might not be due to greater virulence.

PLoS Pathogens is a peer-reviewed journal, and that provides some assurance that the scientific conclusions were based on accepted scientific practices. Unfortunately, published science has the potential to affect society and policy via the popular press, and reviewers need to think about how a publication might be construed by a journalist.

To Byrnes’s credit and the Times’s, readers who ventured a few paragraphs past the alarming headline will read this more moderate assessment of the situation: “Overall, I don’t think it is a large threat at this time. But the fact that it is continuing to spread geographically and the number of cases is rising makes it a concern.”

Whether or not Byrnes et al. were more justified than I can tell in calling this a “increasingly fatal fungal outbreak,” sure, this could be the next black plague. So could any number of currently very rare or unknown pathogens. But based on the science I’ve seen, you shouldn’t be any more worried about C. gattii today than you were last week, when I’m guessing you’d never heard of it.

References

• PLoS: Emergence and Pathogenicity of Highly Virulent Cryptococcus gattii Genotypes in the Northwest United States
• LA Times: Potentially lethal fungus moves south from Canada

Updated at 2010-04-22 at 23:28. I added the words Deadly and Fungus to the post title hoping to get more play. A list of headlines on this story follows.

One rebuttal of the alarm: Ore. DHS questions article statements about deadly fungus
(DHS is Department of Human Services, not Homeland Security)

Alarm and alarm in all the other headlines, though.

• Airborne fungus claiming lives
• New strain of virulent airborne fungi, unique to Oregon, is set to spread
• New Deadly Fungus Found in US, Has Already Killed Six
• Potentially Lethal Airborne Fungus May Spread to California
• Deadly Oregon fungus may spread on West Coast
• Deadly airborne fungus in Oregon set to spread
• Killer fungus seen in Pacific Northwest
• Potentially deadly fungus spreading in US and Canada
• Deadly Oregon fungus may spread on West Coast
• Deadly Fungus In Oregon: New Strain Of Fungus Killed 6 in Oregon
• Fungus Cryptococcus gatti Threat to Healthy People
• New Likely Deadly Fungus Invading US & Canada – Signs & Symptoms
• Emerging Northwest fungal disease develops virulent Oregon strain
• Oregon Fungus Spreading South
• Toxic Airborne Fungus From Oregon Spreading Across West Coast
• New Concerns About Deadly Fungus Found in Oregon
• Deadly strain of airborne fungus spreading among healthy people and animals
• Life threatening tropical fungus seen in Pacific Northwest
• Fungus Spreading Throughout US, Canada
• Killer Fungus Migrates To The US
• Killer Lung Fungus Hits Northwest
• ‘Highly Virulent’ Strain of Killer Fungus Found in Ore

A recent numerical expedition led me to these summary statistics from a 2009 Pew survey on Religion and Public Life.

Looks like a typo to me. At a glance, the breakdown by age seems inconsistent with the aggregate result.

Most (about 85%) of the survey participants fell into one of the three oldest age groups, all of which favored gay marriage at a lower rate than the general population. The 65+ crowd, numbering 507, favored gay marriage at a rate a full 17 points lower than the rate for all ages combined. Although young’ens favor allowing Adam¹ and me (or Autumn and Eve) to get married legally, and they smile with approval at a rate almost 20 points higher than the rate for all ages combined, there are too few of them (only 283) to balance out the manier² old grumps and middle-aged semigrumps.

I have no reason to suspect the Pew folks of vulpigeration, so I tried to find an honest basis for these apparently contradictory figures.

Pew’s full report on survey question Q146a revealed one potential source of slop: the number under “Favor” seems be the sum of two individually rounded percentages: one for “Favor” and one for “Strongly Favor.” The actual survey instrument included both possible answers. Therefore, 39% could mean anything between 38% and 40%. Survey percentages are routinely rounded, but one expects 39% to mean somewhere between 38.5% and 39.5%.

Even allowing for extra slop, the numbers don’t agree. Here’s a tabulation (using the increased slop allowance) that gives the minimum and maximum numbers of favorers in each age group and (by summing) the minimum and maximum number of favorers among those in any age category.

According to these numbers, between 34.8% and 36.8% of 1,980 respondents would be cool with my marrying Adam legally.

According to Pew’s summary chart (at the top), though, between 38% and 40% of 2,010 responses, or between 764 and 804 people, answered “Favor.” That’s quite a bit higher than the breakdown figures show, and even if the 30 people in no age category (who presumably withheld their age or were under 18) all favored gay marriage, the maximum (and an unlikely maximum, because it would require all the rounding and missing information to be skewed favorably) number of favorers is 759.

As another plausible scenario, I calculated a Total percentage based on the age breakdown but weighted according to the actual histogram of age in the U.S. Still no dice. If anyone has an idea, let me know.

¹ For the record, I’m currently Adamless and available

² manier, adjective. Comparative of many; more numerous. To be coined presently. Many and numerous are synonyms; if things can be more numerous, I see no reason they can’t also be more many, or manier.

In today’s number news (State-by-state cremation rates in U.S.), we learn that “slightly more than a third of all persons who died in 2006 were cremated, according to the Cremation Association of North America.” Happily, the article contained the raw data, but only as an alphabetical-by-state table of numbers.

Here’s an illumination, as MapPoint is my amanuensis. Click to embiggen.

Explanation: Pie areas are proportional to the number of deaths; the yellow slice is cremations, the red non-cremations.

Pies for our nation’s two newest states are not shown. Alaska’s looks like a two-thirds size Vermont pie; Hawai’i’s looks like a one-third size Oregon pie.

The Journal of the National Cancer Institute just published the results of a large study evaluating the survival benefit of contralateral prophylactic mastectomy in the surgical treatment of breast cancer. In some mastectomy patients, breast cancer will reoccur in the remaining breast, and that risk can be reduced (but not eliminated) by removing the non-cancerous breast .

Contralateral prophylactic mastectomy lowers post-mastectomy five-year death rate by 30%

The five-year death rate after mastectomy was 11.5% for women who had both breasts removed. It was 16.3% for those who only had the cancerous breast removed. Adding a contralateral prophylactic mastectomy to the original surgery therefore reduced the five-year death rate from 16.3% to 11.5%. Almost a third fewer mastectomy patients died within 5 years when the had chosen to remove the second (healthy) breast, compared to mastectomy patients who had not chosen to remove the second breast. The bilateral mastectomy decreased the 5-year death rate by 29.4%.

This strikes me as a significant benefit. Suppose I have breast cancer and need a mastectomy. I can choose a single mastectomy and have a one-in-6 chance of dying in five years, or I can choose a double mastectomy and have a one-in-9 chance of dying in five years. One-in-9 sounds quite a bit better to me. If 100,000 women with unilateral cancer need mastectomies, performing 100,000 double mastectomies instead of 100,000 unilateral mastectomies will reduce the number of deaths in the first five years from 16,300 to 11,500. About 4,800 fewer women will die within five years.

Contralateral prophylactic mastectomy benefits only 5% of mastectomy patients.

The reporting of this study takes a very different viewpoint. It compares the survival rate, not the death rate, and notes that the bilateral prophylactic mastectomy increases the survival rate from 83.7% to 88.5%, “a difference of less than 5%.” Five percent sounds like a small number, but 5,000 lives saved sounds like a large number.

Point of view

Both statements (lowers by 30%; benefits only 5%) are the same. Only the intent to communicate is different. Whether prophylactic mastectomy is good practice depends not only on the change in five-year survival rate, and I don’t have more information.. For example, how does a double mastectomy (which for 95,000 of the women will not change the five-year death rate outcome) affect a woman’s well-being and general health over time? What is the cost to save these 5,000 lives, and how will the disparity of death rates change more than five years after surgery?

Good news

There is a good piece of news in the study: The study data identified a subgroup of women for whom double mastectomy had an even greater benefit: women 50 and younger with early stage estrogen receptor negative cancer. Removing the second breast had no benefit for women 60 and older, and the benefit for women in their 50s was uncertain, presumably because of the small number of bilateral mastectomies in the sample.

The journalists writing about this study generally downplayed the benefits. It would have been better for them to downplay the benefits on most of the women, but hype the discovery that there is a subgroup of women who might get a substantial benefit from this procedure. If you read the study, or find better summaries of it, you may find that this study can help patients and doctors make wise treatment choices (contralateral mastectomy sometimes among them). With luck and more studies like this, prophylactic mastectomies might in the future go only to those women whom they might help. With even more luck, we’ll improve our diagnosis and prevention of breast cancer and the number of mastectomies will go down.

In July, I griped here about misleading news reports of a higher crash risk among texting truckers than among non-texting ones. I pointed out that the research quoted had not shown an increased crash risk, and in fact observed fewer crashes (zero, in fact) among the texting truckers. The data might suggest a decreased crash risk among texting truckers, I noted. The reason for the confusion was that crashes and near-crashes (which included sudden and possibly crash-preventing maneuvers) hadn’t been separated in the calculations. Nevertheless, the increased crash-or-near-crash risk was widely reported as an increased crash risk. I wondered whether more near crashes might in fact be a positive thing; those who occasionally swerve suddenly might be paying more attention to the road than those who rarely do.

Today, researchers and others are expressing surprise that just-in real crash data doesn’t support what they don’t realize the earlier research didn’t show in the first place. For example:

• Results of Study on Cellphone Use Surprise Researchers
• Cell phone bans do little to reduce crashes, study finds

If researchers (or journalists) are surprised by today’s news, they probably didn’t examine the research very closely. They may have believed the misleading headlines instead.

Disclaimers: I haven’t read all the research, and some studies might have in fact shown an increased crash risk, unlike the one I mentioned. That might be reason for real surprise. In addition, today’s data doesn’t specifically show that less phone use means no fewer accidents, because laws don’t always change behavior. But some of the reports I read did suggest that there was less phone use, yet no lower crash rate, in places that instituted bans.

Ironically, the crash data out today could make things worse. If drivers think phone use while driving is not as unsafe as previously thought, they might be less careful when using a phone while driving, and phone-related crashes might increase. Common sense suggests that multitasking requires greater concentration. If you do use a phone while driving, drive with even more care than usual. In other words, this might be one situation where being wary might have benefits, even when its not warranted by the facts – especially because being somewhat over-cautious while driving has no serious down side, it seems to me. (It’s not like it infringes upon hundreds of millions of people’s civil rights, like acting on other unwarranted fears can and does…)

A headline in today’s Washington Post is “Fewer Americans think Obama has advanced race relations, poll shows.” My statistics students know what I have to say about that: “fewer than what?” Yes, headlines are generally false, and the Post reports the statistics somewhat more carefully in the article. “On the eve of President Obama’s inauguration a year ago, nearly six in 10 Americans said his presidency would advance cross-racial ties. Now, about four in 10 say it has done so.”

According to the actual poll results, a year ago, 60 percent of Americans answered “help” to the question “Do you think Obama’s presidency will do more to (help) or more to (hurt) race relations in this country, or not make much of a difference?” Last week, 40 percent of Americans answered “help” TO A DIFFERENT QUESTION: “Do you think Obama’s presidency has done more to (help) or more to (hurt) race relations in this country, or has it not made much of a difference?”

Big deal. Not a story. Fewer Americans today also think they have died than thought last year that they would die.

Original title: Over 90% of Research Studies Make Me Want to Scream (P < 3E-12).

Shania Twain is in the news today. No, her new album still isn’t out, but her face is in the spotlight. It turns out someone “applied” the latest “research” to “determine” that she has the perfect face, “scientifically” speaking. The distance between her eyes and mouth are precisely 36% of the length of her face, and her interocular distance is exactly 46% of its width. These proportions, according to an article in press at Vision Research, are universally optimal (among low-resolution, mostly Photoshopped images of a few white women).

Garbage. Poppycock. Nonsense. Balderdash. Crap, crap, crap of a research paper, right from sentence 1: “Humans prefer attractive faces over unattractive ones.”

But you came here for the pictures. (more…)

Starting today, funny math is moving to a new and better home, www.lolmath.com. Vulpigeration and other serious number-related topics will stay here.

If you read my last post, you know I’ve been looking at some very fishy survey data from Strategic Visions, LLC. The data seems to stink no matter how anyone looks at it, and mathematicians, statisticians, and programmers have been looking hard and every which way. Instead of throwing yet another heavy mathematical brick at the poor numbers, Let’s see how it stands up to a feather.

You don’t have to read many poll or survey results to be familiar with the phrase “totals may not equal 100% because of rounding.”

So guess what? The numbers in Strategic Visions’ results all add to exactly 100. Oops. That’s a huge red flag. Huge, like big enough to wrap a planet in.

Ok, I didn’t check all their polls. Just the most recent 73, which is how many I checked before stopping. For the record, I didn’t stop because poll #74 added to 99 or 101. That would have been statistical misconduct on my part. I didn’t look at poll #74 or any others, because I wanted to write this up. (If anyone checks further, let me know and I’ll post an update.)

If Strategic Visions were not lying (or being extremely sloppy in a systematic way, which is their only hope of explaining this—see below), the chance the 73 most recent polls would add to 100 is less than 1 in 2,000. (That assumes intentional rounding to minimize 99s and 101s. For any pre-determined rounding rule, however, we’re talking 1 in 10,000,000 or worse. Maybe a real pollster can fill me in on what’s industry practice among those who aren’t lying.)

One-in-two-thousand-ish stuff happens all the time, but believe me, I didn’t find this on a data dredging excursion. I noticed that the first few poll results I saw added to exactly 100, I formulated a plausible hypothesis based on all the evidence at hand, I carried out an experiment, and I calculated the p-value.* Small enough to be incriminating in my home court, and I tend to be a benefit-of-the-doubt guy.

Just two more things. First, the possibility of systematic sloppiness:

The consistent adding-to-100 could be the result of a systematic error, as opposed to cheating. Of the possible excuses, this is the one I suggest SV choose if they decide not to come clean. Logically it can’t be distinguished from lying, and they can attribute it to a whipping boy like the web site designer. (This excuse doesn’t defend against the mountain of mathematical bricks I mentioned earlier, however.) They can say that they made a regrettable decision that to avoid the appearance of error, they calculated one percentage in each survey from the other percentages, not from the survey results. I won’t believe it for a minute (though if they show me their programs or confirm that some commercial product makes this error, I’ll reconsider), but it might get them out of hot water.

And second and last, an example and a bit of the math behind my calculations:

Most survey results are short lists of whole number percentages that express fractions to the nearest whole percentage point. Suppose 600 likely voters were polled in a tight race between Tintin and Dora the Explorer. Tintin had the support of 287 people, Dora was close behind with 286, and the rest of the 600 people surveyed (that would be 27 of them) said they weren’t sure. To the nearest whole percentages, that’s 48% for Tintin, 48% for Dora, and 5% undecided. The sum of the rounded percentages is 101%, and that’s due to honest mathematics, not fraud.

Let me skip some really fun mathematics and tell you that for survey questions that have three answers, the percentages add to 100 most, but by no means all of the time. Exactly how often they don’t add to 100 depends on several factors, two of which matter much at all in this case: the number of people surveyed, and how numbers ending in .5 are rounded to whole numbers. Strategic Visions, LLC’s usual survey sample size is 800, and even if ending-in-.5 numbers are rounded differently in each poll to avoid a sum of 99 or 101 when possible, three-choice result percentages should still add to 99 or 101 at least one time out of ten.

* Not to be taken as evidence of a non-Bayesian persuasion on my part. The frequentist approach seemed to me pretty straightforward and justifiable here, that’s all.

Today’s clicking (especially from fivethirtyeight.com) led me to two strikingly similar declamatory reports about high school student’s knowledge of civics, complete with chart-laden survey results.

“Arizona schools are failing at [a] core academic mission,” concludes this Goldwater Institute policy brief.

“Oklahoma schools are failing at a core academic mission,” announces this Oklahoma Council of Public Affairs article.

When asked to name the first president of the United States, only 26.5% of the Arizona high school students surveyed answered correctly. Only 49.6% could correctly name the two major political parties in the United States. An even smaller percentage of Oklahoma high school students gave correct answers to these and other questions from the U.S. citizenship test study guide. None of the thousands of students surveyed in either state answered all ten questions correctly.

The shocking thing is that these are garbage studies. Made-up numbers, probably. The acme of vulpigeration. Evil. Makes me sick. (Glad I coined the word, though.)

No way these are real studies. Danny Tarlow over at This Number Crunching Life has taken a mathematical hammer to the Oklahoma “study” quite effectively. (The blatant similarity of the Arizona “study” blows away any shred of possibility that the Oklahoma study is legit. I’d love to see Danny’s face when he sees the Arizona report.)

What’s frightening is that this kind of snake oil has far too good a chance of surviving as fact (which it isn’t) and influencing public policy.

The guilty parties? The Goldwater Institute, which as you might guess is a conservative “think” tank. The OCPA, which describes itself as “the flagship of the conservative movement in Oklahoma.” Matthew Ladner, the author of both reports, who is vice president of research for the Goldwater Institute. And last but not least, Strategic Vision, LLC, which Ladner says “conducted” the studies. In my opinion, the word is concocted. Read about them yourself.

[Updated with correct business name: Strategic Vision, LLC.]