Under the headline “Rise in TB Is Linked to Loans From I.M.F.”, Nicholas Bakalar writes for the New York Times today that “The rapid rise in tuberculosis cases in Eastern Europe and the former Soviet Union is strongly associated with the receipt of loans from the International Monetary Fund, a new study has found.”

The study, led by Cambridge University researcher David Stuckler, was published in PLoS Medicine and is online at (URL may wrap):

http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0050143&ct=1.

Cambridge, Schmambridge. First clue: the Times quotes Stuckler: “When you have one correlation, you raise an eyebrow,” Mr. Stuckler said. “But when you have more than 20 correlations pointing in the same direction, you start building a strong case for causality.”

In twenty post-communist countries, the variable “participated in an IMF loan program in year Y” was significantly negatively associated with “TB rate per 100000 people,” whether using rate of cases, of deaths, or of new cases.

After reading the paper and looking at much of the source data, I agree with William Murray, an IMF spokesman also quoted in the article: “This is just phony science.”

Why do I agree with Murray?

Take the supporting table below, for example. It shows all the TB mortality data from “did not participate in an IMF loan program” years: year-to-year percentage changes in TB mortality rates (based on Logs) [sic]. Among the 45 values are 31 0.00s and nothing else close to zero. Almost half the nonzero values are from Poland and Hungary, but—oddly—the change is nonzero in odd-numbered years and zero in even-numbered years. There are four -22.31s, two -18.23s, a 15.42 and a -15.42, a 13.35 and a -13.35, and four stray values, one of which is -69.31. Now I know -0.6931 from calculus (the natural log of ½), and I googled 0.2231: it’s the natural log of 0.8. (There were about four times as many “did participate” country-years, for a total of 200+ data points.)

table

If you haven’t guessed, the data here, which mostly express stable or declining TB mortality, and which found the entire study, and which the authors attribute significantly to non-participation in IMF loan programs, are 4-significant-digit percentage changes between logs of adjacent very small positive integers. The small integers are from the Global Tuberculosis Database, queryable here: http://www.who.int/globalatlas/dataQuery/default.asp. This WHO data is rounded to whole numbers and for the countries and years studied, ranged between 1 and 20.

While this data is crude, I don’t doubt the study’s main finding: among post-communist countries, “participated in an IMF loan program in year Y” was significantly negatively associated with “TB rate per 100000 people.” What I doubt is that the relationship has anything to do with the IMF loan program.

The timeframe studied was 1989 to 2003, and a quick look at the data reveals a pattern to which are the “in an IMF loan program” years for the countries studied. Most countries began participating in 1991, 1992, or 1993, and most countries continued their participation through 2003, the end of the study timeframe. During this time, TB was on the rise, and there’s no question the mid nineties were not a typical period.

While the authors mention many correction strategies and tests to avoid one or another kind of bias, they didn’t mention the way in which “in program” years were distributed as one potential confounder. I can’t see how they ruled it out. There data isn’t there. From 1994 to 1997, there are only 10 “not participating” data points, mostly from Czech Republic, Slovenia, and Poland, which countries were anomolous in having shown no increase in TB during their IMF years. Some countries, Bosnia for example, seem to have been omitted from this part of the analysis, despite having participated in an IMF loan program and data being available from WHO.

The countries studied included Russia, with 140,000,000 people, as well as Estonia, Latvia, Macedonia, Slovenia, Albania, Armenia, Bosnia, Lithuania, counted together having less than 20% of Russia’s population. The authors acknowledge the possibility of ecological fallacy with little investigation. Summary statistics, such as the mean and standard deviation of TB rate among the countries, are unweighted by population, and fail to reflect the real situation. Over one time period quoted, the number resulting from taking the average of each countries TB rate, unweighted for population, went up 30%, but the TB rate among the population under study in fact doubled. Whether this changes any interpretation, I can’t say, but it does make a difference.

Not only am I not a statistician, I’m not an economist, and I have no idea whether the IMF did great things or not in mid-nineties eastern europe and former Soviet Union. But Stuckler and colleagues haven’t convinced me of anything.