One hypothesis weakened

In an earlier post I wondered about the sensitivity of estimates of US foreign aid to the definition of foreign aid; if people included "military involvement" as foreign aid, then their estimates would be biased upwards. But apparently the good people at PIPA already thought of this in an earlier poll (Thanks Andrew, for doing what I was too lazy to do!):

Some have wondered whether the high estimate of foreign aid spending is due to Americans incorrectly including in their estimates the high costs of defending other countries militarily. To determine if this was the case, in June 1996 PIPA presented the following question: US foreign aid includes things like humanitarian assistance, aid to Israel and Egypt, and economic development aid. It does not include the cost of defending other countries militarily, which is paid for through the defense budget. Just based on what you know, please tell me your hunch about what percentage of the federal budget goes to foreign aid. Despite this clarification, the median estimate was 20% and the mean 23%.

Europeans, however, do appear to produce less biased estimates of foreign aid than Americans:

When Europeans are asked how much the government spends on overseas aid from the national budget, approximately one third of respondents do not know. Another third will choose between 1-5 per cent and 5-10 per cent. The smallest proportion will mention less than one per cent.21 The consistent trend across OECD countries is to overestimate the aid effort.

The figures cited appear to be from this report, I think, though the question is not exactly comparable. Most citizens admit they don't know (57% or so). Here's a table (click for larger size):

The correct response is "around 100 Euros per European citizen." (Based on the figures in the table, however, it looks like most Europeans actually underestimate the amount of foreign aid the EU gives - which does not support the conclusion of the other report. I wonder what the results would be if the question were asked in these terms in the USA). Anyway, it seems like the evidence is inconsistent with the hypothesis that high foreign aid estimates are driven by the inclusion of military spending in the results, though the fact that European populations do produce lower estimates of aid spending (even though the questions are not exactly comparable) does suggest that perhaps military spending plays  small role.

Another option: perhaps this is driven in part by national status? "High status" (powerful) countries will tend to have a self-image that includes lots of aid to others. But disaggregated figures for all the EU countries do not appear to be easily available to test this sort of thing (e.g., maybe France, Britain, and Germany produce more incorrect estimates than small, peripheral countries like Latvia and the Czech Republic).

[Update 12/8/2010 - thanks again Andrew: A 1999 Eurobarometer report (p. 11) notes that "Approximately a quarter of Europeans thinks that their government actually contributes to development aid, but does not feel well enough informed to say how much The largest proportions of votes go to the categories « Between 1 and 4% » (14%, -2 since 1996) and « Less than 1% » (10%, -2) Europeans are not far from reality when they make this choice." The question asked then was "We are not talking about humanitarian aid, that is assistance provided in emergency situations like wars. famine, etc, but about development aid Do you think the (NATIONALITY) government helps the people in poor countnes in Afnca, South America Asia etc to develop (I F YES) Roughly how much of its budget do you think the (NATIONALITY) government spends on this aid." The correct answer is "between 1 and 4%". If I'm reading the accompanying table right, Denmark, Finland and Sweden give especially accurate answers - around 40% of people in Denmark give the correct answer.]

Why are estimates of US foreign aid so biased?

A number of people have pointed to the latest reiteration of the fact that Americans do not appear to know what percentage of the budget goes to foreign aid. The median guess is 25% of the total budget, which is far higher than the actual 0.6%. Moreover, as far as I know, for as long as this question has been asked (1995), Americans have always hugely overestimated the percentage of the budget that goes to foreign aid; according to PIPA, the median guess has been about 20%. More educated people guess a bit lower, and less educated people a bit higher, but they mostly err on the high side. But why? As I mentioned in an earlier post, if people estimate such quantities on the basis of unbiased signals, they should converge on the true answer. So what is the source of this bias?

Eric Crampton suggests that voters count a lot of military spending as "foreign aid." This strikes me as plausible. Voters do not have in mind the same technical definition of "foreign aid" that the budget wonks use; they mostly see a large degree of involvement by the US in various countries, some of it justified on "nation building" grounds, which they can easily classify as "foreign aid/involvement." (These are the "signals" that they use to estimate the total amount of aid). And indeed the military accounted for about 23% of federal spending in FY2009 (a bit less this year), depending on how you count, which is close enough to the public guess for "foreign aid."

How would we know if this is what is going on? I wonder if answers to the question fluctuate in ways that are more or less correlated with the foreign wars of the US. Are answers to the question lower in times of peace? (I am too lazy to download the data and crunch it myself. But perhaps some enterprising soul could do it.) Also, has this question been asked in other countries, and does the magnitude of the bias remain constant? Or are the publics of countries with fewer foreign entanglements in war more likely to offer lower guesses of the amount of foreign aid spent? (If anybody kindly points me to easily downloadable date on this, I will make some graphs). I would also like to see a poll that asks this question but primes recipients by explicitly indicating that they are not to count military spending as foreign aid. (E.g., "Just based on what you know, please tell me your hunch about what percentage of the federal budget goes to foreign aid, not counting money spent by the military.") This may well produce a biased estimate, but would it be as biased as the current one? Has some enterprising public opinion researcher asked this question or something similar before?

And I would like to see the question asked in terms of the absolute number of dollars spent. (E.g., "Just based on what you know, please tell me your hunch about how many billions of dollars the Federal government spends on to foreign aid, [not counting money spent by the military]."). Would the estimates be similarly biased upwards? I have a hunch that they might even be biased downwards, and also suspect that asking the question in terms of percentages limits guesses to a degree of coarseness that produces biased estimates. (Foreign aid is 0.6-2.6% of the budget, depending on how you calculate it. Assume people guess the true number based on relatively unbiased signals from the news, including perhaps signals about foreign military involvement, but their guesses are made in 1% increments. Since 0% is an implausible guess, the smallest guess would be 1%, which would inevitably bias the collective estimate upwards, though not necessarily nearly as much as the current estimate. Is this idea too harebrained?)

Another possibility is that answers to this question do not reflect factual beliefs, but rather what Julian Sanchez once called "symbolic beliefs." Here the idea would be that respondents interpret the question as a question about the evaluation of US commitments abroad. The high guesses merely mean "the US spends too much on foreign entanglements," and the 10% median answer to the question of how much the US should spend  merely says something like "whatever it is, halve it." On this view, voters do not really believe that the US should spend 10% on foreign aid, only that it should spend less; educating them about the true amount that the US spends would have only a limited impact on their apparent misperceptions (though could education increase the amount that voters are willing to spend on foreign aid, maybe not to 10%, but perhaps to 3%?). There would be reason to suspect that this is the case if, as Robin Hanson notes, we never see politicians run on increasing foreign aid, even though they could conceivably explain to them that the US actually spends very little on non-military foreign aid.

Could this sort of "symbolic" belief ever be consistently corrected? It would not do to simply tell the voters that the actual value of "foreign aid" is less than 1% of the budget; they might simply adjust their views to say that it should be less, or redefine "foreign aid" to include all sorts of things that the budget analyst would not include (like military spending). Even if the belief were truly a factual and not a symbolic belief, mere provision of information would not necessarily change it: these sorts of quantities are estimated on the basis of signals from the social world of the voters, not merely on the basis of remembered (or misremembered) facts. Since signals are constantly received but mere factual information is not, unless you change the bias in the signals, the public will continue to overestimate "foreign aid" (whatever they actually mean by this).

Other ideas?