How useful are our intelligence agencies? To what degree are they blinded by prejudice and institutional needs?

Intelligence about foreigners — state and non-state actors — is vital for the US government, of course.  But how useful has US intelligence been in practice?  This post is the first in a series examining several perspectives from the large literature on this important subject.  The other chapters are

(2)  About our intelligence agences: the struggle to find an accurate AND institutionally useful narrative
(3)  A major function of our intelligence agencies is to shape the narrative. They do it well, molding history like clay on a wheel

At one extreme we have totalitarian states (in many forms), in which even technically capable intelligence services can become warped so that they maintain the ruling elites’ view of the world — no matter how inaccurate (for examples of this see Christopher Andrew’s articles listed at the end).  At the other end are the idealized intelligence services of fiction (e.g., Ian Flemming’s Secret Service and Tom Clancy’s CIA).  Real-life western agencies fall somewhere in between.  But where?  Histories of the CIA (such as Tim Weiner’s Legacy of Ashes) often highlight the many failures; a balanced accounting is difficult to find.

Perspective one:  the problem of analysis

From “Ethics and Intelligence“, William M. Nolte (Prof Public Policy, U of Maryland), Joint Force Quarterly, 3rd Quarter 2009:

Analysis has its own ethical considerations, and these largely involve applying the desire to bring truth to power. As often noted, on the wall of the Central Intelligence Agency (CIA) headquarters building is the inscription, “Ye shall know the truth, and the truth shall make you free.” Unfortunately, no one has yet developed the formula by which an intelligence analyst, let alone a whole agency or set of agencies, can know the truth, the whole truth, and nothing but the truth. No analyst in my acquaintance ever received a crystal ball from the supply room. Every analyst has been enjoined to emphasize objectivity and to avoid bias, but the reality is that every analyst — in intelligence, in law, in medicine — brings biases to the case at hand. An emergency room physician, knowing that a life-threatening disease may have many of the same symptoms as a simple case of influenza, but experiencing influenza 500 to 1,000 times more frequently than the life-threatening disease, is highly subject to being fooled by the exception to the rule.

In this case, bias, understood as the formation of conclusions based on the accumulation of data, is both unavoidable and efficient. What would happen to waiting times in the emergency room, not to mention to costs, if a doctor overrode the evidence of hundreds of “routine cases” and ordered extensive tests on every patient to rule out the “1 in 100” or “1 in 1,000” occurrence?

Intelligence analysts use bias in the same way and are therefore subject to the same vulnerability to nonlinear or aberrational events. An analyst who in 1990 had approached colleagues preparing a National Intelligence Estimate on the future of the Soviet Union by suggesting that it would simply go out of business, devolving the Baltics and the Central Asian Republics, and renouncing the political monopoly of the Communist Party, would have been hard pressed to provide evidence to support such an outcome. Even as “an alternative outcome,” that panacea of intelligence reformers, would this outcome have had credibility? Or would it have been dismissed, in large part because the analyst would have found it hard to produce “evidence” supporting that alternative?

Much has been made since 2003 of the politicization of analysis, and the collateral mistake of policymakers in “cherry picking” analysis. First and foremost, politicization, that is, the distortion of analysis to fit a desired policy or political outcome, must be considered the cardinal sin of analysis. To a great degree, politicization can only be done within the intelligence agencies. Once the intelligence reaches policymakers, they will read, react to, and interpret the intelligence within the framework of policy preferences, prior experience, and personal intellectual (or ideological) preconceptions, reform efforts notwithstanding. It is at least possible that intelligence analysis can change the mind of a policymaker.

But this cannot be predicted, and it certainly cannot be legislated. Finally, an analyst or even an analytic agency may fall into the same conventional (though incorrect or obsolete) wisdom shared with policymakers. Analysts and policymakers, as the phrase goes, may be “drinking the same Kool-Aid.”

Perspective two:  past success uncovering secrets, failure to solve mysteries

From “Intelligence analysis needs to look backwards before looking forward“, Christopher Andrew (Prof History, Cambridge), Conference on “New Frontiers of Intelligence Analysis: Shared Threats, Diverse Perspectives, New Communities” in Rome, 31 March – 2 April 2004:

Long-term study of the strengths and weaknesses of Western intelligence, based on the methodologies that already exist, has already begun to provide answers which go beyond anything that analysis of only the recent past can provide. To follow the conventional distinction between ‘secrets’ and ‘mysteries’, during the twentieth century we were frequently very good at discovering our opponents’ secrets when it mattered most but more confused than we should have been by the mysteries of what they intended to do.

The American and British success in discovering the capability and deployment of enemy armed forces shortened the Second World War, stabilised a Cold War which might otherwise have turned into hot war, and has since helped to make possible a series of breathtakingly rapid military victories. The importance of intelligence reports on military strengths and capabilities went far beyond the data they provided.

Studies of the Cold War frequently forget the truth of Eisenhower’s dictum that intelligence on ‘what the Soviets did not have’ was often as important as information on what they did. Shortage of reliable intelligence in the early 1950s generated the destabilising American myths of the ‘bomber gap’ and the ‘missile gap’ — the delusion that the Soviet Union was increasingly out-producing the United States in both long-range bombers and ICBMs.

In 1955 US Air Force intelligence estimates calculated that by the end of the decade the Soviet Long-Range Air Force would be more powerful than U.S Strategic Air Command, whose head, General Curtis Le May, became dangerously attracted by the idea of a pre-emptive strike to prevent the Soviet Union achieving nuclear superiority. The introduction of the U-2 spy-plane in 1956, followed 4 years later by the first imagery intelligence (IMINT) from spy satellites, provided proof that the Soviet nuclear strike force was not overtaking that of the United States. The U-2 missions, wrote Eisenhower, ‘provided proof that the horrors of the alleged “bomber gap” and “missile gap” were nothing more than the imaginative creations of irresponsibility’.

Without the IMINT revolution, US policy to the Soviet Union would doubtless have continued to be confused by other destabilising myths about the extent of the Soviet nuclear strike force. It is difficult to exaggerate the importance of the National Technical Means (NTMs) and the analytical tools devised to interpret them, chiefly by US intelligence, to Western policy during the Cold War.

… Of the ‘mysteries’ which confused us during the Second World War and the Cold War, the one we were worst at unravelling was the mindset of our opponents-in particular, the understanding of fanaticism. That remains a serious problem because of one fundamental, largely unnoticed continuity between the threats that faced us in the 20th century and those that face us in the 21st One of the very few to draw attention to that continuity has been Elie Wiesel, Nobel laureate, holocaust survivor and human rights activist. Several years before 9/11 Wiesel said this: ‘The principal challenge of the 21st century is going to be exactly the same as the principal challenge of the 20th century: How do we deal with fanaticism armed with power?’

Tomorrow’s post provides another perspective three:  the struggle to find an accurate AND institutionally useful narrative

For more information

1. “Stalin and Foreign Intelligence“, Christopher Andrew and Julie Elkner, Totalitarian Movements and Political Religions, June 2003
2. “Intelligence, International Relations and ‘Under-theorisation’“, Christopher Andrew, Intelligence and National Security, June 2004

Other posts about intelligence on the FM website

1. The Plame Affair and the Decline of the State, 25 October 2005
2. The new NIE, another small step in the Decline of the State, 10 December 2007
3. When will global oil production peak? Ask the CIA!, 1 May 2008 — If they don’t know this, they’re useless.
4. A must-read book for any American interested in geopolitics, 5 March 2009 — About Legacy of Ashes
5. Another urban legend that will not die: the CIA is the world’s major drug dealer, 11 July 2009
6. Ignatius proposes “A New Deal for The CIA” – perhaps they should sometimes obey our laws, 21 September 2009
7. How the Soviet Menace was over-hyped – and what we can learn from this, 13 October 2009
8. The CIA’s forecast about the Iranian Revolution – and the revolution prediction tool, 6 January 2010
9. The Flynn report, itself a symptom of deep problems in the government establishment, 11 January 2010
10. Stratfor: “The Khost Attack and the Intelligence War Challenge”, 18 January 2010
11. Iran will have the bomb in 5 years (again), 20 January 2010

Afterword

• For more about this website, see the About the FM website page.
• Send emails to fabmaximus at hotmail dot com (note the spam-protected spelling).
• You can subscribe to receive posts by email; see the box on the upper right.