Thursday, May 15, 2008

The impartial intelligence analyst

The Los Angeles Times describes the efforts of US intelligence analysts to come to terms with politics. Or rather to remain free of it. In particular the article follows Thomas Fingar, who was both designated to lead the overall intelligence effort and is the author of the NIE which estimates that Iran has stopped building a nuclear weapon.

One of Fingar's explicit goals was to free analysis from politics. To achieve this he has created explicit reminders the staff to remain so. "Some of Fingar's first moves were scripted in the legislation that created his job. The law called for basic standards, so analysts now wear cards around their necks reminding them to remain 'independent of political considerations.'"

As an illustration of the new policy to remain free of politics, the LAT provides this puzzling example.

Dissent was encouraged. Attempts to goad students into policy debates were rebuffed. As one young analyst went through the mock exercise of briefing a general who was considering an invasion, she offered a pointed warning. "Once you go into a country and take it over," she said, "it would be best to have a plan."

Based upon what, one wonders? Policy presumably. But there we go again.

Fingar's other initiative was to educate analysts in the correct style of thinking. In order to achieve this the new chief analyst is molding an entire generation of intelligence analysts; building the foundations as it were, of how government will see things. The demographics are impressive.

Nearly half the nation's analysts have joined the government since 2001. To speed their development, Fingar required new hires to take a six-week course called Analysis 101."

In order to impress readers with the scientific bent of the new broom which is sweeping the dusty cobwebs out of the musty old rooms, the LAT writes: "Fingar's team assembled a directory of analysts, the first time that had ever been done. They launched classified versions of the Wikipedia and MySpace websites, so analysts from different agencies could collaborate online."

But the use of social networking tools and Wikis does not in and of itself free analysts from politics. Social networking software like MySpace can in fact be amplifiers of politics. They can create a hermetic universe of group thinkers much faster than manual methods. And while Wikis can place a set of facts in plain view, they do not in and of themselves resolve ambiguities. That's why there are "wars" in Wikipedia and locked threads. One of the more interesting questions that might be asked about Fingar's new approach is whether it avoids politics at all. Very often the goal of 'remaining free from politics' is simply a code word for a project to engage in a rival kind of politics.

"What I liked in him was his analytical style," said Richard Clarke, who was one of Fingar's first bosses before becoming a counter-terrorism advisor to Presidents Clinton and Bush. "He was more open, honest and user-friendly than the intentionally obtuse analysts we sometimes get." ...

The key problem Fingar faces in disassociating analysis from politics surfaced during a recent discussion I had among knowledge management developers who were trying to build a better version of the Yellow Pages. Imagine that you are an immigrant with limited English skills whose washing machine had just gone berserk and flooded the basement. You turn to the Yellow Pages and look under "flood", "basement" or "washing machine" and find nothing that is obviously helpful. You get emergency services, building contractors or appliance stores.

What you really want is a repairman and a plumber. If the immigrant knew the correct category to consult he would find hundreds of tradesmen in his area. But how does he find the right category? The entire Yellow Pages business model is designed around categories, advertising space is even managed by the category. The problem is that a knowledge store founded upon categories is most useful when you already know what the categories mean.

In order to avoid this problem and free users from the tyranny of categories, some developers wanted to work towards creating a network of links among the different ontologies so that the hapless hypothetical immigrant might gradually be led from the idea of flooding to basement flooding and eventually to the right providers. The key problem lay in defining the links between one set of objects and another. The problem was, in practice, of finding ways to progressively constrain the weights among the multiplicity of possible links in order to converge on the right set of associations.

One developer proposed to solve the problem by heading a wiki effort to construct links of appropriate weight between ontologies. In other words, coding a trail of breadcrumbs from one point to another. Others proposed implementing a neural learning network which would track the paths that people traced through Yellow Pages categories in order to empirically learn where, when faced with a recalcitrant washing machine, most sufferers finished up.

But whether deductively or inductively semantics implicitly reared its ugly head. The facts had to be put into a context to make sense. It was never enough to merely observe that a washing machine had flooded a given basement. It order to get to the next step it was vital to establish what the homeowner's utility function was. Depending on the homeowner's preferences, the next call could be to the movers, the demolition man, the travel agent or to the plumber.

In such a universe, what does it mean when an analyst is obliged to wear a card around your neck saying 'I will not think of policy'? And what happens when intelligence estimates, such as Fingar's NIE can produce a political reaction from Iran? The LAT itself hints that Fingar's NIE may have set up a self unfulfilling prophecy. By downplaying the urgency of Iran's nuclear threat, the NIE could have itself weakened UN sanctions against Teheran, making it more likely for Iran to restart what was "stopped".

Even those who defended the report's findings faulted the way it was put together. Fingar's boss, Director of National Intelligence J. Michael McConnell, testified in February that the report had caused such confusion that if he could rewrite it, he would "do some things differently." ...

Fingar said the Iran intelligence report emphasized the halt in warhead work because that was the newest finding. He attributes the attacks to anger among hard-liners that the report didn't conform to their preconceived views.

"The unhappiness with the finding -- namely that the evil Iranians might be susceptible to diplomacy -- adroitly turned into an ad hominem assault," Fingar said. "Why do we have an intelligence community if all you want are cheerleaders?"

The lasting impact of the report on Iran policy has been unclear. Weeks after its release, the U.N. approved new sanctions against Tehran, but they fell far short of what the Bush administration wanted.

Intelligence and policy may prove themselves to be inextricably linked for as long as intelligence influences what policy is made. There may in fact be no such thing as a value-free judgment especially in ambiguous situations, where the value itself may be in the judgment.




The Belmont Club is supported largely by donations from its readers.

22 Comments:

Blogger eggplant said...

Thomas Fingar's style of analysis reminds me of the MSM's standard reply to the accusation of bias, i.e.

"We're presenting the facts based upon our own best judgement of the truth. How can the simple truth be biased?"

5/15/2008 11:11:00 AM  
Blogger Wretchard said...

I think the problem is misdefined. What intelligence analysts really want to remain free from -- and this is the situation in forecasting in general -- is not politics but responsibility.

The question is 'how do I make an assessment without getting it wrong'. Any useful forecasting can only be done in the context of knowledge that there will always be consequences. But bureaucratically, no one wants to be tasked with that burden.

5/15/2008 11:24:00 AM  
Blogger Katchoo said...

And while Wikis can place a set of facts in plain view, they do not in and of themselves resolve ambiguities. That's why there are "wars" in Wikipedia and locked threads.

The alternative is far worse. In the Conservapedia, which is a kind of mini-Wikipedia but corrected to undo what they perceive is a liberal bias, the article on Operation Iraqi Freedom lists the casualties by "Coalition Troops" and "Enemy Casualties". In other words, when Wikipedia merely lists the Iraqi casualties, they are being "liberal" because they don't label them the enemy.

5/15/2008 11:34:00 AM  
Blogger Aristides said...

Wretchard, I wrote a post over at The American Scene which is germane to this thread. (It's here.) Read the articles on Bruce Bueno de Mesquita that I link to. I think you'll find them interesting.

Also, I'm not sure if you recall, but I also suggested a while back that "public domain" has gotten so large that a private consortium of concerned citizens could operate as intel analysts, should they create and maintain sub-networks connecting (ideally) the top 25 experts on any given nation, people, subject or whatever, and accumulate their analyses and insights on a single website.

I got this idea from the policy paper Information Operations: Putting the 'I' back in DIME, which I summarized here. I still thinks it's a good idea.

5/15/2008 11:41:00 AM  
Blogger Wretchard said...

Aristedes,

The whole area of inquiry that you touch is one of the hinges on which many things will turn. There are unresolved issues in that regard and it's a safe bet that people are working on them.

5/15/2008 11:46:00 AM  
Blogger hdgreene said...

Did Mr. Fingar call the Iranians evil? Let's see.

"The unhappiness with the finding -- namely that the evil Iranians might be susceptible to diplomacy -- adroitly turned into an ad hominem assault," Fingar said.

Hmm. Or is that the clever use of irony by Mr. Fingar? Maybe the Iranians are not the evil ones; maybe it is the unhappy ones, those who issue ad hominem assaults, who are evil. Perhaps he uses Sarcasm. Which means the Iranians, far from being evil, are susceptible to diplomacy. But this sounds like he is promoting a policy position through the adroit use of sarcasm. Is that allowed? Perhaps we should think of Mr. Fingar as the Shadow National Security Advisor.

Mr. Fingar sounds like a typical left intellectual. They don't have policy preferences, just superior insight and knowledge. They are "the reality based community" because they can make two plus two equal their preferred sum simply by using a clever retort and changing the subject.

Is Mr. Fingar advising the Obama campaign? We'll find out when he's fired.

5/15/2008 11:49:00 AM  
Blogger Peter Grynch said...

This got me thinking: The problem is not that what you know is wrong, the problem is that you don't know that what you know is wrong!

There we go, the whole divide between liberals and conservatives summed up in one sentence.

5/15/2008 11:58:00 AM  
Blogger DougLoss said...

Read Ken Timmerman's "Shadow Warriors" to find out about Fingar. He's part of the CIA group that's essentially treasonous, doing its best to subvert and frustrate any effort of the Bush administration that the CIA leftwing bureaucrats don't like.

5/15/2008 12:04:00 PM  
Blogger eggplant said...

Hdgreene said:

"Or is that the clever use of irony by Mr. Fingar? Maybe the Iranians are not the evil ones... Mr. Fingar sounds like a typical left intellectual. They don't have policy preferences, just superior insight and knowledge."

I mentioned this in an earlier thread and will refer to it as the "Moonbat Shahada":

"War is terrorism. The United States is involved in war. Therefore the United States practices terrorism and is no more moral than any other terrorist."

This is the moonbat core belief. If you assume that Mr. Fingar buys into the Moonbat Shahada then his behavior can be easily understood.

5/15/2008 12:22:00 PM  
Blogger Insufficiently Sensitive said...

With the CIA and State Department in hands considered unreliable by the Administration - who's supposed to be their boss - there's little wonder that Dick Cheney did his best to set up alternate intelligence sources in hopes of piercing their PC fog. As an experienced executive, Cheney would understand the enormous importance of uncorrupted intelligence.

And it's no wonder that they would hate him for the attempt.

5/15/2008 12:23:00 PM  
Blogger Mark said...

This comment has been removed by the author.

5/15/2008 12:40:00 PM  
Blogger Mark said...

Thank you hdgreene for an entertaining post. . . . And let's dwell further upon Mr. Fingar's superior insight:

The DOD, in 2003 I believe, floated the idea of a PAM (Policy Analysis Market) that would have provided the U.S. military and intelligence (sic) agencies with access to market responses regarding various events. Trading in these events, as in the case of commoditites futures, would have produced price movements that would provide more insight into proposed policy decisions than could be gained from traditional analysis alone. Why rely entirely or even primarily on analysts when there are folks with boots on the ground, and in the banks, who have money on the table and will make bets accordingly?

So, who among the intelligence analyst community is betting his own money 'with some confidence' etc. that Iran is not actively pursuing nuclear weapons capabilities? Judging from his own report, as a whole, I suspect that Fingar would not be betting his family farm, or federal pension.

Money talks. S**t walks . . . . except when the public is paying the salaries of the analysts.

5/15/2008 12:46:00 PM  
Blogger Whitehall said...

The analytical approach here is well-trod in the artificial intelligence community. They too have for decades tried to structure "knowledge" into general trees and structures.

They've had little luck, at least on the global knowledge thing.

In fact, my conclusion about why AI is a deadend here relates to the CIA's problem.

What is the difference between a machine and a human? My answer is "will" or "intent." Does a machine have values or goals? No.

Likewise, a CIA analyst must be motivated by intent. We're paying them to help protect the citizenry but they too often have goals, values, and intents that don't match that.

There can be no automated wisdom.

5/15/2008 12:56:00 PM  
Blogger Martin said...

This post and the article it is about are depressing.

I've staffed policy people in State and local govt in the US for my whole career, 34 years and counting.

A good adviser cannot and should not try to divorce politics or policy from his technical work. Trying to do so makes the technical work shoddy and irrelevant or worse. It is also impossible to fully do.

The trick is to recognize that you as an analyst are not MAKING policy, you have to accept the policy and political directions you are given, and dedicate yourself to helping advance them, working with your policy superiors. You can recommend changes in policy, but in the end the decisions are made by those vested with that authority, not you.

If you can't work to support your organization's policies because of disagreements, you inform your superiors and request transfer to an area where you can be supportive; if all else fails, you leave.

This is not rocket science, but apparently it's beyong the US government's senior intelligence (sic) managers.

5/15/2008 01:18:00 PM  
Blogger Genji said...

Wretchard... one begins to get more of an idea of what it is that you do for a living... Personally, when I hear the word 'ontology' I reach for my revolver... Perhaps should get your clients to google 'Cyc' and/or 'Jorn Barger' - either should be enough to send them running for the hills.

'We just need a bit of AI to handle this part....(insane laughter)'

5/15/2008 01:24:00 PM  
Blogger RWE said...

I recall hearing that when both India and Pakistan conducted an unforecast series of nuclear weapons tests in the 1990’s U.S. intelligence analysts were said to have ignored the clear statements of intent by the two countries’ leaders because “Those people are politicians and politicians all lie.”

Admittedly, this was during the Clinton Admin. But aside from the rather clear lying that was going on in U.S. presidential circles at the time, the nuclear tests were especially upsetting to Bill Clinton, whose foreign policy was based on the “everything’s coming up roses, so it is time to party hardy” and “I’d like to buy the world a Coke” theories of diplomacy.

Analysts may be able to ignore policy - but they will never be able to ignore the rampart mythology that courses through bureaucracies and ultimately defines the outlook on just about everything.

5/15/2008 01:39:00 PM  
Blogger hdgreene said...

Mark, I noticed that in the profession of Middle East Expert you thrive by being wrong. This has been true for as long as I have been following the region. Way back when it was still called the McNeil-Lehrer Report on PBS. The "expert" would sit there while the Palestinian spoke of ending the Israeli occupation of Israel. Of course he'd leave the "of Israel" part off, but that is what he meant. Instead of clarifying what was said, the expert would nod in agreement. So, I figured the expert was wrong.

Of course he was only "wrong" from the point of view of an American living in the Midwest thinking in terms of US interest. But if he's looking for a cushy position at a Saudi funded think tank, why, he was -- if I may paraphrase the President of Iran here -- "dead, rotting corpse, right."

So they bet their little hard scrabble farms and hope to settle into a plush little Oasis. That was my impression, at least. Also, I want to practice my ad hominem attacks.

5/15/2008 01:59:00 PM  
Blogger Fat Man said...

If Thomas Fingar is teaching the rookies (fresh from their indoctrinations in the ivy league) how to do analysis, we are in deep kim-chi. I'll second eggplant.

Like the MSM, this is guy who is working for the Democrat agenda (sell out the US as fast as possible, so we can have socialized medicine) but is completely unable see and understand how intensely partisan he is.

One of George Bush's gravest errors was not having the entire intelligence apparatus hanged after 9/11, from George Tenet on down they all had failed to do their utmost.* They then spent the next 8 years repaying that act of mercy with betrayal and sabotage. No hell is deep enough for them.

*In 1756, the British Admiralty sent Admiral John Byng to prevent the French from taking Minorca. Byng arrived when the island was already under siege, and, after an indecisive naval engagement, withdrew without relieving the siege. Byng was court-martialed and hanged for "failure to do his utmost."

French author Francois Marie Arouet (1694 - 1778, "Voltaire") had his fictional character Candide witness such a hanging in the eponymous novel and remark:

"Dans ce pay-ci, il est bon de tuer de temps en temps un amiral pour encourager les autres."
"In this country it is good to kill an admiral from time to time, to encourage the others."

5/15/2008 02:37:00 PM  
Blogger Derek Kite said...

I must say that this post is disconcerting and unsurprising at the same time.

Fingar is political. If he thinks he isn't, he is a fool.

How would Fingar, or anyone in the CIA, define an excellent analyst?

A team player? Bah.

Experience? Again, Bah. Experience could mean being wrong dozens of time without recognizing it.

How about being right about what you specialize in. And not being wrong about the same thing more than once.

Unfortunately, that would disqualify 95% of analysts, including most of the upper echelons.

So the solution is to define excellence as something 95% of analysts can meet. Makes the situation that exists in most colleges where you can get 85-90% marks, and are nowhere close to being capable of doing what you are ostensibly learning.

Ugly.

Derek

5/15/2008 09:03:00 PM  
Blogger section9 said...

What's often not understood about the NIE was that it is supposed to be a classified document.

I will give you dimes, dollars, to doughnuts that Fingar coordinated the writing of the NIE sure in the knowledge that it was going to leak to the Times and the Post.

The notion that intelligence analysts are supposed to divorce political considerations from what they do is simply a lie.

5/16/2008 06:42:00 AM  
Blogger Ralph Hitchens said...

Based on my 20 years in the intelligence analysis business, I think the vast majority of analysts are politically "subdued," i.e., undoubtedly have some partisan identification but keep it under wraps most of the time. Those who don't stand out clearly, and usually don't get to the table with the big dogs. I believe wretchard hit the nail in his first comment, that nearly all analysts dread having to publish unambiguous conclusions in many cases because no one wants to be seen as having gotten it wrong. And by the way, fat man, it wasn't the analysts who deserved hanging after 9/11, it was the DO and the Feebs. Had the former shared info about participants in the now-famous Malaysia meeting with the latter, odds are two of the al-Q terrorists would have been nabbed early and the plot unravelled. (Just imagine what America would be like today, if 9/11 hadn't happened!) Section9 is right, however, that analysts can't completely divorce political concerns from their end product -- it was pretty obvious that the KJs of the Iran NIE would be leaked, so why not leak it officially right off the bat? Which means that there must have been an unusually high degree of unanimity around the table. As for analytical surprises like the Indian & Pakistani nuclear tests, we simply didn't know enough and were reduced to guessing. Not enough transparency into test preparations, and therefore it became a simple exercise in "will they (defy world opinion) or won't they?" We delayed guessing until it became a fact -- back to what wretchard pointed out.

5/16/2008 11:56:00 AM  
Blogger Fat Man said...

Ralph: If we had hanged or at least fired, them all. Then none of the guilty would have escaped.

There is no upside in letting any of them carry on. Letting Fingar reproduce is a violation of the Darwinian imperative.

5/17/2008 07:26:00 PM  

Post a Comment

Links to this post:

Create a Link

<< Home


Powered by Blogger