Blogs 2

It's interesting to compare the previous post at the Belmont Club, "Blogs" with a very long piece in the New York Times describing a changes to counterterrorist intelligence gathering. The convergence of ideas between what has historically happened within some intel agencies since 9/11 and the Belmont Club piece is amazing. Of course, the NYT is describing how some of those ideas have been implemented within US intelligence to achieve what it calls "open source spying". (This is one of the areas where the NYT gets it wrong, or at least where its subject matter -- spying -- starts to distort the description of what is really a general process. But we are getting ahead of ourselves.) First, let's read how intel is gathered through the process of ... blogging.

---

Andrus argued that the real power of the Internet comes from the boom in self-publishing: everyday people surging online to impart their thoughts and views ... logs, Andrus noted, had the same effect: they leveraged the wisdom of the crowd. When a blogger finds an interesting tidbit of news, he posts a link to it, along with a bit of commentary. Then other bloggers find that link and, if they agree it’s an interesting news item, post their own links pointing to it. This produces a cascade effect. Whatever the first blogger pointed toward can quickly amass so many links pointing in its direction that it rockets to worldwide notoriety in a matter of hours. ...

Spies, Andrus theorized, could take advantage of these rapid, self-organizing effects. If analysts and agents were encouraged to post personal blogs and wikis on Intelink [a classified network] — linking to their favorite analyst reports or the news bulletins they considered important — then mob intelligence would take over.

This will all be very familiar to anyone who has been watching blogs and the Internet. Now in completely unclassified world, blogs arise spontaneously. In any area one can think of. However, some institutions, such as Global Voices, actually encourage the formation of blogs in Third World countries. This is the equivalent of seeding sensors. Or if you like, deploying agents. They may not be "secret agents", and could simply be describing daily life in Kazakhstan, for example, but in principle, it is a sensor with a particular sensitivity. The one concept that the NYT article highlights which the Belmont Club post "Blogs" did not, is the use of Wikis to aggregate the "state of knowledge". However, "Blogs" uses another concept that of virtual "threads" to which any document on the Internet belongs. These threads can be tracked and indexed over a web service which any document joins or leaves over the course of its life. It is arguably a similar concept and in some ways more subtle than the Wiki. But anyway here's how the Wiki concept was used within the intelligence network.

Fingar and Wertheimer are now testing whether a wiki could indeed help analysts do their job. In the fall of 2005, they joined forces with C.I.A. wiki experts to build a prototype of something called Intellipedia, a wiki that any intelligence employee with classified clearance could read and contribute to. To kick-start the content, C.I.A. analysts seeded it with hundreds of articles from nonclassified documents like the C.I.A. World Fact Book. In April, they sent out e-mail to other analysts inviting them to contribute, and sat back to see what happened.

By this fall, more than 3,600 members of the intelligence services had contributed a total of 28,000 pages. Chris Rasmussen, a 31-year-old “knowledge management” engineer at the National Geospatial-Intelligence Agency, spends part of every day writing or editing pages. Rasmussen is part of the younger generation in the intelligence establishment that is completely comfortable online; he regularly logs into a sprawling, 50-person chat room with other Intellipedians, and he also blogs about his daily work for all other spies to read.

But that is no panacea. Here the spy Wiki runs into the same problem described in the Belmont's "Posts". How can collection be counterchecked? The NYT article never says how other than to claim that enough eyes scanning Wiki entries will ferret out an error. In Belmont's "Blogs", the principle of "discovery" focuses eyes on a thread and makes error spotting statistically more likely in that any truly knowledgeable person writing on a particular subject will very probably become aware of a thread in which he will see any error. And if he does, the enhanced messaging will make it possible to suggest a correction. Here's how the NYT article describes the spy wiki, Intellipedia.

Yet Intellipedia also courts the many dangers of wikis — including the possibility of error. What’s to stop analysts from posting assertions that turn out to be false? Fingar admits this will undoubtedly happen. But if there are enough people looking at an entry, he says, there will always be someone to catch any grave mistakes.

One problem was that even with the Intellipedia, analysts could write their blogs in ignorance of each others work. This had two effects. First analyst/bloggers would still feel they were working alone, without the encouragement and input from others; and second information -- whether of error or new developments -- discovered by the one were not always known to the others.

There was never a tipping point — “never a moment when two people who never knew each other could begin discussing something,” as Clay Shirky, a professor at New York University who was hired to consult on the project, explained to me. For the intelligence agencies to benefit from “social software,” he said, they need to persuade thousands of employees to begin blogging and creating wikis all at once. And that requires a cultural sea change: persuading analysts, who for years have survived by holding their cards tightly to their chests, to begin openly showing their hands online.

Is it possible to reconcile the needs of secrecy with such a radically open model for sharing? Certainly, there would be merit in a system that lets analysts quickly locate like-minded colleagues around the world to brainstorm new ideas about how the Iraqi insurgency will evolve. But the intelligence agencies also engage in covert operations that ferret out truly incendiary secrets: the locations of Iranian nuclear facilities, say, or the name of a Qaeda leader in Pakistan. Is this the sort of information that is safe to share widely in an online network?

This is where Belmont's "Blogs" with its concept of automatic discovery, coupled with the ability of the user to limit his "discoverability" may have inherent advantages over the models described by the NYT. The Belmont concept was designed so that bloggers could find related information even while they were writing, enabling them to potentially change the course of their analysis in midstream. But the key Belmont idea, that of defining a blogger interface standard which will allow for discovery and messaging between threads is not described in the NYT article on spy-blogging at all. It is an essential part of any effort to allow blogs to fully take their place within the collection-analysis-distribution cycle.

None of this is to argue for the superiority of one concept over the other. Far from it. This post, like "Blogs" is actually part of a thread that includes the NYT article. In that context, it is the perfection of the thread, rather than the rival merits of the fibers, that we should attempt to focus on. On the contrary, it extremely gratifying to see how very similar ideas can arise in different contexts like the US intelligence agencies and the mind of one individual. One of the reasons many mathematicians believe in Supreme Being is their conviction that they have never "invented" a structure that didn't already exist in the abstract before they became aware of it. Somewhere out there exists the Perfect Blog -- or at least a better blog architecture and sooner or later someone will define it and revolutionize the Internet even further.