Monday, December 04, 2006

Blogs 2

It's interesting to compare the previous post at the Belmont Club, "Blogs" with a very long piece in the New York Times describing a changes to counterterrorist intelligence gathering. The convergence of ideas between what has historically happened within some intel agencies since 9/11 and the Belmont Club piece is amazing. Of course, the NYT is describing how some of those ideas have been implemented within US intelligence to achieve what it calls "open source spying". (This is one of the areas where the NYT gets it wrong, or at least where its subject matter -- spying -- starts to distort the description of what is really a general process. But we are getting ahead of ourselves.) First, let's read how intel is gathered through the process of ... blogging.

Andrus argued that the real power of the Internet comes from the boom in self-publishing: everyday people surging online to impart their thoughts and views ... logs, Andrus noted, had the same effect: they leveraged the wisdom of the crowd. When a blogger finds an interesting tidbit of news, he posts a link to it, along with a bit of commentary. Then other bloggers find that link and, if they agree it’s an interesting news item, post their own links pointing to it. This produces a cascade effect. Whatever the first blogger pointed toward can quickly amass so many links pointing in its direction that it rockets to worldwide notoriety in a matter of hours. ...

Spies, Andrus theorized, could take advantage of these rapid, self-organizing effects. If analysts and agents were encouraged to post personal blogs and wikis on Intelink [a classified network] — linking to their favorite analyst reports or the news bulletins they considered important — then mob intelligence would take over.

This will all be very familiar to anyone who has been watching blogs and the Internet. Now in completely unclassified world, blogs arise spontaneously. In any area one can think of. However, some institutions, such as Global Voices, actually encourage the formation of blogs in Third World countries. This is the equivalent of seeding sensors. Or if you like, deploying agents. They may not be "secret agents", and could simply be describing daily life in Kazakhstan, for example, but in principle, it is a sensor with a particular sensitivity. The one concept that the NYT article highlights which the Belmont Club post "Blogs" did not, is the use of Wikis to aggregate the "state of knowledge". However, "Blogs" uses another concept that of virtual "threads" to which any document on the Internet belongs. These threads can be tracked and indexed over a web service which any document joins or leaves over the course of its life. It is arguably a similar concept and in some ways more subtle than the Wiki. But anyway here's how the Wiki concept was used within the intelligence network.

Fingar and Wertheimer are now testing whether a wiki could indeed help analysts do their job. In the fall of 2005, they joined forces with C.I.A. wiki experts to build a prototype of something called Intellipedia, a wiki that any intelligence employee with classified clearance could read and contribute to. To kick-start the content, C.I.A. analysts seeded it with hundreds of articles from nonclassified documents like the C.I.A. World Fact Book. In April, they sent out e-mail to other analysts inviting them to contribute, and sat back to see what happened.

By this fall, more than 3,600 members of the intelligence services had contributed a total of 28,000 pages. Chris Rasmussen, a 31-year-old “knowledge management” engineer at the National Geospatial-Intelligence Agency, spends part of every day writing or editing pages. Rasmussen is part of the younger generation in the intelligence establishment that is completely comfortable online; he regularly logs into a sprawling, 50-person chat room with other Intellipedians, and he also blogs about his daily work for all other spies to read.

But that is no panacea. Here the spy Wiki runs into the same problem described in the Belmont's "Posts". How can collection be counterchecked? The NYT article never says how other than to claim that enough eyes scanning Wiki entries will ferret out an error. In Belmont's "Blogs", the principle of "discovery" focuses eyes on a thread and makes error spotting statistically more likely in that any truly knowledgeable person writing on a particular subject will very probably become aware of a thread in which he will see any error. And if he does, the enhanced messaging will make it possible to suggest a correction. Here's how the NYT article describes the spy wiki, Intellipedia.

Yet Intellipedia also courts the many dangers of wikis — including the possibility of error. What’s to stop analysts from posting assertions that turn out to be false? Fingar admits this will undoubtedly happen. But if there are enough people looking at an entry, he says, there will always be someone to catch any grave mistakes.

One problem was that even with the Intellipedia, analysts could write their blogs in ignorance of each others work. This had two effects. First analyst/bloggers would still feel they were working alone, without the encouragement and input from others; and second information -- whether of error or new developments -- discovered by the one were not always known to the others.

There was never a tipping point — “never a moment when two people who never knew each other could begin discussing something,” as Clay Shirky, a professor at New York University who was hired to consult on the project, explained to me. For the intelligence agencies to benefit from “social software,” he said, they need to persuade thousands of employees to begin blogging and creating wikis all at once. And that requires a cultural sea change: persuading analysts, who for years have survived by holding their cards tightly to their chests, to begin openly showing their hands online.

Is it possible to reconcile the needs of secrecy with such a radically open model for sharing? Certainly, there would be merit in a system that lets analysts quickly locate like-minded colleagues around the world to brainstorm new ideas about how the Iraqi insurgency will evolve. But the intelligence agencies also engage in covert operations that ferret out truly incendiary secrets: the locations of Iranian nuclear facilities, say, or the name of a Qaeda leader in Pakistan. Is this the sort of information that is safe to share widely in an online network?

This is where Belmont's "Blogs" with its concept of automatic discovery, coupled with the ability of the user to limit his "discoverability" may have inherent advantages over the models described by the NYT. The Belmont concept was designed so that bloggers could find related information even while they were writing, enabling them to potentially change the course of their analysis in midstream. But the key Belmont idea, that of defining a blogger interface standard which will allow for discovery and messaging between threads is not described in the NYT article on spy-blogging at all. It is an essential part of any effort to allow blogs to fully take their place within the collection-analysis-distribution cycle.

None of this is to argue for the superiority of one concept over the other. Far from it. This post, like "Blogs" is actually part of a thread that includes the NYT article. In that context, it is the perfection of the thread, rather than the rival merits of the fibers, that we should attempt to focus on. On the contrary, it extremely gratifying to see how very similar ideas can arise in different contexts like the US intelligence agencies and the mind of one individual. One of the reasons many mathematicians believe in Supreme Being is their conviction that they have never "invented" a structure that didn't already exist in the abstract before they became aware of it. Somewhere out there exists the Perfect Blog -- or at least a better blog architecture and sooner or later someone will define it and revolutionize the Internet even further.


Blogger RWE said...

When launching a space mission you have to become proficent in listening to multiple separate "threads", called "nets" and making decsions based on all that information.

There will be "Net 1," which is the master countdown net, and also nets covering the payload activities, the launch range, weather measurement, the satellite control network, etc. Then there are the phone calls from indivdual to individual separate from the nets and usually some radio communications as well.

Perhaps one approach would be to have the ability to run all such related nets or threads simultaneously and separately - but to enable them all to be viewed simultaneously if desired.

Simultaneous threads happen here to a rather limited extent, and even contain links to yet other threads, but viewing them all simultaneously is not very feasible without a lot more hardware. It need not be that way, given even existing technology.

Development of such "Launch Director" blogging software could facilitate such non-recreational use of the concept. And it might cause people to think and utilize the the blogging concept in different ways as well.

12/04/2006 03:36:00 PM  
Blogger wretchard said...

A friend wrote to ask whether "Blogs" was written after I had read the NYT article. I wrote on the threads and discovery concept weeks before in a correspondence with a developer, but a lot of discussion must have already been taking place on this subject of which I was unaware. However the NYT article did convince me that there may be something valid in the general "Blogs" concept which can be exploited

12/04/2006 03:51:00 PM  
Blogger rasqual said...

I've long wished that usenet -- now buried under a mountain of unindexed web forums -- would have evolved into a tighter citation medium. It wouldn't "quote" in the traditional sense; newsreaders of the future would be doing SQL joins, of sorts, between one post and others. Not a perfect analogy. Every word is indexed and any quote is checksummed to the original. The instantaneous checksum would be stored with new messages, and if the underlying "quote" changed, the disparity would flag the citation as a low-confidence quote.

I just don't think quotes should need to be checked in the future. I think quotes should be instantly accessed instances of the original, and whether the original has changed since it was "quoted" should be capable of being determined.

Now imagine a usenet thread with this kind of "quoting" instead of the usual literal string quotes. Radically different. Not flat text at all.

12/04/2006 09:40:00 PM  
Blogger orlandoslug said...

Yes, I believe in God...

the beauty around me and the eloquence are inexplicable, and can be somewhat mimmicked; but, to what end?

imagine the inner workings of the brain, the dendrites, axions and synapses...
...certain pathways are more robust, the ones that are most trodden and used, these are the connections that are reliable, quickest and easiest to use, that we build upon, and fall back to again and again, as known territory, that have weathered the experiences of time...

knowledge is only as good as your ability to access it... our brain has made connections, and has organized millions of bits of information, each with its own geospacial zip code within the gray matter, organized by interconnections along with other characteristics, perhaps phonetic sounds...

similarly, if one believed that collective wisdom / preponderance of evidence often point to the truth, one could identify words (or groups / patterns of words), of not only of high interconnectivity (node-iness); but, also proximity to well worn pathways as exemplified by high traffic / bandwidth to other important nodes...mathematical equations could be used to derive the relative signal strength which has made it from the corpus luteum, as an indicator of the importance of this connection... that I've lost you with my pseudo-science, I wish I could just draw you a picture, which by the way, we could also rank as a "memory" by the number of times accessed; although, this would involve bulkier, harder to communicate ideas....... this, overlays / filters could be used to say identify recent, newly formed strong signals; or, perhaps filter this corporate knowledge for geographical location of user platforms (say to determine what may be considered "truth" in arab speaking countries), etc. etc.

...would it take alot to propagate such a system and maintain it? (I doubt a poindexter in a small, dimly lit room could keep up with something like this!) Probably, because it's still just a dumb machine, and because alot of what we learn must first start out as a theory, to which we then find the facts to substantiate...

...additionally, could not such an all scanning system not be used for nefarious purposes?? (imagine a small company shanghai-ing technology to say use voice recognition to identify key words (try: gather & protest, bible & worship, guns & shooting, allah & infidel, etc.) used over a cell phone, and subsequently monitoring the contents of that call...scary!!

...but rather than opine over oppenheimer's remorse, before ever taking any action...
...we must remember dark forces do exist...
...if they might, we must!

12/05/2006 01:54:00 AM  
Blogger krishna_kirti said...

One important concept in security is what is called Need To Know (NTK). Just having a security clearance isn't enough. In order to make sure classified information doesn't get into the wrong hands, NTK demands that only the right hands have access to information--only what they need. It's the concept of a white list of who needs the info instead of a blacklist of who doesn't need it.

As regards to unclassified material, the other problem with such a wiki is that you put one piece of unclassified information together with another piece of unclassified material, you might have to classify them anyway because they "connect the dots".

I might be wrong, but an agency-wide wiki sounds like a night-mare for people who review and classify documents.

12/05/2006 02:29:00 AM  
Blogger The Flying Spaghetti Monster (Tyler Cassell) said...


12/05/2006 07:13:00 AM  
Blogger Sparks fly said...

Info Wars.




The riddle.


12/06/2006 12:40:00 AM  
Blogger orlandoslug said...

ahem...perhaps corpus callosum is more fitting in my pseudo-science commentary above...

Kris, the slug has debated whether a completely open discussion of "connect the dot" theories could in fact be giving free insight to forces that would oppose us...

...however, I always rationalized in my mind that anyone monitoring such things would in fact be human and able to cogitate on such things themselves (this could be a big fallacy, if in fact they thought they were being monitored!). Also, I've always hoped that we might drive our worst enemies so far underground, (in order to avoid detection), that they in fact became ineffective...

12/06/2006 03:41:00 AM  
Blogger rasqual said...

It's the "emergent properties" thing, whereby combinations of simple things end up giving rise to properties that can't be explained as the sum of the parts. Something beyond synergy -- "complexity". There's a "duh" factor here, of course -- but yeah, the Wiki thing sounds like quite a nightmare -- especially the moment someone requests an FOIA and some hapless sot is tasked with redacting it (the beloved black bars on so much FOIA material). "Which date's edit would you like us to prepare for you, sir?" Geesh.

12/06/2006 10:14:00 PM  
Anonymous Anonymous said...

Excellent piece, very interesting, it seems too smart, or maybe obvious, for our Intell people to do, I wonder...

absurd thought -
God of the Universe says
ignore enemies' lies

1/18/2007 03:38:00 PM  
Blogger AtomicSEO said...

Atomic SEO I really like this blog when I first signed up here i wasn't sure were to post...

2/18/2007 08:43:00 PM  
Blogger AtomicSEO said...

Atomic SEO I really like this blog when I first signed up here i wasn't sure were to post...

2/18/2007 09:14:00 PM  
Blogger John Tiniakos said...

Hey you have a very good blog.

If by any chance you need weight loss help check out by blog: Weight Loss Tips

2/26/2007 02:50:00 PM  

Post a Comment

Links to this post:

Create a Link

<< Home

Powered by Blogger