Select Page

In Search of the Perfect Search

The issues associated with Electronically Discoverable Information (ESI) hang over the legal profession like the threat of Katrina II hangs over New Orleans. Lets face it: most judges and attorneys would do anything to avoid confronting the complexities of ESI. However, judges are good at forcing lawyers to face up to bad stuff, so it’s impossible to avoid the subject.

Of course, in a huge case involving large sums of money it’s no problem hiring a consulting firm that does all the work for the lawyers, and guides them every step of the way. However, that’s only 1 case in 100, if that. What about all the “little cases,” where expensive consultants are not an option?

The answer, not surprisingly, is the “keyword search.” After all, if we can search a trillion documents using Google, why not use key word search to find documents relevant to litigation.

Sadly, key word search is not very reliable. For example, if you have a million documents how would you formulate a key word search that would be certain to collect documents that relate to to people under age 12?  In fact, recently the courts have noted the shortcomings of key word searching and criticized its use.

An excellent article in the April 2009 ABA Journal discusses these issues in some detail, and the good news is that some “very smart people” are working on a solution. The bad news, as I said, is that key word search is not very reliable, and this poses a problem until the problem is solved by these masterminds. To quote from the article:

It would be the ultimate discovery for e-discovery: a perfect method to turn terabytes of digital data into a collection of case-relevant documents.

Three years ago, a handful of lawyers and scientists started the quest, a project to save litigation from being bur­ied in an avalanche of electronic documents. Since then, the Text Retrieval Confer­ence Legal Track has been using different types of computer searches to wade through huge piles of digital in­formation, hoping to get closer to a complete picture of what is issue-important in a computer’s data stores.

The good news: The TREC Legal Track team believes it is close to finding a protocol that can work. The bad: The project also found disturbing problems with the way lawyers work today.

And the harshest conclusion: Key­word searching—what most law­yers use to find litigation documents—misses the majority of relevant documents. Or as Jason Baron, one of the Legal Track study coordinators, puts it, “Lawyers need to understand that the way they have been searching for electronic documents has some serious flaws.”

So, just what is the “TREC Legal Track”? Well, the TREC Legal Track web page has a number of documents that discuss the efforts underway. The papers are, based on my perusal, not for the faint of heart – they involve technical assessments of the effectiveness of various search. Lets hope we can get by on old-fashioned key word searches until the happy day arrives when the TREC folks, or someone, finds that protocol.

ESI and Admissibility

After writing the post immediately below it occurred to me that although there is much talk about the discovery of electronically stored evidence (ESI), the admissibility of ESI is addressed far less often. In fact, in the two day conference I linked to in that post, the topic is not even mentioned.

For the interested, there are two important starting places for this topic. The first is the 101 page decision in Lorraine v. Markel American Insurance Company by Magistrate Judge Paul Grimm (one of the “rock star judges” mentioned in the ABA article), and the second is The Next Frontier: Admissibility of Electronic Evidence (Listrom, Harlan, Ferguson and Redis). (Note: this last link is on the ABA website and appears to require an ABA membership user name/password; as yet I am unable to locate a copy anywhere else).