The Hypothesis: Simply put electronic discovery is wrong. It is not 100% accurate, far from it. It’s inherently inaccurate. It misses out lots of data. The legal system should find other, more accurate methods.
Out of the ten of thousands of files collected from any one computer only a fraction of those are going to be taken out of the computer for full ED processing.
Out of the small percent that do get processed tens, hundreds or possibly thousands will fail due to errors, corroption, encryption, or other problems. Eventually, after the initial cull and the ED processing, a small set of data will be loaded into a review platform. Once the data is reviewed even greater cuts will be made with huge percentages of documents removed through keyword searching.
This means that out of the 10,000s of documents originally available only a few thousands or a couple of hundred, per computer, will even be given the opportunity to be reviewed by teams of lawyers.
This is where the errors really start.
The bulk of reviews are conducted not by partners or senior associates, but by junior staff. Sometimes contractors brought in for this purpose. These junior staff people are working quickly, under pressure, with a requirement to review their assigned documents as quickly as possible. The subject matter will often be new to them, the review platform may also be new to them, they are junior, under pressure, possibly tired, and working for long hours. We don’t let people drive trucks in these conditions, let alone make decisions on multi-million/multi- billion dollar cases.
Even a highly experienced litigator, working short hours on a subject they know will make errors. It’s just a statistical fact. The junior staff, working in the conditions described will make a lot more errors; and these errors will not be even. The type of error that a person makes on one day will not be the same on another day, and different people will make different errors. This makes the QC of the errors difficult.
Does this means that electronic discovery is wrong? Is it fundamentally flawed, so much so that a better method should be used?
No; quite the reverse.
Electronic discovery has errors, but so do must other systems of evidence. Fingerprints are not, as many people think, 100% reliable, but require human interpretation.
Witness statements are staggeringly inaccurate. A person who has witnessed a shooting or a road accident, will be traumatized, upset, angry, and influenced by the police (even accidentally through leading questions “Did you see the suspect get into a green car?”). The witness will have their own prejudices about what they saw, what they think they saw, and what they think happened. Witness statements are, in short, not 100% reliable.
Even DNA, the trump card in any investigation has errors, in all levels. The science is sound but humans make mistakes.
Does this mean that DNA, fingerprints and witness statements should not be used because there can be errors? Of course not.
The issue is not that mistakes are made, but that they are understood and accounted for.
Is the electronic discovery process less accurate than DNA or fingerprints, certainly? But does that matter? The issue of accuracy must also be understood in terms of what is being measured.
If a forensic scientist makes an incorrect decision with their fingerprint or DNA analysis than they can state that Person A killed Person B, rather than Person A did not kill Person B (and this has happened, on more than one occasion). This is a massive error.
If ED processing is conducted incorrectly then a junk document, Document A may be reviewed by a lawyer when it was supposed to left out, or a document that was supposed to be reviewed was not.
Incorrect ED processing is very different. It does not change an accounting spreadsheet from profit to a loss, it does not make perfectly legal business transactions into a multinational fraud. ED processing does not interpret the data.
An ED processing guru does not go to court and state “Due to the MD5 value of this document I have concluded that the money was moved from the US, to Switzerland, then back to BVI, for the purposes of tax avoidance, defrauding the US government of 7.5% of the gross amount in tax, meaning that the suspect only paid 11.5% gross. The net cost to the US government is $3.5 million. With that money the suspect invested in property, making a net loss over the year, for the investment, but he still retains assets of $2.5 million…..”
No, an ED guru should go to court and say “I have provided the documents as best as I can, not 100% of documents, but the best sample I can, and here is how….”
Everything else is up to the accountants, lawyers, and other expert witnesses. This does not demean what an ED person does, but rather allows them to put it into perspective what they are doing and asses the risk.
If they are collecting hundreds of millions of documents (which is entirely possible), and thousands are not processed that is reasonable, expected, and required.
Electronic Discovery has errors, but it does not make it wrong.