Science Found Wanting in Nation’s Crime Labs

The arti­cle below hints at harsh crit­i­cism of foren­sic sci­ence in gen­eral. Comparing other foren­sic dis­ci­plines to DNA analy­sis, and an appar­ent lack of “objectivity”.

Let’s be clear on one thing, ALL of foren­sic sci­ence entails some amount of sub­jec­tiv­ity, even the holy grail that is DNA. I per­son­ally was involved in a homi­cide case in Phoenix Arizona, where a DNA report had to be “amended”. The amend­ment had to do with the “likely-hood ratio” of the DNA pro­file found on case evi­dence NOT being from the sus­pect fell from about 1 in 4 tril­lion peo­ple, to 1 in 100. The rea­son for the change, a DNA analyst’s SUBJECTIVE inter­pre­ta­tion of instru­ment data.

Certainly there are occa­sional mis­takes. Some labs need bet­ter pro­ce­dures, even ones that are accred­ited. Some ana­lysts need a bet­ter atti­tude than “I fol­lowed my pro­ce­dures, there­fore every­thing has to be per­fect in my results/conclusions”.

But does any­one really think the appro­pri­ate solu­tion is fed­eral gov­ern­ment over­sight? Whenever lawyers and politi­cians try to leg­is­late sci­en­tific real­ity, the results are catastrophic.

There cer­tainly needs to be more inde­pen­dence in foren­sic sci­ence, but I don’t see police giv­ing up their strangle-hold on crime labs.

It all comes down to our legal sys­tem. We don’t have a legal sys­tem that tries to find the truth, we have two sides argu­ing to con­vince the trier of fact that their ver­sion of events is the truth. Other coun­tries have an inde­pen­dent foren­sic agency that ana­lyzes the evi­dence, and reports their find­ings to the trier of fact, and isn’t called as a wit­ness by either side. The foren­sic expert is called upon by judges to help them under­stand the evi­dence. They are allowed to explore all per­ti­nent details and evidence.

Our cur­rent sys­tem has the police sci­en­tists ana­lyz­ing the evi­dence that the police want ana­lyzed (and some­times pros­e­cu­tors), to help build their case. The sys­tem is inher­ently flawed in that the foren­sic sci­en­tist can not be com­pletely objec­tive, because they are not given all the evi­dence to inves­ti­gate to come to an impar­tial conclusion.

If we keep our cur­rent legal sys­tem, that of an adver­sar­ial sys­tem, then there needs to be an adver­sar­ial com­po­nent to the sci­en­tific evi­dence admit­ted dur­ing court pro­ceed­ings. Granted that’s more costly, and the major­ity of the time, a defense expert is likely to come to the same, or very sim­i­lar, con­clu­sion as the pros­e­cu­tion expert, but that’s the check-and-balance sys­tem our coun­try needs, if it is to keep our cur­rent legal system.

Multiple times I’ve had the oppor­tu­nity to speak to pros­e­cu­tion experts about a case after it was adju­di­cated. Quite often I talk about evi­dence they weren’t even aware of. Sometimes they are shocked, per­haps even angry, that they based their opin­ion on only the lim­ited evi­dence the police/prosecution wanted them to have access to. How can a sci­en­tist reach a cor­rect con­clu­sion with only fil­tered information?

If two experts are allowed to work “oppo­site” the­o­ries of the case, it is more likely the trier of fact will be able to sort through the data, and come to an appro­pri­ate verdict.

Original arti­cle posted here.

By SOLOMON MOORE
Published: February 4, 2009

Forensic evi­dence that has helped con­vict thou­sands of defen­dants for nearly a cen­tury is often the prod­uct of shoddy sci­en­tific prac­tices that should be upgraded and stan­dard­ized, accord­ing to accounts of a draft report by the nation’s pre-eminent sci­en­tific research group.

The report by the National Academy of Sciences is to be released this month. People who have seen it say it is a sweep­ing cri­tique of many foren­sic meth­ods that the police and pros­e­cu­tors rely on, includ­ing fin­ger­print­ing, firearms iden­ti­fi­ca­tion and analy­sis of bite marks, blood spat­ter, hair and handwriting.

The report says such analy­ses are often han­dled by poorly trained tech­ni­cians who then exag­ger­ate the accu­racy of their meth­ods in court. It con­cludes that Congress should cre­ate a fed­eral agency to guar­an­tee the inde­pen­dence of the field, which has been dom­i­nated by law enforce­ment agen­cies, say foren­sic pro­fes­sion­als, schol­ars and sci­en­tists who have seen review copies of the study. Early review­ers said the report was still sub­ject to change.

The result of a two-year review, the report fol­lows a series of widely pub­li­cized crime lab­o­ra­tory fail­ures, includ­ing the case of Brandon Mayfield, a lawyer from Portland, Ore., and Muslim con­vert who was wrongly arrested in the 2004 ter­ror­ist train bomb­ing in Madrid that killed 191 peo­ple and wounded 2,000.

American exam­in­ers matched Mr. Mayfield’s fin­ger­print to those found at the scene, although Spanish author­i­ties even­tu­ally con­vinced the Federal Bureau of Investigation that its fin­ger­print iden­ti­fi­ca­tion meth­ods were faulty. Mr. Mayfield was released, and the fed­eral gov­ern­ment set­tled with him for $2 million.

In 2005, Congress asked the National Academy to assess the state of the foren­sic tech­niques used in court pro­ceed­ings. The report’s find­ings are not bind­ing, but they are expected to be highly influential.

“This is not a judi­cial rul­ing; it is not a law,” said Michael J. Saks, a psy­chol­ogy and law pro­fes­sor at Arizona State University who pre­sented fun­da­men­tal weak­nesses in foren­sic evi­dence to the acad­emy. “But it will be used by oth­ers who will make law or will argue cases.”

Legal experts expect that the report will give ammu­ni­tion to defense lawyers seek­ing to dis­credit foren­sic pro­ce­dures and expert wit­nesses in court. Lawyers could also use the find­ings in their attempts to over­turn con­vic­tions based on spu­ri­ous evi­dence. Judges are likely to use the find­ings to raise the bar for admis­si­bil­ity of cer­tain types of foren­sic evi­dence and to rein in exag­ger­ated expert testimony.

The report may also drive fed­eral leg­is­la­tion if Congress adopts its rec­om­men­da­tions. Senator Richard C. Shelby, Republican of Alabama, who has pushed for foren­sic reform, said, “My hope is that this report will pro­vide an objec­tive and unbi­ased per­spec­tive of the crit­i­cal needs of our crime labs.”

Forensics, which devel­oped within law enforce­ment insti­tu­tions — and have been mythol­o­gized on tele­vi­sion shows from “Quincy, M.E.” to “CSI: Miami” — suf­fers from a lack of inde­pen­dence, the report found.

The report’s most con­tro­ver­sial rec­om­men­da­tion is the estab­lish­ment of a fed­eral agency to finance research and train­ing and pro­mote uni­ver­sal stan­dards in foren­sic sci­ence, a dis­ci­pline that spans anthro­pol­ogy, biol­ogy, chem­istry, physics, med­i­cine and law. The report also calls for tougher reg­u­la­tion of crime laboratories.

In an effort to mit­i­gate law enforce­ment oppo­si­tion to the report, which has already delayed its pub­li­ca­tion, the draft focuses on sci­en­tific short­com­ings and pol­icy changes that could improve foren­sics. It is largely silent on strictly legal issues to avoid over­step­ping its bounds.

Perhaps the most pow­er­ful exam­ple of the National Academy’s prior influ­ence on foren­sic sci­ence was a 2004 report dis­cred­it­ing the F.B.I. tech­nique of match­ing the chem­i­cal sig­na­tures of lead in bul­lets at a crime scene to sim­i­lar bul­lets pos­sessed by a sus­pect. As a result, the agency had to notify hun­dreds of peo­ple who poten­tially had been wrong­fully convicted.

In its cur­rent draft report, the National Academy wrote that the field suf­fered from a reliance on out­moded and untested the­o­ries by ana­lysts who often have no back­ground in sci­ence, sta­tis­tics or other empir­i­cal disciplines.

Although it is not sub­ject to sig­nif­i­cant crit­i­cism in the report, the advent of DNA pro­fil­ing clearly set the agenda. DNA evi­dence is pre­sented in less than 10 per­cent of all vio­lent crimes but has rev­o­lu­tion­ized the entire sci­ence of foren­sics. While DNA test­ing has helped to free more than 200 wrong­fully con­victed peo­ple, “DNA was a shock to police cul­ture and cre­ated an alter­na­tive sci­en­tific model, which pro­moted stan­dard­iza­tion, trans­parency and a higher level of pre­ci­sion,” said Paul Giannelli, a foren­sic sci­ence expert at Case Western Reserve University School of Law who pre­sented his research to the National Academy. Enforcement offi­cials, Mr. Giannelli said, “chose to say they never make mis­takes, but they have lit­tle sci­en­tific sup­port, and this report could blow them out of the water.”

Peter J. Neufeld, a co-director of the Innocence Project, a non­profit group that uses DNA evi­dence to exon­er­ate the wrong­fully con­victed, pre­sented to the acad­emy a study of trial tran­scripts of 137 con­vic­tions that were over­turned by DNA evi­dence and found that 60 per­cent included false or mis­lead­ing state­ments regard­ing blood, hair, bite mark, shoe print, soil, fiber and fin­ger­print analyses.

The courts have long strug­gled with the proper role of sci­en­tific evi­dence. In a 1993 land­mark deci­sion, Daubert v. Merrell Dow Pharmaceuticals, the Supreme Court held that sci­en­tific tes­ti­mony had to meet an objec­tive stan­dard. Federal courts have occa­sion­ally excluded evi­dence like hand­writ­ing and hair analysis.

Donald Kennedy, a Stanford sci­en­tist who helped select the report’s authors, said fed­eral law enforce­ment agen­cies resented “inter­ven­tion” of main­stream sci­ence — espe­cially the National Academy — in the courts.

He said the National Institute of Justice, a research arm of the Justice Department, tried to derail the foren­sic study by refus­ing to finance it and demand­ing to review the find­ings before pub­li­ca­tion. A bipar­ti­san vote in Congress in 2005 broke the impasse with a $1.5 mil­lion appropriation.

Mr. Shelby also accused the National Institute of Justice of try­ing to infil­trate the foren­sic study panel with lob­by­ists for pri­vate DNA analy­sis com­pa­nies, who were seek­ing to limit the research to DNA studies.

The National Institute of Justice said it would not com­ment until the report was released. But a pre­view of poten­tial turf wars played out in the pre­sen­ta­tions to the National Academy in December 2007. A foren­sic expert from the Secret Service blasted the F.B.I. for devel­op­ing ques­tion­able tech­niques “on an ad-hoc basis, with­out proper research.”

He said the Secret Service wanted the National Academy “to send a mes­sage to the entire foren­sic sci­ence com­mu­nity that this type of method devel­op­ment is not accept­able practice.”

Everyone inter­viewed for this arti­cle agreed that the report would be a force of change in the foren­sics field.

One per­son who has reviewed the draft and who asked not to be iden­ti­fied because of promises to keep the con­tents con­fi­den­tial said: “I’m sure that every defense attor­ney in the coun­try is wait­ing for this report to come out. There are going to be chal­lenges to fin­ger­prints and firearms evi­dence and the gen­eral lack of empir­i­cal ground­ing. It’s going to be big.”