Study: Many courts fail to screen out unreliable psychology tests
When psychological tests are presented as evidence in court, they are required to meet certain criteria. As with any scientific evidence, psychological tests must be “generally accepted as reliable in the relevant scientific community” (psychologists and psychiatrists).
Judges are supposed to be the gatekeepers of scientific evidence. That means ensuring that evidence that is based on junk science, or unproven, novel science, does not get admitted. However, a recent study found that many courts fail to limit the admissibility of psychological tests that have been largely discredited.
Psychological tests are frequently used in criminal cases, along with family law and other cases. They can be used in an attempt to convince the jury that the defendant is, or is not, mentally ill. That can have a serious impact on how the case proceeds.
“There’s a lot of stuff that looks like it’s junk and should be filtered out by the courts, but it’s not being filtered out,” said an Arizona State University psychology professor who co-authored the study, which appeared in the journal Psychological Science in the Public Interest.
It’s not entirely the fault of courts failing to filter out junk psychology tests. Attorneys shoulder part of the responsibility, as the study found that attorneys had challenged the admission of psychological tests only 3% of the time.
What psychological tests are being admitted?
For the study, researchers examined hundreds of different tests that were used in 876 court cases that took place in the U.S. between 2016 and 2018. Of the tests that had been admitted into evidence, a third had never been reviewed in the field of psychology’s most prominent manuals. That seems to indicate that they did not meet the “generally accepted” standard for admissibility.
Moreover, of those tests that had been reviewed, only 40% had received favorable reviews. Almost 25% had actually been deemed unreliable. These tests should never have been admitted.
The good news is that the most common test used in the court cases studied was the Minnesota Multiphasic Personality Inventory. This test is highly regarded and has been positively reviewed in most of the professional literature. It is fair to say that, for most uses, the inventory is generally accepted by the psychological community.
More troubling was the second most common test. The Rorschach “ink blot” test, first developed in 1921, is less accepted. Many practitioners believe that it is dangerously ambiguous and that the results are highly subjective. It does have its defenders, but it probably cannot be considered to be generally accepted in the field.
This isn’t the only junk science being used in courtrooms
In 2009, the National Research Council issued a landmark report on the state of forensic science used in courts. It determined that a great many common forensic techniques lack any serious scientific underpinning.
That report concluded that “testimony based on faulty forensic science analyses may have contributed to wrongful convictions of innocent people.” Indeed, according to the National Registry of Exonerations, false or misleading forensic evidence has been a factor in nearly a quarter of all DNA-verified exonerations.
Then, in 2016, the President’s Council of Advisors on Science and Technology (PCAST) issued another report challenging the scientific basis for many common forensic science techniques. It also found that forensic analysts often overstate the certainty of their conclusions when testifying in court.
Famously, fingerprint matching has proven to be unreliable. Brandon Mayfield, an Oregon attorney, was held as a witness in 2004 in response to the Madrid train bombings which killed 191 people. An FBI computer matched Mayfield’s fingerprints to prints found at the bombing site, and FBI analysts later wrongly confirmed that those prints belonged to him. The FBI apologized to Mayfield. Nevertheless, fingerprint analysts regularly tout the high reliability of their “science.”
Arizona’s Standard for Admission of Expert Testimony
In Arizona, as elsewhere, judges are the gatekeepers of what evidence is presented. Arizona uses the Daubert standard, derived from the United States Supreme Court case Daubert v. Merrell Dow Pharmaceuticals Inc., which requires a judge to determine whether an expert’s scientific testimony is based on scientifically valid reasoning that can be applied to the facts. In determining whether the methodology is valid, the following factors are considered: (1) whether the theory or technique can be and has been tested; (2) whether it has been subjected to peer review and publication; (3) known error rate; (4) that there are standards existing and maintained for controlling its operation; and (5) that it is widely accepted within the relevant scientific community.
Daubert is not the only consideration, however. Judges are supposed to consider other rules of evidence, such as the prohibition on character evidence in most cases, outlined in Arizona Rule of Evidence 404, and the risk of unfair prejudice in admitting certain types of evidence, outlined in Arizona Rule of Evidence 403.
Although fortunately psychological testing is typically not admitted in criminal cases in most Arizona courts, other forms of junk science routinely are. Junk science should never be used against people in court. Yet many lawyers fail to challenge its admissibility, and courts fail to weed it out on their own.