Michigan v Mitchell - a bad Daubert decision

From February 24-27, 2026, a hearing was held in the city of Detroit to completely exclude a firearm examiner's testimony. Arguments against the field of firearm examination were given by David Faigman and Nicolas Scurich. Faigman is a law professor with a Master's in Psychology. Scurich is a Psychology professor. These practitioners and teachers of psychology and law fly around the country and provide testimony against the use of forensic science in courts. Their goal is the complete exclusion of forensic analyst testimony, not allowing them to say anything about any of the work that was performed in the lab. Michigan v. Mitchell specifically involved the comparison of toolmarks left by a firearm. In this post, I will start to discuss a bit about why I think this decision was wrong on a couple of levels, and why firearm and toolmark examiners should be given more credit as true practitioners of a science with a track record of holding bad examiners accountable. This will be a multi-part post, as otherwise the post will become unwieldy.

The first argument I would like to tackle is one given by Scurich relating an inconclusive decision as skipping a hard question on a test. I have seen this argument multiple times and it is a complete strawman and mischaracterization of inconclusive decisions. First, you need some background information. In toolmark examination, as well as many forensics fields, there are three main conclusions that will appear in a report. These three conclusions will be considered only after the items to be examined are determined to be suitable for comparison. Especially with fired bullets, damage is acquired as the bullet hits objects, and there are times the bullet is so damaged that no useful information can be obtained. This would be unsuitable, and no comparison will take place. If the fired bullet is suitable for comparison, the comparison process will begin. An examiner will look at the so-called class, subclass, and individual characteristics to start narrowing down the population of firearms that could have fired the bullet in question. Each step along the way reduces the population of firearms that could have fired the bullet. Class characteristics are very straightforward, sometimes literally being that a bullet is too big to fit down the barrel. Think of the baby toy where you put the same shaped pegs in the same shaped hole (I will bring up this fact again in a later post).

Admittedly, firearms examiners in the past have mainly been worried about the concept of "false identifications." A false identification is a miscarriage of justice. This is when a firearms examiner — either through a true mistake, extreme bias, or bad morals through "dry labbing" (essentially making up a result without doing any examination) — says that a fired bullet or fired cartridge case was fired in the suspect firearm when it was not. In my mind as well, this is absolutely the worst outcome, and it makes sense that most studies were concerned with addressing this topic. Because of this, past studies have been really focused on trying to find out if examiners can successfully relate fired bullets and cartridge cases to firearms in the most difficult examples the study designer can think of. To accomplish this, the study designers will obtain "consecutively manufactured" firearm parts, like barrels, and use notoriously poor marking ammunition (a study called Ames II used this strategy, for example). Consecutively manufactured parts are made one right after the other, each one being cut by the same tool. The point of this is to provide confidence that even under extreme conditions, when a firearms examiner says an item was fired from a firearm, there is strong evidence this is correct (we do not want false identifications). The problem with creating these extremely difficult (and frankly not at all like casework scenarios) conditions is that the studies will have a high "inconclusive" rate. An inconclusive is when you do not have enough information (data) on the fired bullet or fired cartridge case to confidently make a determination if the evidence was or was not fired in the firearm. In this scenario, it would be irresponsible to make any other conclusion. Because of the lack of data on the evidence, saying identification or elimination would just be a guess. This would have a very high chance of resulting in a false identification or false elimination.

With that background information out of the way, we can now get to the strawman argument. Scurich testified — when discussing inconclusive conclusions in studies — that (per the court transcripts): "The false positive error rate is not credible because what it doesn't tell you is that 33 percent of the time the examiners said inconclusive, or to put it kind of tongue in cheek, 'I don't know.' And not only does the 1 percent not account for those responses, but it actually counts them as affirmatively correct responses. The problem with that is at a very basic level, when you're studying aptitude, and this applies generally to any discipline, if you let students skip the questions that they don't want to answer, you've I think obviously not measured their true ability. This is actually worse than just skipping the difficult questions because it counts skipping the questions as correct, it doesn't just set them aside." For context, this is only in reference to one particular study generally called "Ames I." This is a strawman because no one would disagree with how this argument is presented, but the presentation is wrong.

To me this shows either an accidental misunderstanding of what an inconclusive is, or a purposeful misrepresentation. The examiner is not "skipping a question" when they make an inconclusive decision. The actual true representation of this scenario, using the same test example, would be this: the teacher gives out a test, but the test prints poorly, and some of the questions are not fully legible. The students bring up that they can't read some of the questions on the test. What Scurich would want from his students, in this example, is to guess what the answer is, even though they cannot read the full question. If they don't guess, he will count it wrong anyway. As a concrete example, it would be like getting the question 2 + ___ + ___ = ___ and being expected to guess the right answer. This will grossly underestimate how well his students know the material, because they are being forced to guess on questions they can not read. Obviously, when a person's future is dependent on the guess, this is unreasonable. The only correct response would be to not answer, because you do not have enough information to give an answer. An inconclusive is the only appropriate response.

I think part of what this represents is law and psychology experts trying to also be forensic experts, and specifically firearms examiner experts, a highly specialized field. They think that because they are experts in one field, and are highly educated, they can read some studies and apply their law and psychology expertise to also be firearms examination experts. What they do not understand, or conveniently forget, is that the samples in these tests are not the same samples. Even though the test firearms are from the same group of firearms, due to manufacturing tolerances, a dynamic firing process, and sometimes damage from the firing process, each examiner is not looking at the exact same sample. One examiner can get a perfectly readable test, while the next will get a test where some of the questions are smeared and not readable. It would be unfair to say they are wrong because they got a smeared test. This natural variance is something a firearms examiner is trained on and tests for personally, but it seems to be a concept that is missed during this testimony.

I do think that if Scurich and Faigman both spent time going through a full training program, like the National Firearms Examiner Academy through the ATF (in an unbiased fashion), they would gain a solid understanding. They would also have to grapple with the fact that they would be able to consistently and correctly relate firearms evidence to the firearms that fired them (provided they can identify patterns, which not everyone can, but I'll let the psychology experts explain what that phenomenon is). I know our most recent trainee in the firearms and toolmarks unit was skeptical of the process, coming from a pharmaceutical lab. Through training and objective observation though, he saw that the process works. If you truly do not agree with a discipline, my lab will let you transfer. So, he would have had the opportunity to transfer if he felt immoral about the testing process. But back to Scurich and Faigman. If they completed an accredited training program, they would then be able to go out into the world and find the actual bad actors in the world of firearms examination. There are bad actors. There are examiners that dry lab. There are examiners that do not follow policies and procedures. There are examiners that are biased to the point it alters their work product. It is actually in the best interest of firearms examiners to find these examiners, because allowing them to continue jeopardizes public confidence. And examiners do go out and perform this function. The best person to expose a bad examination is another trained examiner.

This the first of several posts regarding this decision that I will make. I will cover the repeatability and reproducibility arguments they make, I will cover the 2016 (ten years old) PCAST report, I will cover why a complete testimony exclusion is always the wrong decision, and I will cover instances where firearms examiners hold each other accountable.