Michigan v Mitchell - a bad Daubert decision #2

You can objectively see the wear on this tool

This is the second post in a series about the Michigan vs Mitchell Daubert decision of 2026.

In the first post I highlighted that the people brought in to testify for the complete exclusion of firearms examination evidence were college professors of law and psychology. These professors have never designed an original study to try to support their hypothesis that there is no value to firearms examination evidence. They have done literature reviews, and recalculated the data based on their subjective decisions on what data means and the best way to calculate that information.

On the side defending the continued use of firearms examination evidence, which has been studied and used for over 100 years, were two current firearms examiners and a professor who has designed and conducted an original research study to test firearms evidence. The professor who took part in the scientific method, designing and conducting a study to test a hypothesis, was Max Guyll. Guyll is an associate professor from the School of Interdisciplinary Forensics at Arizona State University. Guyll holds a master's in mechanical engineering (very applicable to the metal machining element of firearms examination) and a PhD in clinical psychology. His study, published May 8, 2023, in PNAS (Proceedings of the National Academy of Sciences), titled "Validity of forensic cartridge-case comparisons," highlighted that when examiners made a conclusive decision (identification or elimination), they were over 99% accurate. You can find the full study here: https://www.pnas.org/doi/10.1073/pnas.2210428120 and a summary here: https://news.asu.edu/20230622-discoveries-asu-study-sheds-new-light-inconclusive-gun-evidence. It also highlighted that there was a trend: when an examiner made an inconclusive decision (this is where there is not enough information to make a decision), it was more supportive of an elimination. The data from the study showed that the majority of inconclusive decisions were true eliminations. This is information that a defense attorney could use, but I almost never get called to testify about my inconclusive decisions. To me, this finding makes logical sense. It is easy to explain why something is there, but much harder to explain why something is NOT there. If someone asks me why a car is parked somewhere, there is a pretty limited set of reasons why the car is there. It was either driven there or towed there. If you see an empty parking spot, and ask why a car isn't there, the possibilities for why a car isn't there are much greater without additional information. The list of possible reasons is long: maybe a car just left, maybe there was a shopping cart blocking the spot, maybe there was no shade, maybe there was broken glass there, maybe no one goes to that store, maybe a truck was unloading product, maybe it's assigned parking. It's the same with fired bullets and fired cartridge cases. It's easy when a corresponding pattern of toolmarks exists. I can take a photo or video of evidence and show objectively that there are corresponding toolmarks. I don't need to explain the existence of the corresponding toolmark because they exist and I can show it to you. To explain why a corresponding pattern does not exist is harder, as there are a lot of possibilities, especially when participating in a study where you have limited information. When participating in a study, you have limited options: you do not have a firearm to study, you cannot create more specimens by performing more test-fires, you cannot investigate further how well the firearm creates toolmarks, and you cannot test different ammunition compositions. Because of all these unknowns, when an examiner sees a lack of data (toolmarks), and has limited information, the best decision is to be inconclusive. To make a definitive statement of elimination, an examiner needs to be able to account for more unknowns, which means an elimination generally requires more data. This is why some labs have policies and procedures that do not allow an elimination unless a firearm is present. This also means that an examiner from one of these labs will always conclude either identification or inconclusive during one of these studies or tests, because they do not have a firearm to study. But the defense experts (who have not conducted their own original study) will argue that because inconclusive decisions exist, the whole field of firearms examination should be discredited and barred from courts, even though the study above demonstrates that inconclusive decisions provide useful information for the defense proposition.

Even if you buy into the argument presented by the defense experts that you can not make categorical conclusions with firearms examination, a complete barring of firearms comparison evidence makes no logical sense, as useful information is being left out of the case. As I have described in previous posts, there are aspects of firearms comparisons that are completely objective, and require no subjective judgment. When a firearms manufacturer creates a firearm, there are certain design elements that they decide they want for functionality or aesthetics. Take a firing pin with a hemispherical end (half a baseball shape): it will be created using a turning cutting process and finished by tumbling. The shape, cutting process, and finishing process are all decisions that were made before any firing pin was created, and they already limit the total population of firearms that need to be considered by a firearms examiner. The same limiting factors apply (using a pistol as an example) to the slide, extractor, ejector, and barrel. A firearms manufacturer is also going to decide what caliber (think size of the bullet) it will manufacture a line of firearms in. Combine all of these factors, and the population of firearms that need to be considered continues to dwindle. When a firearms examiner makes a combination of objective observations—a 9mm Luger caliber fired cartridge case; with a turned, hemispherical firing pin impression, with a drag mark; with a circular firing pin aperture of a certain diameter, with shearing; that has an extractor mark at about 3 o'clock, that has a comma-shaped ejector mark about 8 o'clock; with fine, dense chamber marks that start about ⅓ of the way up the case wall—those observations greatly limit the total population of firearms that could have produced the fired cartridge case. This means that during an examination, a firearms examiner quickly goes from considering any firearm in the world to a much smaller population of firearms. This is pertinent information, even before any subjective conclusion has been made.

Now we can move to what are called subclass marks. These are coarse, continuous marks created by a specific tool in a specific state of wear. As described above, manufacturers decide the best method for cutting the metal to make a certain part. One cutting method, called broaching, is frequently used to cut the breech face of slides in pistols. If no other processes follow broaching, and if the broaching tool picks up a substantial chip in the cutting surface, the gap the chip creates will leave the same coarse and continuous line in multiple, consecutively manufactured slides (which is why studies use consecutively manufactured parts) unless the cutting surface on the broach wears down enough that the chip no longer influences the metal cutting. Firearms examiners are trained to look for and eliminate these marks before they make an identification conclusion. While these marks are not useful for making an identification, they do further limit the total population of firearms that could have produced the fired cartridge case. The firearm would have to be part of a small group manufactured by one specific broach (tool) with that specific chip, before the tool wore down so much that the chip no longer influenced the cutting. Again, this is an objective mark that can be photographed and shown to a jury. You could measure the width of the mark if you wanted a numerical value. Although it remains circumstantial, because no conclusive (identification or elimination) decision is being made, it is potentially useful information. When combined with other circumstantial evidence, it could provide benefit to the jury.

The arrows and blue line indicate some of the random variability of metal cutting illustrated by the termination of striae. There is also a lack of corresponding striae when looking at one end of the barrel compared to the other, further demonstrating a lack of precise control over the metal cutting process.

Now we can move on to some objective information about "individual" marks. These are microscopic marks that are created during short-lived, uncontrolled, random events that are the result of metal cutting. There are many articles about metal cutting and manufacturing that cover issues like chip formation, side flow, built-up edge, vibration, chatter, tool wear, cutting lubricants, CNC programming, and many more. As an example, while writing this post, I went to Google Scholar and looked up metal-cutting chip formation. There were "about 344,000" results. So, there has clearly been a lot of research surrounding this phenomenon and how to control it. Also consider that while a part needs a certain level of surface finish, it does not need to be perfect. It needs to be good enough to work, while balancing the cost, so the firearms manufacturer can make a profit. Identical, perfect surfaces do not exist yet. Even if they did, they would be incredibly costly and inefficient. One example of this is trying to make a flawless sphere for a fusion reactor. You can read about the effort to make a batch of these spheres here: https://www.bbc.com/news/business-64553796. As the article shows, this is not what firearms manufacturers are doing. So even outside of the forensic realm, this phenomenon of surface features created while cutting metal has been studied and provides some of the foundation for toolmark examination. This also satisfies the general acceptance needs of courts, as surface topography has been studied and used by engineers and manufacturers, and it informs their policies, procedures and quality assurance programs. The fact that a quality assurance program needs to exist in manufacturing in the first place supports that there is variability that needs to be constantly monitored. A certain number of checks and measurements will be made, at both intermittent steps and the final product, to make sure that a tool has not worn too much, or that a machine is operating appropriately. The fact that manufacturing tolerances exist, where it's okay for a part to be a certain amount too small or too big, also supports the point that there is variability in the metal cutting process.

Everything above is either something objective that I could show an untrained individual through photographs, or a general concept that exists outside the specific field of forensic toolmark examination. With this much objectivity, barring an examiner from at least discussing all the class characteristics, subclass characteristics, and microscopic topography that do not eliminate the firearm in question is not supported by evidence. It is not scientific to ignore that all this information exists and helps limit the number of potential firearms. But the law and the courts are not scientific processes. The decisions made are themselves ultimately subjective. Which makes the criticism about expert subjective decision making ironic, coming from the law expert making a subjective ruling.