Saturday, August 29, 2009

Pseudoscience in Amanda Knox Trial

Methods of the Polizia PseudoScientifica A Knife, a Clasp, a Glow

By Mark Waterbury

Italian Translation

We have had time to consider the methods of the Polizia Scientifica in the Knox/Sollecito trial, and they have come to look, not merely unscientific, but pseudoscientific. That means pretend science, it is not the real thing. We see this, because a consistent pattern has emerged.

It is not a pattern of innocent mistakes, made under the stress of time. Neither is it one of incompetence, for these people mostly know what they are doing. Neither does it appear to be blatant framing by splashing blood, planting evidence, or manufacturing data. That type of lie would produce clear results, and might also leave clear tracks. These results are anything but clear. And who wants to leave tracks? Certainly not the Polizia Scientifica.

What appears to be the pattern in the Knox/Sollecito trial is something like this: The Polizia Scientifica plucks an item from the field. They subject it to of some sort of “scientific” testing. Then they “cherry pick” the results, and present these special results in the complete absence of anything to compare them with. That is, without any control experiments. This modus operandi is compatible with pseudoscience. It appears to be an attempt to create the appearance of scientific certainty, while producing absolutely unscientific results. Who are they trying to fool? The judge and jury.

“...pseudoscience is any subject that appears superficially to be scientific, or whose proponents state that it is scientific, but which nevertheless contravenes the testability requirement...”

What is this “testability requirement?” It means that if you can't test it, if you can't repeat the experiment to see if it happens again, it isn't science. It is pseudoscience. Take the kitchen knife DNA. The testing performed by Stefanoni destroyed the sample. Not a trace was left. Her experiment cannot be tested, cannot be reproduced. It can never be checked to see if it was a real result, or a fraction of a flyspeck of contamination from her lab. It was, “I did this experiment. These are the results. Trust me.”

Here's the procedure in more detail. Follow along. There will be a quiz at the end.

Step 1: Collect some stuff from the crime scene, or thereabouts. A kitchen knife, a clasp, a Luminol glow. No need to be picky here. It's not the item itself, it's the science you do to it that makes it cry “guilty.” That's why it didn't matter that the kitchen knife was a randomly chosen implement, as shown in “The Magic Knife.” A can opener from Raffaele's drawer would probably have Amanda's DNA on the handle, it just wouldn't sound as impressive.

Step 2: Perform tests on the item. What kind of tests? Scientific Tests, of course. The tests need to have the veneer of impressive, and barely comprehensible science upon them. For this, DNA profiling cannot be beaten.

Step 3: “Cherry pick” the results. This means, pick the ones that support your theory, and leave the ones that don't. Don't like a result? No problem. Don't report it. Discard those DNA peaks that don't fit. Didn't find anything that incriminates the defendant? No problem. Go back 46 days later, and pick up some more stuff from the scene.

Step 4: Present these results floating freely in an ominous air of suspicion and guilt. Leave out any kind of reference, any kind of comparison that might show how uninformative and mundane they really are. Avoid, in other words, any kind of control tests .

Now you have the impressive sounding results to support the prosecutor's tales.

Let's talk some more about control tests, because this is such a fundamental matter, that it is often lost in the grass. An experiment without a control is like a thermometer without a scale. You see the mercury inside, but you have nothing to compare it with. You can understand that it is meaningless, it is data without context, or, you can point at it and make up any temperature you want.

“Scientific controls are a vital part of the scientific method , since they can eliminate or minimize unintended influences such as researcher bias .”

Careful scientists take steps to keep their own researcher bias from influencing results. They perform double blind experiments, for instance, in which even they don't know which specimen is which, to prevent themselves from knowing what the results “should be.” That way, when they get a result, they can trust it, because they know they didn't unconsciously choose it.

Let's walk through the Polizia PseudoScientifica process for three items.

A Knife
The kitchen knife was retrieved from Raffaele's kitchen drawer some 5 days after the murder. It was reportedly taken to the police station, sat on a detective's desk for a day or so, and then was mailed to the lab in an ordinary box. Hardly the kind of careful handling you would expect of evidence slated for conventional DNA testing, let alone the hypersensitive LCN profiling. The blade DNA profiling was performed by an improvised, non-reproducible, never-validated method with deficiencies described in detail in LCN DNA Parts I and II. In fact we can add extremely poor evidence handling to the list of 9 deficiencies compiled in Part II to make it 10 testing deficiencies.

The low copy number test on the kitchen knife was performed without any negative controls. In negative control tests you perform the DNA multiplication and profiling, without adding any sample to the system. Often when this is done, as if by magic, a DNA profile emerges. It has come from a minute amount of contamination from the equipment, or the laboratory. These negative controls are essential to performance of LCN work, as shown in Part II of this series. They were not performed by Stefanoni.

Perhaps even more important for the knife DNA, no control experiments were run to follow the handling of the item from the field through to the laboratory. That is, to see if other, random objects retrieved from the same drawer and handled in the same, unprofessional way, might also appear to have DNA on them. It would be interesting to hear the prosecution spinning a sinister implication out of DNA found on a can opener. Perhaps one can use canned peas for satanic rituals. Would Meredith's DNA be found on a spoon from the same drawer? How about Filomena's? Would the spoon then be cast as the murder weapon, whether it matches any wounds or not?

All this is preposterous of course. But think about it. We have no way of knowing what the supposed knife DNA means, or where it came from, because no comparison tests of any kind were performed.

Retrieval of the bra clasp by the Polizia Scientifica.

A Clasp
Making sense out of the bra clasp DNA, which reportedly includes profiles from approximately five people, would be a challenge even with careful, rigorous control experiments. Without them, it is hopeless. Ask yourself this simple question. What would happen to any random object left on the same floor and kicked about for 46 days? Especially an object with cloth attached , making it a virtual dust mop. It would be covered with dust, and the DNA that comes with that dust. Raffaele was at the apartment visiting Amanda on several occasions. The presence of his DNA there means nothing.

Control experiments to check for this would have been simple. The clasp was retrieved from a pile of debris left by the fastidious investigators in Meredith's room, shown in the picture at the right. Testing a few other items from that pile to see if they, too, had picked up DNA dust from the floor would tell us whether there was anything special about the clasp. Of course, that wasn't done.

So we have “Raffaele's DNA was found on Meredith's bra clasp,” rather than, “Raffaele's DNA, along with DNA from lots of other people, was found at various random locations throughout Amanda's apartment, which he visited several times before the murder.” The first phrase sounds incriminating. The second, accurate phrase, shows how meaningless this test result is without a control experiment.

The handling of the clasp when it was retrieved from the scene is shown in the video above. The investigators, dressed in fancy white outfits, seem to play some kind of game with it. Why the outfits? They do nothing to prevent mixing contamination of the material at the scene. As shown in the closeup picture, the outfits, and their gloves, quickly become contaminated by DNA from various sources at the scene which can then be transferred to the evidence.

A Glow
Luminol glowing footprints were found in a hallway, and some may have been Amanda's, it is hard to know for sure because they were only compared with her feet, and found to be “compatible.” Again, no controls. Meredith, Laura, Filomena, none of the other resident's feet were compared to these footprints. The footprints were tested for blood, and it came out negative. No blood. So, why are they important? Amanda lived there, after all.

Amanda's DNA was said to be found in one of these footprints. Did they also test a meter away from the footprints, to see if her DNA was all over the apartment where she lived? No. That would have been another control experiment. Was the DNA actually associated with the footprint, or did it just happen to be there, because the resident's DNA was all over their apartment, as people's DNA usually is? We will never know. They skipped the control experiments, and presented results without any reference.

It's getting hot in here. Just look at that thermometer without any scale!

These are just three examples, there are many more. Enough to discern a clear pattern, that the methods of the Polizia Scientifica are compatible with pseudoscience, and are a consistent attempt to mislead the judge and jurors.

Now for the quiz:
1. If you wanted to perform honest tests to search for truth, to learn what has really happened, would you exclude control experiments? (Yes /No).

2. If you wanted to perform tests that appear scientific, but are actually intended to make an innocent person look guilty, would you perform control experiments? (Yes/No)

3. Did the Polizia Scientifica perform control tests in the Knox/Sollecito investigation? (Yes/No)

If you answered “No, No, No!” then, you've got the point. If you didn't, stop and think if you would like to be investigated with these same kinds of tactics.

Special thanks to my molecular biologist friend, Dr. K. for reviewing these articles.