Can a test prove two things are identical?
I have a (possibly irrational) fear of genetically modified/engineered foods. I thought I would disclose this before I plunge into this topic, agitated by this Washington Post article about the FDA stance that (1) does not require labeling of genetically modified foods; and (2) forbids labeling of non genetically modified foods as such.
***
How is this topic related to statistics? Consider how you might answer the question: is this new food going to harm us? Clearly, some form of testing is needed to compare the natural version and the genetically modified version. Are they the same or are they different?
According to the FDA, this is an open-and-shut case. Their scientists have determined that genetically modified salmon is not "materially" different from other salmon, by which they mean there are no "biologically relevant" variations between the two versions of salmon.
The level of confidence expressed in this conclusion is astounding and, I have to say, un-statistical. I seriously wonder if there are statisticians involved in this decision at all.
***
Notice that the scientists issued a qualified statement: no "biologically relevant" variations instead of no variations at all. What do they mean by "biologically relevant"? The article does not tell.
In fact, "biologically relevant" is according to whatever metrics these scientists deem important to examine. Fat content, mercury content, etc. Presumably anything that they consider to have health implications. Therein lies the rub. This analysis presumes that we know exactly which parts of salmon are related to our health, and which parts aren't. In my opinion, there is no way to know because the genetically modified salmon is an innovation.
The general lesson is that tests won't tell us about things that they are not designed to measure. The design phase of testing is even more important than any subsequent analysis.
***
One wonders if a properly designed randomized controlled trial is being planned, and if not, why not. One branch would only eat natural salmon and the other branch would only eat modified salmon. Would we find differences in clinical outcomes?
The value of such trials is that if the clinical outcomes turn out to be different, we know something unexpected is happening, and we can go back and search for the biological cause of such a difference. Without such trials, our knowledge is limited by what dimensions the scientists attempted to test. But what we fear is not the usual suspects but the surprises.
***
Under the traditional statistical testing framework (discussed in Chapter 5 of my book), one would set up a hypothesis that the two versions are the same, and venture to seek significant evidence of difference to reject that so-called "null hypothesis". If insufficient evidence exists for rejection, the conclusion is that there is insufficient evidence to prove that the two versions are different. This statement is not the same as that the versions are identical.
There is a reason why we do not like to assert that two things are the same. The two things can be different along any number of dimensions -- a fish is a complex organism; one cannot possibly have exhausted all such comparisons.
One dimension was clearly not considered, namely, the genetic makeup of the salmons. Given that one version has been given a gene (which quickens its growth), it cannot possibly be the case that the two versions are identical. This difference is judged to be not biologically relevant by someone in a position of power.
***
Even if one trusts the FDA scientists, there is still absolutely no reason why marketers of natural salmon should be forbidden from advertising their fish as not having genetic modification. In the article, we were fed a bunch of legalistic, Orwellian mumbo jumbo about why this is so, none of which convincing. The last section of the article ("No 'Hormone free' either") is a must-read, and unintentionally hilarious.
If the scientific evidence is as clearcut as the FDA is saying, they should have no reason to fear a public rebuke unless they think the public is stupid. Withholding information and taking choice away from consumers are horrible ideas.
