Troubling Gluten Testing Data Released by Nima: But Hold the Phone
Yesterday the Nima Sensor team released testing data that has caused me a great deal of concern. They are reporting that 24% of packaged foods tested by consumers and uploaded to the Nima app have tested gluten detected. They are also reporting in a new graphic that it is very rare for the Nima to test positive for gluten at levels below 20 parts per million.
BUT this is not the complete story. The Nima is NOT a fully validated ELISA. It is a lateral flow device and there are limitations to these types of testing devices. Of very real concern are false positive results. According to Thomas Grace of Bia Diagnostics, “These methods should never be used for finished product validation in and of themselves, but only in conjunction with a fully certified and validated method such as an ELISA…Something as “innocent” as a pinch of salt or a little vinegar can be all it takes for an LFD to produce a false negative or false positive result.” For more information on LFDs and my interview with Thom and Adrian Rogers from Romer Labs see https://www.glutenfreewatchdog.org/news/lateral-flow-devices-nima-gluten-tox-home-ez-gluten-to-test-food-for-gluten-manufacturer-consumer-use/
Back in January after seeing some early testing data from Nima I emailed the Nima team writing, “If this was my data and I was using a “new” device with a “new” antibody I would wonder why my data was so different from other published and unpublished data.” I suggested to the Nima team that if the percentage of packaged food products labeled gluten-free and testing at/above 20 parts per million of gluten is substantially higher than percentages published in the peer-reviewed scientific literature using a fully validated assay then they should consider the possibility of false positives.
So what is the testing data using the R5 ELISA—an assay scientifically validated by the Prolamin Work Group of Codex via a multi-laboratory ring trial—telling us?
- A study by the FDA published in the journal Food Chemistry (Food Chemistry 169 (2015) 120–126) tested 275 foods labeled gluten-free. 3 samples (1.1%) of tested foods had gluten levels greater than 20 ppm of gluten.
- A study by Gluten Free Watchdog published in the journal European Journal of Clinical Nutrition tested 158 products labeled gluten-free. 8 samples (5.1 %) of tested foods had gluten levels greater than or equal to 20 ppm of gluten.
- According to Bia Diagnostics, LLC they have tested over 14,000 foods labeled gluten-free during the past 2 years. Less than 5% are testing at/above 5 ppm of gluten. Data on the percentage of products testing at/above 20 ppm of gluten is not readily available.
- Note: A third US study was published in the Journal of Food Protection. While 16 (20.5%) of products labeled gluten-free tested at levels at or above 20 ppm of gluten, the study was flawed. Gluten Free Watchdog communicated at length with the study authors. During the course of our discussions it was determined that the researchers inadvertently used the wrong assay to assess foods for gluten. They meant to use the sandwich R5 ELISA but used the competitive R5 ELISA instead. A competitive ELISA is used to assess foods and ingredients that are fermented or hydrolyzed. The study write-up wrongly states that the sandwich R5 ELISA was used.
Nima: 24% testing “gluten detected”
FDA: 1.1% testing at/above 20 ppm
Gluten Free Watchdog: 5.1% testing at/above 20 ppm
Bia Diagnostics: Less than 5% testing at/above 5 ppm
“One of these things is not like the other…”
The bottom line and what I tweeted yesterday to Nima, “If 24% of packaged labeled gluten-free foods are testing “gluten found” & it is “rare” for gluten to be detected below 20 ppm then you should strongly consider false positives. Data to date from peer-reviewed studies published in scientific literature by FDA & Gluten Free Watchdog finding between 1- 5% labeled gluten-free foods testing at/above 20 ppm.”
Gluten Free Watchdog is currently in the process of assessing the Nima with Bia Diagnostics. For information on the strengths and weaknesses of Nima please see the videos and articles posted at https://www.glutenfreewatchdog.org/news/category/nima-sensor/
A couple of additional comments based on comments we’ve received after this post was published…
- Some of you have wondered about bias. Might consumers test products that they are concerned about or that caused a reaction. This is possible and would be referred to as sampling bias. At the very least the Nima team should put the results into context just as they would have to in a scientific paper. That said, at Gluten Free Watchdog almost all products tested are subscriber requests. It is likely that our results are potentially biased as well BUT we are still seeing “only” 5% of products testing at/above 20 ppm of gluten.
- A false positive result refers to more than just matrix effects (i.e., the composition of the food). The Nima website provides instructions for testing various food products that may or may not be followed by consumers. These directions are often not exact (e.g., pea size amount of food, small amount of food, dilute with water–in other words no measurements are provided). As we reported yesterday for the product that tested low gluten using the Nima but less than 5 ppm using the R5 ELISA: the product contains vinegar and according to the Nima website pure vinegar can not be tested because the acidity of the vinegar damages the test; it is brightly colored and the Nima website recommends diluting brightly colored food with water before testing; and it contains tannins/polyphenols (when testing with the fully validated R5 ELISA the recommendation is to add skim milk powder to the extraction solution to prevent false positives and false negatives).