Commentary

Reader letters on our 2026 six-application validation study

Three letters with Initiative responses

Helena Weiss, PhD, MPH, RD
Published April 19, 2026

The Initiative received 43 reader messages in the two weeks following the publication of DAI-VAL-2026-01 on 2 April 2026. Most were straightforward clarification requests or expressions of support and are not reproduced here. A smaller set raised substantive questions worth a public response. We reproduce three below, in each case with the writer’s permission and in edited form.

Letter 1 — why only 180 meals?

From a reader in public-health nutrition (Europe). “Why use only 180 meals? Validation studies in this field have ranged up to several thousand meal-images. A sample of 180 seems small for a study that aims to make comparative claims across six applications and four nutritional outcomes.”

Initiative response. The sample size of 180 meals was set against a pre-registered power calculation for the primary comparisons: per-application, per-outcome MAPE with a target 95% CI half-width of ±3 percentage points, and pairwise equivalence testing against a ±20% margin for energy. Under those targets, 180 meals provide adequate precision for the primary analyses.¹ The reader is correct that a larger sample would narrow the confidence intervals and increase power for secondary comparisons — for example, between-application pairwise tests on protein or fat — and we have noted this in the limitations section of the primary paper. The choice of 180 reflects a trade-off between precision, the logistical feasibility of a fully weighed-food reference protocol, and the pre-registration commitment to not adjust n post-hoc based on preliminary results. A follow-up snapshot in the next evaluation cycle is planned with an expanded meal set, which will partially address the reader’s concern.²

Letter 2 — results reporting format

From a reader in clinical dietetics (US). “The Bland-Altman limits of agreement reported in Figure 3 are for energy only. Were equivalent analyses conducted for protein, fat, and carbohydrate, and if so where can one see them?”

Initiative response. They were. Per-outcome Bland-Altman plots for protein, fat, and carbohydrate are in Supplementary Figures S3 through S5 of the primary paper. We elected to place only the energy Bland-Altman plot in the main text because energy is the most commonly reported outcome in this literature and because placing four Bland-Altman plots in the main text would have produced a crowded figure-reading experience. The supplementary figures use the same axes and scales as Figure 3 for comparability. A reader interested in all four outcomes at once may find Supplementary Figure S6, a four-panel composite, more convenient.³

Letter 3 — on the headline ordering

From a reader in industry (vendor-affiliated). “Your Table 2 appears to order the six applications from lowest to highest energy MAPE. Readers will inevitably interpret this as a ranking. Is that the paper’s intent?”

Initiative response. Table 2 is ordered alphabetically by application name. The apparent energy-MAPE ordering in the reader’s observation is, to the extent it exists, a coincidence of the alphabetical order; it does not reflect a deliberate ranking choice by the authors. The paper, by pre-registered design, does not rank the applications. It reports per-application agreement statistics with confidence intervals and leaves interpretation to the reader. We recognize that readers may nonetheless read a ranking into the numbers, and we accept that this is an unavoidable feature of publishing comparative data; we have declined to reorder the table or to add a “ranking” column, because either intervention would misrepresent what the pre-registered analysis actually does.⁴

Editorial note

We think reader letters are a valuable and under-used genre in the digital-health validation literature. A validation study that does not invite critical reading is, in our view, incomplete. We intend to continue publishing reader responses and our own replies in this commentary section. Submissions may be sent through the contact form.⁵

References