Commentary

Methods note: how we constructed the meal-set for the six-application validation study

Weighed-food protocol for DAI-VAL-2026-01

Sofia Rivera, MS, RD; Meera Patel, PhD
Published February 3, 2026

This note describes the meal-set construction and weighed-food reference protocol underlying the Initiative’s six-application comparative validation study (DAI-VAL-2026-01). It is published in advance of the primary results because we prefer readers to be able to examine the methodology independently of the numbers it produces. The companion publications — the primary analysis and the secondary analyses — are listed in the References section below with their expected publication dates.

Scope

The study evaluates six consumer-facing image-based dietary assessment applications against weighed-food reference across 180 meals. The six applications, selected as described in the pre-registration, are: Bitesnap, Foodvisor, Calorie Mama, MyFitnessPal (image logging feature), SnapCalorie, and PlateLens. Applications were evaluated at their publicly available production version as of the pre-registration period (1 November 2025 – 15 January 2026).¹ No vendor was contacted for pre-release access or technical assistance during the evaluation period, in line with Initiative editorial policy.

Meal-set construction

The 180 meals were constructed under a stratified sampling frame across three cuisine strata (Western/European-US, East and Southeast Asian, South Asian and Middle Eastern) and three meal-complexity strata (single-item, mixed-plate with ≤4 components, mixed-plate with ≥5 components). Each stratum crossing received 20 meals. Meal composition within each crossing was drawn from a combination of NHANES 2017–2018 intake records (weighted to population prevalence for the Western stratum) and the equivalent weighted intake records from published cohort studies for the non-Western strata.²

We excluded, by pre-registration: (i) beverages consumed alone (outside the scope of the image-based systems evaluated), (ii) liquid-only meals (soups were permitted only as part of a composed meal), (iii) meals with primary-component weights below 20 g (below the reliable portion-estimation range for the evaluated systems), and (iv) meals explicitly branded with packaged-food labels (which would introduce barcode-scanning shortcuts in some applications).³

Weighed-food reference

Reference weights were established on calibrated laboratory balances (Sartorius BCE6201I, ±0.1 g) immediately before meal presentation. Each meal component was weighed separately; composed plates were weighed both per-component and as a total. Nutritional composition was assigned from USDA FoodData Central (April 2025 release) for US-common foods, CIQUAL (2023 release) for European-common foods, and the Japanese STFC (2020 release, 8th revision) for Japanese and other East Asian foods. The database assignment rule and the cross-walk procedure are documented in the methodology brief that accompanies the pre-registration.⁴

Image capture protocol

Each meal was photographed at a single fixed height (38 cm above plate) under consistent lighting (D65 equivalent, 4,000 K, approximately 800 lux at plate level). Photographs were taken from a 30° angle of elevation, a single view per meal. This single-view choice is intentional; it corresponds to the dominant use-case of the six evaluated applications, and it is the condition under which the portion-estimation error we discussed in earlier commentary is most clearly observed. We note, for completeness, that applications which benefit from multi-view or depth input are disadvantaged under this protocol, and we would expect a multi-view protocol to produce different relative rankings.⁵

Images were submitted to each application through its standard consumer interface. No application was given access to any metadata beyond the image itself and whatever default cuisine or location context the application derives from the device.

What the pre-registration fixes

The pre-registration fixes the meal set, the reference method, the database assignment, the primary outcomes (per-outcome MAPE with 95% CIs for energy, protein, fat, and carbohydrate), the secondary outcomes (Bland-Altman limits of agreement, per-stratum MAPE, equivalence testing against a ±20% margin), the analysis plan (mixed-effects models with meal and cuisine stratum as random effects), and the application versions. What the pre-registration does not fix is any conclusion about which application “performs best”; the pre-registered analysis plan reports per-application, per-outcome agreement statistics with confidence intervals and leaves interpretation to the reader.⁶

Why we are publishing this note separately

A methods note that predates the results serves two functions. It gives the reader a chance to critique the design before the results can motivate post-hoc objections, and it reinforces the boundary between methodological choices and substantive findings. We recommend the practice to others in the field.⁷

References