A photo-based calorie tracker does two separate jobs, not one. The first is identification: figuring out what is on the plate. The second is portion estimation: figuring out how much of it there is. Most of the public conversation about accuracy focuses on the first job, but the second is where almost all the error actually comes from.
Identification is the easy part
Modern vision models are good at recognizing food. Grilled chicken, rice, and broccoli on a plate gets classified correctly the overwhelming majority of the time. This part of the problem is close to solved industry-wide, and has been for a few years.
Portion size is the hard part
Once the app knows what is on the plate, it has to estimate the mass. This is fundamentally a 3D-from-2D problem: a single photo compresses depth into a flat image, so the app is estimating volume from a shape it can only partially see, then converting that volume to a weight using an assumed density for that food, then converting weight to calories using a nutrition database entry.
Every step in that chain adds error:
- Volume from a photo. A photo does not carry true depth information, so volume is inferred from apparent size, plate context, and camera angle. Camera distance and angle alone can shift the estimate meaningfully.
- Density assumptions. A gram of rice and a gram of rice pudding are very different foods with very different calorie density, and the model has to guess which one it is looking at from appearance alone.
- Database entries. Even a correct weight estimate is only as accurate as the nutrition data it gets multiplied against, and that data is not always consistent between sources.
This is why two AI food-scanning apps can look at the same photo and land on meaningfully different calorie estimates. They are not disagreeing about what the food is. They are making different guesses about how much of it there is.
What actually reduces the error
The most effective techniques are unglamorous: adding depth signal beyond a single flat photo, cross-checking a portion estimate against known reference objects on the plate, and calibrating against real, weighed ground-truth meals rather than trusting the raw model output. None of this fully closes the gap. Anyone telling you their app is simply "accurate," full stop, is skipping the part of the problem that actually matters.
How Arcal approaches it
This is the exact problem Arcal's camera system is built around. On supported iPhones it runs the wide and ultrawide lenses together at capture time, so the model has genuine off-frame spatial context for the plate instead of a single tightly cropped shot it has to guess around. That does not eliminate the underlying 2D-to-3D estimation problem above. Nothing does, yet. It is a real attempt at giving the model better input rather than asking you to compensate for a bad photo, and when an estimate is still off, fixing it is our job on the backend, not something we ask you to manually correct meal by meal. See the full feature list or join the waitlist.