MyAiPlate: A Mobile Application for Personalized Nutrition Using YOLOv8 Food Recognition and a Hybrid LSTM-DQN Model for Real-Time Meal Plan Adaptation

Abstract

Keeping a balanced and individualized diet plays an important role in the prevention of chronic diseases as well as in maintaining optimal wellness. The accurate monitoring of daily calorie consumption and abiding by the restrictions of certain diets is key to achieving good health. Nevertheless, existing dieting apps are not dynamic enough; they only serve as online diaries incapable of adjusting according to the behaviour of the user. A multi-model machine learning architecture was developed for the MyAiPlate application to address this gap. A YOLOv8 object detection model, fine-tuned on a real-world dataset of 1,169 labelled food images, was implemented to visually recognize food items and estimate caloric intake from user-captured images. The proposed machine learning components were rigorously evaluated for performance and user safety. The YOLOv8 food recognition model was validated using unseen, real-world meal images. On the held-out validation set the model achieved 95.16% precision, 94.42% recall, 97.51% mAP@50, and 75.57% mAP@50-95. In addition to encouraging user participation, feedback was provided through a conversational interface within the application. Traditional diet tracking tools do not have a conversational approach and cannot answer specific user inquiries in a context-aware manner. An AI Nutrition Chatbot was implemented using Large Language Models (LLMs) based on transformers from OpenRouter, which offered real-time, context-aware, personalized nutritional advice based on meal entries. The conversational interface was tested using 50 queries, including 25 challenging nutritional queries and 25 common diet prompts to ensure that the LLM could deliver reliable results without hallucinations.