Food Health Checker

Today we will be looking at a sample Food Health Checker app that assesses the healthiness of packaged foods based on the given image.

Built using Blazor web app(server) and Semantic Kernel SDK along with Azure OpenAI.

Try it now here: Food Health Checker App

You can find the source code here: GitHub Repository

Disclaimer

This is a sample app for demo purposes only and can be inaccurate. It is not a substitute for professional medical advice.

Overview

• Image Analysis: Users can upload an image of packaged food, which the AI model analyzes for contents.
• Health Assessment: The app lists all the ingredients and nutrition charts found in the given image and provides a health assessment based on the AI’s analysis of the uploaded image.

How It Works

The user can either provide a link to a hosted image URL or upload an image of the packaged food. When an image is successfully uploaded it will be visible just below the Check Health button.

Clicking the check health button will display a list of ingredients and the nutritional values present in the image along with the assessment done by the AI based on the present ingredients and nutri-values.

In the backend, we are using Semantic kernel to connect our Blazor app with the AI services.

To generate the above results we are using two different prompts sequentially. The first prompt works as an OCR that tries to get the list of ingredients and nutritional values while the second prompt gives an analysis based on the results obtained from the first prompt.

Currently, both prompts are using the same model but we could also assign different models for each prompt based on the requirements.

Local Setup

1. Clone the github repository Food checker git repo
2. API keys for either Azure OpenAI or OpenAI vision models are required. You can either set them locally through user secrets (or appsettings.json) or temporarily through the Blazor UI. (An error message will be displayed if the config isn’t set up properly)

Before running the app make sure you have .Net 8 installed.

NOTE - DataURIs aren’t properly supported yet so local images can’t be used. Use an Hosted image URL instead

Currently, this app is also available on HuggingFace spaces so you could duplicate this app using the duplicate space option. (Requires a HuggingFace account)

The app also includes some hardcoded examples to show what kind of results are expected (generated using gpt-4o model).

Click on any of the See Results buttons to display the results for the respective image.

Wrapping up

This was just an overview for what the app does.The sample demonstrates just a glimpse of what’s available in the semantic kernel SDK.

We can further extend this app by adding different services like Google’s Gemini or OpenSource models, adding more plugins, use filters cache the results, make it more robust and secure.

In the next blog we will be looking at the semantic kernel side of the app more in-depth.