Smart Mall Navigation Assistant

Shoppers in large malls waste time finding products and staff can't answer every query. Existing navigation solutions are either expensive digital signage or paper maps that go out of date. No affordable intelligent hardware solution existed at the store level.

A Raspberry Pi-based handheld and kiosk unit running a local LLM pipeline at the edge. OpenAI Whisper captures the user's voice query, converts it to text, and passes it to a constrained Gemini-powered intent parser that extracts what the user wants. The actual product location, recommendations, and information are served from a local structured database — not from the LLM directly. The hardware stays cheap and deployable. The intelligence lives inside.

An LLM is great at language and terrible at being a source of truth. Early on I let Gemini answer location questions directly — it confidently invented aisles that didn't exist. The fix was constraint: the LLM parses intent only, then I look the actual location up in my own verified data store. Never trust a language model to remember where the cereal is.

Latency is a UX feature. A two second round-trip feels broken on a kiosk where someone is standing and waiting. I added local caching for common queries which cut response time dramatically for the 20% of questions that cover 80% of real usage.

Good hardware UX is mostly about hiding complexity. The user sees a device that listens and answers. They don't see the Whisper transcription, intent parsing, database lookup, and response formatting happening in under a second. Making all of that invisible is the actual engineering challenge.