Teaching our on-device assistant to say 'I don't know'

A smarter first step

Every request to the FamiliaSync assistant now starts with a small intent classifier trained on-device. It replaces the hand-written pattern rules that used to decide what you meant. The pipeline behind it is unchanged in spirit: your text is routed to the right tool, or handed to Needle 2 - the on-device model bundled with the app - when a request needs open-ended reasoning.

The training corpus was researched, not generated in bulk: 105 intents backed by roughly 2,400 example utterances carrying the slots real commands need - dates, times, assignees, quantities, amounts, categories, and recurrence. Thirty-five of those intents map to real tools in the app.

Training the 'no' cases

Here's the part we didn't expect to matter most: 70 of the 105 intents are documented-only. They cover things the assistant can't do yet - notes, stores, loans, meter readings - and they exist so that an unsupported request falls through to the model for a plain-language answer instead of being forced into an unrelated tool. Training the 'no' cases turned out to be as important as training the 'yes' cases.

A confidence gate sits on top: if the classifier isn't sure, it doesn't route at all - the model handles the request instead. And everything happens on-device. Training is lazy and local, inference is local, and no query ever leaves the phone.

Real phrasing broke it within hours

Shipping the router was the easy part. Real queries surfaced three misroutes the same day, and each one taught us something:

• "what are todays plan" fell below the confidence threshold - and a stray keyword let the model invent a meal called 'Sri, Ram' from family names in the prompt. Loose keywords were stripped, and 'today's plan' phrasings now map to listing events.
• "what is my meal plan for today" confidently matched the wrong tool - a read question nearly overwrote the shopping list, stopped only by the confirmation gate. That tool now requires an explicit generate signal, and the corpus grew question-form examples.
• The meal-plan answer that did work dumped raw record IDs and the entire week. It now renders meal names and accepts filters like 'today' or 'lunch'.

From commands to conversation

The bigger shift: the assistant can now ask back. Tell it 'create a reminder' and instead of guessing, it asks what to remind you about - then completes the action from your reply. Tools can request a missing argument mid-flow, and partially collected details carry over between turns.

Follow-ups exposed gaps we didn't know existed. When people answer in their own words, dates arrive as 'friday 6pm', 'tonight', or '19th spetember 2026' - so the date parser now handles casual formats, including typo-tolerant month names. And when someone answers 'anyone' or 'doesn't matter' to a member prompt, the event is created unassigned rather than silently picking the first family member.

“An assistant that knows when it doesn't know is more trustworthy than one that always guesses.”