Coming Soon
Camera-First AI Assistant · Personal Project
A camera-first iOS assistant: point at anything, tap to select it, and ask what it is, how it works, or what to do next — answered in under two seconds by Gemma running on Cerebras. A second entry point adds a full chat and voice assistant with memory, a browsable capture journal, and tool access. One person across the whole stack: on-device segmentation, the backend orchestrator, and the tool-confirmation layer between them.
Swift / SwiftUI
Metal
Python
FastAPI
Google Cloud Run
Cerebras
Gemma
ElevenLabs
MCPPerception vs. Cognition
If a task has a correct answer computable from pixels alone — segmentation, OCR, barcode decode, similarity — it runs on-device with Apple Vision. Anything needing world knowledge or reasoning goes to Gemma, and only Gemma's voice ever reaches the user.
A Sticker, Not a Spinner
On-device segmentation isolates the tapped object, a Metal dot-dissolve shader disintegrates the background while the object stays sharp, then it floats onto a paper-like dot-grid canvas as a die-cut sticker with a semantic label pill.
Hard Latency Budgets
Tap ripple under 50 ms, initial mask under 500 ms, first useful model result under 2 seconds, first spoken audio under 3 seconds — enforced as explicit performance budgets, not aspirations.
Live Assist Never Freezes
Live Assist is a distinct mode inside the camera tab — an ongoing voice conversation about whatever the camera currently sees, deliberately built to never freeze the UI, unlike the tap-and-ask flow.
One Voice Stack, Two Tabs
Chat's Voice Mode reuses the same VoiceConversationService and SpeechService as Camera's Live Assist rather than reimplementing STT, endpointing, and barge-in — but wires its own AudioPipeline instance so the two tabs' audio never collides.
Model Race
A long press on Send races the selected Cerebras model against an OpenRouter model on the same question, client-driven with two parallel calls, with an "adopt this answer" action to keep the better response.
Nothing Writes Without a Preview
Write-capable tools require both read_only:false and an explicit allow_write:true. When the model selects one, the orchestrator sends an immutable action preview the user must confirm before exchanging it for a signed, five-minute execution token — editing or replaying it invalidates the signature.
No Implicit Trust From Metadata
A connected MCP server's own tool descriptions are treated as descriptive text, not as a source of permission. Write tools stay narrowly allowlisted, each with the least-privilege credentials its specific operations need.
A Personal-Project Safety Net
A Pub/Sub-triggered Cloud Function watches the project's budget alerts and, if spend ever reaches the cap, detaches billing from the GCP project entirely — a hard stop for a self-funded prototype, deliberately blunter than anything you'd ship to production.
