Navigation aid · working title
A browser app that turns a webcam into spatial audio, so you can hear where obstacles are.
A proof of concept and a playground. I wanted to see how far you can get with a laptop webcam, a free open-source depth model and a pair of headphones. Could you hear the shape of a room?
It was an excuse to get hands-on with tech I hadn’t combined before: monocular depth estimation, AI inference running entirely in the browser on WebGPU, and 3D spatial audio. Navigation for blind and low-vision people was the idea that tied it together, and a good reason to take the safety side seriously, even in a side project.
Like a car’s parking sensor, but for walking, and with directional sound.
camera → depth model → sectors → selection → beep timing → spatial audio
“Silence must never mean all clear.”
If the model stops producing results for 500 ms, a distinct alarm plays. A second check catches main-thread freezes the timer can’t see. A pause cue plays before audio stops when the tab is hidden, a screen wake lock keeps the device awake, and the volume slider can’t mute the alarms.
Transformers.js runs the ONNX model on WebGPU in fp16, falling back to 8-bit WASM on machines without WebGPU. Video never leaves the device, with no server or install.
Beeps are scheduled ahead on the audio clock (20 ms tick, 60 ms lookahead), so the rhythm stays steady when frame rate varies, with no stutters or catch-up bursts.
Every frame logs capture, inference, processing and audio timing. The app shows median and p95 latency and exports sessions as CSV, so frame-to-sound delay can be measured.
All decision logic (sector math, voice selection, beep timing, watchdog, statistics) lives in pure TypeScript modules, covered by 43 passing unit tests.
A control panel adjusts sector count, cutoff, max voices, beep speed range, arc width, pitch mode and volume while the app runs.