I got tired of waiting 500ms every time I typed something into my app's command bar.
When you just want to type "Algorithms exam tomorrow at 9am" and hit Enter, sending a request to a cloud LLM feels broken. You feel the network lag, it fails on bad campus Wi-Fi, and paying for API calls just to figure out if a sentence is a calendar event or a to-do item makes no sense.
So for Stedy
, I ripped the cloud call out of the hot path and moved classification into the browser.
The setup is pretty simple:
Deterministic TypeScript code extracts dates, time ranges, and known course names.
ADS-1.5M, a small INT8-quantized model (~1.5M parameters), runs in WebAssembly via onnxruntime-web/wasm to classify the intent (EVENT, TASK, or NOTE) and category.
Once the .int8.onnx file and vocab.json are cached in CacheStorage, a forward pass takes less than 10ms locally:
import * as ort from "onnxruntime-web/wasm" ; const session = await ort . InferenceSession .…