I just released AnswerSearch in open source. This is a Swift Mac app I put on the Mac App Store a few months ago. Source code on GitHub at ancientcomputing/answersearch-oss. Some background on what it is and why I'm releasing it first, then the part that might be useful to other Swift developers: using Apple's on-device Foundation Model to decide when to call the cloud.

I built AnswerSearch as a Google web search killer. Why search with Google (or Bing, or any other old-school search engine) for web links, then sort through endless webpages (and ads) just to answer a question? That experience is so 1999 when ChatGPT or Claude can give you the answer along with the links.

Of course, you won't want your boomer parents using ChatGPT or Claude. All the bells and whistles mean you'll end up as their tech support.

So AnswerSearch is a deliberately simple UI: ask a question, get the answer, call it a day. Even a caveman can use it (ahem).

Unfortunately, it turns out the idea doesn't really work as a product. You have to bring your own API key, which is a hassle and makes it a poor deal for regular people. OpenAI is free so replacing it with tokenomics doesn't make sense.

The on-device Apple Foundation Model is free with your Mac, but can only be used for questions that don't need a web search.

Net: AnswerSearch isn't going anywhere as a product. I'm releasing the code because there's no reason to keep it to myself. You can read exactly what the app does with user inputs, build it yourself without the App Store, fork it, borrow parts of it or make it your own thing (for said boomer parents?).

So here is the part of the implementation that may interesting. To save tokens, the on-device model answers the simple questions itself.. For everything else, it routes the question to your cloud AI provider. Essentially, the on-device model is working as a "decision model", before Jev became all the rage.

The routing is a triage step: the on-device model classifies the question and self-reports a confidence label (high, medium or low) using guided generation. Apple's API doesn't expose a numeric score. Only "high", plus a few other checks, keeps it local:

- **High confidence:** it answers locally. It's free, private and instant.

- **Low confidence:** it hands the question to the cloud provider, which can search the web.

Two source files cover how the confidence works: Sources/AnswerSearchKit/AppleLocalAnswerer.swift and LocalPrompts.swift.

The confidence is self-reported by a small model, so it is a heuristic, not a calibrated probability. The category list and the info flags act as guardrails around it. The design favors precision over recall: it will send some answerable questions to the cloud rather than risk a wrong or stale local answer.

The full design is described in docs/annotations/02-local-ai.md

Note: This isn't self-promotion. I'm just letting you know the source is there to read, reuse and ultimately to have fun with.

submitted by /u/AdventurousKeys
[link] [comments]

Menu

AnswerSearch is open-sourced