Menu

Miktam Bible | Part 2 — An AI Study Assistant That Runs on the Phone
📰

Miktam Bible | Part 2 — An AI Study Assistant That Runs on the Phone

DEV Community: reactnative·Ai2th·about 2 hours ago
#YDntMB9C
#devtoreactnative#devto#dev#model#verses
Reading 0:00
15s threshold

Ai2th

Posted on Nov 15

Edited on Nov 15

Most AI features in apps are a thin wrapper around a cloud API. Miktam Bible's study assistant is different: the language model runs on the phone , and every answer is grounded in verses pulled from the offline database.

🧠 The model

The assistant runs SmolLM2-135M , a small model quantized to 4 bits in GGUF format (about a 105 MB one-time download), through llama.rn , the React Native binding for llama.cpp. The settings are chosen for phones:

context = await initLlama ( { model : nativePath , n_ctx : 2048 , // Mobile-safe context window n_batch : 256 , // Mobile-safe batch size n_threads : 4 , // CPU threads for inference n_gpu_layers : 0 , // 0 = CPU only use_mlock : false , use_mmap : true , // Memory-map the model file } ) ;

Enter fullscreen mode

Exit fullscreen mode

CPU-only with a memory-mapped model file keeps it working across the wide range of Android hardware.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.