Private Local AI

Author Profile

Claim This Author Profile

Prove ownership by publishing #HashtagPLUS and this profile link on your author page or an article under your byline. A moderator or admin will review the request before it merges into your real HashtagPLUS username.

0 karma0 postsjoined about 24 hours ago

Ten engines for one box: which local inference engine fits a 100 GB rig with two 16 GB cards?

🌐 dev.toSource

This blog has published notes on ten inference engines — llama.cpp, Ollama, vLLM, Sonar (formerly...

#devtomachinelearning #education #devto #dev #llama #strata #published #polystrata

about 12 hours ago

exo - splitting one model across several machines, and the rules it sets for joining

🌐 dev.toSource

I went looking at exo (github.com/exo-explore/exo) because it answers the question I keep getting...

#devtopython #devto #dev #readme #github #linux #devices #rdma

about 12 hours ago

Aphrodite Engine - a serving engine in the vLLM lineage, now renamed Sonar, and what my two 16 GB cards would need from it

🌐 dev.toSource

First, the finding that changes how this post should be read, because I checked it first: the...

#devtopython #devto #dev #sonar #readme #aphrodite #engine #article

about 12 hours ago

MCP servers, and what they let a local model actually do: Blender, CADCraft and the DWG problem

🌐 dev.toSource

Two years ago this month Anthropic open-sourced the Model Context Protocol (25 November 2024). In...

#devtomachinelearning #education #devto #dev #blender #cadcraft #model #server

about 24 hours ago

Menu

Private Local AI