Menu

Post image 1
Post image 2
Post image 3
Post image 4
1 / 4
475

How to Setup a Local Coding Agent on macOS

#ikyle#gguf#gemma#llama#models#model
Reading 0:00
15s threshold

I'd had my internet fail a few times recently leaving me stranded without a coding agent, and so when I saw the "Gemma 4 now runs 2x faster with MTP" Multi-Token Prediction update for Gemma 4 I decided to have a go at getting it running. I wanted a local coding agent setup that: was fast enough to actually use on my Mac worked through an OpenAI compatible API (so I could use it in other tools) and preferably could handle screenshots/images when needed, so I can feed it screenshots of what it has made. And I did! This video is realtime. And shows the agent responding at a perfectly usable speed. After a bit of testing the final setup I ended up with is: llama.cpp built with Metal on macOS Gemma 4 26B-A4B in GGUF format A Q8 MTP draft model for speculative decoding The Gemma 4 multimodal projector Pi as the terminal coding agent This was tested on an Apple M1 Max with 64 GB unified memory, running macOS 15.7.7. The Model The main model is: gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More