Menu

Post image 1
Post image 2
1 / 2
0

Mr.PERFECT---TO PERFORM AGENTIC TASKS USING LOCAL LLM

DEV Community·Rajab Baig·3 months ago
#XqHiROGV
#dev#local#perfect#agent#server#github
Reading 0:00
15s threshold

GitHub “Finish-Up-A-Thon” Challenge Submission This is a submission for the GitHub Finish-Up-A-Thon Challenge What I Built What I Built: Agent Mr. Perfect The Backstory I started with a folder full of local LLM tools and a desire and wish to run AI model locally completely offline for accuracy, privacy and speed. However, for a long time due to my workings, the project was just a "server in a box"—I could chat with it, but it couldn't do anything. It was a powerful engine with no wheels. This challenge gave me the push to build the "wheels": a custom agentic layer I call Agent Mr. Perfect. The "Before": A Local LLM Server Before this submission, my project was essentially a local hosting setup. The Tech: I was using text-generation-webui as a backend to run quantized models like Gemma (via llama.cpp). It run perfectly on only CPU 20 GB RAM and 1TD SSD and NVME device. The Workflow: It was a "Prompt-In, Text-Out" system.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More