Menu

Post image 1
Post image 2
1 / 2
0

Colibrì proof-of-concept gains frontier-level 1.5-TB AI model — novel approach runs on only 25GB of RAM and shows promise for local AI setups

Latest from Tom's Hardware ·Bruno Ferreira·2 months ago
#WMYEIQCz
Reading 0:00
15s threshold

Running LLMs and agents in home lab setups is steadily gaining popularity due to the rising cost of AI bot subscriptions and concerns about data privacy. Unfortunately, an Nvidia NVL72 rack is ever so slightly out of the financial reach of most people, so enthusiasts have to make do with models that can run in limited amounts of memory. Italian engineer Vincenzo (aka JustVugg) seemingly wanted to have his cake and eat it, so he created ColibrÌ to run the 744-billion-parameter 1.5-TB GLM-5.2 model on a modest CPU, a mere 25 GB of RAM, and a 1 GB/s virtual NVMe drive.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More