Menu

Post image 1
Post image 2
Post image 3
Post image 4
Post image 5
Post image 6
Post image 7
Post image 8
1 / 8
0

GitHub - Sparticle62ops/pssa: A custom AI architecture being developed in rust

Hacker News·about 16 hours ago
#1WIj9oHX
Reading 0:00
15s threshold

PSSA: a plastic state-space architecture PSSA is a small language model that is not a transformer. It reads text one token at a time through a recurrent state-space layer, keeps a bank of episodic memories it can look things up in, and rewrites part of its own weights while it runs. It is written in Rust from scratch, with no PyTorch, no TensorFlow, and no ML framework of any kind underneath it. At matched parameters and on the same corpus, it learns faster than a transformer and generates text about twelve times quicker on the same CPU. Why Rust, and why that is not the point Not for speed points, and not because the language makes the architecture better. PSSA needed per-token weight updates, a memory bank written during the forward pass, and a scalar reference path that every batched kernel could be differentiated against. Expressing that inside an autograd framework meant fighting the framework at every step, so the linear algebra is written directly instead.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More