Menu

Locally trained VLA model with minimal attention based vision encoder. [p]
📰

Locally trained VLA model with minimal attention based vision encoder. [p]

Reading 0:00
15s threshold

Hi y’all. Thought Id share the progress on my current local VLA agent project! using agent and language somewhat loosely here. Technically it is an agent, and technically it does adhere to text prompts. However very minimally.

Video Link: https://www.reddit.com/r/learnmachinelearning/comments/1xq23zi/local_rnn_style_vla_model_project/

Video Description:

The leftmost display shows what the models sees, The yellow triangle marker on screen shows which shape is NOT the target shape on that run (counter intuitive, I know). Importantly, the model does not see it, it is only to make it easier to see if the model is performing correctly. The upper right display shows the models visual attention, in which I discuss later. And the bottom right is the per step state matrix cell values. A small 128x128 view of the whole 768x768 matrix.

Preamble:

Recently, I have been extremely fascinated by machine learning and AI, for obvious reasons.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.