Menu

1 / 2

Testing Long-Horizon Coherence in MusicGen: A Real-Data Mech Interp Pipeline

DEV Community·Amey Muke·7 months ago
#moU6hMw4
#why#ai#recurrence#real#features
Reading 0:00
15s threshold

Testing Long-Horizon Coherence in MusicGen: A Real-Data Mech Interp Pipeline

I have been working on a mechanistic interpretability experiment for music generation models.

The big motivating question is simple:

Do autoregressive music models have internal features that track long-horizon musical structure, or are they mostly stitching together locally plausible audio?

More concretely, I want to test whether a model like MusicGen has residual-stream or sparse-autoencoder features that say something like: "this motif should come back later," "this tension should resolve," or "this section is setting up a later recurrence."

That sounds exciting, but let me be clear upfront:

This post does not claim that I found foresight circuits.

What I have so far is a real-data pipeline, a benchmark slice, cached activations, recurrence proposals, and a set of artifacts that make the next causal experiments possible.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.