Model & weights: BrunoHays/embeddinggemma-2-nullspace-audio
I wanted to share an experiment on google/embeddinggemma-2 (577M) for MAEB, the 30-task audio benchmark on the MTEB leaderboard . By projecting intermediate Conformer layers into the unused null space of the model's final projection matrix using only PCA, the model's macro score jumps from 49.3% to 54.4% (+5.1 points) , moving from Rank #26 to #1 on MAEB audio-only (and #16 to #1 on full MAEB ), ahead of 8B-9B multimodal models.
1. Last-layer filtering in text-aligned audio encoders
Across the MAEB leaderboard, LLM-backed Omni embedders score 95-99% on multilingual text-to-audio retrieval, but drop to 1%-6% on unsupervised audio clustering and ~20% on vocal emotion ( CREMA-D ). Aligning an audio encoder to a text LLM filters the final layers: lexical content ( what was said ) is kept, while paralinguistic and acoustic traits ( timbre, pitch, emotion ) are discarded.
2. Probing the 12 Conformer layers
Layer-wise speech studies ( Pasad et al.…