Vuk Rosić
Machine learning research: optimizers, LLM architecture, memory and world models. Every result comes with code that reproduces it.
Papers
- Recursive Latent Reasoning by Topological ContractionReasoning by repeatedly refining a latent state.
- Analysis and Design of Novel Optimizers for Neural Networks
- Auto-Selection between Linear and Full Attention Layers for Every Token
Results with code
-
A fixed word-to-expert table trains as well as a learned router, and stays balanced up to about 26 experts per chosen expert; sending “the”, “,” and “.” to the shared expert raises the limit to about 42.2026-11-14
- Training one network on two losses added together hurts the thing with the smaller losspaper in progressAcross 519 shared networks in four worlds, the smaller-loss term's extra error follows P − 1 ≈ 0.15 Λ0.75, where Λ is the loss ratio.2026-11-12
-
Keying an LLM's n-gram memory by the set of words, not their order, so it recalls facts asked in new words.2026-10-30
-
When making every row of Muon's update the same length helps, and where to apply it in recent open LLMs.2026-10-19
-
Storing momentum along random directions instead of the strongest (SVD) ones gives the same benefit with less memory.2026-10-20
-
Per-head Muon on q and k leads early and the lead shrinks; head crowding explains the late loss.2026-10-21
-
A grid of Newton–Schulz-type formulas, a picker page and coefficients for each target.2026-10-18
-
Letting each token decide how many tokens to read.2026-10-22
-
Deleting the attention value matrices from some layers of an LLM.2026-10-22
Papers in progress
- started 2026-11-12
- started 2026-11-12