Linear oscillatory state-space models

A common critique of machine learning research is that the results of many papers are based on trial-and-error, often representing only minor modifications of existing approaches and without any proper justification of the design decisions. The typical justification for publication is that a model resulted in a good score on some established benchmark, with the validity of these benchmarks being a topic on its own. Even from an engineering perspective this is obviously highly undesirable.

I was therefore very delighted by a preliminary reading of linear oscillatory state-space models presented by T. Konstantin Rusch and Daniela Rus at ICLR 2025. This publication combines several desirable qualities.

  1. It is based on a justified theoretical model
  2. The approach strongly inspired by physics and neurobiology
  3. It exercises mathematical rigour1
  4. It includes empirical evidence
  5. It provides a reference implementation
  6. The approach could have substantial impact in multiple domains
  7. It is actually an enjoyable read!

Leaving aside the formal perspective, the ability to model very long-range interactions on multivariate sequences with high accuracy using a theoretically well-grounded model makes the publication appealing especially for domains where explainability is important.


1. I have not verified the claims, but as opposed to theoretical contributions in other publications, given sufficient time the proofs actually appear rather digestable mainly drawing from Calculus and Linear Algebra

Interesting Stuff

Me writing about Tech stuff