It's out! We published the xLSTM on Arxiv.
We finally published the xLSTM on arxiv.
I posted about this on x and linkedin:
The #xLSTM is finally live! What an exciting day!
— Maximilian Beck (@maxmbeck) May 8, 2024
How far do we get in language modeling with the LSTM compared to State-of-the-Art LLMs?
I would say pretty, pretty far!
How? We extend the LSTM with Exponential Gating and parallelizable Matrix Memory!https://t.co/Z2xaH0wfji pic.twitter.com/8fXj8XPWqv
Thanks @srush_nlp for this compelling collection of recent RNN-based Language Models!
— Maximilian Beck (@maxmbeck) May 8, 2024
I think now you have to update this list with the #xLSTM 😉
I agree, naming conventions are always hard...
In our paper we try to stick to the original LSTM formulation from the 1990s: https://t.co/prFJA7kPvp pic.twitter.com/Xe6R32pNsO