BlinkDL/RWKV-LM

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

328 best points · 107 comments · 3 Hacker News posts · 14,772 GitHub stars · View on GitHub

Hacker News discussions

• RWKV RNN: Better than ChatGPT? (328 points, 107 comments, by pffft8888, 2023-03-23)
• RWKV-LM: RNN with transformer-level performance, without using attention (5 points, 0 comments, by azhenley, 2023-02-27)
• RWKV-LM: RNN with transformer-level performance, without using attention (4 points, 0 comments, by vletal, 2022-05-11)