Collection by 
Aug 9, 2025
Collections
Discover the best community collections!
Collections including paper arxiv:2305.13048
Collection by 
Jan 4, 2025
-
Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning
Paper • 2310.20587 • Published • 18 -
JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention
Paper • 2310.00535 • Published • 2 -
Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla
Paper • 2307.09458 • Published • 12 -
The Impact of Depth and Width on Transformer Language Model Generalization
Paper • 2310.19956 • Published • 10
Collection by 
Feb 23, 2024
Collection by 
Jul 18, 2024
-
The Impact of Depth and Width on Transformer Language Model Generalization
Paper • 2310.19956 • Published • 10 -
Retentive Network: A Successor to Transformer for Large Language Models
Paper • 2307.08621 • Published • 173 -
RWKV: Reinventing RNNs for the Transformer Era
Paper • 2305.13048 • Published • 21 -
Attention Is All You Need
Paper • 1706.03762 • Published • 120
Collection by Mar 28, 2024
Collection by 
Feb 18, 2025
-
Nemotron-4 15B Technical Report
Paper • 2402.16819 • Published • 46 -
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Paper • 2402.19427 • Published • 57 -
RWKV: Reinventing RNNs for the Transformer Era
Paper • 2305.13048 • Published • 21 -
Reformer: The Efficient Transformer
Paper • 2001.04451 • Published • 2
Collection by 
Aug 1, 2024
-
Attention Is All You Need
Paper • 1706.03762 • Published • 120 -
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Paper • 1810.04805 • Published • 26 -
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Paper • 1907.11692 • Published • 10 -
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Paper • 1910.01108 • Published • 22
Collection by Aug 9, 2025
Collection by Mar 28, 2024
Collection by Jan 4, 2025
-
Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning
Paper • 2310.20587 • Published • 18 -
JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention
Paper • 2310.00535 • Published • 2 -
Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla
Paper • 2307.09458 • Published • 12 -
The Impact of Depth and Width on Transformer Language Model Generalization
Paper • 2310.19956 • Published • 10
Collection by Feb 18, 2025
-
Nemotron-4 15B Technical Report
Paper • 2402.16819 • Published • 46 -
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Paper • 2402.19427 • Published • 57 -
RWKV: Reinventing RNNs for the Transformer Era
Paper • 2305.13048 • Published • 21 -
Reformer: The Efficient Transformer
Paper • 2001.04451 • Published • 2
Collection by Feb 23, 2024
Collection by Aug 1, 2024
-
Attention Is All You Need
Paper • 1706.03762 • Published • 120 -
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Paper • 1810.04805 • Published • 26 -
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Paper • 1907.11692 • Published • 10 -
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Paper • 1910.01108 • Published • 22
Collection by Jul 18, 2024
-
The Impact of Depth and Width on Transformer Language Model Generalization
Paper • 2310.19956 • Published • 10 -
Retentive Network: A Successor to Transformer for Large Language Models
Paper • 2307.08621 • Published • 173 -
RWKV: Reinventing RNNs for the Transformer Era
Paper • 2305.13048 • Published • 21 -
Attention Is All You Need
Paper • 1706.03762 • Published • 120