Models
Datasets
Spaces
Buckets new
Docs
Enterprise
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2312.12742

Order Matters in the Presence of Dataset Imbalance for Multilingual Learning

Paper • 2312.06134 • Published Dec 11, 2023 • 3
Efficient Monotonic Multihead Attention

Paper • 2312.04515 • Published Dec 7, 2023 • 8
Contrastive Decoding Improves Reasoning in Large Language Models

Paper • 2309.09117 • Published Sep 17, 2023 • 40
Exploring Format Consistency for Instruction Tuning

Paper • 2307.15504 • Published Jul 28, 2023 • 8

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

Paper • 2312.15166 • Published Dec 23, 2023 • 61
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU

Paper • 2312.12456 • Published Dec 16, 2023 • 45
Cached Transformers: Improving Transformers with Differentiable Memory Cache

Paper • 2312.12742 • Published Dec 20, 2023 • 13
Mini-GPTs: Efficient Large Language Models through Contextual Pruning

Paper • 2312.12682 • Published Dec 20, 2023 • 9

Model Architectures

togethercomputer/StripedHyena-Hessian-7B

Text Generation • 8B • Updated Mar 27, 2024 • 76 • 66
Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention

Paper • 2312.08618 • Published Dec 14, 2023 • 13
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention

Paper • 2312.07987 • Published Dec 13, 2023 • 41
LLM360: Towards Fully Transparent Open-Source LLMs

Paper • 2312.06550 • Published Dec 11, 2023 • 57

Efficient LLM Inference on CPUs

Paper • 2311.00502 • Published Nov 1, 2023 • 7
Exponentially Faster Language Modelling

Paper • 2311.10770 • Published Nov 15, 2023 • 119
Cached Transformers: Improving Transformers with Differentiable Memory Cache

Paper • 2312.12742 • Published Dec 20, 2023 • 13
LLM in a flash: Efficient Large Language Model Inference with Limited Memory

Paper • 2312.11514 • Published Dec 12, 2023 • 264

Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering

Paper • 2204.04581 • Published Apr 10, 2022 • 1
Retrieval-Augmented Multimodal Language Modeling

Paper • 2211.12561 • Published Nov 22, 2022 • 1
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

Paper • 2212.10511 • Published Dec 20, 2022 • 1
Memorizing Transformers

Paper • 2203.08913 • Published Mar 16, 2022 • 2

aMUSEd: An Open MUSE Reproduction

Paper • 2401.01808 • Published Jan 3, 2024 • 31
From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations

Paper • 2401.01885 • Published Jan 3, 2024 • 28
SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity

Paper • 2401.00604 • Published Dec 31, 2023 • 6
LARP: Language-Agent Role Play for Open-World Games

Paper • 2312.17653 • Published Dec 24, 2023 • 33

Cached Transformers: Improving Transformers with Differentiable Memory Cache

Paper • 2312.12742 • Published Dec 20, 2023 • 13
ProTIP: Progressive Tool Retrieval Improves Planning

Paper • 2312.10332 • Published Dec 16, 2023 • 8
Paloma: A Benchmark for Evaluating Language Model Fit

Paper • 2312.10523 • Published Dec 16, 2023 • 13
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Paper • 2406.17557 • Published Jun 25, 2024 • 102

Trellis Networks for Sequence Modeling

Paper • 1810.06682 • Published Oct 15, 2018 • 1
ProSG: Using Prompt Synthetic Gradients to Alleviate Prompt Forgetting of RNN-like Language Models

Paper • 2311.01981 • Published Nov 3, 2023 • 1
Gated recurrent neural networks discover attention

Paper • 2309.01775 • Published Sep 4, 2023 • 10
Inverse Approximation Theory for Nonlinear Recurrent Neural Networks

Paper • 2305.19190 • Published May 30, 2023 • 1

LLM architecture

The Impact of Depth and Width on Transformer Language Model Generalization

Paper • 2310.19956 • Published Oct 30, 2023 • 10
Retentive Network: A Successor to Transformer for Large Language Models

Paper • 2307.08621 • Published Jul 17, 2023 • 173
RWKV: Reinventing RNNs for the Transformer Era

Paper • 2305.13048 • Published May 22, 2023 • 21
Attention Is All You Need

Paper • 1706.03762 • Published Jun 12, 2017 • 121

TRAMS: Training-free Memory Selection for Long-range Language Modeling

Paper • 2310.15494 • Published Oct 24, 2023 • 2
A Long Way to Go: Investigating Length Correlations in RLHF

Paper • 2310.03716 • Published Oct 5, 2023 • 10
YaRN: Efficient Context Window Extension of Large Language Models

Paper • 2309.00071 • Published Aug 31, 2023 • 82
Giraffe: Adventures in Expanding Context Lengths in LLMs

Paper • 2308.10882 • Published Aug 21, 2023 • 1

Order Matters in the Presence of Dataset Imbalance for Multilingual Learning

Paper • 2312.06134 • Published Dec 11, 2023 • 3
Efficient Monotonic Multihead Attention

Paper • 2312.04515 • Published Dec 7, 2023 • 8
Contrastive Decoding Improves Reasoning in Large Language Models

Paper • 2309.09117 • Published Sep 17, 2023 • 40
Exploring Format Consistency for Instruction Tuning

Paper • 2307.15504 • Published Jul 28, 2023 • 8

aMUSEd: An Open MUSE Reproduction

Paper • 2401.01808 • Published Jan 3, 2024 • 31
From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations

Paper • 2401.01885 • Published Jan 3, 2024 • 28
SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity

Paper • 2401.00604 • Published Dec 31, 2023 • 6
LARP: Language-Agent Role Play for Open-World Games

Paper • 2312.17653 • Published Dec 24, 2023 • 33

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

Paper • 2312.15166 • Published Dec 23, 2023 • 61
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU

Paper • 2312.12456 • Published Dec 16, 2023 • 45
Cached Transformers: Improving Transformers with Differentiable Memory Cache

Paper • 2312.12742 • Published Dec 20, 2023 • 13
Mini-GPTs: Efficient Large Language Models through Contextual Pruning

Paper • 2312.12682 • Published Dec 20, 2023 • 9

Cached Transformers: Improving Transformers with Differentiable Memory Cache

Paper • 2312.12742 • Published Dec 20, 2023 • 13
ProTIP: Progressive Tool Retrieval Improves Planning

Paper • 2312.10332 • Published Dec 16, 2023 • 8
Paloma: A Benchmark for Evaluating Language Model Fit

Paper • 2312.10523 • Published Dec 16, 2023 • 13
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Paper • 2406.17557 • Published Jun 25, 2024 • 102

Model Architectures

togethercomputer/StripedHyena-Hessian-7B

Text Generation • 8B • Updated Mar 27, 2024 • 76 • 66
Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention

Paper • 2312.08618 • Published Dec 14, 2023 • 13
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention

Paper • 2312.07987 • Published Dec 13, 2023 • 41
LLM360: Towards Fully Transparent Open-Source LLMs

Paper • 2312.06550 • Published Dec 11, 2023 • 57

Trellis Networks for Sequence Modeling

Paper • 1810.06682 • Published Oct 15, 2018 • 1
ProSG: Using Prompt Synthetic Gradients to Alleviate Prompt Forgetting of RNN-like Language Models

Paper • 2311.01981 • Published Nov 3, 2023 • 1
Gated recurrent neural networks discover attention

Paper • 2309.01775 • Published Sep 4, 2023 • 10
Inverse Approximation Theory for Nonlinear Recurrent Neural Networks

Paper • 2305.19190 • Published May 30, 2023 • 1

Efficient LLM Inference on CPUs

Paper • 2311.00502 • Published Nov 1, 2023 • 7
Exponentially Faster Language Modelling

Paper • 2311.10770 • Published Nov 15, 2023 • 119
Cached Transformers: Improving Transformers with Differentiable Memory Cache

Paper • 2312.12742 • Published Dec 20, 2023 • 13
LLM in a flash: Efficient Large Language Model Inference with Limited Memory

Paper • 2312.11514 • Published Dec 12, 2023 • 264

LLM architecture

The Impact of Depth and Width on Transformer Language Model Generalization

Paper • 2310.19956 • Published Oct 30, 2023 • 10
Retentive Network: A Successor to Transformer for Large Language Models

Paper • 2307.08621 • Published Jul 17, 2023 • 173
RWKV: Reinventing RNNs for the Transformer Era

Paper • 2305.13048 • Published May 22, 2023 • 21
Attention Is All You Need

Paper • 1706.03762 • Published Jun 12, 2017 • 121

Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering

Paper • 2204.04581 • Published Apr 10, 2022 • 1
Retrieval-Augmented Multimodal Language Modeling

Paper • 2211.12561 • Published Nov 22, 2022 • 1
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

Paper • 2212.10511 • Published Dec 20, 2022 • 1
Memorizing Transformers

Paper • 2203.08913 • Published Mar 16, 2022 • 2

TRAMS: Training-free Memory Selection for Long-range Language Modeling

Paper • 2310.15494 • Published Oct 24, 2023 • 2
A Long Way to Go: Investigating Length Correlations in RLHF

Paper • 2310.03716 • Published Oct 5, 2023 • 10
YaRN: Efficient Context Window Extension of Large Language Models

Paper • 2309.00071 • Published Aug 31, 2023 • 82
Giraffe: Adventures in Expanding Context Lengths in LLMs

Paper • 2308.10882 • Published Aug 21, 2023 • 1

Company

TOS Privacy About Careers

Website

Models Datasets Spaces Pricing Docs