Models
Datasets
Spaces
Buckets new
Docs
Enterprise
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2308.14711

Papers - Custom Layers - Feedforward Neural Network (FFN)

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

Paper • 2402.15627 • Published Feb 23, 2024 • 36
One Wide Feedforward is All You Need

Paper • 2309.01826 • Published Sep 4, 2023 • 34
Fast Feedforward Networks

Paper • 2308.14711 • Published Aug 28, 2023 • 3
Memory Layers at Scale

Paper • 2412.09764 • Published Dec 12, 2024 • 5

Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts

Paper • 2009.10622 • Published Sep 22, 2020 • 1
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

Paper • 2401.15947 • Published Jan 29, 2024 • 53
MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

Paper • 2401.04081 • Published Jan 8, 2024 • 74
MoE-Infinity: Activation-Aware Expert Offloading for Efficient MoE Serving

Paper • 2401.14361 • Published Jan 25, 2024 • 2

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models

Paper • 2310.16795 • Published Oct 25, 2023 • 27
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference

Paper • 2308.12066 • Published Aug 23, 2023 • 4
Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference

Paper • 2303.06182 • Published Mar 10, 2023 • 1
EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate

Paper • 2112.14397 • Published Dec 29, 2021 • 1

Papers - ETH Zurich

I-Design: Personalized LLM Interior Designer

Paper • 2404.02838 • Published Apr 3, 2024 • 2
Scaling MLPs: A Tale of Inductive Bias

Paper • 2306.13575 • Published Jun 23, 2023 • 17
Fast Feedforward Networks

Paper • 2308.14711 • Published Aug 28, 2023 • 3
How Good Are Low-bit Quantized LLaMA3 Models? An Empirical Study

Paper • 2404.14047 • Published Apr 22, 2024 • 45

Scaling MLPs: A Tale of Inductive Bias

Paper • 2306.13575 • Published Jun 23, 2023 • 17
Trap of Feature Diversity in the Learning of MLPs

Paper • 2112.00980 • Published Dec 2, 2021 • 2
Understanding the Spectral Bias of Coordinate Based MLPs Via Training Dynamics

Paper • 2301.05816 • Published Jan 14, 2023 • 1
RaftMLP: How Much Can Be Done Without Attention and with Less Spatial Locality?

Paper • 2108.04384 • Published Aug 9, 2021 • 1

Papers - Custom Layers - Feedforward Neural Network (FFN)

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

Paper • 2402.15627 • Published Feb 23, 2024 • 36
One Wide Feedforward is All You Need

Paper • 2309.01826 • Published Sep 4, 2023 • 34
Fast Feedforward Networks

Paper • 2308.14711 • Published Aug 28, 2023 • 3
Memory Layers at Scale

Paper • 2412.09764 • Published Dec 12, 2024 • 5

Papers - ETH Zurich

I-Design: Personalized LLM Interior Designer

Paper • 2404.02838 • Published Apr 3, 2024 • 2
Scaling MLPs: A Tale of Inductive Bias

Paper • 2306.13575 • Published Jun 23, 2023 • 17
Fast Feedforward Networks

Paper • 2308.14711 • Published Aug 28, 2023 • 3
How Good Are Low-bit Quantized LLaMA3 Models? An Empirical Study

Paper • 2404.14047 • Published Apr 22, 2024 • 45

Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts

Paper • 2009.10622 • Published Sep 22, 2020 • 1
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

Paper • 2401.15947 • Published Jan 29, 2024 • 53
MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

Paper • 2401.04081 • Published Jan 8, 2024 • 74
MoE-Infinity: Activation-Aware Expert Offloading for Efficient MoE Serving

Paper • 2401.14361 • Published Jan 25, 2024 • 2

Scaling MLPs: A Tale of Inductive Bias

Paper • 2306.13575 • Published Jun 23, 2023 • 17
Trap of Feature Diversity in the Learning of MLPs

Paper • 2112.00980 • Published Dec 2, 2021 • 2
Understanding the Spectral Bias of Coordinate Based MLPs Via Training Dynamics

Paper • 2301.05816 • Published Jan 14, 2023 • 1
RaftMLP: How Much Can Be Done Without Attention and with Less Spatial Locality?

Paper • 2108.04384 • Published Aug 9, 2021 • 1

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models

Paper • 2310.16795 • Published Oct 25, 2023 • 27
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference

Paper • 2308.12066 • Published Aug 23, 2023 • 4
Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference

Paper • 2303.06182 • Published Mar 10, 2023 • 1
EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate

Paper • 2112.14397 • Published Dec 29, 2021 • 1

Company

TOS Privacy About Careers

Website

Models Datasets Spaces Pricing Docs