| base_model: katanemo/Arch-Router-1.5B | |
| base_model_relation: quantized | |
| quantized_by: ArtusDev | |
| <style> | |
| .container-dark { | |
| font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; | |
| line-height: 1.6; | |
| color: #d4d4d4; | |
| } | |
| a { | |
| color: #569cd6; | |
| text-decoration: none; | |
| font-weight: 600; | |
| } | |
| a:hover { | |
| text-decoration: underline; | |
| } | |
| .card-dark { | |
| background-color: #252526; | |
| border-radius: 12px; | |
| padding: 24px; | |
| margin-bottom: 20px; | |
| box-shadow: 0 4px 12px rgba(0,0,0,0.3); | |
| border: 1px solid #3c3c3c; | |
| } | |
| .card-dark h1 { | |
| font-size: 2.2em; | |
| color: #ffffff; | |
| text-align: center; | |
| margin-bottom: 10px; | |
| } | |
| .card-dark.card-dark-title h1 { | |
| font-size: 1.5em; | |
| } | |
| .card-dark .subtitle { | |
| text-align: center; | |
| font-size: 1.1em; | |
| color: #a0a0a0; | |
| } | |
| .card-dark h2 { | |
| font-size: 1.5em; | |
| margin-top: 0; | |
| padding-bottom: 10px; | |
| border-bottom: 1px solid #3c3c3c; | |
| color: #c586c0; | |
| } | |
| .card-dark h3 { | |
| font-size: 1.2em; | |
| color: #d4d4d4; | |
| } | |
| .styled-table { | |
| display: table; | |
| border: none; | |
| width: 100%; | |
| font-size: 0.95em; | |
| margin-bottom: 0px; | |
| } | |
| .styled-table thead th { | |
| background-color: #333333; | |
| color: #c586c0; | |
| text-align: left; | |
| } | |
| .styled-table th { | |
| padding: 12px 15px; | |
| } | |
| .styled-table td { | |
| padding: 0; | |
| } | |
| .styled-table table, .styled-table th, .styled-table td { | |
| border-left: none; | |
| border-right: none; | |
| border-bottom: none; | |
| } | |
| .styled-table td { | |
| border-bottom: 1px solid #3c3c3c; | |
| } | |
| .styled-table tbody tr { | |
| transition: background-color 0.1s ease; | |
| } | |
| .styled-table tbody tr:hover { | |
| background-color: #3a3a3a; | |
| } | |
| .styled-table tr:last-child td { | |
| border-bottom: none; | |
| } | |
| .styled-table td a { | |
| display: block; | |
| padding: 12px 15px; | |
| } | |
| .styled-table td a.fake-link { | |
| text-decoration:none; | |
| color:inherit; | |
| } | |
| details { | |
| margin-top: 20px; | |
| border: 1px solid #3c3c3c; | |
| border-radius: 8px; | |
| overflow: hidden; | |
| } | |
| summary { | |
| cursor: pointer; | |
| padding: 12px 18px; | |
| background-color: #6A5ACD; | |
| font-weight: 600; | |
| display: flex; | |
| align-items: center; | |
| gap: 10px; | |
| justify-content: space-between; | |
| list-style: none; | |
| } | |
| summary::-webkit-details-marker { | |
| display: none; | |
| } | |
| summary:hover { | |
| filter: brightness(1.1); | |
| } | |
| summary::after { | |
| content: ''; | |
| display: inline-block; | |
| width: 8px; | |
| height: 8px; | |
| border-bottom: 2px solid white; | |
| border-right: 2px solid white; | |
| transform: rotate(45deg); | |
| transition: transform 0.3s ease; | |
| } | |
| details[open] > summary::after { | |
| transform: rotate(225deg); | |
| } | |
| .details-content { | |
| padding: 18px; | |
| } | |
| .btn-purple { | |
| display: inline-block; | |
| background-color: #6A5ACD; | |
| color: white !important; | |
| padding: 12px 24px; | |
| border-radius: 8px; | |
| text-decoration: none; | |
| font-weight: 600; | |
| transition: background-color 0.3s ease, transform 0.2s ease; | |
| text-align: center; | |
| } | |
| .btn-purple:hover { | |
| background-color: #7B68EE; | |
| transform: translateY(-2px); | |
| } | |
| </style> | |
| <div class="container-dark"> | |
| <div class="card-dark card-dark-title"> | |
| <h1>ArtusDev/katanemo_Arch-Router-1.5B-EXL3</h1> | |
| <p class="subtitle"> | |
| EXL3 quants of <a href="https://huggingface.co/katanemo/Arch-Router-1.5B" target="_blank">katanemo/Arch-Router-1.5B</a> using <a href="https://github.com/turboderp-org/exllamav3/" target="_blank">exllamav3</a> for quantization. | |
| </p> | |
| </div> | |
| <div class="card-dark"> | |
| <h2>Quants</h2> | |
| <table class="styled-table"> | |
| <thead> | |
| <tr> | |
| <th>Quant</th> | |
| <th>BPW</th> | |
| <th>Head Bits</th> | |
| <th>Size (GB)</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr> | |
| <td><a href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/2.5bpw_H6" target="_blank">2.5_H6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/2.5bpw_H6" target="_blank">2.5</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/2.5bpw_H6" target="_blank">6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/2.5bpw_H6" target="_blank">1.07</a></td> | |
| </tr> | |
| <tr> | |
| <td><a href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/3.0bpw_H6" target="_blank">3.0_H6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/3.0bpw_H6" target="_blank">3.0</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/3.0bpw_H6" target="_blank">6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/3.0bpw_H6" target="_blank">1.16</a></td> | |
| </tr> | |
| <tr> | |
| <td><a href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/3.5bpw_H6" target="_blank">3.5_H6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/3.5bpw_H6" target="_blank">3.5</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/3.5bpw_H6" target="_blank">6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/3.5bpw_H6" target="_blank">1.24</a></td> | |
| </tr> | |
| <tr> | |
| <td><a href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/4.0bpw_H6" target="_blank">4.0_H6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/4.0bpw_H6" target="_blank">4.0</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/4.0bpw_H6" target="_blank">6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/4.0bpw_H6" target="_blank">1.32</a></td> | |
| </tr> | |
| <tr> | |
| <td><a href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/4.5bpw_H6" target="_blank">4.5_H6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/4.5bpw_H6" target="_blank">4.5</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/4.5bpw_H6" target="_blank">6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/4.5bpw_H6" target="_blank">1.40</a></td> | |
| </tr> | |
| <tr> | |
| <td><a href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/5.0bpw_H6" target="_blank">5.0_H6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/5.0bpw_H6" target="_blank">5.0</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/5.0bpw_H6" target="_blank">6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/5.0bpw_H6" target="_blank">1.48</a></td> | |
| </tr> | |
| <tr> | |
| <td><a href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/6.0bpw_H6" target="_blank">6.0_H6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/6.0bpw_H6" target="_blank">6.0</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/6.0bpw_H6" target="_blank">6</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/6.0bpw_H6" target="_blank">1.65</a></td> | |
| </tr> | |
| <tr> | |
| <td><a href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/8.0bpw_H8" target="_blank">8.0_H8</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/8.0bpw_H8" target="_blank">8.0</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/8.0bpw_H8" target="_blank">8</a></td> | |
| <td><a class="fake-link" href="https://huggingface.co/ArtusDev/katanemo_Arch-Router-1.5B-EXL3/tree/8.0bpw_H8" target="_blank">2.03</a></td> | |
| </tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| <div class="card-dark"> | |
| <h2>How to Download and Use Quants</h2> | |
| <p>You can download quants by targeting specific size using the Hugging Face CLI.</p> | |
| <details> | |
| <summary>Click for download commands</summary> | |
| <div class="details-content"> | |
| <b>1. Install huggingface-cli:</b> | |
| <pre><code>pip install -U "huggingface_hub[cli]"</code></pre> | |
| <b>2. Download a specific quant:</b> | |
| <pre><code>huggingface-cli download ArtusDev/katanemo_Arch-Router-1.5B-EXL3 --revision "5.0bpw_H6" --local-dir ./</code></pre> | |
| </div> | |
| </details> | |
| <p>EXL3 quants can be run with any inference client that supports EXL3, such as <a href="https://github.com/theroyallab/tabbyapi" target="_blank"><b>TabbyAPI</b></a>. Refer to <a href="https://github.com/theroyallab/tabbyAPI/wiki/01.-Getting-Started" target="_blank">documentation</a> for set up instructions.</p> | |
| </div> | |
| <div class="card-dark"> | |
| <h2>Quant Requests</h2> | |
| <div style="text-align: center; margin-top: 25px;"> | |
| <a href="https://huggingface.co/ArtusDev/requests-exl/discussions/new?title=[MODEL_NAME_HERE]&description=[MODEL_HF_LINK_HERE]" class="btn-purple" target="_blank">Request EXL3 Quants</a> | |
| </div> | |
| <p class="subtitle"> | |
| See <a href="https://huggingface.co/ArtusDev/requests-exl" target="_blank">EXL community hub</a> for request guidelines. | |
| </p> | |
| </div> | |
| </div> | |