caiovicentino1/Qwen2.5-3B-EOQ-Dynamic-compressed

Qwen3.5-9B EOQ Dynamic (Mixed-Bit Compressed)

EOQ Dynamic compressed version of Qwen/Qwen3.5-9B.

Per-tensor mixed-bit quantization inspired by Unsloth Dynamic 2.0: spends more bits on sensitive tensors (attention o_proj Q6) and fewer on robust ones (MLP gate/up Q3).

Verified Benchmark (NVIDIA H100 80GB)

Method	Avg Bits	PPL (WikiText-2)	Delta	tok/s
FP16 (baseline)	16	6.37	---	29.4
Uniform Q4	4.0	8.80	+2.42	30.2
EOQ Dynamic	3.7	7.26	+0.89	29.7
Uniform Q5	5.0	7.09	+0.71	29.9

Dynamic achieves Q5-level quality at 3.7-bit average -- 1.54 PPL better than Uniform Q4.

Bit Allocation

Tensor Type	Bits	Params	Share
MLP gate/up	Q3	3,221M	36%
MLP down	Q4	1,611M	18%
Attn Q/K/V + embed	Q5	2,567M	29%
Attn O	Q6	1,554M	17%
Norms/biases	FP16	0.3M	0%

Usage

Version: ImageMagick 7.1.2-13 Q16-HDRI aarch64 23522 https://imagemagick.org Copyright: (C) 1999 ImageMagick Studio LLC License: https://imagemagick.org/license/ Features: Cipher DPC HDRI Modules Delegates (built-in): bzlib heic jng jpeg lcms ltdl lzma png tiff webp xml zlib zstd Compiler: clang (17.0.0) Usage: import [options ...] [ file ]

Image Settings: -adjoin join images into a single multi-image file -border include window border in the output image -channel type apply option to select image channels -colorspace type alternate image colorspace -comment string annotate image with comment -compress type type of pixel compression when writing the image -define format:option define one or more image format options -density geometry horizontal and vertical density of the image -depth value image depth -descend obtain image by descending window hierarchy -display server X server to contact -dispose method layer disposal method -dither method apply error diffusion to image -delay value display the next image after pausing -encipher filename convert plain pixels to cipher pixels -endian type endianness (MSB or LSB) of the image -encoding type text encoding type -filter type use this filter when resizing an image -format "string" output formatted image characteristics -frame include window manager frame -gravity direction which direction to gravitate towards -identify identify the format and characteristics of the image -interlace type None, Line, Plane, or Partition -interpolate method pixel color interpolation method -label string assign a label to an image -limit type value Area, Disk, Map, or Memory resource limit -monitor monitor progress -page geometry size and location of an image canvas -pause seconds seconds delay between snapshots -pointsize value font point size -quality value JPEG/MIFF/PNG compression level -quiet suppress all warning messages -regard-warnings pay attention to warning messages -repage geometry size and location of an image canvas -respect-parentheses settings remain in effect until parenthesis boundary -sampling-factor geometry horizontal and vertical sampling factor -scene value image scene number -screen select image from root window -seed value seed a new sequence of pseudo-random numbers -set property value set an image property -silent operate silently, i.e. don't ring any bells -snaps value number of screen snapshots -support factor resize support: > 1.0 is blurry, < 1.0 is sharp -synchronize synchronize image to storage device -taint declare the image as modified -transparent-color color transparent color -treedepth value color tree depth -verbose print detailed information about the image -virtual-pixel method Constant, Edge, Mirror, or Tile -window id select window with this id or name root selects whole screen

Image Operators: -annotate geometry text annotate the image with text -colors value preferred number of colors in the image -crop geometry preferred size and location of the cropped image -encipher filename convert plain pixels to cipher pixels -extent geometry set the image size -geometry geometry preferred size or location of the image -help print program options -monochrome transform image to black and white -negate replace every pixel with its complementary color -quantize colorspace reduce colors in this colorspace -resize geometry resize the image -rotate degrees apply Paeth rotation to the image -strip strip image of all profiles and comments -thumbnail geometry create a thumbnail of the image -transparent color make this color transparent within the image -trim trim image edges -type type image type

Miscellaneous Options: -debug events display copious debugging information -help print program options -list type print a list of supported option arguments -log format format of debugging information -version print version information

By default, 'file' is written in the MIFF image format. To specify a particular image format, precede the filename with an image format name and a colon (i.e. ps:image) or specify the image type as the filename suffix (i.e. image.ps). Specify 'file' as '-' for standard input or output.

Links

• GitHub: https://github.com/caiovicentino/eoq-quantization
• Base model: Qwen/Qwen3.5-9B
• Uniform Q5 version: caiovicentino1/Qwen3.5-9B-EOQ-Q5-compressed