Hugging Face's logo

MengqiLei
/
hyper-align

Text Generation
Transformers
hyperlm_qwen3
hypergraph
large-language-models
qwen3
projector
hyper-align
hypergraph-as-language
Model card Files
xet
 Community
MengqiLei commited on
Commit
bec5839
·
verified ·
1 Parent(s): a42150c

Add Hyper-Align projector checkpoint

Browse files
Files changed (3) hide show
  1. README.md +61 -0
  2. config.json +85 -0
  3. mm_projector.bin +3 -0
README.md CHANGED
@@ -1,3 +1,64 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: transformers
4
+ base_model: Qwen/Qwen3-8B
5
+ tags:
6
+ - hypergraph
7
+ - large-language-models
8
+ - qwen3
9
+ - projector
10
+ - hyper-align
11
+ - hypergraph-as-language
12
+ - arxiv:2605.21858
13
+ inference: false
14
  ---
15
+
16
+ # Hyper-Align
17
+
18
+ This repository contains the released **Hyper-Align** projector checkpoint for the paper **Hypergraph as Language**.
19
+
20
+ Hyper-Align is a hypergraph-native alignment framework that makes high-order association structures directly consumable by a frozen large language model. This checkpoint uses the **HIDT-O** hypergraph serialization protocol and the **HIP** projector to map hypergraph incidence information into the token space of **Qwen/Qwen3-8B**.
21
+
22
+ > This repository does not include the base LLM or text encoder weights. Users must download those models from their official Hugging Face repositories.
23
+
24
+ ## Files
25
+
26
+ ```text
27
+ config.json
28
+ mm_projector.bin
29
+ ```
30
+
31
+ - `config.json` defines the Hyper-Align wrapper and HIP projector configuration.
32
+ - `mm_projector.bin` contains the trained projector weights.
33
+
34
+ Keep both files in the same checkpoint directory. The evaluation code uses `config.json` to instantiate the Hyper-Align model wrapper and projector before loading `mm_projector.bin`.
35
+
36
+ ## License
37
+
38
+ The Hyper-Align code and released projector checkpoint are distributed under the Apache License 2.0.
39
+
40
+ The base LLM and embedding model are not redistributed here. Users must comply with the licenses and terms of the corresponding upstream model repositories:
41
+
42
+ - `Qwen/Qwen3-8B`
43
+ - `Qwen/Qwen3-Embedding-0.6B`
44
+
45
+ ## Citation
46
+
47
+ If you use this checkpoint, please cite:
48
+
49
+ ```bibtex
50
+ @misc{lei2026hypergraphlanguage,
51
+ title={Hypergraph as Language},
52
+ author={Mengqi Lei and Guohuan Xie and Shihui Ying and Shaoyi Du and Jun-Hai Yong and Siqi Li and Yue Gao},
53
+ year={2026},
54
+ eprint={2605.21858},
55
+ archivePrefix={arXiv},
56
+ primaryClass={cs.CL},
57
+ doi={10.48550/arXiv.2605.21858}
58
+ }
59
+ ```
60
+
61
+ ## Links
62
+
63
+ - Paper: https://arxiv.org/abs/2605.21858
64
+ - Code: https://github.com/Mengqi-Lei/Hypergraph-as-Language
config.json ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 151643,
8
+ "consistency_start_step": 0,
9
+ "consistency_warmup_steps": 0,
10
+ "eos_token_id": 151645,
11
+ "freeze_mm_mlp_adapter": false,
12
+ "head_dim": 128,
13
+ "hidden_act": "silu",
14
+ "hidden_size": 4096,
15
+ "htp_num_layers": 1,
16
+ "htp_semantic_core_dim": 384,
17
+ "htp_structure_sidecar_dim": 64,
18
+ "hypergraph_semantic_dim": 1024,
19
+ "hypergraph_structure_dim": 24,
20
+ "initializer_range": 0.02,
21
+ "intermediate_size": 12288,
22
+ "lambda_ord": 0.01,
23
+ "lambda_rel": 0.01,
24
+ "layer_types": [
25
+ "full_attention",
26
+ "full_attention",
27
+ "full_attention",
28
+ "full_attention",
29
+ "full_attention",
30
+ "full_attention",
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention",
36
+ "full_attention",
37
+ "full_attention",
38
+ "full_attention",
39
+ "full_attention",
40
+ "full_attention",
41
+ "full_attention",
42
+ "full_attention",
43
+ "full_attention",
44
+ "full_attention",
45
+ "full_attention",
46
+ "full_attention",
47
+ "full_attention",
48
+ "full_attention",
49
+ "full_attention",
50
+ "full_attention",
51
+ "full_attention",
52
+ "full_attention",
53
+ "full_attention",
54
+ "full_attention",
55
+ "full_attention",
56
+ "full_attention",
57
+ "full_attention",
58
+ "full_attention",
59
+ "full_attention",
60
+ "full_attention"
61
+ ],
62
+ "max_position_embeddings": 40960,
63
+ "max_window_layers": 36,
64
+ "mm_hidden_size": 1048,
65
+ "mm_projector_type": "htp",
66
+ "mm_use_graph_special_token": false,
67
+ "mm_use_graph_start_end": false,
68
+ "model_type": "hyperlm_qwen3",
69
+ "num_attention_heads": 32,
70
+ "num_hidden_layers": 36,
71
+ "num_key_value_heads": 8,
72
+ "projector_incidence_mode": "sample_real",
73
+ "rms_norm_eps": 1e-06,
74
+ "rope_scaling": null,
75
+ "rope_theta": 1000000,
76
+ "sliding_window": null,
77
+ "tie_word_embeddings": false,
78
+ "torch_dtype": "float32",
79
+ "transformers_version": "4.53.0",
80
+ "tune_mm_mlp_adapter": true,
81
+ "use_cache": true,
82
+ "use_mm_proj": true,
83
+ "use_sliding_window": false,
84
+ "vocab_size": 151936
85
+ }
mm_projector.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f63c2682d95f77abdae117e63974032b1415c58dda8cbdf3e9f8ef7188670a12
3
+ size 10937990