Update README.md

README.md

@@ -1,3 +1,126 @@
----
-license: apache-2.0
----
+
+# A Prompt-based Knowledge Graph Foundation Model for Universal In-Context Reasoning
+<div align="center">
+  <img src="Logo.jpg" width="200">
+</div>
+
+> Extensive knowledge graphs (KGs) have been constructed to facilitate knowledge-driven tasks across various scenarios. However, existing work usually develops separate reasoning models for different KGs, lacking the ability to generalize and transfer knowledge across diverse KGs and reasoning settings. In this paper, we propose a prompt-based KG foundation model via in-context learning, namely **KG-ICL**, to achieve a universal reasoning ability. Specifically, we introduce a prompt graph centered with a query-related example fact as context to understand the query relation. To encode prompt graphs with the generalization ability to unseen entities and relations in queries, we first propose a unified tokenizer that maps entities and relations in prompt graphs to predefined tokens. Then, we propose two message passing neural networks to perform prompt encoding and KG reasoning, respectively. We conduct evaluation on 43 different KGs in both transductive and inductive settings. Results indicate that the proposed KG-ICL outperforms baselines on most datasets, showcasing its outstanding generalization and universal reasoning capabilities.
+
+![image](overview.png)
+
+# Updates
+
+- **2024-10-24**: CPU version: Now, KG-ICL can be run on CPU devices without the need for a GPU. 
+- **2024-10-14**: We have released the code and datasets for KG-ICL.
+- **2024-09-26**: Our paper has been accepted by NeurIPS 2024! 
+
+# Instructions
+
+A quick instruction is given for readers to reproduce the whole process.
+Please use python 3.9 to run the code.
+
+
+## Install dependencies
+
+    pip install torch==2.2.0 --index-url https://download.pytorch.org/whl/cu118
+    pip install torch-scatter==2.1.2 torch-sparse==0.6.18 torch-geometric==2.4.0 -f https://data.pyg.org/whl/torch-2.2.0+cu118.html
+    pip install ninja easydict pyyaml tqdm
+
+If the numpy version is not compatible, please install the following version:
+    
+    pip uninstall numpy
+    pip install numpy==1.24.0
+
+We use the ``rspmm`` kernel. Please make sure your ``CUDA_HOME`` variable is set properly to avoid potential compilation errors, eg
+    
+    export CUDA_HOME=/usr/local/cuda-11.8/
+
+If you do not have GPUs, or the ``rspmm`` kernel is not compiled successfully, please set the hyperparameter ``use_rspmm`` to ``False``.
+
+
+## Dataset
+
+Release datasets
+
+    unzip datasets.zip
+
+Process datasets
+
+    cd datasets
+    chmod +x process.sh
+    ./process.sh
+
+## Quick Start
+
+> If you have any difficulty or question in running code and reproducing experimental results, please email to yncui.nju@gmail.com.
+
+For pre-training
+    
+    cd src
+    python pretrain.py
+
+The checkpoints will be stored in the ``./chechpoint/pretrain/`` fold.
+
+For evaluation, please replace the checkpoint path and test dataset path in the following shell script:
+    
+    cd shell
+    chmod +x test.sh
+    ./test.sh
+
+If you want to inference for a specific dataset, please replace the checkpoint path and evaluation dataset path:
+    
+    cd src
+    python evaluation.py --checkpoint_path ./checkpoint/pretrain/kg_icl_6l --test_dataset_list [dataset_name]
+
+For fine-tuning, please replace the checkpoint path and fine-tune dataset path in the following shell script:
+
+    cd shell
+    chmod +x finetune.sh
+    ./finetune.sh
+
+
+
+## Results
+We have released three versions of the model, including KG-ICL-4L, KG-ICL-5L, KG-ICL-6L. 
+They are training with 4, 5, 6 layers of KG encoder, respectively.
+If your device has limited memory, you can choose the KG-ICL-4L model.
+If you have enough memory, you can choose the KG-ICL-6L model.
+Here are the results of the three models:
+
+### MRR:
+
+| Model | Inductive | Fully-Inductive | Transductive | Average |
+| --- | --- | --- | --- | --- |
+| Supervised SOTA | 0.466 | 0.210 | 0.365 | 0.351 |
+| ULTRA (pretrain) | 0.513 | 0.352 | 0.329 | 0.396 |
+| ULTRA (finetune) | 0.528 | 0.350 | 0.384 | 0.421 |
+| KG-ICL-4L (pretrain) | 0.550 | 0.434 | 0.328 | 0.433 |
+| KG-ICL-5L (pretrain) | 0.554 | 0.438 | 0.346 | 0.441 |
+| KG-ICL-6L (pretrain) | 0.550 | 0.442 | 0.350 | 0.443 |
+| KG-ICL-6L (finetune) | 0.592 | 0.444 | 0.413 | 0.481 |
+
+### Hits@10:
+
+| Model | Inductive | Fully-Inductive | Transductive | Average |
+| --- | --- | --- | --- | --- |
+| Supervised SOTA | 0.607 | 0.347 | 0.511 | 0.493 |
+| ULTRA (pretrain) | 0.664 | 0.536 | 0.479 | 0.557 |
+| ULTRA (finetune) | 0.684 | 0.542 | 0.548 | 0.590 |
+| KG-ICL-4L (pretrain) | 0.696 | 0.622 | 0.471 | 0.590 |
+| KG-ICL-5L (pretrain) | 0.705 | 0.635 | 0.501 | 0.608 |
+| KG-ICL-6L (pretrain) | 0.706 | 0.642 | 0.504 | 0.611 |
+| KG-ICL-6L (finetune) | 0.738 | 0.640 | 0.566 | 0.644 |
+
+
+## Citation
+If you find the repository helpful, please cite the following [paper](http://arxiv.org/abs/2410.12288)
+```bibtex
+@inproceedings{cui2024prompt,
+  title = { A Prompt-based Knowledge Graph Foundation Model for Universal In-Context Reasoning },
+  author = { Cui, Yuanning and 
+            Sun, Zequn and 
+            Hu, Wei },
+  booktitle = { NeurIPS },
+  year = { 2024 }
+}
+