Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
| # HTML title for the application | |
| TITLE = """<h1 align="center" id="space-title">FAIR Chemistry Leaderboard</h1>""" | |
| # Main introduction text | |
| INTRODUCTION_TEXT = """ | |
| # Welcome to the FAIR Chemistry Leaderboard! π§ͺ | |
| This space hosts comprehensive leaderboards across different chemical domains including molecules, catalysts, and materials. | |
| Select a dataset below to view the corresponding leaderboard. | |
| """ | |
| # OMol25 introduction text | |
| OMOL_INTRODUCTION_TEXT = """ | |
| ## 𧬠OMol25 | |
| This leaderboard evaluates performance on the **Open Molecules 2025 (OMol25)** datasetβa diverse, high-quality collection that uniquely combines elemental, chemical, and structural diversity. | |
| π **Learn more:** [OMol25 Paper](https://arxiv.org/pdf/2505.08762) | |
| ### Benchmarks | |
| **S2EF (Structure to Energy and Forces)** | |
| - Test and validation sets across different molecular categories | |
| **Evaluations.** Downstream chemistry tasks that evaluate practical applications: | |
| - **Ligand Pocket:** Protein-ligand interaction energy as a proxy for binding energy | |
| - **Ligand Strain:** Ligand-strain energy crucial for understanding protein-ligand binding | |
| - **Conformers:** Identifying the lowest energy conformer | |
| - **Protonation:** Energy differences between protonated structures (proxy for pKa prediction) | |
| - **Distance Scaling:** Short and long range intermolecular interactions | |
| - **IE/EA:** Ionization energy and electron affinity | |
| - **Spin Gap:** Energy differences between varying spin states | |
| ## π Getting Started | |
| Ready to submit your model? Check out our steps for running benchmarks and generating prediction files: | |
| π **[Submission Documentation](https://fair-chem.github.io/omol/)** | |
| """ | |
| # OC20 introduction text | |
| OC20_INTRODUCTION_TEXT = """ | |
| ## βοΈ OC20 | |
| This leaderboard evaluates performance on the **Open Catalyst 2020 (OC20)** datasetβa large-scale dataset for catalyst discovery containing DFT relaxations across a wide variety of adsorbate-catalyst combinations. | |
| π **Learn more:** [OC20 Paper](https://arxiv.org/abs/2010.09990) | |
| ### Benchmarks | |
| **S2EF (Structure to Energy and Forces)** | |
| - Predict energy and per-atom forces given an atomic structure | |
| **IS2RE (Initial Structure to Relaxed Energy)** | |
| - Predict the relaxed energy given the initial structure | |
| Both tasks are evaluated across four test splits: | |
| - **ID:** In-domain test set | |
| - **OOD-Ads:** Out-of-domain adsorbates | |
| - **OOD-Cat:** Out-of-domain catalysts | |
| - **OOD-Both:** Out-of-domain adsorbates and catalysts | |
| π **[Submission Documentation](https://fair-chem.github.io/oc20-1/)** | |
| """ | |
| # Submission instructions | |
| SUBMISSION_TEXT = """ | |
| ## How to Submit | |
| ### To submit your model predictions: | |
| - **π Step 1:** Generate prediction files for the appropriate task (see [here](https://fair-chem.github.io/summary-5/) for details) | |
| - **π Step 2:** Sign in with Hugging Face | |
| - **π Step 3:** Fill in the submission metadata (name, organization, contact info, etc.) | |
| - **π― Step 4:** Select the evaluation type that matches your prediction file | |
| - **π€ Step 5:** Upload your file and click Submit Eval | |
| - **β±οΈ Step 6:** Wait for the evaluation to complete and see the "β " message in the Status bar | |
| **π Submission Limits:** Users are limited to **5 successful submissions per month** for each evaluation type. | |
| ### β οΈ Important Notes: | |
| - β **File Format:** Ensure your prediction file format matches the expected format for the selected evaluation (.npz for S2EF and .json for Evaluations) | |
| - π **Privacy:** Your email will be stored privately and only used for communication regarding your submission | |
| - π **Results:** Results will appear on the leaderboard after successful validation | |
| - β±οΈ **Wait Time:** Remain on the page until you see the "Success" message. Evaluations can take several minutes, please be patient | |
| - ποΈ **Removal:** If you wish to have your model removed from the leaderboard, please reach out to mshuaibi@meta.com with the model name and submission date | |
| ### π¬ Need Help? | |
| This leaderboard is actively being developed and we welcome any feedback and contributions! | |
| **π Contact us:** | |
| - π [GitHub Issues](https://github.com/facebookresearch/fairchem) | |
| - π¬ [Discussion Forum](https://huggingface.co/spaces/facebook/fairchem_leaderboard/discussions) | |
| """ | |
| # Citation information | |
| CITATION_BUTTON_LABEL = "Copy the following snippet to cite these results" | |
| OMOL_CITATION_TEXT = r""" | |
| ```latex | |
| @article{levine2025open, | |
| title={The open molecules 2025 (omol25) dataset, evaluations, and models}, | |
| author={Levine, Daniel S and Shuaibi, Muhammed and Spotte-Smith, Evan Walter Clark and Taylor, Michael G and Hasyim, Muhammad R and Michel, Kyle and Batatia, Ilyes and Cs{'a}nyi, G{'a}bor and Dzamba, Misko and Eastman, Peter and others}, | |
| journal={arXiv preprint arXiv:2505.08762}, | |
| year={2025} | |
| } | |
| ``` | |
| """ | |
| OC20_CITATION_TEXT = r""" | |
| ```latex | |
| @article{chanussot2021open, | |
| title={Open catalyst 2020 (OC20) dataset and community challenges}, | |
| author={Chanussot, Lowik and Das, Abhishek and Gober, Siddharth and Lavril, Thibaut and Shuaibi, Muhammed and Riviere, Morgane and Tran, Kevin and Devereaux, Javier and Zitnick, C Lawrence}, | |
| journal={ACS Catalysis}, | |
| volume={11}, | |
| number={10}, | |
| pages={6059--6072}, | |
| year={2021}, | |
| publisher={ACS Publications} | |
| } | |
| ``` | |
| """ | |
| # Table configuration | |
| PRE_COLUMN_NAMES = ["Model", "Organization", "Energy Conserving", "Training Set"] | |
| OC20_PRE_COLUMN_NAMES = ["Model", "Organization", "Energy Conserving", "Total Energy Model", "Training Set"] | |
| POST_COLUMN_NAMES = ["Submission date"] | |
| OMOL_TYPES = ["markdown", "str", "bool", "str"] | |
| OC20_TYPES = ["markdown", "str", "bool", "bool", "str"] | |
| def model_hyperlink(model_link: str, paper_link: str, model_name: str) -> str: | |
| """Create a hyperlink for model names in the leaderboard.""" | |
| # Check if we have valid links | |
| has_model_link = model_link and model_link.strip() != "" | |
| has_paper_link = paper_link and paper_link.strip() != "" | |
| if not has_model_link and not has_paper_link: | |
| return model_name | |
| if has_model_link and not has_paper_link: | |
| return f'<a target="_blank" href="{model_link}" style="color: var(--link-text-color); text-decoration: underline;text-decoration-style: dotted;" title="Download model">{model_name}</a>' | |
| if not has_model_link and has_paper_link: | |
| return f'{model_name} <a target="_blank" href="{paper_link}" style="color: var(--link-text-color); text-decoration: underline;text-decoration-style: dotted;" title="Read paper">π</a>' | |
| return f'<a target="_blank" href="{model_link}" style="color: var(--link-text-color); text-decoration: underline;text-decoration-style: dotted;" title="Download model">{model_name}</a> <a target="_blank" href="{paper_link}" style="color: var(--link-text-color); text-decoration: underline;text-decoration-style: dotted;" title="Read paper">π</a>' | |