Initial CASCADE HF Space deployment

.gitattributes

@@ -0,0 +1,3 @@
+*.hdf5 filter=lfs diff=lfs merge=lfs -text
+*.p filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,8 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.env
+.DS_Store
+*.sqlite3
+results/
+*.tar.gz

DEPLOYMENT_GUIDE.md

@@ -0,0 +1,207 @@
+# CASCADE → Hugging Face Spaces: Deployment Guide
+
+## Overview
+
+The HF Space replaces Django with Flask but keeps the exact same Redis job-queue
+pattern and frontend JS. The app is mounted at `/cascade_v1/` so the live URL is:
+
+```
+https://patonlab-cascade.hf.space/cascade_v1/predict/
+```
+
+The Space is named `cascade` on HF — the `_v1` is a subpath in Flask, not the repo name.
+This means you can add `cascade_v2` later under the same Space without creating a new repo.
+
+```
+hf_space/
+├── Dockerfile               # Python 3.7, Redis, TF1.15, Keras 2.2.5
+├── requirements.txt
+├── start.sh                 # Redis → worker → gunicorn
+├── app.py                   # Flask Blueprint mounted at /cascade_v1
+├── worker.py                # Redis consumer, runs GNN inference
+├── setup_assets.sh          # One-time script: copies binaries from nova app
+├── README.md                # HF Space card (YAML front-matter required)
+├── .gitattributes           # Git LFS tracking for .hdf5 and .p files
+├── .gitignore
+├── NMR_Prediction/
+│   ├── __init__.py
+│   ├── apply.py             # ← copied + path-fixed by setup_assets.sh
+│   ├── genConf.py           # ← copied by setup_assets.sh
+│   ├── valid.py             # new (simple RDKit SMILES check)
+│   ├── preprocessor.p       # ← copied by setup_assets.sh
+│   └── schnet_edgeupdate/
+│       └── best_model.hdf5  # ← copied by setup_assets.sh (31 MB)
+├── nfp/                     # ← copied by setup_assets.sh
+├── static/                  # ← copied by setup_assets.sh
+│   ├── main.css
+│   ├── sketch.css
+│   ├── JSmol.min.js
+│   ├── images/
+│   ├── j2s/
+│   ├── jquery/
+│   ├── js/
+│   └── jsme/
+└── templates/
+    └── cascade/
+        ├── base.html
+        ├── home.html
+        ├── predict.html
+        ├── results.html
+        └── about.html
+```
+
+---
+
+## Step 1 — Copy assets from the nova app
+
+Run the setup script (copies model weights, inference code, static files, and
+applies the preprocessor path fix automatically):
+
+```bash
+cd ~/Documents/huggingface/cascade/hf_space
+bash setup_assets.sh
+```
+
+You should see a ✓ for each of the 5 asset groups. Afterwards confirm:
+
+```bash
+ls NMR_Prediction/   # apply.py  genConf.py  preprocessor.p  schnet_edgeupdate/  valid.py  __init__.py
+ls nfp/              # __init__.py  layers/  models/  preprocessing/
+ls static/           # main.css  JSmol.min.js  j2s/  jsme/  jquery/  js/  images/
+```
+
+---
+
+## Step 2 — Test locally with Docker
+
+```bash
+cd ~/Documents/huggingface/cascade/hf_space
+
+# Build (takes ~5 min first time — TF1.15 + RDKit are large)
+docker build -t cascade-local .
+
+# Run
+docker run -p 7860:7860 cascade-local
+
+# Open in browser
+open http://localhost:7860/cascade_v1/predict/
+```
+
+**What to check:**
+- Page loads, JSME editor opens
+- Submitting `CC(=O)O` (acetic acid) returns a task_id (check browser Network tab)
+- `check_task` eventually returns the results HTML with 2D SVG and shift table
+- Worker logs appear in the Docker terminal: `Processing task ...`
+
+**Common first-run issues:**
+
+| Symptom | Fix |
+|---|---|
+| `ModuleNotFoundError: nfp` | Check `nfp/` is in the root of hf_space, not nested inside another folder |
+| `OSError: preprocessor.p not found` | Re-run `setup_assets.sh` — the path fix may not have applied |
+| `keras load_model` custom layer error | Check custom_objects in worker.py match the nfp layer class names exactly |
+| Redis connection refused | Redis takes ~1 s to start; the worker retries automatically — wait a moment |
+| 500 on `/cascade_v1/predict/` | Check `templates/cascade/predict.html` exists |
+| JSME editor blank | Check `static/jsme/jsme.nocache.js` was copied across |
+
+---
+
+## Step 3 — Create the Hugging Face Space
+
+```bash
+# Install HF CLI if needed
+pip install huggingface_hub
+
+# Login (needs a write token from hf.co/settings/tokens)
+huggingface-cli login
+
+# Create the Space — named 'cascade', Docker SDK, public, under patonlab org
+huggingface-cli repo create cascade \
+    --type space \
+    --space-sdk docker \
+    --organization patonlab
+
+# This creates: https://huggingface.co/spaces/patonlab/cascade
+# Live URL will be: https://patonlab-cascade.hf.space/cascade_v1/predict/
+```
+
+---
+
+## Step 4 — Push to Hugging Face
+
+```bash
+cd ~/Documents/huggingface/cascade/hf_space
+
+git init
+git lfs install          # Required — .hdf5 and .p are tracked via Git LFS
+
+# .gitattributes is already written; just add it
+git add .gitattributes
+git remote add origin https://huggingface.co/spaces/patonlab/cascade
+
+git add .
+git commit -m "Initial CASCADE HF Space deployment"
+git push origin main
+```
+
+The Space builds automatically. Build logs at:
+`https://huggingface.co/spaces/patonlab/cascade` → Logs tab
+
+Build time is typically **8–12 minutes** (TF1.15 + RDKit are large).
+
+---
+
+## Step 5 — Verify the live Space
+
+Once the build is green:
+
+1. `https://patonlab-cascade.hf.space/cascade_v1/predict/` — page loads
+2. Submit `CC(=O)O` (acetic acid) — result in ~10–15 s
+3. Submit `CC1=CC(=CC(=C1)O)C` — result in ~30–45 s
+4. Test the JSME drawing tool → OK → predicted shifts labelled on 2D structure
+
+---
+
+## Step 6 — Update the GitHub README
+
+Update `patonlab/CASCADE` on GitHub to point to the new URL:
+
+```markdown
+The production web server can be found here:
+https://patonlab-cascade.hf.space/cascade_v1/predict/
+```
+
+---
+
+## Ongoing maintenance
+
+**Pushing updates:**
+```bash
+cd ~/Documents/huggingface/cascade/hf_space
+git add . && git commit -m "your message" && git push origin main
+```
+HF rebuilds automatically on each push.
+
+**Checking logs:** HF Space → Logs tab (worker output streams there in real time)
+
+**Cold starts:** The free HF tier sleeps after inactivity. First request after sleep
+takes ~30 s. Upgrade to HF Pro ($9/mo) for persistent Spaces if needed.
+
+**Redis:** Results have a 1-hour TTL and do not persist across container restarts.
+This is fine for interactive use.
+
+---
+
+## What changed from the original Django app
+
+| Original (nova) | New HF Space |
+|---|---|
+| Django 2.1 | Flask 2.0 with Blueprint at `/cascade_v1` |
+| `uwsgi` | `gunicorn` |
+| `django.template` tags | Plain Jinja2 (`/static/...` paths) |
+| `ALLOWED_HOSTS = ['129.82.62.62']` | Flask `HOST=0.0.0.0` |
+| Hard-coded `SECRET_KEY` | Not needed (no sessions/auth) |
+| `db.sqlite3` | Removed (unused) |
+| `cascade/preprocessor.p` path | `NMR_Prediction/preprocessor.p` |
+| Model loaded in Django views at startup | Loaded once in worker.py at startup |
+| Manual file copies + sed fix | Automated via `setup_assets.sh` |

Dockerfile

@@ -0,0 +1,26 @@
+FROM --platform=linux/amd64 python:3.7-slim
+
+# System dependencies
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    redis-server \
+    libxrender1 \
+    libxext6 \
+    libgl1-mesa-glx \
+    libglib2.0-0 \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+# Install Python dependencies
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy all application code
+COPY . .
+
+# HF Spaces requires port 7860
+EXPOSE 7860
+
+# Startup: Redis + worker + Flask
+CMD ["bash", "start.sh"]

NMR_Prediction/apply.py

@@ -0,0 +1,255 @@
+import sys,io,os
+import pandas as pd
+import numpy as np
+import gzip, pickle, argparse, warnings
+import pickle
+import math
+
+from tqdm import tqdm
+
+from rdkit import Chem
+from rdkit.Chem import AllChem
+from rdkit.Chem import ForwardSDMolSupplier
+
+from itertools import islice
+
+from nfp.preprocessing import MolAPreprocessor, GraphSequence
+from .genConf import genConf
+
+import keras
+import keras.backend as K
+
+from keras.callbacks import ModelCheckpoint, CSVLogger, LearningRateScheduler
+
+from keras.layers import (Input, Embedding, Dense, BatchNormalization,
+                                 Concatenate, Multiply, Add)
+
+from keras.models import Model, load_model
+
+from nfp.layers import (MessageLayer, GRUStep, Squeeze, EdgeNetwork,
+                               ReduceBondToPro, ReduceBondToAtom,
+                               GatherAtomToBond, ReduceAtomToPro)
+from nfp.models import GraphModel
+
+def to_C(atom):
+    neighbors = [x.GetAtomicNum() for x in atom.GetNeighbors()]
+    if 6 in neighbors:
+        return True
+    else:
+        return False
+
+def Mol_iter(df):
+    for index,r in df.iterrows():
+        yield(r['Mol'], r['atom_index'])
+
+def preprocess_C(mols, preprocessor, keep_all_cf=False):
+    mols_id = []
+    confs_id = []
+    mols_conf = []
+    for i,m in enumerate(mols):
+        try:
+            mol,ids,nr = genConf(m, rms=-1, nc=200, efilter=10.0, rmspost=0.5)
+        except Exception as e:
+            msg = "cannot generate FF conformers"
+            print(msg)
+            mols_conf.append(None)
+            continue
+
+        mols_i = [Chem.RWMol(mol) for x in ids]
+        if keep_all_cf:
+            for m_i,id in zip(mols_i, ids):
+                m_i.SetProp('E', str(id[0]))
+                m_i.SetProp('_Name', '{}_{}'.format(i, id[1]))
+                for ia in range(mol.GetNumAtoms()):
+                    m_i.GetConformer().SetAtomPosition(ia, mol.GetConformer(id[1]).GetAtomPosition(ia))
+            mols_conf.extend(mols_i)
+
+        else:
+            id = ids[0]
+            m.SetProp('E', str(id[0]))
+            m.SetProp('_Name', '{}_{}'.format(i, id[1]))
+
+    df = []
+    for m in mols_conf:
+        if m:
+            E = float(m.GetProp('E'))
+            m_id, cf_id = [ int(x) for x in m.GetProp('_Name').split('_') ]
+            Cs = [x for x in m.GetAtoms() if x.GetAtomicNum()==6]
+            C_index = np.array([x.GetIdx() for x in Cs]).astype(int)
+            df.append([m_id, m, m.GetNumAtoms(), C_index, E, cf_id])
+
+    df = pd.DataFrame(df, columns=['mol_id', 'Mol', 'n_atoms', 'atom_index', 'relative_E', 'cf_id'])
+
+    inputs = preprocessor.predict(Mol_iter(df))
+
+    return inputs, df, mols_conf
+
+class RBFSequence(GraphSequence):
+    def process_data(self, batch_data):
+        batch_data['distance_rbf'] = self.rbf_expansion(batch_data['distance'])
+
+        offset = self._compute_stacked_offsets(
+            batch_data['n_pro'], batch_data['n_atom'])
+
+        offset = np.where(batch_data['atom_index']>=0, offset, 0)
+        batch_data['atom_index'] += offset
+
+        del batch_data['n_atom']
+        del batch_data['n_bond']
+        del batch_data['distance']
+        return batch_data
+
+    def _compute_stacked_offsets(self, sizes, repeats):
+        return np.repeat(np.cumsum(np.hstack([0, sizes[:-1]])), repeats)
+
+    def rbf_expansion(self, distances, mu=0, delta=0.1, kmax=256):
+        k = np.arange(0, kmax)
+        logits = -(np.atleast_2d(distances).T - (-mu + delta * k))**2 / delta
+        return np.exp(logits)
+
+def evaluate_C(inputs, preprocessor, model):
+    batch_size = 32
+    evaluate_sequence = RBFSequence(inputs, batch_size=batch_size)
+
+    predicted = []
+
+    for x in evaluate_sequence:
+        out = model.predict_on_batch(x)
+        out = np.concatenate(out)
+        predicted.extend(out)
+
+    return predicted
+
+def predict_NMR_C(smiles, model):
+
+    mols = [Chem.MolFromSmiles(smiles)]
+    for m in mols: AllChem.EmbedMolecule(m, useRandomCoords=True)
+    mols = [Chem.AddHs(m, addCoords=True) for m in mols]
+
+
+    with open(os.path.join('NMR_Prediction', 'preprocessor.p'), 'rb') as ft:
+        preprocessor = pickle.load(ft)['preprocessor']
+
+    inputs, df, mols = preprocess_C(mols, preprocessor, True)
+
+    if len(inputs) == 0:
+        raise RuntimeError('Failed to find any conformer for the given molecule')
+
+    predicted = evaluate_C(inputs, preprocessor, model)
+
+    spread_df = pd.DataFrame([], columns=['mol_id', 'atom_index', 'relative_E', 'cf_id'])
+    for _,r in df.iterrows():
+        mol_id=[r.mol_id]*len(r.atom_index)
+        cf_id = [r.cf_id]*len(r.atom_index)
+        E = [r.relative_E]*len(r.atom_index)
+        df_mol = pd.DataFrame(data={'mol_id': mol_id, 'atom_index': r.atom_index, 'relative_E':E,  'cf_id': cf_id})
+        spread_df = pd.concat([spread_df, df_mol], sort=True)
+    spread_df['predicted'] = predicted
+    spread_df['b_weight'] = spread_df.relative_E.apply(lambda x: math.exp(-x/(0.001987*298.15)))
+
+    df_group = spread_df.set_index(['mol_id', 'atom_index', 'cf_id']).groupby(level=[0,1])
+    final = []
+    for (m_id, a_id),df in df_group:
+        weighted_shift = df.apply(lambda x: x['b_weight']*x['predicted'], axis=1).sum()/df.b_weight.sum()
+        final.append([m_id, a_id, weighted_shift])
+
+    final = pd.DataFrame(final, columns=['mol_id', 'atom_index', 'Shift'])
+    final['atom_index'] = final.atom_index.apply(lambda x: x+1)
+    final = final.round(2).astype(dtype={'atom_index':'int'})
+    spread_df['atom_index'] = spread_df.atom_index.apply(lambda x: x+1)
+    spread_df = spread_df.round(2)
+
+    return(mols, final, spread_df)
+
+
+def preprocess_H(mols, preprocessor, keep_all_cf=False):
+    mols_id = []
+    confs_id = []
+    mols_conf = []
+    for i,m in enumerate(mols):
+        try:
+            mol,ids,nr = genConf(m, rms=-1, nc=200, efilter=10.0, rmspost=0.5)
+        except Exception as e:
+            msg = "cannot generate FF conformers"
+            print(msg)
+            mols_conf.append(None)
+            continue
+
+        mols_i = [Chem.RWMol(mol) for x in ids]
+        if keep_all_cf:
+            for m_i,id in zip(mols_i, ids):
+                m_i.SetProp('E', str(id[0]))
+                m_i.SetProp('_Name', '{}_{}'.format(i, id[1]))
+                for ia in range(mol.GetNumAtoms()):
+                    m_i.GetConformer().SetAtomPosition(ia, mol.GetConformer(id[1]).GetAtomPosition(ia))
+            mols_conf.extend(mols_i)
+
+        else:
+            id = ids[0]
+            m.SetProp('E', str(id[0]))
+            m.SetProp('_Name', '{}_{}'.format(i, id[1]))
+
+    df = []
+    for m in mols_conf:
+        if m:
+            E = float(m.GetProp('E'))
+            m_id, cf_id = [ int(x) for x in m.GetProp('_Name').split('_') ]
+            Hs = [x for x in m.GetAtoms() if x.GetAtomicNum()==1]
+            H_index = np.array([x.GetIdx() for x in Hs]).astype(int)
+            df.append([m_id, m, m.GetNumAtoms(), H_index, E, cf_id])
+
+    df = pd.DataFrame(df, columns=['mol_id', 'Mol', 'n_atoms', 'atom_index', 'relative_E', 'cf_id'])
+
+    inputs = preprocessor.predict(Mol_iter(df))
+
+    return inputs, df, mols_conf
+
+def evaluate_H(inputs, preprocessor, model):
+    batch_size = 32
+    evaluate_sequence = RBFSequence(inputs, batch_size=batch_size)
+
+    predicted = []
+
+    for x in evaluate_sequence:
+        out = model.predict_on_batch(x)
+        out = np.concatenate(out)
+        predicted.extend(out)
+
+    return predicted
+
+def predict_NMR_H(smiles, model):
+
+    mols = [Chem.MolFromSmiles(smiles)]
+    for m in mols: AllChem.EmbedMolecule(m, useRandomCoords=True)
+    mols = [Chem.AddHs(m, addCoords=True) for m in mols]
+
+
+    with open(os.path.join('NMR_Prediction', 'preprocessor.p'), 'rb') as ft:
+        preprocessor = pickle.load(ft)['preprocessor']
+
+    inputs, df, mols = preprocess_H(mols, preprocessor, True)
+    predicted = evaluate_H(inputs, preprocessor, model)
+
+    spread_df = pd.DataFrame([], columns=['mol_id', 'atom_index', 'relative_E', 'cf_id'])
+    for _,r in df.iterrows():
+        mol_id=[r.mol_id]*len(r.atom_index)
+        cf_id = [r.cf_id]*len(r.atom_index)
+        E = [r.relative_E]*len(r.atom_index)
+        df_mol = pd.DataFrame(data={'mol_id': mol_id, 'atom_index': r.atom_index, 'relative_E':E,  'cf_id': cf_id})
+        spread_df = pd.concat([spread_df, df_mol], sort=True)
+    spread_df['predicted'] = predicted
+    spread_df['b_weight'] = spread_df.relative_E.apply(lambda x: math.exp(-x/(0.001987*298.15)))
+
+    df_group = spread_df.set_index(['mol_id', 'atom_index', 'cf_id']).groupby(level=[0,1])
+    final = []
+    for (m_id, a_id),df in df_group:
+        weighted_shift = df.apply(lambda x: x['b_weight']*x['predicted'], axis=1).sum()/df.b_weight.sum()
+        final.append([m_id, a_id, weighted_shift])
+
+    final = pd.DataFrame(final, columns=['mol_id', 'atom_index', 'Shift'])
+    final['atom_index'] = final.atom_index.apply(lambda x: x+1)
+    final = final.round(2).astype(dtype={'atom_index':'int'})
+    spread_df['atom_index'] = spread_df.atom_index.apply(lambda x: x+1)
+    spread_df = spread_df.round(2)
+
+    return(mols, final, spread_df)

NMR_Prediction/genConf.py

@@ -0,0 +1,243 @@
+#!/usr/bin/python
+from __future__ import print_function, absolute_import
+
+#######################################################################
+# Molecular conformer generator in progress
+# genConf.py -isdf file_input.sdf -osdf file_output.sdf
+# -n number_of_conformers (optional, if not specified is based
+# on the nomber of rotable bonds) -rtpre rms_threshold_pre_opt(optional)
+# -rtpost rms_threshold_post_opt(optional) -e energy_window (optional, Kcal/mol)
+# -t number_of_threads (if not specify 1)
+#######################################################################
+
+## known issues / to-do list
+## logging doesn't work properly
+## need to print out rotatable bond atoms for reference
+## comparison of printing vs. old full monte - more options
+## comparison of printing vs. GoodVibes
+## tabulation of results in addition to sdf output file
+## pythonify where possible
+## currently RMS but torsion_list would be better?
+
+from rdkit import Chem
+from rdkit.Chem import AllChem
+from concurrent import futures
+import argparse, logging, os, sys, time, copy
+logger = logging.getLogger(__name__)
+
+# PHYSICAL CONSTANTS
+GAS_CONSTANT, PLANCK_CONSTANT, BOLTZMANN_CONSTANT, SPEED_OF_LIGHT, AVOGADRO_CONSTANT, AMU_to_KG, atmos = 8.3144621, 6.62606957e-34, 1.3806488e-23, 2.99792458e10, 6.0221415e23, 1.66053886E-27, 101.325
+# UNIT CONVERSION
+j_to_au = 4.184 * 627.509541 * 1000.0
+
+# version number
+__version__ = "1.0.1"
+
+# Formatted output to command line and log file
+class Logger:
+    # Designated initializer
+    def __init__(self, filein, ext):
+        # check to see if already exists
+        if os.path.exists(filein+"."+ext):
+            var = input("\no  Logger file %s already exists! OK to proceed? (Y/N) " % (filein+"."+ext))
+            if var.lower().strip() == "n" or var.lower().strip() == "no":
+                logger.error("\n   OK. Exiting gracefully ...\n"); sys.exit(1)
+        # Create the log file at the input path
+        self.log = open(filein+"."+ext, 'w' )
+
+    # Write a message to the log
+    def Write(self, message):
+        # Print the message
+        logger.info(message)
+        # Write to log
+        self.log.write(message + "\n")
+
+    # Write a message only to the log and not to the terminal
+    def Writeonlyfile(self, message):
+        # Write to log
+        self.log.write(message)
+
+    # Write a fatal error, finalize and terminate the program
+    def Fatal(self, message):
+        # Print the message
+        logger.error(message+"\n")
+        # Write to log
+        self.log.write(message + "\n")
+        # Finalize the log
+        self.Finalize()
+        # End the program
+        sys.exit(1)
+
+    # Finalize the log file
+    def Finalize(self):
+        self.log.close()
+
+dashedline = "   ------------------------------------------------------------------------------------------------------------------"
+emptyline = "   |                                                                                                                     |"
+normaltermination = "\n   -----------------       N   O   R   M   A   L      T   E   R   M   I   N   A   T   I   O   N      ----------------\n"
+leftcol=97
+rightcol=12
+
+asciiArt = "     ___       ___                                    ___          ___          ___                   ___ \n    /  /\\     /__/\\                                  /__/\\        /  /\\        /__/\\         ___     /  /\\\n   /  /:/_    \\  \\:\\                                |  |::\\      /  /::\\       \\  \\:\\       /  /\\   /  /:/_\n  /  /:/ /\\    \\  \\:\\   ___     ___  ___     ___    |  |:|:\\    /  /:/\\:\\       \\  \\:\\     /  /:/  /  /:/ /\\\n /  /:/ /:/___  \\  \\:\\ /__/\\   /  /\\/__/\\   /  /\\ __|__|:|\\:\\  /  /:/  \\:\\  _____\\__\\:\\   /  /:/  /  /:/ /:/_\n/__/:/ /://__/\\  \\__\\:\\\\  \\:\\ /  /:/\\  \\:\\ /  /://__/::::| \\:\\/__/:/ \\__\\:\\/__/::::::::\\ /  /::\\ /__/:/ /:/ /\\\n\\  \\:\\/:/ \\  \\:\\ /  /:/ \\  \\:\\  /:/  \\  \\:\\  /:/ \\  \\:\\~~\\__\\/\\  \\:\\ /  /:/\\  \\:\\~~\\~~\\//__/:/\\:\\\\  \\:\\/:/ /:/\n \\  \\::/   \\  \\:\\  /:/   \\  \\:\\/:/    \\  \\:\\/:/   \\  \\:\\       \\  \\:\\  /:/  \\  \\:\\  ~~~ \\__\\/  \\:\\\\  \\::/ /:/\n  \\  \\:\\    \\  \\:\\/:/     \\  \\::/      \\  \\::/     \\  \\:\\       \\  \\:\\/:/    \\  \\:\\          \\  \\:\\\\  \\:\\/:/\n   \\  \\:\\    \\  \\::/       \\__\\/        \\__\\/       \\  \\:\\       \\  \\::/      \\  \\:\\          \\__\\/ \\  \\::/\n    \\__\\/     \\__\\/                                  \\__\\/        \\__\\/        \\__\\/                 \\__\\/\n  "
+
+# algorithm to generate nc conformations
+def genConf(m, nc, rms, efilter, rmspost):
+    nr = int(AllChem.CalcNumRotatableBonds(m))
+    #m = Chem.AddHs(m)
+    Chem.AssignAtomChiralTagsFromStructure(m, replaceExistingTags=True)
+    if not nc: nc = 3**nr
+
+    print(dashedline+"\n   |    "+("FULL_MONTE search").ljust(leftcol)+("|").rjust(rightcol))
+    #log.Write("   | o  "+("COMP: "+str(Params.COMP)+" degrees").ljust(leftcol)+("|").rjust(rightcol))
+    #log.Write("   | o  "+("LEVL: "+str(Params.LEVL)+" force field").ljust(leftcol)+("|").rjust(rightcol))
+    #log.Write("   | o  "+("DEMX: "+str(Params.DEMX)+" kcal/mol").ljust(leftcol)+("|").rjust(rightcol))
+    print("   | o  "+("EWIN: "+str(efilter)+" kcal/mol").ljust(leftcol)+("|").rjust(rightcol))
+    print("   | o  "+("MCNV: "+str(nr)+" ROTATABLE BONDS").ljust(leftcol)+("|").rjust(rightcol))
+    #log.Write("   |    "+torstring.ljust(leftcol)+("|").rjust(rightcol))
+    print("   | o  "+("STEP: "+str(nc)+" (ESTIMATED CONFORMER SPACE: "+str(nr**3)+")").ljust(leftcol)+("|").rjust(rightcol))
+    print(dashedline+"\n")
+
+    if not rms: rms = -1
+    ids=AllChem.EmbedMultipleConfs(m, numConfs=nc, pruneRmsThresh=rms, randomSeed=1, useExpTorsionAnglePrefs=True, useBasicKnowledge=True)
+
+
+    if len(ids)== 0:
+        ids = m.AddConformer(m.GetConformer, assignID=True)
+
+    diz = []
+    diz2 = []
+    diz3 = []
+    for id in ids:
+        prop = AllChem.MMFFGetMoleculeProperties(m, mmffVariant="MMFF94s")
+        ff = AllChem.MMFFGetMoleculeForceField(m, prop, confId=id)
+        ff.Minimize()
+        en = float(ff.CalcEnergy())
+        econf = (en, id)
+        diz.append(econf)
+
+    if efilter != "Y":
+        n, diz2 = energy_filter(m, diz, efilter)
+    else:
+        n = m
+        diz2 = diz
+
+    if rmspost != None and n.GetNumConformers() > 1:
+        o, diz3 = postrmsd(n, diz2, rmspost)
+    else:
+        o = n
+        diz3 = diz2
+
+    return o, diz3, nr
+
+# filter conformers based on relative energy
+def energy_filter(m, diz, efilter):
+    print("o  FILTERING CONFORMERS BY ENERGY CUTOFF: "+str(efilter)+" kcal/mol")
+    diz.sort()
+    mini = float(diz[0][0])
+    sup = mini + efilter
+    n = Chem.Mol(m)
+    n.RemoveAllConformers()
+    n.AddConformer(m.GetConformer(int(diz[0][1])))
+    nid = []
+    ener = []
+    nid.append(int(diz[0][1]))
+    ener.append(float(diz[0][0])-mini)
+    del diz[0]
+    for x,y in diz:
+        if x <= sup:
+            #print("   KEEP - Erel:", x-mini)
+            n.AddConformer(m.GetConformer(int(y)))
+            nid.append(int(y))
+            ener.append(float(x-mini))
+        else:
+            #print("   REMOVE - Erel:", x-mini)
+            break
+    diz2 = list(zip(ener, nid))
+    print("   KEEPING "+str(len(ener))+" CONFORMERS")
+    return n, diz2
+
+# filter conformers based on geometric RMS
+def postrmsd(n, diz2, rmspost):
+    print("o  FILTERING CONFORMERS BY RMS: "+str(rmspost))
+    diz2.sort(key=lambda x: x[0])
+    o = Chem.Mol(n)
+    confidlist = [diz2[0][1]]
+    enval = [diz2[0][0]]
+    nh = Chem.RemoveHs(n)
+    del diz2[0]
+    for z,w in diz2:
+        confid = int(w)
+        p=0
+        for conf2id in confidlist:
+            #print(confid, conf2id)
+            rmsd = AllChem.GetBestRMS(nh, nh, prbId=confid, refId=conf2id)
+            if rmsd < rmspost:
+                p=p+1
+                break
+        if p == 0:
+            confidlist.append(int(confid))
+            enval.append(float(z))
+    diz3 = list(zip(enval, confidlist))
+    print("   KEEPING "+str(len(enval))+" CONFORMERS")
+    return o, diz3
+
+# conformational search / handles parallel threads if more than one structure is defined
+def csearch(log, suppl, writer, args):
+    with futures.ProcessPoolExecutor(max_workers=args.threads) as executor:
+        jobs = []
+        nm = []
+        numMol=0
+        for mol in suppl:
+            numMol = numMol+1
+            if mol is not None:
+                nm.append(mol.GetProp('_Name'))
+                job = executor.submit(genConf, mol, args.nconf, args.rmspre, args.cutoff, args.rmspost)
+                jobs.append(job)
+            else:
+                log.Write("ERROR: Impossible to generate conformers for molecule number " + str(numMol))
+        widgets = ["Generating conformations; ", progressbar.Percentage(), " ", progressbar.ETA(), " ", progressbar.Bar()]
+
+        # If there are several structures use progressbar
+        if len(jobs) > 1:
+            for job in futures.as_completed(jobs):
+                mol,ids,nr = job.result()
+
+        for j in range(0, len(jobs)):
+            mol,ids,nr = jobs[j].result()
+            name = nm[j]
+            for en,id in ids:
+                mol.SetProp('_Name', name)
+
+                if args.printproperty == True:
+                    mol.SetProp('ConfId', str(id))
+                    mol.SetProp('ConfEnergies', str(en)+ " kcal/mol")
+                    mol.SetProp('Rotatable Bonds Number', str(nr))
+                writer.write(mol, confId=id)
+    writer.close()
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description='Molecular conformer generator')
+    parser.add_argument('-isdf', required=True, help='sdf input file')
+    parser.add_argument('-osdf', required=True, help='sdf output file')
+    parser.add_argument('-nconf', type=int, required=False, help='number of conformers')
+    parser.add_argument('-rmspre', type=float, required=False, help='rms threshold pre optimization')
+    parser.add_argument('-rmspost', type=float, required=False, default=0.2, help='rms threshold post minimization')
+    parser.add_argument('-cutoff', type=float, required=False, default=10.0, help='energy window')
+    parser.add_argument('-printproperty', action='store_true', default=True, help='Print molecule properties (energy and rotable bond number)')
+    parser.add_argument('-threads', type=int, required=False, default=1, help='number of threads')
+    args = parser.parse_args()
+
+    # Check that the input structure exists and has the correct format
+    if os.path.exists(args.isdf) and os.path.splitext(args.isdf)[1] in [".sdf"]:
+        inp = args.isdf; out = args.osdf
+        filename = os.path.splitext(inp)[0]
+
+    # Define input and outputs
+    log = Logger(filename,"cs"); writer = Chem.SDWriter(out); suppl = Chem.SDMolSupplier(inp, removeHs=False)
+    log.Write("\no  Extracting structure from "+args.isdf+" ...")
+
+    # conformational search
+    search = csearch(log, suppl, writer, args)
+
+    end = time.strftime("%H:%M:%S", time.localtime())
+    log.Write(asciiArt+end); log.Write(normaltermination); log.Finalize()

NMR_Prediction/preprocessor.p

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9160321e192de2a5ddf706be7b048a634b79e8d928feea6b1558151349bcb21
+size 4556

NMR_Prediction/schnet_edgeupdate/best_model.hdf5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:056b2da63696eb4a8f319ed52c0a9f8e20a0971d5f471b9a89f8468ab9ee5180
+size 32938712

NMR_Prediction/valid.py

@@ -0,0 +1,9 @@
+from rdkit import Chem
+
+
+def validate_smiles(smiles: str) -> bool:
+    """Return True if the SMILES is parseable by RDKit and non-empty."""
+    if not smiles or not smiles.strip():
+        return False
+    mol = Chem.MolFromSmiles(smiles.strip())
+    return mol is not None

README.md

@@ -0,0 +1,32 @@
+---
+title: CASCADE NMR Predictor
+emoji: 🧪
+colorFrom: blue
+colorTo: indigo
+sdk: docker
+app_port: 7860
+pinned: false
+license: mit
+---
+
+# CASCADE – NMR Chemical Shift Prediction
+
+**CASCADE** (**C**hemic**A**l **S**hift **CA**lculation with **DE**ep learning) predicts ¹H and ¹³C NMR chemical shifts for organic molecules using a stereochemically- and conformationally aware 3D Message Passing Graph Neural Network.
+
+Developed in the [Paton group](https://patonlab.colostate.edu) at Colorado State University.
+
+## Usage
+
+Open the app, enter a SMILES string (or use the Draw button), select ¹H or ¹³C, and click Submit. Predicted shifts are Boltzmann-weighted averages over an MMFF94s conformer ensemble.
+
+**Scope:** Neutral organic molecules containing C, H, N, O, S, P, F, Cl.
+
+## Citation
+
+Guan, Y.; Sowndarya, S. V. S.; Gallegos, L. C.; St. John, P. C.; Paton, R. S.
+*Chem. Sci.* **2021**, *12*, 12012–12026.
+[DOI: 10.1039/D1SC03343C](https://doi.org/10.1039/D1SC03343C)
+
+## Source code
+
+[github.com/patonlab/CASCADE](https://github.com/patonlab/CASCADE)

app.py

@@ -0,0 +1,178 @@
+"""
+CASCADE – Flask app for HF Spaces
+Mounted at /cascade_v1/
+"""
+
+import os
+import json
+import uuid
+
+import redis
+from flask import Flask, Blueprint, request, jsonify, render_template, send_file, abort, redirect, Response
+
+from rdkit import Chem
+from rdkit.Chem import AllChem
+from rdkit.Chem.Draw import rdMolDraw2D
+
+from NMR_Prediction.valid import validate_smiles
+
+bp = Blueprint("cascade", __name__)
+
+redis_client = redis.StrictRedis(
+    host="localhost", port=6379, db=0, decode_responses=True
+)
+
+# ── Pages ─────────────────────────────────────────────────────────────────────
+@bp.route("/")
+@bp.route("")
+@bp.route("/predict/")
+@bp.route("/predict")
+@bp.route("/home/")
+@bp.route("/about/")
+def predict():
+    return render_template("cascade/predict.html")
+
+
+# ── Job submission ────────────────────────────────────────────────────────────
+def _submit_job(smiles, type_):
+    if not validate_smiles(smiles):
+        return jsonify({"message": "Input molecule is not allowed", "task_id": None})
+    task_id = uuid.uuid4().hex
+    redis_client.set(f"task_detail_{task_id}",
+                     json.dumps({"smiles": smiles, "type_": type_}))
+    redis_client.rpush("task_queue", task_id)
+    return task_id
+
+
+@bp.route("/predict_NMR_C/", methods=["POST"])
+def predict_NMR_C():
+    result = _submit_job(request.form["smiles"], request.form["type_"])
+    if not isinstance(result, str):
+        return result
+    return jsonify({"message": "Molecule submitted to C queue", "task_id": result})
+
+
+@bp.route("/predict_NMR_H/", methods=["POST"])
+def predict_NMR_H():
+    result = _submit_job(request.form["smiles"], request.form["type_"])
+    if not isinstance(result, str):
+        return result
+    return jsonify({"message": "Molecule submitted to H queue", "task_id": result})
+
+
+# ── check_task ────────────────────────────────────────────────────────────────
+@bp.route("/check_task/")
+def check_task():
+    raw = redis_client.get(f"task_result_{request.args['task_id']}")
+    if not raw:
+        return "running", 200
+    result = json.loads(raw)
+    if "errMessage" in result:
+        return "Error1", 200
+    return "done", 200
+
+
+# ── get_result ────────────────────────────────────────────────────────────────
+@bp.route("/get_result/")
+def get_result():
+    task_id = request.args["task_id"]
+    raw = redis_client.get(f"task_result_{task_id}")
+    if not raw:
+        return jsonify({"error": "Result not found"}), 404
+
+    result = json.loads(raw)
+    if "errMessage" in result:
+        return jsonify({"error": result["errMessage"]}), 500
+
+    smiles  = result["smiles"]
+    nucleus = result.get("type_", "C")  # "C" or "H"
+
+    weighted_shift_txt = result["weightedShiftTxt"]
+    shift_map = {}
+    for item in filter(None, weighted_shift_txt.split(";")):
+        parts = item.split(",")
+        if len(parts) == 2:
+            shift_map[int(parts[0])] = parts[1]
+
+    if nucleus == "H":
+        # ¹H: draw with explicit H so H atoms are visible
+        mol = Chem.MolFromSmiles(smiles)
+        mol = Chem.AddHs(mol)
+        AllChem.Compute2DCoords(mol)
+        mol_draw = rdMolDraw2D.PrepareMolForDrawing(mol, kekulize=True)
+        n_label  = mol.GetNumAtoms()
+        drawer   = rdMolDraw2D.MolDraw2DSVG(700, 500)
+    else:
+        # ¹³C: skeletal structure, no explicit H
+        mol = Chem.MolFromSmiles(smiles)
+        AllChem.Compute2DCoords(mol)
+        mol_draw = rdMolDraw2D.PrepareMolForDrawing(mol, kekulize=True)
+        n_label  = mol.GetNumAtoms()
+        drawer   = rdMolDraw2D.MolDraw2DSVG(600, 450)
+
+    opts = drawer.drawOptions()
+    for atom_1idx, shift_val in shift_map.items():
+        atom_0idx = atom_1idx - 1
+        if atom_0idx < n_label:
+            opts.atomLabels[atom_0idx] = shift_val
+    opts.clearBackground = False
+    opts.bondLineWidth = 1
+    opts.padding = 0.15
+    opts.additionalAtomLabelPadding = 0.1
+
+    drawer.DrawMolecule(mol_draw)
+    drawer.FinishDrawing()
+    svg = drawer.GetDrawingText().replace("svg:", "").replace(":svg", "")
+
+    return jsonify({
+        "svg": svg,
+        "smiles": smiles,
+        "nucleus": nucleus,
+        "conf_sdfs": result.get("conf_sdfs", []),
+        "weightedShift": weighted_shift_txt,
+        "confShift": result["confShiftTxt"],
+        "relative_E": result["relative_E"],
+        "taskId": task_id,
+    })
+
+
+# ── Download as CSV ───────────────────────────────────────────────────────────
+@bp.route("/download/<task_id>/")
+def download(task_id):
+    raw = redis_client.get(f"task_result_{task_id}")
+    if not raw:
+        abort(404)
+    result = json.loads(raw)
+    if "errMessage" in result:
+        abort(404)
+
+    nucleus = result.get("type_", "C")
+    header = f"Atom Index,Predicted {'1H' if nucleus == 'H' else '13C'} Shift (ppm)"
+    lines = [header]
+    for item in filter(None, result["weightedShiftTxt"].split(";")):
+        parts = item.split(",")
+        if len(parts) == 2:
+            lines.append(f"{parts[0]},{parts[1]}")
+
+    return Response(
+        "\n".join(lines),
+        mimetype="text/csv",
+        headers={"Content-Disposition": f"attachment; filename=cascade_{task_id[:8]}.csv"}
+    )
+
+
+def create_app():
+    app = Flask(__name__, static_folder="static", template_folder="templates")
+    app.register_blueprint(bp, url_prefix="/cascade_v1")
+
+    @app.route("/")
+    @app.route("/cascade_v1")
+    def root():
+        return redirect("/cascade_v1/predict/")
+
+    return app
+
+app = create_app()
+
+if __name__ == "__main__":
+    app.run(host="0.0.0.0", port=7860, debug=False)

nfp/__init__.py

@@ -0,0 +1,15 @@
+from .layers import *
+from .models import *
+
+custom_layers = {
+    'MessageLayer': MessageLayer,
+    'EdgeNetwork': EdgeNetwork,
+    'ReduceAtomToMol': ReduceAtomToMol,
+    'ReduceBondToAtom': ReduceBondToAtom,
+    'GatherAtomToBond': GatherAtomToBond,
+    'GRUStep': GRUStep,
+    'Embedding2D': Embedding2D,
+    'Squeeze': Squeeze,
+    'GraphModel': GraphModel,
+    'masked_mean_squared_error': masked_mean_squared_error
+}

nfp/layers/__init__.py

@@ -0,0 +1,3 @@
+from .layers import *
+from .utils import *
+from .wrappers import *

nfp/layers/layers.py

@@ -0,0 +1,445 @@
+from keras.engine import Layer
+
+from keras import activations
+from keras import initializers
+from keras import regularizers
+from keras import constraints
+from keras.layers import Lambda
+
+import tensorflow as tf
+import keras.backend as K
+
+class MessageLayer(Layer):
+    """ Implements the matrix multiplication message functions from Gilmer
+    2017. This could probably be implemented as a series of other layers, but
+    this is more convenient.
+
+    """
+
+    def __init__(self, dropout=0., reducer=None, **kwargs):
+        """ 
+
+        dropout : float between 0 and 1
+            Whether to apply dropout to individual messages before they are
+            reduced to each incoming atom.
+
+        reducer : ['sum', 'mean', 'max', or 'min']
+            How to collect incoming messages for each atom. In this library,
+            I'm careful to only have messages be a function of the sending
+            atom, so we can sort the connectivity matrix by recieving atom.
+            That lets us use the `segment_*` methods from tensorflow, instead
+            of the `unsorted_segment_*` methods.
+
+        """
+
+        self.dropout = dropout
+        self.reducer = reducer
+
+        reducer_dict = {
+            None: tf.segment_sum,
+            'sum': tf.segment_sum,
+            'mean': tf.segment_mean,
+            'max': tf.segment_max,
+            'min': tf.segment_min
+        }
+
+        self._reducer = reducer_dict[reducer]
+
+        super(MessageLayer, self).__init__(**kwargs)
+    
+
+    def call(self, inputs, training=None):
+        """ Perform a single message passing step, returing the summed messages
+        for each recieving atom.
+
+        Inputs are [atom_matrix, bond_matrix, connectivity_matrix]
+
+        atom_matrix : (num_atoms_in_batch, d)
+            The input matrix of current hidden states for each atom
+
+        bond_matrix : (num_bonds_in_batch, d, d)
+            A matrix of current edge features, with each edge represented as a
+            (dxd) matrix.
+
+        connectivity : (num_bonds_in_batch, 2)
+            A matrix of (a_i, a_j) pairs that indicates the bond in bond_matrix
+            connecting atom_matrix[a_j] to atom_matrix[a_i].
+            The first entry indicates the recieving atom.
+
+        """
+
+        atom_matrix, bond_matrix, connectivity = inputs
+
+        # Gather the atom matrix so that each reciever node corresponds with
+        # the given bond_matrix entry
+        atom_gathered = tf.gather(atom_matrix, connectivity[:, 1])
+
+        # Multiply the bond matrices by the gathered atom matrices
+        messages = K.batch_dot(bond_matrix, atom_gathered)
+
+        # Add dropout on a message-by-message basis if desired
+        def add_dropout():
+            if 0. < self.dropout < 1.:
+                return K.dropout(messages, self.dropout)
+            else:
+                return messages
+
+        dropout_messages = K.in_train_phase(
+            add_dropout(), messages, training=training)
+
+        # Sum each message along the (sorted) reciever nodes
+        summed_message = self._reducer(dropout_messages, connectivity[:, 0])
+
+        return summed_message
+
+
+    def compute_output_shape(self, input_shape):
+        """ Computes the shape of the output, which should be the same
+        dimension as the first input, that atom hidden state """
+        
+        assert input_shape and len(input_shape) == 3
+        assert input_shape[0][-1]  # atom hidden state dimension must be specified
+        return input_shape[0]
+
+
+    def get_config(self):
+        config = {
+            'dropout': self.dropout,
+            'reducer': self.reducer,
+        }
+        base_config = super(MessageLayer, self).get_config()
+        return dict(list(base_config.items()) + list(config.items()))
+
+
+class GatherAtomToBond(Layer):
+    """ Reshapes the atom matrix (num_atoms_in_batch, d) to the bond matrix
+    (num_bonds_in_batch, d) by reindexing according to which atom is involved
+    in each bond.
+
+    index : 0 or 1
+        whether to gather the sending atoms (1) or recieving atoms (0) for each
+        bond.
+
+    """
+
+    def __init__(self, index, **kwargs):
+        self.index = index
+        super(GatherAtomToBond, self).__init__(**kwargs)
+    
+    def call(self, inputs):
+        atom_matrix, connectivity = inputs
+        return  tf.gather(atom_matrix, connectivity[:, self.index])
+
+    def compute_output_shape(self, input_shape):
+        """ Computes the shape of the output,
+        which should be the shape of the atom matrix with the length
+        of the bond matrix """
+        
+        assert input_shape and len(input_shape) == 2
+        return input_shape[0]
+        
+    def get_config(self):
+        config = {
+            'index': self.index,
+        }
+        base_config = super(GatherAtomToBond, self).get_config()
+        return dict(list(base_config.items()) + list(config.items()))
+
+class Reducer(Layer):
+    """ Superclass for reducing methods. 
+    
+    reducer : ['sum', 'mean', 'max', or 'min']
+        How to collect elements for each atom or molecule. In this library,
+        I'm careful to only have messages be a function of the sending
+        atom, so we can sort the connectivity matrix by recieving atom.
+        That lets us use the `segment_*` methods from tensorflow, instead
+        of the `unsorted_segment_*` methods.
+
+    """
+
+    def __init__(self, reducer=None, **kwargs):
+
+        self.reducer = reducer
+
+        reducer_dict = {
+            None: tf.segment_sum,
+            'sum': tf.segment_sum,
+            'unsorted_sum': tf.unsorted_segment_sum,
+            'mean': tf.segment_mean,
+            'unsorted_mean': tf.unsorted_segment_mean,
+            'max': tf.segment_max,
+            'min': tf.segment_min
+        }
+
+        self._reducer = reducer_dict[reducer]
+
+        super(Reducer, self).__init__(**kwargs)
+
+    def get_config(self):
+        config = {'reducer': self.reducer}
+        base_config = super(Reducer, self).get_config()
+        return dict(list(base_config.items()) + list(config.items()))
+
+    def compute_output_shape(self, input_shape):
+        assert input_shape and len(input_shape) == 2
+        # Output shape is (n_graphs, atom_dim)
+        return input_shape[0]
+
+
+class ReduceAtomToMol(Reducer):
+    """ Sum over all atoms in each molecule.
+
+    Inputs
+
+    atom_matrix : (num_atoms_in_batch, d)
+        atom hidden states for each atom in the batch
+
+    node_graph_indices : (num_atoms_in_batch,)
+        A scalar for each atom representing which molecule in the batch the
+        atom belongs to. This is generated by the preprocessor class, and
+        essentially looks like [0, 0, 0, 1, 1] for a batch with a 3 atom
+        molecule and a 2 atom molecule.
+    """
+
+    def call(self, inputs):
+
+        atom_matrix, node_graph_indices = inputs
+        return self._reducer(atom_matrix, node_graph_indices)
+
+class ReduceBondToPro(Reducer):
+    """
+    Sums over bonds acoording to bond_index to get target bond properties
+    
+    Inputs
+
+    bond_matrix : (num_bonds_in_batch, d)
+        bond hidden states for each bond in the batch
+
+    bond_index : (bond_atoms_in_batch, )
+        A scalar for each bond representing which number in the target property 
+        the bond links to. This is generated by the preprocessor class, and
+        essentially looks like [-1,-1,0,0,-1,-1,1....]
+    """
+
+    def call(self, inputs):
+        
+        bond_matrix, bond_index, n_pro = inputs
+        num_segments = tf.reduce_sum(n_pro)
+        return self._reducer(bond_matrix, bond_index, num_segments)
+
+    def compute_output_shape(self, input_shape):
+        assert input_shape and len(input_shape) == 3
+        return input_shape[0]
+
+class ReduceAtomToPro(Reducer):
+    """
+    Sums over atoms acoording to atom_index to get target atom properties
+    
+    Inputs
+
+    atom_matrix : (num_atoms_in_batch, d)
+        atom hidden states for each atom in the batch
+
+    atom_index : (atom_atoms_in_batch, )
+        A scalar for each atom representing which number in the target property 
+        the atom links to. This is generated by the preprocessor class, and
+        essentially looks like [-1,-1,0,0,-1,-1,1....]
+    """
+
+    def call(self, inputs):
+        
+        atom_matrix, atom_index, n_pro = inputs
+        num_segments = tf.reduce_sum(n_pro)
+        return self._reducer(atom_matrix, atom_index, num_segments)
+
+    def compute_output_shape(self, input_shape):
+        assert input_shape and len(input_shape) == 3
+        return input_shape[0]
+
+class ReduceBondToAtom(Reducer):
+
+    """ Sums over the incoming messages from all sender atoms.
+
+    Inputs: 
+    
+    bond_matrix : (num_bonds_in_batch, d)
+        A matrix of messages coming from each sender atom; one row for each
+        bond/edge.
+
+    connectivity : (num_bonds_in_batch, 2)
+        A matrix of (a_i, a_j) pairs that indicates the bond in bond_matrix
+            connecting atom_matrix[a_j] to atom_matrix[a_i].
+        The first entry indicates the recieving atom.
+
+    Again, I'm careful to only have the messages be a function of the sending
+    node, such that we can use sorted methods in performing the reduction.
+
+    """
+
+    def call(self, inputs):
+
+        bond_matrix, connectivity = inputs
+        return self._reducer(bond_matrix, connectivity[:, 0])
+
+
+class Squeeze(Layer):
+    """ Keras forces inputs to be a vector per entry, so this layer squeezes
+    them to a single dimension.
+
+    I.e., node_graph_indices will have shape (num_atoms_in_batch, 1), while its
+    easier to work with a vector of shape (num_atoms_in_batch,)
+    """
+
+    def call(self, inputs):
+        return K.squeeze(inputs, 1)
+
+    def compute_output_shape(self, input_shape):
+        return input_shape[:-1]
+
+
+class Embedding2D(Layer):
+    """ Keras typically wants to embed items as a single vector, while for the
+    matrix multiplication method of Gilmer 2017 we need a matrix for each bond
+    type. This just implements that fairly simple extension of the traditional
+    embedding layer.
+    """
+
+    def __init__(self, input_dim, output_dim,
+                 embeddings_initializer='uniform',
+                 embeddings_regularizer=None,
+                 embeddings_constraint=None,
+                 **kwargs):
+        
+        super(Embedding2D, self).__init__(**kwargs)
+
+        self.input_dim = input_dim
+        self.output_dim = output_dim
+        
+        self.embeddings_initializer = initializers.get(embeddings_initializer)
+        self.embeddings_regularizer = regularizers.get(embeddings_regularizer)
+        self.embeddings_constraint = constraints.get(embeddings_constraint)
+        
+    def build(self, input_shape):
+
+        self.embeddings = self.add_weight(
+            shape=(self.input_dim, self.output_dim, self.output_dim),
+            initializer=self.embeddings_initializer,
+            name='bond_embedding_weights', 
+            regularizer=self.embeddings_regularizer,
+            constraint=self.embeddings_constraint)
+        
+        self.built = True
+        
+    def call(self, inputs):
+        return tf.nn.embedding_lookup(self.embeddings, inputs)
+        
+    def compute_output_shape(self, input_shape):
+        return (input_shape[0], self.output_dim, self.output_dim)
+
+    def get_config(self):
+        config = {
+            'input_dim': self.input_dim,
+            'output_dim': self.output_dim,
+            'embeddings_initializer':
+            initializers.serialize(self.embeddings_initializer),
+            'embeddings_regularizer':
+            regularizers.serialize(self.embeddings_regularizer),
+            'embeddings_constraint':
+            constraints.serialize(self.embeddings_constraint),
+        }
+        base_config = super(Embedding2D, self).get_config()
+        return dict(list(base_config.items()) + list(config.items()))
+
+
+class EdgeNetwork(Layer):
+    """ A layer to embed (bond_type, distance) pairs as a NxN matrix. 
+    
+    Inputs:
+    units : dimension of the output matrix
+    bond_classes : number of unique bonds
+
+    First perfoms a 1-hot encoding of the bond_type, then passes the
+    (*one_hot_encoding, distance) vector to a dense layer. This is the "Edge
+    Network" message described by Gilmer, 2017.
+    
+    """
+    
+    def __init__(self, units, bond_classes,
+                 activation=None,
+                 use_bias=True,
+                 kernel_initializer='glorot_uniform',
+                 bias_initializer='zeros',
+                 kernel_regularizer=None,
+                 bias_regularizer=None,
+                 activity_regularizer=None,
+                 kernel_constraint=None,
+                 bias_constraint=None,
+                 **kwargs):
+        
+        super(EdgeNetwork, self).__init__(**kwargs)
+        self.units = units
+        self.bond_classes = bond_classes
+        
+        self.activation = activations.get(activation)
+        self.use_bias = use_bias
+        self.kernel_initializer = initializers.get(kernel_initializer)
+        self.bias_initializer = initializers.get(bias_initializer)
+        self.kernel_regularizer = regularizers.get(kernel_regularizer)
+        self.bias_regularizer = regularizers.get(bias_regularizer)
+        self.activity_regularizer = regularizers.get(activity_regularizer)
+        self.kernel_constraint = constraints.get(kernel_constraint)
+        self.bias_constraint = constraints.get(bias_constraint)
+        
+    def build(self, input_shape):
+        
+        self.kernel = self.add_weight(shape=(self.bond_classes + 1, self.units**2),
+                                      initializer=self.kernel_initializer,
+                                      name='kernel',
+                                      regularizer=self.kernel_regularizer,
+                                      constraint=self.kernel_constraint)
+        if self.use_bias:
+            self.bias = self.add_weight(shape=(self.units**2,),
+                                        initializer=self.bias_initializer,
+                                        name='bias',
+                                        regularizer=self.bias_regularizer,
+                                        constraint=self.bias_constraint)
+        else:
+            self.bias = None
+            
+        self.built = True
+
+    def call(self, inputs):
+        
+        bond_type, distance = inputs
+        bond_type_onehot = tf.one_hot(tf.squeeze(bond_type), self.bond_classes)
+        stacked_inputs = tf.concat([bond_type_onehot, distance], 1)
+        
+        output = K.dot(stacked_inputs, self.kernel)
+        if self.use_bias:
+            output = K.bias_add(output, self.bias, data_format='channels_last')
+        if self.activation is not None:
+            output = self.activation(output)
+            
+        output = tf.reshape(output, [-1, self.units, self.units])
+        return output
+    
+    def compute_output_shape(self, input_shape):
+        return (input_shape[0][0], self.units, self.units)
+    
+    def get_config(self):
+        config = {
+            'units': self.units,
+            'bond_classes': self.bond_classes,
+            'activation': activations.serialize(self.activation),
+            'use_bias': self.use_bias,
+            'kernel_initializer': initializers.serialize(self.kernel_initializer),
+            'bias_initializer': initializers.serialize(self.bias_initializer),
+            'kernel_regularizer': regularizers.serialize(self.kernel_regularizer),
+            'bias_regularizer': regularizers.serialize(self.bias_regularizer),
+            'activity_regularizer':
+                regularizers.serialize(self.activity_regularizer),
+            'kernel_constraint': constraints.serialize(self.kernel_constraint),
+            'bias_constraint': constraints.serialize(self.bias_constraint)
+        }
+        base_config = super(EdgeNetwork, self).get_config()
+        return dict(list(base_config.items()) + list(config.items()))

nfp/layers/utils.py

@@ -0,0 +1,90 @@
+""" Unused for the moment, I believe. """
+
+import tensorflow as tf
+
+def get_shape(tensor):
+    """Returns the tensor's shape.
+    Each shape element is either:
+    - an `int`, when static shape values are available, or
+    - a `tf.Tensor`, when the shape is dynamic.
+    Args:
+    tensor: A `tf.Tensor` to get the shape of.
+    Returns:
+    The `list` which contains the tensor's shape.
+    """
+
+    shape_list = tensor.shape.as_list()
+    if all(s is not None for s in shape_list):
+        return shape_list
+    
+    shape_tensor = tf.shape(tensor)
+    return [shape_tensor[i] if s is None else s for i, s in
+            enumerate(shape_list)]
+
+
+def repeat(tensor, repeats, axis=0):
+    """Repeats a `tf.Tensor`'s elements along an axis by custom amounts.
+    Equivalent to Numpy's `np.repeat`.
+    `tensor and `repeats` must have the same numbers of elements along `axis`.
+    Args:
+    tensor: A `tf.Tensor` to repeat.
+    repeats: A 1D sequence of the number of repeats per element.
+    axis: An axis to repeat along. Defaults to 0.
+    name: (string, optional) A name for the operation.
+    Returns:
+    The `tf.Tensor` with repeated values.
+    """
+
+    cumsum = tf.cumsum(repeats)
+    range_ = tf.range(cumsum[-1])
+
+    indicator_matrix = tf.cast(tf.expand_dims(range_, 1) >= cumsum, tf.int32)
+    indices = tf.reduce_sum(indicator_matrix, reduction_indices=1)
+
+    shifted_tensor = _axis_to_inside(tensor, axis)
+    repeated_shifted_tensor = tf.gather(shifted_tensor, indices)
+    repeated_tensor = _inside_to_axis(repeated_shifted_tensor, axis)
+
+    shape = tensor.shape.as_list()
+    shape[axis] = None
+    repeated_tensor.set_shape(shape)
+
+    return repeated_tensor
+
+
+def _axis_to_inside(tensor, axis):
+    """Shifts a given axis of a tensor to be the innermost axis.
+    Args:
+        tensor: A `tf.Tensor` to shift.
+        axis: An `int` or `tf.Tensor` that indicates which axis to shift.
+    Returns:
+        The shifted tensor.
+    """
+
+    axis = tf.convert_to_tensor(axis)
+    rank = tf.rank(tensor)
+
+    range0 = tf.range(0, limit=axis)
+    range1 = tf.range(tf.add(axis, 1), limit=rank)
+    perm = tf.concat([[axis], range0, range1], 0)
+
+    return tf.transpose(tensor, perm=perm)
+
+
+def _inside_to_axis(tensor, axis):
+    """Shifts the innermost axis of a tensor to some other axis.
+    Args:
+        tensor: A `tf.Tensor` to shift.
+        axis: An `int` or `tf.Tensor` that indicates which axis to shift.
+    Returns:
+        The shifted tensor.
+    """
+
+    axis = tf.convert_to_tensor(axis)
+    rank = tf.rank(tensor)
+
+    range0 = tf.range(1, limit=axis + 1)
+    range1 = tf.range(tf.add(axis, 1), limit=rank)
+    perm = tf.concat([range0, [0], range1], 0)
+
+    return tf.transpose(tensor, perm=perm)

nfp/layers/wrappers.py

@@ -0,0 +1,45 @@
+"""
+These classes wrap the GRUCell and LSTMCell keras layers, allowing weight-tying
+between RNNs for each atom. Essentially, Keras expects inputs to an RNN to be
+of the shape (batch_size, sequence_length, features), while our inputs are of
+shape (num_atoms_per_batch, features). 
+
+Peter, 3/20/2018
+
+I should revisit whether these are needed post graph_nets update.
+
+Peter, 11/9/2018.
+
+"""
+
+from keras.layers import GRUCell, LSTMCell
+
+class GRUStep(GRUCell): 
+    def build(self, input_shape):
+        GRUCell.build(self, input_shape[0])
+
+    def call(self, inputs):
+        """ Inputs should be [message, previous_state], returns [next_state]
+        """
+        return GRUCell.call(self, inputs[0], [inputs[1]])[0]
+
+    def compute_output_shape(self, input_shape):
+        return input_shape[0]
+
+
+class LSTMStep(LSTMCell): 
+    def build(self, input_shape):
+        LSTMCell.build(self, input_shape[0])
+
+    def call(self, inputs):
+        """ Inputs should be [message, previous_state, previous_memory], 
+        returns [next_state, next_memory]
+        """
+        outputs = LSTMCell.call(self, inputs[0], [inputs[1], inputs[2]])
+        return [outputs[0], outputs[1][1]]
+
+    def compute_mask(self, inputs, mask):
+        return [None] * 2
+        
+    def compute_output_shape(self, input_shape):
+        return [input_shape[0], input_shape[0]]

nfp/models/__init__.py

@@ -0,0 +1,2 @@
+from .models import *
+from .losses import *

nfp/models/losses.py

@@ -0,0 +1,19 @@
+""" Keras loss functions will choke on NaN inputs, which we'll often have for
+unspecified inputs. These are just a couple simple loss functions that mask NaN
+values in both the test and predicted tensors when computing the cost. """
+
+import keras.backend as K
+import tensorflow as tf
+
+def masked_mean_squared_error(y_true, y_pred):
+    mask = tf.is_finite(y_true)
+    y_true_mask = tf.boolean_mask(y_true, mask)
+    y_pred_mask = tf.boolean_mask(y_pred, mask)
+    return K.mean(K.square(y_pred_mask - y_true_mask), axis=-1)
+
+
+def masked_mean_absolute_error(y_true, y_pred):
+    mask = tf.is_finite(y_true)
+    y_true_mask = tf.boolean_mask(y_true, mask)
+    y_pred_mask = tf.boolean_mask(y_pred, mask)
+    return K.mean(K.abs(y_pred_mask - y_true_mask), axis=-1)

nfp/models/models.py

@@ -0,0 +1,12 @@
+from keras.models import Model
+
+class GraphModel(Model):
+    """ This is a simple modification of the Keras `Model` class to avoid
+    checking each input for a consistent batch_size dimension. Should work as
+    of keras-team/keras#11548.
+
+    """
+
+    def _standardize_user_data(self, *args, **kwargs):
+        kwargs['check_array_lengths'] = False
+        return super(GraphModel, self)._standardize_user_data(*args, **kwargs)

nfp/preprocessing/__init__.py

@@ -0,0 +1,4 @@
+from .preprocessor import *
+from .features import *
+from .scaling import *
+from .sequence import *

nfp/preprocessing/features.py

@@ -0,0 +1,128 @@
+class Tokenizer(object):
+    """ A class to turn arbitrary inputs into integer classes. """
+
+    def __init__(self):
+        # the default class for an unseen entry during test-time
+        self._data = {'unk': 1}
+        self.num_classes = 1
+        self.train = True
+        self.unknown = []
+
+    def __call__(self, item):
+        """ Check to see if the Tokenizer has seen `item` before, and if so,
+        return the integer class associated with it. Otherwise, if we're
+        training, create a new integer class, otherwise return the 'unknown'
+        class.
+
+        """
+        try:
+            return self._data[item]
+
+        except KeyError:
+            if self.train:
+                self._add_token(item)
+                return self(item)
+
+            else:
+                # Record the unknown item, then return the unknown label
+                self.unknown += [item]
+                return self._data['unk']
+
+    def _add_token(self, item):
+        self.num_classes += 1
+        self._data[item] = self.num_classes
+
+# The rest of the methods in this module are specific functions for computing
+# atom and bond features. New ones can be easily added though, and these are
+# passed directly to the Preprocessor class.
+
+def get_ring_size(obj, max_size=12):
+    if not obj.IsInRing():
+        return 0
+    else:
+        for i in range(max_size):
+            if obj.IsInRingSize(i):
+                return i
+        else:
+            return 'max'
+
+def atom_features(atom):
+    return atom.GetAtomicNum()
+
+
+def atom_features_v1(atom):
+    """ Return an integer hash representing the atom type
+    """
+
+    return str((
+        atom.GetSymbol(),
+        atom.GetDegree(),
+        atom.GetTotalNumHs(),
+        atom.GetImplicitValence(),
+        atom.GetIsAromatic(),
+    ))
+
+
+def atom_features_v2(atom):
+
+    props = ['GetChiralTag', 'GetDegree', 'GetExplicitValence',
+             'GetFormalCharge', 'GetHybridization', 'GetImplicitValence',
+             'GetIsAromatic', 'GetNoImplicit', 'GetNumExplicitHs',
+             'GetNumImplicitHs', 'GetNumRadicalElectrons', 'GetSymbol',
+             'GetTotalDegree', 'GetTotalNumHs', 'GetTotalValence']
+
+    atom_type = [getattr(atom, prop)() for prop in props]
+    atom_type += [get_ring_size(atom)]
+
+    return str(tuple(atom_type))
+
+
+def bond_features_v1(bond, **kwargs):
+    """ Return an integer hash representing the bond type.
+
+    flipped : bool
+        Only valid for 'v3' version, whether to swap the begin and end atom types
+
+    """
+
+    return str((
+        bond.GetBondType(),
+        bond.GetIsConjugated(),
+        bond.IsInRing(),
+        sorted([
+            bond.GetBeginAtom().GetSymbol(),
+            bond.GetEndAtom().GetSymbol()]),
+        ))
+
+
+def bond_features_v2(bond, **kwargs):
+
+    return str((
+        bond.GetBondType(),
+        bond.GetIsConjugated(),
+        bond.GetStereo(),
+        get_ring_size(bond),
+        sorted([
+            bond.GetBeginAtom().GetSymbol(),
+            bond.GetEndAtom().GetSymbol()]),
+        ))
+
+
+def bond_features_v3(bond, flipped=False):
+
+    if not flipped:
+        start_atom = atom_features(bond.GetBeginAtom())
+        end_atom = atom_features(bond.GetEndAtom())
+
+    else:
+        start_atom = atom_features(bond.GetEndAtom())
+        end_atom = atom_features(bond.GetBeginAtom())
+
+    return str((
+        bond.GetBondType(),
+        bond.GetIsConjugated(),
+        bond.GetStereo(),
+        get_ring_size(bond),
+        bond.GetEndAtom().GetSymbol(),
+        start_atom,
+        end_atom))

nfp/preprocessing/preprocessor.py

@@ -0,0 +1,734 @@
+import logging, sys
+
+import numpy as np
+from tqdm import tqdm
+from scipy.linalg import eigh
+
+from rdkit import Chem
+from rdkit.Chem import MolFromSmiles, MolToSmiles, AddHs
+
+from . import features
+from .features import Tokenizer
+import time
+
+
+class SmilesPreprocessor(object):
+    """ Given a list of SMILES strings, encode these molecules as atom and
+    connectivity feature matricies.
+
+    Example:
+    >>> preprocessor = SmilesPreprocessor(explicit_hs=False)
+    >>> inputs = preprocessor.fit(data.smiles)
+    """
+
+    def __init__(self, explicit_hs=True, atom_features=None, bond_features=None):
+        """
+
+        explicit_hs : bool
+            whether to tell RDkit to add H's to a molecule.
+        atom_features : function
+            A function applied to an rdkit.Atom that returns some
+            representation (i.e., string, integer) for the Tokenizer class.
+        bond_features : function
+            A function applied to an rdkit Bond to return some description.
+
+        """
+
+        self.atom_tokenizer = Tokenizer()
+        self.bond_tokenizer = Tokenizer()
+        self.explicit_hs = explicit_hs
+
+        if atom_features is None:
+            atom_features = features.atom_features
+
+        if bond_features is None:
+            bond_features = features.bond_features_v1
+
+        self.atom_features = atom_features
+        self.bond_features = bond_features
+
+
+    def fit(self, smiles_iterator):
+        """ Fit an iterator of SMILES strings, creating new atom and bond
+        tokens for unseen molecules. Returns a dictionary with 'atom' and
+        'connectivity' entries """
+        return list(self.preprocess(smiles_iterator, train=True))
+
+
+    def predict(self, smiles_iterator):
+        """ Uses previously determined atom and bond tokens to convert a SMILES
+        iterator into 'atom' and 'connectivity' matrices. Ensures that atom and
+        bond classes commute with previously determined results. """
+        return list(self.preprocess(smiles_iterator, train=False))
+
+
+    def preprocess(self, smiles_iterator, train=True):
+
+        self.atom_tokenizer.train = train
+        self.bond_tokenizer.train = train
+
+        for smiles in tqdm(smiles_iterator):
+            yield self.construct_feature_matrices(smiles)
+
+
+    @property
+    def atom_classes(self):
+        """ The number of atom types found (includes the 0 null-atom type) """
+        return self.atom_tokenizer.num_classes + 1
+
+
+    @property
+    def bond_classes(self):
+        """ The number of bond types found (includes the 0 null-bond type) """
+        return self.bond_tokenizer.num_classes + 1
+
+
+    def construct_feature_matrices(self, smiles):
+        """ construct a molecule from the given smiles string and return atom
+        and bond classes.
+
+        Returns
+        dict with entries
+        'n_atom' : number of atoms in the molecule
+        'n_bond' : number of bonds in the molecule
+        'atom' : (n_atom,) length list of atom classes
+        'bond' : (n_bond,) list of bond classes
+        'connectivity' : (n_bond, 2) array of source atom, target atom pairs.
+
+        """
+
+        mol = MolFromSmiles(smiles)
+        if self.explicit_hs:
+            mol = AddHs(mol)
+
+        n_atom = len(mol.GetAtoms())
+        n_bond = 2 * len(mol.GetBonds())
+
+        # If its an isolated atom, add a self-link
+        if n_bond == 0:
+            n_bond = 1
+
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        bond_index = 0
+
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        for n, atom in enumerate(atoms):
+
+            # Atom Classes
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+
+            start_index = atom.GetIdx()
+
+            for bond in atom.GetBonds():
+                # Is the bond pointing at the target atom
+                rev = bond.GetBeginAtomIdx() != start_index
+
+                # Bond Classes
+                bond_feature_matrix[n] = self.bond_tokenizer(
+                    self.bond_features(bond, flipped=rev))
+
+                # Connectivity
+                if not rev:  # Original direction
+                    connectivity[bond_index, 0] = bond.GetBeginAtomIdx()
+                    connectivity[bond_index, 1] = bond.GetEndAtomIdx()
+
+                else:  # Reversed
+                    connectivity[bond_index, 0] = bond.GetEndAtomIdx()
+                    connectivity[bond_index, 1] = bond.GetBeginAtomIdx()
+
+                bond_index += 1
+
+
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'connectivity': connectivity,
+        }
+
+
+class ConnectivityAPreprocessor(object):
+    """ Given a list of SMILES strings, encode these molecules as atom and
+    connectivity feature matricies.
+
+    Example:
+    >>> preprocessor = SmilesPreprocessor(explicit_hs=False)
+    >>> inputs = preprocessor.fit(data.smiles)
+    """
+
+    def __init__(self, explicit_hs=True, atom_features=None, bond_features=None):
+        """
+
+        explicit_hs : bool
+            whether to tell RDkit to add H's to a molecule.
+        atom_features : function
+            A function applied to an rdkit.Atom that returns some
+            representation (i.e., string, integer) for the Tokenizer class.
+        bond_features : function
+            A function applied to an rdkit Bond to return some description.
+
+        """
+
+        self.atom_tokenizer = Tokenizer()
+        self.bond_tokenizer = Tokenizer()
+        self.explicit_hs = explicit_hs
+
+        if atom_features is None:
+            atom_features = features.atom_features_v1
+
+        if bond_features is None:
+            bond_features = features.bond_features_v1
+
+        self.atom_features = atom_features
+        self.bond_features = bond_features
+
+
+    def fit(self, smiles_iterator):
+        """ Fit an iterator of SMILES strings, creating new atom and bond
+        tokens for unseen molecules. Returns a dictionary with 'atom' and
+        'connectivity' entries """
+        return list(self.preprocess(smiles_iterator, train=True))
+
+
+    def predict(self, smiles_iterator):
+        """ Uses previously determined atom and bond tokens to convert a SMILES
+        iterator into 'atom' and 'connectivity' matrices. Ensures that atom and
+        bond classes commute with previously determined results. """
+        return list(self.preprocess(smiles_iterator, train=False))
+
+
+    def preprocess(self, smiles_iterator, train=True):
+
+        self.atom_tokenizer.train = train
+        self.bond_tokenizer.train = train
+
+        for smiles in tqdm(smiles_iterator):
+            yield self.construct_feature_matrices(smiles)
+
+
+    @property
+    def atom_classes(self):
+        """ The number of atom types found (includes the 0 null-atom type) """
+        return self.atom_tokenizer.num_classes + 1
+
+
+    @property
+    def bond_classes(self):
+        """ The number of bond types found (includes the 0 null-bond type) """
+        return self.bond_tokenizer.num_classes + 1
+
+
+    def construct_feature_matrices(self, smiles):
+        """ construct a molecule from the given smiles string and return atom
+        and bond classes.
+
+        Returns
+        dict with entries
+        'n_atom' : number of atoms in the molecule
+        'n_bond' : number of bonds in the molecule
+        'atom' : (n_atom,) length list of atom classes
+        'bond' : (n_bond,) list of bond classes
+        'connectivity' : (n_bond, 2) array of source atom, target atom pairs.
+
+        """
+
+        mol = MolFromSmiles(smiles)
+        if self.explicit_hs:
+            mol = AddHs(mol)
+
+        n_atom = len(mol.GetAtoms())
+        n_bond = 2 * len(mol.GetBonds())
+
+        # If its an isolated atom, add a self-link
+        if n_bond == 0:
+            n_bond = 1
+
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        bond_index = 0
+
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        for n, atom in enumerate(atoms):
+
+            # Atom Classes
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+
+            start_index = atom.GetIdx()
+
+            for bond in atom.GetBonds():
+                # Is the bond pointing at the target atom
+                rev = bond.GetBeginAtomIdx() != start_index
+
+                # Bond Classes
+                bond_feature_matrix[n] = self.bond_tokenizer(
+                    self.bond_features(bond, flipped=rev))
+
+                # Connectivity
+                if not rev:  # Original direction
+                    connectivity[bond_index, 0] = bond.GetBeginAtomIdx()
+                    connectivity[bond_index, 1] = bond.GetEndAtomIdx()
+
+                else:  # Reversed
+                    connectivity[bond_index, 0] = bond.GetEndAtomIdx()
+                    connectivity[bond_index, 1] = bond.GetBeginAtomIdx()
+
+                bond_index += 1
+
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'connectivity': connectivity,
+        }
+
+
+class MolPreprocessor(SmilesPreprocessor):
+    """ I should refactor this into a base class and separate
+    SmilesPreprocessor classes. But the idea is that we only need to redefine
+    the `construct_feature_matrices` method to have a working preprocessor that
+    handles 3D structures.
+
+    We'll pass an iterator of mol objects instead of SMILES strings this time,
+    though.
+
+    """
+
+    def __init__(self, n_neighbors, cutoff, **kwargs):
+        """ A preprocessor class that also returns distances between
+        neighboring atoms. Adds edges for non-bonded atoms to include a maximum
+        of n_neighbors around each atom """
+
+        self.n_neighbors = n_neighbors
+        self.cutoff = cutoff
+        super(MolPreprocessor, self).__init__(**kwargs)
+
+
+    def construct_feature_matrices(self, mol):
+        """ Given an rdkit mol, return atom feature matrices, bond feature
+        matrices, and connectivity matrices.
+
+        Returns
+        dict with entries
+        'n_atom' : number of atoms in the molecule
+        'n_bond' : number of edges (likely n_atom * n_neighbors)
+        'atom' : (n_atom,) length list of atom classes
+        'bond' : (n_bond,) list of bond classes. 0 for no bond
+        'distance' : (n_bond,) list of bond distances
+        'connectivity' : (n_bond, 2) array of source atom, target atom pairs.
+
+        """
+
+        n_atom = len(mol.GetAtoms())
+
+        # n_bond is actually the number of atom-atom pairs, so this is defined
+        # by the number of neighbors for each atom.
+        #if there is cutoff,
+        distance_matrix = Chem.Get3DDistanceMatrix(mol)
+
+        if self.n_neighbors <= (n_atom - 1):
+            n_bond = self.n_neighbors * n_atom
+        else:
+            # If there are fewer atoms than n_neighbors, all atoms will be
+            # connected
+            n_bond = distance_matrix[(distance_matrix < self.cutoff) & (distance_matrix != 0)].size
+
+        if n_bond == 0: n_bond = 1
+
+        # Initialize the matrices to be filled in during the following loop.
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        bond_distance_matrix = np.zeros(n_bond, dtype=np.float32)
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        # Hopefully we've filtered out all problem mols by now.
+        if mol is None:
+            raise RuntimeError("Issue in loading mol")
+
+        # Get a list of the atoms in the molecule.
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        # Here we loop over each atom, and the inner loop iterates over each
+        # neighbor of the current atom.
+        bond_index = 0  # keep track of our current bond.
+        for n, atom in enumerate(atoms):
+
+            # update atom feature matrix
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+
+            # if n_neighbors is greater than total atoms, then each atom is a
+            # neighbor.
+            if (self.n_neighbors + 1) > len(mol.GetAtoms()):
+                neighbor_end_index = len(mol.GetAtoms())
+            else:
+                neighbor_end_index = (self.n_neighbors + 1)
+
+            distance_atom = distance_matrix[n, :]
+            cutoff_end_index = distance_atom[distance_atom < self.cutoff].size
+
+            end_index = min(neighbor_end_index, cutoff_end_index)
+
+            # Loop over each of the nearest neighbors
+
+            neighbor_inds = distance_matrix[n, :].argsort()[1:end_index]
+            if len(neighbor_inds)==0: neighbor_inds = [n]
+            for neighbor in neighbor_inds:
+
+                # update bond feature matrix
+                bond = mol.GetBondBetweenAtoms(n, int(neighbor))
+                if bond is None:
+                    bond_feature_matrix[bond_index] = 0
+                else:
+                    rev = False if bond.GetBeginAtomIdx() == n else True
+                    bond_feature_matrix[bond_index] = self.bond_tokenizer(
+                        self.bond_features(bond, flipped=rev))
+
+                distance = distance_matrix[n, neighbor]
+                bond_distance_matrix[bond_index] = distance
+
+                # update connectivity matrix
+                connectivity[bond_index, 0] = n
+                connectivity[bond_index, 1] = neighbor
+
+                bond_index += 1
+        print(connectivity)
+
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'distance': bond_distance_matrix,
+            'connectivity': connectivity,
+        }
+
+
+class MolBPreprocessor(MolPreprocessor):
+    """
+    This is a subclass of Molpreprocessor that preprocessor molecule with
+    bond property target
+    """
+    def __init__(self, **kwargs):
+        """
+        A preprocessor class that also returns bond_target_matrix, besides the bond matrix
+        returned by MolPreprocessor. The bond_target_matrix is then used as ref to reduce molecule
+        to bond property
+        """
+        super(MolBPreprocessor, self).__init__(**kwargs)
+
+    def construct_feature_matrices(self, entry):
+        """
+        Given an entry contining rdkit molecule, bond_index and for the target property,
+        return atom
+        feature matrices, bond feature matrices, distance matrices, connectivity matrices and bond
+        ref matrices.
+
+        returns
+        dict with entries
+        see MolPreproccessor
+        'bond_index' : ref array to the bond index
+        """
+        mol, bond_index_array = entry
+
+        n_atom = len(mol.GetAtoms())
+        n_pro = len(bond_index_array)
+
+        # n_bond is actually the number of atom-atom pairs, so this is defined
+        # by the number of neighbors for each atom.
+        #if there is cutoff,
+        distance_matrix = Chem.Get3DDistanceMatrix(mol)
+
+        if self.n_neighbors <= (n_atom - 1):
+            n_bond = self.n_neighbors * n_atom
+        else:
+            # If there are fewer atoms than n_neighbors, all atoms will be
+            # connected
+            n_bond = distance_matrix[(distance_matrix < self.cutoff) & (distance_matrix != 0)].size
+
+        if n_bond == 0: n_bond = 1
+
+        # Initialize the matrices to be filled in during the following loop.
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        bond_distance_matrix = np.zeros(n_bond, dtype=np.float32)
+        bond_index_matrix = np.full(n_bond, -1, dtype='int')
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        # Hopefully we've filtered out all problem mols by now.
+        if mol is None:
+            raise RuntimeError("Issue in loading mol")
+
+        # Get a list of the atoms in the molecule.
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        # Here we loop over each atom, and the inner loop iterates over each
+        # neighbor of the current atom.
+        bond_index = 0  # keep track of our current bond.
+        for n, atom in enumerate(atoms):
+            # update atom feature matrix
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+
+            # if n_neighbors is greater than total atoms, then each atom is a
+            # neighbor.
+            if (self.n_neighbors + 1) > len(mol.GetAtoms()):
+                neighbor_end_index = len(mol.GetAtoms())
+            else:
+                neighbor_end_index = (self.n_neighbors + 1)
+
+            distance_atom = distance_matrix[n, :]
+            cutoff_end_index = distance_atom[distance_atom < self.cutoff].size
+
+            end_index = min(neighbor_end_index, cutoff_end_index)
+
+            # Loop over each of the nearest neighbors
+
+            neighbor_inds = distance_matrix[n, :].argsort()[1:end_index]
+            if len(neighbor_inds)==0: neighbor_inds = [n]
+            for neighbor in neighbor_inds:
+
+                # update bond feature matrix
+                bond = mol.GetBondBetweenAtoms(n, int(neighbor))
+                if bond is None:
+                    bond_feature_matrix[bond_index] = 0
+                else:
+                    rev = False if bond.GetBeginAtomIdx() == n else True
+                    bond_feature_matrix[bond_index] = self.bond_tokenizer(
+                        self.bond_features(bond, flipped=rev))
+                    try:
+                        bond_index_matrix[bond_index] = bond_index_array.tolist().index(bond.GetIdx())
+                    except:
+                        pass
+
+                distance = distance_matrix[n, neighbor]
+                bond_distance_matrix[bond_index] = distance
+
+                # update connectivity matrix
+                connectivity[bond_index, 0] = n
+                connectivity[bond_index, 1] = neighbor
+
+                bond_index += 1
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'n_pro': n_pro,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'distance': bond_distance_matrix,
+            'connectivity': connectivity,
+            'bond_index': bond_index_matrix,
+        }
+
+class MolAPreprocessor(MolPreprocessor):
+    """
+    This is a subclass of Molpreprocessor that preprocessor molecule with
+    bond property target
+    """
+    def __init__(self, **kwargs):
+        """
+        A preprocessor class that also returns bond_target_matrix, besides the bond matrix
+        returned by MolPreprocessor. The bond_target_matrix is then used as ref to reduce molecule
+        to bond property
+        """
+        super(MolAPreprocessor, self).__init__(**kwargs)
+
+    def construct_feature_matrices(self, entry):
+        """
+        Given an entry contining rdkit molecule, bond_index and for the target property,
+        return atom
+        feature matrices, bond feature matrices, distance matrices, connectivity matrices and bond
+        ref matrices.
+
+        returns
+        dict with entries
+        see MolPreproccessor
+        'bond_index' : ref array to the bond index
+        """
+        mol, atom_index_array = entry
+
+        n_atom = len(mol.GetAtoms())
+        n_pro = len(atom_index_array)
+
+        # n_bond is actually the number of atom-atom pairs, so this is defined
+        # by the number of neighbors for each atom.
+        #if there is cutoff,
+        distance_matrix = Chem.Get3DDistanceMatrix(mol)
+
+        if self.n_neighbors <= (n_atom - 1):
+            n_bond = self.n_neighbors * n_atom
+        else:
+            # If there are fewer atoms than n_neighbors, all atoms will be
+            # connected
+            n_bond = distance_matrix[(distance_matrix < self.cutoff) & (distance_matrix != 0)].size
+
+        if n_bond == 0: n_bond = 1
+
+        # Initialize the matrices to be filled in during the following loop.
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        bond_distance_matrix = np.zeros(n_bond, dtype=np.float32)
+        atom_index_matrix = np.full(n_atom, -1, dtype='int')
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        # Hopefully we've filtered out all problem mols by now.
+        if mol is None:
+            raise RuntimeError("Issue in loading mol")
+
+        # Get a list of the atoms in the molecule.
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        # Here we loop over each atom, and the inner loop iterates over each
+        # neighbor of the current atom.
+        bond_index = 0  # keep track of our current bond.
+        for n, atom in enumerate(atoms):
+            # update atom feature matrix
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+            try:
+                atom_index_matrix[n] = atom_index_array.tolist().index(atom.GetIdx())
+            except:
+                pass
+            # if n_neighbors is greater than total atoms, then each atom is a
+            # neighbor.
+            if (self.n_neighbors + 1) > len(mol.GetAtoms()):
+                neighbor_end_index = len(mol.GetAtoms())
+            else:
+                neighbor_end_index = (self.n_neighbors + 1)
+
+            distance_atom = distance_matrix[n, :]
+            cutoff_end_index = distance_atom[distance_atom < self.cutoff].size
+
+            end_index = min(neighbor_end_index, cutoff_end_index)
+
+            # Loop over each of the nearest neighbors
+
+            neighbor_inds = distance_matrix[n, :].argsort()[1:end_index]
+            if len(neighbor_inds)==0: neighbor_inds = [n]
+            for neighbor in neighbor_inds:
+
+                # update bond feature matrix
+                bond = mol.GetBondBetweenAtoms(n, int(neighbor))
+                if bond is None:
+                    bond_feature_matrix[bond_index] = 0
+                else:
+                    rev = False if bond.GetBeginAtomIdx() == n else True
+                    bond_feature_matrix[bond_index] = self.bond_tokenizer(
+                        self.bond_features(bond, flipped=rev))
+
+                distance = distance_matrix[n, neighbor]
+                bond_distance_matrix[bond_index] = distance
+
+                # update connectivity matrix
+                connectivity[bond_index, 0] = n
+                connectivity[bond_index, 1] = neighbor
+
+                bond_index += 1
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'n_pro': n_pro,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'distance': bond_distance_matrix,
+            'connectivity': connectivity,
+            'atom_index': atom_index_matrix,
+        }
+
+
+# TODO: rewrite this
+# class LaplacianSmilesPreprocessor(SmilesPreprocessor):
+#     """ Extends the SmilesPreprocessor class to also return eigenvalues and
+#     eigenvectors of the graph laplacian matrix.
+#
+#     Example:
+#     >>> preprocessor = SmilesPreprocessor(
+#     >>>     max_atoms=55, max_bonds=62, max_degree=4, explicit_hs=False)
+#     >>> atom, connectivity, eigenvalues, eigenvectors = preprocessor.fit(
+#             data.smiles)
+#     """
+#
+#     def preprocess(self, smiles_iterator, train=True):
+#
+#         self.atom_tokenizer.train = train
+#         self.bond_tokenizer.train = train
+#
+#         for smiles in tqdm(smiles_iterator):
+#             G = self._mol_to_nx(smiles)
+#             A = self._get_atom_feature_matrix(G)
+#             C = self._get_connectivity_matrix(G)
+#             W, V = self._get_laplacian_spectral_decomp(G)
+#             yield A, C, W, V
+#
+#
+#     def _get_laplacian_spectral_decomp(self, G):
+#         """ Return the eigenvalues and eigenvectors of the graph G, padded to
+#         `self.max_atoms`.
+#         """
+#
+#         w0 = np.zeros((self.max_atoms, 1))
+#         v0 = np.zeros((self.max_atoms, self.max_atoms))
+#
+#         w, v = eigh(nx.laplacian_matrix(G).todense())
+#
+#         num_atoms = len(v)
+#
+#         w0[:num_atoms, 0] = w
+#         v0[:num_atoms, :num_atoms] = v
+#
+#         return w0, v0
+#
+#
+#     def fit(self, smiles_iterator):
+#         results = self._fit(smiles_iterator)
+#         return {'atom': results[0],
+#                 'connectivity': results[1],
+#                 'w': results[2],
+#                 'v': results[3]}
+#
+#
+#     def predict(self, smiles_iterator):
+#         results = self._predict(smiles_iterator)
+#         return {'atom': results[0],
+#                 'connectivity': results[1],
+#                 'w': results[2],
+#                 'v': results[3]}
+
+
+def get_max_atom_bond_size(smiles_iterator, explicit_hs=True):
+    """ Convienence function to get max_atoms, max_bonds for a set of input
+    SMILES """
+
+    max_atoms = 0
+    max_bonds = 0
+    for smiles in tqdm(smiles_iterator):
+        mol = MolFromSmiles(smiles)
+        if explicit_hs:
+            mol = AddHs(mol)
+        max_atoms = max([max_atoms, len(mol.GetAtoms())])
+        max_bonds = max([max_bonds, len(mol.GetBonds())])
+
+    return dict(max_atoms=max_atoms, max_bonds=max_bonds*2)
+
+
+def canonicalize_smiles(smiles, isomeric=True, sanitize=True):
+    try:
+        mol = MolFromSmiles(smiles, sanitize=sanitize)
+        return MolToSmiles(mol, isomericSmiles=isomeric)
+    except Exception:
+        pass

nfp/preprocessing/scaling.py

@@ -0,0 +1,40 @@
+import numpy as np
+
+from sklearn.preprocessing import RobustScaler
+from scipy import sparse
+from sklearn.utils import check_array
+from sklearn.utils.validation import FLOAT_DTYPES
+from sklearn.preprocessing.data import _handle_zeros_in_scale
+
+class RobustNanScaler(RobustScaler):
+    
+    def _check_array(self, X, copy):
+        """Makes sure centering is not enabled for sparse matrices."""
+        X = check_array(X, accept_sparse=('csr', 'csc'), copy=self.copy,
+                        estimator=self, dtype=FLOAT_DTYPES, force_all_finite=False)
+
+        if sparse.issparse(X):
+            if self.with_centering:
+                raise ValueError(
+                    "Cannot center sparse matrices: use `with_centering=False`"
+                    " instead. See docstring for motivation and alternatives.")
+        return X
+
+    
+    def fit(self, X, y=None):
+        if sparse.issparse(X):
+            raise TypeError("RobustScaler cannot be fitted on sparse inputs")
+        X = self._check_array(X, self.copy)
+        if self.with_centering:
+            self.center_ = np.nanmedian(X, axis=0)
+
+        if self.with_scaling:
+            q_min, q_max = self.quantile_range
+            if not 0 <= q_min <= q_max <= 100:
+                raise ValueError("Invalid quantile range: %s" %
+                                 str(self.quantile_range))
+
+            q = np.nanpercentile(X, self.quantile_range, axis=0)
+            self.scale_ = (q[1] - q[0])
+            self.scale_ = _handle_zeros_in_scale(self.scale_, copy=False)
+        return self

nfp/preprocessing/sequence.py

@@ -0,0 +1,115 @@
+import numpy as np
+from random import shuffle
+
+from keras.utils import Sequence
+
+
+class GraphSequence(Sequence):
+    
+    def __init__(self, inputs, y=None, batch_size=1, shuffle=True,
+                 final_batch=True):
+        """ A keras.Sequence generator to be passed to model.fit_generator. (or
+        any other *_generator method.) Returns (inputs, y) tuples where
+        molecule feature matrices have been stitched together. Offsets the
+        connectivity matrices such that atoms are indexed appropriately.
+
+        batch_size: number of molecules per batch
+        shuffle : whether to shuffle the input data
+        final_batch : whether to include the final, incomplete batch
+
+        """
+        self._inputs = inputs
+        self._y = np.asarray(y) if y is not None else None
+        self._input_keys = list(inputs[0].keys())
+        self.batch_size = batch_size
+        self.shuffle = shuffle
+        self.final_batch = final_batch
+        
+
+    def __len__(self):
+        """ Total number of batches """
+        if self.final_batch:
+            return int(np.ceil(len(self._inputs) / float(self.batch_size)))
+        else:
+            return int(np.floor(len(self._inputs) / float(self.batch_size)))
+
+    def on_epoch_end(self):
+        if self.shuffle:
+            indices = np.arange(0, len(self._inputs))
+            np.random.shuffle(indices)
+            self._inputs = [self._inputs[i] for i in indices]
+            if self._y is not None:
+                self._y = self._y[indices]
+    
+    def __getitem__(self, idx):
+        """ Calculate the feature matrices for a whole batch (with index `i` <
+        self.__len__). This involves adding offsets to the indices for each
+        atom in the connectivity matrix; such that atoms and bonds in later
+        molecules still refer to the correct atoms.
+
+        """
+        batch_indexes = idx * self.batch_size + np.arange(0, self.batch_size)
+        batch_indexes = batch_indexes[batch_indexes < len(self._inputs)]
+
+        batch_data = {
+            key: self._concat([self._inputs[i][key] for i in batch_indexes])
+            for key in self._input_keys}
+    
+        # Offset the connectivity matrix to account for the multiple graphs per
+        # batch
+        offset = _compute_stacked_offsets(
+            batch_data['n_atom'], batch_data['n_bond'])
+
+        batch_data['connectivity'] += offset[:, np.newaxis]
+        
+        # Compute graph indices with shape (n_atom,) that indicate to which
+        # molecule each atom belongs.
+        n_graphs = len(batch_indexes)
+        batch_data['node_graph_indices'] = np.repeat(
+            np.arange(n_graphs), batch_data['n_atom'])
+
+        batch_data = self.process_data(batch_data)
+
+        # Keras takes to options, one (x, y) pairs, or just (x,) pairs if we're
+        # doing predictions. Here, if we've specified a y matrix, we return the
+        # x,y pairs for training, otherwise just return the x data.
+        if self._y is not None:
+            return (batch_data, np.concatenate(self._y[batch_indexes]).reshape(-1,1))
+
+        else:
+            return batch_data
+        
+    def process_data(self, batch_data):
+        """ function to add additional processing to batch data before returning """
+
+        # These aren't used currently, so I pop them. But we might need them at
+        # a later time.
+        del batch_data['n_atom']
+        del batch_data['n_bond']
+        
+        return batch_data
+
+    
+    def _concat(self, to_stack):
+        """ function to stack (or concatentate) depending on dimensions """
+
+        if np.asarray(to_stack[0]).ndim >= 2:
+            return np.concatenate(to_stack)
+        
+        else:
+            return np.hstack(to_stack)
+
+
+def _compute_stacked_offsets(sizes, repeats):
+    """ Computes offsets to add to indices of stacked np arrays.
+    When a set of np arrays are stacked, the indices of those from the second on
+    must be offset in order to be able to index into the stacked np array. This
+    computes those offsets.
+
+    Args:
+        sizes: A 1D sequence of np arrays of the sizes per graph.
+        repeats: A 1D sequence of np arrays of the number of repeats per graph.
+    Returns:
+        The index offset per graph.
+    """
+    return np.repeat(np.cumsum(np.hstack([0, sizes[:-1]])), repeats)

nfp/preprocessing/test.py

@@ -0,0 +1,734 @@
+import logging, sys
+
+import numpy as np
+from tqdm import tqdm
+from scipy.linalg import eigh
+
+from rdkit import Chem
+from rdkit.Chem import MolFromSmiles, MolToSmiles, AddHs
+
+from nfp.preprocessing import features
+from nfp.preprocessing.features import Tokenizer
+import time
+
+
+class SmilesPreprocessor(object):
+    """ Given a list of SMILES strings, encode these molecules as atom and
+    connectivity feature matricies.
+
+    Example:
+    >>> preprocessor = SmilesPreprocessor(explicit_hs=False)
+    >>> inputs = preprocessor.fit(data.smiles)
+    """
+
+    def __init__(self, explicit_hs=True, atom_features=None, bond_features=None):
+        """
+
+        explicit_hs : bool
+            whether to tell RDkit to add H's to a molecule.
+        atom_features : function
+            A function applied to an rdkit.Atom that returns some
+            representation (i.e., string, integer) for the Tokenizer class.
+        bond_features : function
+            A function applied to an rdkit Bond to return some description.
+
+        """
+
+        self.atom_tokenizer = Tokenizer()
+        self.bond_tokenizer = Tokenizer()
+        self.explicit_hs = explicit_hs
+
+        if atom_features is None:
+            atom_features = features.atom_features_v1
+
+        if bond_features is None:
+            bond_features = features.bond_features_v1
+
+        self.atom_features = atom_features
+        self.bond_features = bond_features
+
+
+    def fit(self, smiles_iterator):
+        """ Fit an iterator of SMILES strings, creating new atom and bond
+        tokens for unseen molecules. Returns a dictionary with 'atom' and
+        'connectivity' entries """
+        return list(self.preprocess(smiles_iterator, train=True))
+
+
+    def predict(self, smiles_iterator):
+        """ Uses previously determined atom and bond tokens to convert a SMILES
+        iterator into 'atom' and 'connectivity' matrices. Ensures that atom and
+        bond classes commute with previously determined results. """
+        return list(self.preprocess(smiles_iterator, train=False))
+
+
+    def preprocess(self, smiles_iterator, train=True):
+
+        self.atom_tokenizer.train = train
+        self.bond_tokenizer.train = train
+
+        for smiles in tqdm(smiles_iterator):
+            yield self.construct_feature_matrices(smiles)
+
+
+    @property
+    def atom_classes(self):
+        """ The number of atom types found (includes the 0 null-atom type) """
+        return self.atom_tokenizer.num_classes + 1
+
+
+    @property
+    def bond_classes(self):
+        """ The number of bond types found (includes the 0 null-bond type) """
+        return self.bond_tokenizer.num_classes + 1
+
+
+    def construct_feature_matrices(self, smiles):
+        """ construct a molecule from the given smiles string and return atom
+        and bond classes.
+
+        Returns
+        dict with entries
+        'n_atom' : number of atoms in the molecule
+        'n_bond' : number of bonds in the molecule 
+        'atom' : (n_atom,) length list of atom classes
+        'bond' : (n_bond,) list of bond classes
+        'connectivity' : (n_bond, 2) array of source atom, target atom pairs.
+
+        """
+
+        mol = MolFromSmiles(smiles)
+        if self.explicit_hs:
+            mol = AddHs(mol)
+
+        n_atom = len(mol.GetAtoms())
+        n_bond = 2 * len(mol.GetBonds())
+
+        # If its an isolated atom, add a self-link
+        if n_bond == 0:
+            n_bond = 1
+        
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        bond_index = 0
+
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        for n, atom in enumerate(atoms):
+
+            # Atom Classes
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+
+            start_index = atom.GetIdx()
+
+            for bond in atom.GetBonds():
+                # Is the bond pointing at the target atom
+                rev = bond.GetBeginAtomIdx() != start_index
+
+                # Bond Classes
+                bond_feature_matrix[n] = self.bond_tokenizer(
+                    self.bond_features(bond, flipped=rev))
+
+                # Connectivity
+                if not rev:  # Original direction
+                    connectivity[bond_index, 0] = bond.GetBeginAtomIdx()
+                    connectivity[bond_index, 1] = bond.GetEndAtomIdx()
+
+                else:  # Reversed
+                    connectivity[bond_index, 0] = bond.GetEndAtomIdx()
+                    connectivity[bond_index, 1] = bond.GetBeginAtomIdx()
+
+                bond_index += 1
+
+
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'connectivity': connectivity,
+        }
+    
+
+class ConnectivityAPreprocessor(object):
+    """ Given a list of SMILES strings, encode these molecules as atom and
+    connectivity feature matricies.
+
+    Example:
+    >>> preprocessor = SmilesPreprocessor(explicit_hs=False)
+    >>> inputs = preprocessor.fit(data.smiles)
+    """
+
+    def __init__(self, explicit_hs=True, atom_features=None, bond_features=None):
+        """
+
+        explicit_hs : bool
+            whether to tell RDkit to add H's to a molecule.
+        atom_features : function
+            A function applied to an rdkit.Atom that returns some
+            representation (i.e., string, integer) for the Tokenizer class.
+        bond_features : function
+            A function applied to an rdkit Bond to return some description.
+
+        """
+
+        self.atom_tokenizer = Tokenizer()
+        self.bond_tokenizer = Tokenizer()
+        self.explicit_hs = explicit_hs
+
+        if atom_features is None:
+            atom_features = features.atom_features_v1
+
+        if bond_features is None:
+            bond_features = features.bond_features_v1
+
+        self.atom_features = atom_features
+        self.bond_features = bond_features
+
+
+    def fit(self, smiles_iterator):
+        """ Fit an iterator of SMILES strings, creating new atom and bond
+        tokens for unseen molecules. Returns a dictionary with 'atom' and
+        'connectivity' entries """
+        return list(self.preprocess(smiles_iterator, train=True))
+
+
+    def predict(self, smiles_iterator):
+        """ Uses previously determined atom and bond tokens to convert a SMILES
+        iterator into 'atom' and 'connectivity' matrices. Ensures that atom and
+        bond classes commute with previously determined results. """
+        return list(self.preprocess(smiles_iterator, train=False))
+
+
+    def preprocess(self, smiles_iterator, train=True):
+
+        self.atom_tokenizer.train = train
+        self.bond_tokenizer.train = train
+
+        for smiles in tqdm(smiles_iterator):
+            yield self.construct_feature_matrices(smiles)
+
+
+    @property
+    def atom_classes(self):
+        """ The number of atom types found (includes the 0 null-atom type) """
+        return self.atom_tokenizer.num_classes + 1
+
+
+    @property
+    def bond_classes(self):
+        """ The number of bond types found (includes the 0 null-bond type) """
+        return self.bond_tokenizer.num_classes + 1
+
+
+    def construct_feature_matrices(self, smiles):
+        """ construct a molecule from the given smiles string and return atom
+        and bond classes.
+
+        Returns
+        dict with entries
+        'n_atom' : number of atoms in the molecule
+        'n_bond' : number of bonds in the molecule 
+        'atom' : (n_atom,) length list of atom classes
+        'bond' : (n_bond,) list of bond classes
+        'connectivity' : (n_bond, 2) array of source atom, target atom pairs.
+
+        """
+
+        mol = MolFromSmiles(smiles)
+        if self.explicit_hs:
+            mol = AddHs(mol)
+
+        n_atom = len(mol.GetAtoms())
+        n_bond = 2 * len(mol.GetBonds())
+
+        # If its an isolated atom, add a self-link
+        if n_bond == 0:
+            n_bond = 1
+        
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        bond_index = 0
+
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        for n, atom in enumerate(atoms):
+
+            # Atom Classes
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+
+            start_index = atom.GetIdx()
+
+            for bond in atom.GetBonds():
+                # Is the bond pointing at the target atom
+                rev = bond.GetBeginAtomIdx() != start_index
+
+                # Bond Classes
+                bond_feature_matrix[n] = self.bond_tokenizer(
+                    self.bond_features(bond, flipped=rev))
+
+                # Connectivity
+                if not rev:  # Original direction
+                    connectivity[bond_index, 0] = bond.GetBeginAtomIdx()
+                    connectivity[bond_index, 1] = bond.GetEndAtomIdx()
+
+                else:  # Reversed
+                    connectivity[bond_index, 0] = bond.GetEndAtomIdx()
+                    connectivity[bond_index, 1] = bond.GetBeginAtomIdx()
+
+                bond_index += 1
+
+
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'connectivity': connectivity,
+        }
+
+
+class MolPreprocessor(SmilesPreprocessor):
+    """ I should refactor this into a base class and separate
+    SmilesPreprocessor classes. But the idea is that we only need to redefine
+    the `construct_feature_matrices` method to have a working preprocessor that
+    handles 3D structures. 
+
+    We'll pass an iterator of mol objects instead of SMILES strings this time,
+    though.
+    
+    """
+
+    def __init__(self, n_neighbors, cutoff, **kwargs):
+        """ A preprocessor class that also returns distances between
+        neighboring atoms. Adds edges for non-bonded atoms to include a maximum
+        of n_neighbors around each atom """
+
+        self.n_neighbors = n_neighbors
+        self.cutoff = cutoff
+        super(MolPreprocessor, self).__init__(**kwargs)
+
+
+    def construct_feature_matrices(self, mol):
+        """ Given an rdkit mol, return atom feature matrices, bond feature
+        matrices, and connectivity matrices.
+
+        Returns
+        dict with entries
+        'n_atom' : number of atoms in the molecule
+        'n_bond' : number of edges (likely n_atom * n_neighbors)
+        'atom' : (n_atom,) length list of atom classes
+        'bond' : (n_bond,) list of bond classes. 0 for no bond
+        'distance' : (n_bond,) list of bond distances
+        'connectivity' : (n_bond, 2) array of source atom, target atom pairs.
+            
+        """
+
+        n_atom = len(mol.GetAtoms())
+
+        # n_bond is actually the number of atom-atom pairs, so this is defined
+        # by the number of neighbors for each atom.
+        #if there is cutoff, 
+        distance_matrix = Chem.Get3DDistanceMatrix(mol)
+
+        if self.n_neighbors <= (n_atom - 1):
+            n_bond = self.n_neighbors * n_atom
+        else:
+            # If there are fewer atoms than n_neighbors, all atoms will be
+            # connected
+            n_bond = distance_matrix[(distance_matrix < self.cutoff) & (distance_matrix != 0)].size
+
+        if n_bond == 0: n_bond = 1
+
+        # Initialize the matrices to be filled in during the following loop.
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        bond_distance_matrix = np.zeros(n_bond, dtype=np.float32)
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        # Hopefully we've filtered out all problem mols by now.
+        if mol is None:
+            raise RuntimeError("Issue in loading mol")
+        
+        # Get a list of the atoms in the molecule.
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        # Here we loop over each atom, and the inner loop iterates over each
+        # neighbor of the current atom.
+        bond_index = 0  # keep track of our current bond.
+        for n, atom in enumerate(atoms):
+            
+            # update atom feature matrix
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+            
+            # if n_neighbors is greater than total atoms, then each atom is a
+            # neighbor.
+            if (self.n_neighbors + 1) > len(mol.GetAtoms()):
+                neighbor_end_index = len(mol.GetAtoms())
+            else:
+                neighbor_end_index = (self.n_neighbors + 1)
+
+            distance_atom = distance_matrix[n, :]
+            cutoff_end_index = distance_atom[distance_atom < self.cutoff].size
+
+            end_index = min(neighbor_end_index, cutoff_end_index)
+
+            # Loop over each of the nearest neighbors
+
+            neighbor_inds = distance_matrix[n, :].argsort()[1:end_index]
+            if len(neighbor_inds)==0: neighbor_inds = [n]
+            for neighbor in neighbor_inds:
+                
+                # update bond feature matrix
+                bond = mol.GetBondBetweenAtoms(n, int(neighbor))
+                if bond is None:
+                    bond_feature_matrix[bond_index] = 0
+                else:
+                    rev = False if bond.GetBeginAtomIdx() == n else True
+                    bond_feature_matrix[bond_index] = self.bond_tokenizer(
+                        self.bond_features(bond, flipped=rev))
+
+                distance = distance_matrix[n, neighbor]
+                bond_distance_matrix[bond_index] = distance
+                
+                # update connectivity matrix
+                connectivity[bond_index, 0] = n
+                connectivity[bond_index, 1] = neighbor
+                
+                bond_index += 1
+
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'distance': bond_distance_matrix,
+            'connectivity': connectivity,
+        }
+
+
+class MolBPreprocessor(MolPreprocessor):
+    """
+    This is a subclass of Molpreprocessor that preprocessor molecule with
+    bond property target
+    """
+    def __init__(self, **kwargs):
+        """
+        A preprocessor class that also returns bond_target_matrix, besides the bond matrix
+        returned by MolPreprocessor. The bond_target_matrix is then used as ref to reduce molecule
+        to bond property
+        """
+        super(MolBPreprocessor, self).__init__(**kwargs)
+
+    def construct_feature_matrices(self, entry):
+        """
+        Given an entry contining rdkit molecule, bond_index and for the target property, 
+        return atom 
+        feature matrices, bond feature matrices, distance matrices, connectivity matrices and bond
+        ref matrices.
+
+        returns
+        dict with entries
+        see MolPreproccessor
+        'bond_index' : ref array to the bond index
+        """
+        mol, bond_index_array = entry
+        
+        n_atom = len(mol.GetAtoms())
+        n_pro = len(bond_index_array)
+
+        # n_bond is actually the number of atom-atom pairs, so this is defined
+        # by the number of neighbors for each atom.
+        #if there is cutoff, 
+        distance_matrix = Chem.Get3DDistanceMatrix(mol)
+
+        if self.n_neighbors <= (n_atom - 1):
+            n_bond = self.n_neighbors * n_atom
+        else:
+            # If there are fewer atoms than n_neighbors, all atoms will be
+            # connected
+            n_bond = distance_matrix[(distance_matrix < self.cutoff) & (distance_matrix != 0)].size
+
+        if n_bond == 0: n_bond = 1
+
+        # Initialize the matrices to be filled in during the following loop.
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        bond_distance_matrix = np.zeros(n_bond, dtype=np.float32)
+        bond_index_matrix = np.full(n_bond, -1, dtype='int')
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        # Hopefully we've filtered out all problem mols by now.
+        if mol is None:
+            raise RuntimeError("Issue in loading mol")
+        
+        # Get a list of the atoms in the molecule.
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        # Here we loop over each atom, and the inner loop iterates over each
+        # neighbor of the current atom.
+        bond_index = 0  # keep track of our current bond.
+        for n, atom in enumerate(atoms):
+            # update atom feature matrix
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+            
+            # if n_neighbors is greater than total atoms, then each atom is a
+            # neighbor.
+            if (self.n_neighbors + 1) > len(mol.GetAtoms()):
+                neighbor_end_index = len(mol.GetAtoms())
+            else:
+                neighbor_end_index = (self.n_neighbors + 1)
+
+            distance_atom = distance_matrix[n, :]
+            cutoff_end_index = distance_atom[distance_atom < self.cutoff].size
+
+            end_index = min(neighbor_end_index, cutoff_end_index)
+
+            # Loop over each of the nearest neighbors
+
+            neighbor_inds = distance_matrix[n, :].argsort()[1:end_index]
+            if len(neighbor_inds)==0: neighbor_inds = [n]
+            for neighbor in neighbor_inds:
+                
+                # update bond feature matrix
+                bond = mol.GetBondBetweenAtoms(n, int(neighbor))
+                if bond is None:
+                    bond_feature_matrix[bond_index] = 0
+                else:
+                    rev = False if bond.GetBeginAtomIdx() == n else True
+                    bond_feature_matrix[bond_index] = self.bond_tokenizer(
+                        self.bond_features(bond, flipped=rev))
+                    try:
+                        bond_index_matrix[bond_index] = bond_index_array.tolist().index(bond.GetIdx())
+                    except:
+                        pass
+
+                distance = distance_matrix[n, neighbor]
+                bond_distance_matrix[bond_index] = distance
+                 
+                # update connectivity matrix
+                connectivity[bond_index, 0] = n
+                connectivity[bond_index, 1] = neighbor
+                
+                bond_index += 1
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'n_pro': n_pro,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'distance': bond_distance_matrix,
+            'connectivity': connectivity,
+            'bond_index': bond_index_matrix,
+        }
+
+class MolAPreprocessor(MolPreprocessor):
+    """
+    This is a subclass of Molpreprocessor that preprocessor molecule with
+    bond property target
+    """
+    def __init__(self, **kwargs):
+        """
+        A preprocessor class that also returns bond_target_matrix, besides the bond matrix
+        returned by MolPreprocessor. The bond_target_matrix is then used as ref to reduce molecule
+        to bond property
+        """
+        super(MolAPreprocessor, self).__init__(**kwargs)
+
+    def construct_feature_matrices(self, entry):
+        """
+        Given an entry contining rdkit molecule, bond_index and for the target property, 
+        return atom 
+        feature matrices, bond feature matrices, distance matrices, connectivity matrices and bond
+        ref matrices.
+
+        returns
+        dict with entries
+        see MolPreproccessor
+        'bond_index' : ref array to the bond index
+        """
+        mol, atom_index_array = entry
+        
+        n_atom = len(mol.GetAtoms())
+        n_pro = len(atom_index_array)
+
+        # n_bond is actually the number of atom-atom pairs, so this is defined
+        # by the number of neighbors for each atom.
+        #if there is cutoff, 
+        distance_matrix = Chem.Get3DDistanceMatrix(mol)
+
+        if self.n_neighbors <= (n_atom - 1):
+            n_bond = self.n_neighbors * n_atom
+        else:
+            # If there are fewer atoms than n_neighbors, all atoms will be
+            # connected
+            n_bond = distance_matrix[(distance_matrix < self.cutoff) & (distance_matrix != 0)].size
+
+        if n_bond == 0: n_bond = 1
+
+        # Initialize the matrices to be filled in during the following loop.
+        atom_feature_matrix = np.zeros(n_atom, dtype='int')
+        bond_feature_matrix = np.zeros(n_bond, dtype='int')
+        bond_distance_matrix = np.zeros(n_bond, dtype=np.float32)
+        atom_index_matrix = np.full(n_atom, -1, dtype='int')
+        connectivity = np.zeros((n_bond, 2), dtype='int')
+
+        # Hopefully we've filtered out all problem mols by now.
+        if mol is None:
+            raise RuntimeError("Issue in loading mol")
+        
+        # Get a list of the atoms in the molecule.
+        atom_seq = mol.GetAtoms()
+        atoms = [atom_seq[i] for i in range(n_atom)]
+
+        # Here we loop over each atom, and the inner loop iterates over each
+        # neighbor of the current atom.
+        bond_index = 0  # keep track of our current bond.
+        for n, atom in enumerate(atoms):
+            # update atom feature matrix
+            atom_feature_matrix[n] = self.atom_tokenizer(
+                self.atom_features(atom))
+            try:
+                atom_index_matrix[n] = atom_index_array.tolist().index(atom.GetIdx())
+            except:
+                pass 
+            # if n_neighbors is greater than total atoms, then each atom is a
+            # neighbor.
+            if (self.n_neighbors + 1) > len(mol.GetAtoms()):
+                neighbor_end_index = len(mol.GetAtoms())
+            else:
+                neighbor_end_index = (self.n_neighbors + 1)
+
+            distance_atom = distance_matrix[n, :]
+            cutoff_end_index = distance_atom[distance_atom < self.cutoff].size
+
+            end_index = min(neighbor_end_index, cutoff_end_index)
+
+            # Loop over each of the nearest neighbors
+
+            neighbor_inds = distance_matrix[n, :].argsort()[1:end_index]
+            if len(neighbor_inds)==0: neighbor_inds = [n]
+            for neighbor in neighbor_inds:
+                
+                # update bond feature matrix
+                bond = mol.GetBondBetweenAtoms(n, int(neighbor))
+                if bond is None:
+                    bond_feature_matrix[bond_index] = 0
+                else:
+                    rev = False if bond.GetBeginAtomIdx() == n else True
+                    bond_feature_matrix[bond_index] = self.bond_tokenizer(
+                        self.bond_features(bond, flipped=rev))
+
+                distance = distance_matrix[n, neighbor]
+                bond_distance_matrix[bond_index] = distance
+                 
+                # update connectivity matrix
+                connectivity[bond_index, 0] = n
+                connectivity[bond_index, 1] = neighbor
+                
+                bond_index += 1
+        return {
+            'n_atom': n_atom,
+            'n_bond': n_bond,
+            'n_pro': n_pro,
+            'atom': atom_feature_matrix,
+            'bond': bond_feature_matrix,
+            'distance': bond_distance_matrix,
+            'connectivity': connectivity,
+            'atom_index': atom_index_matrix,
+        }
+
+
+# TODO: rewrite this                                
+# class LaplacianSmilesPreprocessor(SmilesPreprocessor):
+#     """ Extends the SmilesPreprocessor class to also return eigenvalues and
+#     eigenvectors of the graph laplacian matrix.
+#
+#     Example:
+#     >>> preprocessor = SmilesPreprocessor(
+#     >>>     max_atoms=55, max_bonds=62, max_degree=4, explicit_hs=False)
+#     >>> atom, connectivity, eigenvalues, eigenvectors = preprocessor.fit(
+#             data.smiles)
+#     """
+#
+#     def preprocess(self, smiles_iterator, train=True):
+#
+#         self.atom_tokenizer.train = train
+#         self.bond_tokenizer.train = train
+#
+#         for smiles in tqdm(smiles_iterator):
+#             G = self._mol_to_nx(smiles)
+#             A = self._get_atom_feature_matrix(G)
+#             C = self._get_connectivity_matrix(G)
+#             W, V = self._get_laplacian_spectral_decomp(G)
+#             yield A, C, W, V
+#
+#
+#     def _get_laplacian_spectral_decomp(self, G):
+#         """ Return the eigenvalues and eigenvectors of the graph G, padded to
+#         `self.max_atoms`.
+#         """
+#
+#         w0 = np.zeros((self.max_atoms, 1))
+#         v0 = np.zeros((self.max_atoms, self.max_atoms))
+#
+#         w, v = eigh(nx.laplacian_matrix(G).todense())
+#
+#         num_atoms = len(v)
+#
+#         w0[:num_atoms, 0] = w
+#         v0[:num_atoms, :num_atoms] = v
+#
+#         return w0, v0
+#
+#
+#     def fit(self, smiles_iterator):
+#         results = self._fit(smiles_iterator)
+#         return {'atom': results[0], 
+#                 'connectivity': results[1],
+#                 'w': results[2],
+#                 'v': results[3]}
+#
+#
+#     def predict(self, smiles_iterator):
+#         results = self._predict(smiles_iterator)
+#         return {'atom': results[0], 
+#                 'connectivity': results[1],
+#                 'w': results[2],
+#                 'v': results[3]}
+
+
+def get_max_atom_bond_size(smiles_iterator, explicit_hs=True):
+    """ Convienence function to get max_atoms, max_bonds for a set of input
+    SMILES """
+
+    max_atoms = 0
+    max_bonds = 0
+    for smiles in tqdm(smiles_iterator):
+        mol = MolFromSmiles(smiles)
+        if explicit_hs:
+            mol = AddHs(mol)
+        max_atoms = max([max_atoms, len(mol.GetAtoms())])
+        max_bonds = max([max_bonds, len(mol.GetBonds())])
+
+    return dict(max_atoms=max_atoms, max_bonds=max_bonds*2)
+
+
+def canonicalize_smiles(smiles, isomeric=True, sanitize=True):
+    try:
+        mol = MolFromSmiles(smiles, sanitize=sanitize)
+        return MolToSmiles(mol, isomericSmiles=isomeric)
+    except Exception:
+        pass

requirements.txt

@@ -0,0 +1,24 @@
+flask==2.0.3
+Werkzeug==2.0.3
+redis==3.5.3
+gunicorn==20.1.0
+
+# RDKit — last version with a Python 3.7 / manylinux amd64 wheel
+rdkit-pypi==2022.3.5
+
+# TF1 CPU — pinned to match the trained model
+tensorflow-cpu==1.15.0
+keras==2.2.5
+h5py==2.10.0
+
+# protobuf must be pinned — versions >=3.21 break TF1.15
+protobuf==3.20.3
+
+# Scientific stack — pinned for Python 3.7 / TF1 compatibility
+numpy==1.16.6
+pandas==0.25.3
+scipy==1.3.3
+scikit-learn==0.21.3
+tqdm==4.64.1
+
+# nfp is vendored locally — no PyPI entry needed

setup_assets.sh

@@ -0,0 +1,50 @@
+#!/bin/bash
+# CASCADE HF Space — Asset Setup Script
+# Run once from your terminal: bash setup_assets.sh
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+NOVA="$SCRIPT_DIR/../NNPredictor/webapp_NMR/Predictor"
+HF="$SCRIPT_DIR"
+
+echo "=== CASCADE HF Space Asset Setup ==="
+echo "Source: $NOVA"
+echo "Target: $HF"
+echo ""
+
+# ── 1. Python inference code (with preprocessor path fix) ──────────────────
+echo "[1/5] Copying inference code..."
+sed "s|os.path.join('cascade', 'preprocessor.p')|os.path.join('NMR_Prediction', 'preprocessor.p')|g" \
+    "$NOVA/NMR_Prediction/apply.py" > "$HF/NMR_Prediction/apply.py"
+cp "$NOVA/NMR_Prediction/genConf.py" "$HF/NMR_Prediction/genConf.py"
+echo "    apply.py   ✓  (preprocessor path fixed)"
+echo "    genConf.py ✓"
+
+# ── 2. Preprocessor pickle ─────────────────────────────────────────────────
+echo "[2/5] Copying preprocessor..."
+cp "$NOVA/NMR_Prediction/preprocessor.p" "$HF/NMR_Prediction/preprocessor.p"
+echo "    preprocessor.p ✓  ($(du -sh "$HF/NMR_Prediction/preprocessor.p" | cut -f1))"
+
+# ── 3. Model weights ───────────────────────────────────────────────────────
+echo "[3/5] Copying model weights..."
+mkdir -p "$HF/NMR_Prediction/schnet_edgeupdate"
+cp "$NOVA/NMR_Prediction/schnet_edgeupdate/best_model.hdf5" \
+   "$HF/NMR_Prediction/schnet_edgeupdate/best_model.hdf5"
+echo "    best_model.hdf5 ✓  ($(du -sh "$HF/NMR_Prediction/schnet_edgeupdate/best_model.hdf5" | cut -f1))"
+
+# ── 4. Vendored nfp package ────────────────────────────────────────────────
+echo "[4/5] Copying nfp package..."
+cp -r "$NOVA/nfp/." "$HF/nfp/"
+echo "    nfp/ ✓  ($(find "$HF/nfp" -type f | wc -l | tr -d ' ') files)"
+
+# ── 5. Static assets ───────────────────────────────────────────────────────
+echo "[5/5] Copying static assets..."
+cp -r "$NOVA/NMR_Prediction/static/NMR_Prediction/." "$HF/static/"
+echo "    static/ ✓  ($(find "$HF/static" -type f | wc -l | tr -d ' ') files)"
+
+echo ""
+echo "=== Setup complete! ==="
+echo "Total hf_space size: $(du -sh "$HF" | cut -f1)"
+echo ""
+echo "Next: docker build -t cascade-local . && docker run -p 7860:7860 cascade-local"
+echo "Then: open http://localhost:7860/cascade_v1/predict/"

start.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+set -e
+
+echo "Starting Redis..."
+redis-server --daemonize yes --loglevel warning
+
+echo "Starting CASCADE worker..."
+python worker.py &
+
+echo "Starting Flask app..."
+gunicorn app:app \
+    --bind 0.0.0.0:7860 \
+    --workers 2 \
+    --timeout 120 \
+    --log-level info

static/css/doc.css

@@ -0,0 +1,276 @@
+html{
+	font-family: "Titillium",  Cambria, Helvetica, Arial, Geneva, sans-serif !important;
+	padding: 20px 5% 20px 5% !important;
+	background: #29999c;
+}
+
+body {
+	clear: both;
+	box-shadow: 2px 2px 3px #444;
+	border-radius: 3px;
+	padding: 20px !important;
+	margin: 0px;
+	background: #f9f9f9;
+	min-width: 650px;
+}
+
+table {
+	border-collapse: collapse;
+}
+
+a, a:link, a:active{color:#29999C !important;}
+a:visited{color:#55299C !important;}
+a:hover{color:#3EB9BD !important;}
+
+
+h1 {
+	text-align: center;
+	color: #29999c;
+	background: white;
+	border-radius: 3px 3px 0px 0px;
+	margin: -20px -20px 0px -20px;
+	border-bottom: 1px solid #888;
+}
+
+h1, h1 a, h2 a{
+		font-family: "Titillium",  Cambria, Helvetica, Arial, Geneva, sans-serif !important;
+}
+
+.version {
+	font-family: sans-serif;
+	font-size: smaller;
+	float: right;
+}
+
+pre {
+	padding: 30px;
+	font-family: courier;
+	max-width: 70vw;
+	overflow-x: auto;
+}
+
+input {
+	border: 1px solid;
+}
+
+iframe{
+	border:none;
+	padding: 0px;
+	margin: 0px;
+    
+	box-shadow: 2px 2px 3px #888;
+}
+
+select {
+	border: 1px solid;
+}
+
+textarea {
+	font-family: monospace;
+}
+
+
+#molsource-box{
+	min-width: 30%;
+	width: 600px;
+	height: 350px;
+}
+.sketcher-frame {
+	margin: -2px;
+	resize: none;
+	min-width: 370px;
+	min-height: 370px;
+	height: 100%;
+	width: 100%;
+	overflow: hidden;
+}
+
+.resizable{
+	border: none;
+	padding: 0px 8px 8px 0px;
+	border: 0px;
+	resize: both;
+	width: 510px;
+	height: 510px;
+	min-width: 370px;
+	min-height: 370px;
+	overflow: auto;
+	background-color: transparent;
+}
+
+ul.horizontalmenu {
+	margin-top: 0px;
+	margin-bottom: 0px;
+	padding-top: 2px;
+	padding-bottom: 2px;
+	padding-left: 6px;
+	background-color: #dddddd;
+}
+
+ul.horizontalmenu li {
+	display: inline;
+	list-style-type: none;
+	padding-right: 6px;
+}
+
+input[type="checkbox"], input[type="button"] {
+	cursor: pointer;
+}
+
+input[type="number"] {
+	width: 70px;
+	text-indent: 2px;
+}
+
+.table-layout {
+	vertical-align: top;
+}
+
+.table-layout li {
+	display: table-row;
+}
+.table-layout li >* {
+	display: table-cell;
+	padding: 2px;
+}
+
+label {
+	cursor: pointer;
+}
+
+.darkbox {
+	display: inline-block;
+	margin-left: 10px;
+	margin-right: 10px;
+	background-color: transparent;
+}
+
+.scrollablebox {
+	width: 90%;
+	height:350px;
+	white-space: nowrap;
+	overflow: scroll;
+	margin-bottom: 20px;
+	margin-top: 0px;
+	margin-left: 2px;
+	margin-right: 2px;
+}
+
+#submitButton {
+	margin-top: -20px;
+}
+
+td { vertical-align: top; }
+li { list-style-type: none; }
+
+.molecule-div {
+	background-color: #bababa;
+	margin-left: 20px;
+	margin-right: 0px;
+	font-size: 11px;
+}
+
+div#serviceForm {
+	margin: 10px;
+	background-color: #ddd;
+	padding: 4px;
+	clear: none;
+	display: inline-block;
+	font-family: sans-serif;
+	font-size: 12px;
+}
+
+div#serviceForm input {
+	border: none;
+	margin: 2px;
+	margin-left: 0px;
+}
+
+.editorheader {
+    float: left;
+    margin-left: 10px;
+    margin-right: 10px;
+    margin-top: 5px;
+    margin-bottom: 5px;
+    background-color: transparent;
+}
+
+.left10 {
+	margin-left: 10px;
+}
+
+#convertStructureInputHeader {
+	display: inline-block;
+	margin-right: 10px;
+	background-color: #ddd;
+	font-size: 12px;
+	font-family: sans-serif;
+	margin-bottom: 20px;
+}
+
+.bordered {
+	box-shadow: 2px 2px 3px #444;
+}
+
+/* APIDOC style */
+.rightSide {
+	float: right;
+}
+
+div.table {
+	border: 1px solid;
+	display: table;
+	width: 100%;
+	margin-bottom: 14px;
+}
+
+div.api-header div.api-body {
+	display: table-row;
+}
+
+.api-body, .api-header, div.inner-api-table {
+	padding-left: 1em;
+	padding-right: 1em;
+}
+
+div.api-body p:last-of-type {
+	margin-bottom: 1em;
+}
+
+.api-header {
+	background-color:  #29999C;
+	color: #FFFFFF;
+	font-weight: bold;
+}
+
+span.api-method-returntype:before {
+	content: "Returns: ";
+}
+
+div code {
+	background-color: inherit;
+	border: none;
+}
+
+div.inner-api-table {
+	display: block;
+	background-color: lightblue;
+	border: none;
+	overflow-x: auto;
+}
+
+/* categories.html and items.html */
+table.cell-borrder-vertical tr > td {
+	padding-top: 0.5em;
+}
+
+table.cell-border-vertical tr > td, table.cell-border-vertical tr > th {
+	border-left: 1px solid;
+	padding-left: 15px;
+	padding-right: 15px;
+}
+
+table.cell-border-vertical tr > td:first-of-type, table.cell-border-vertical tr > th:first-of-type {
+	border-left: none;
+	padding-left: initial;
+}

static/j2s/J/Jmol.properties

@@ -0,0 +1,3 @@
+Jmol.___JmolDate="$Date: 2018-11-28 19:07:11 -0600 (Wed, 28 Nov 2018) $"
+Jmol.___fullJmolProperties="src/org/jmol/viewer/Jmol.properties"
+Jmol.___JmolVersion="14.29.29" 

static/j2s/J/adapter/readers/aflow/AFLOWReader.js

@@ -0,0 +1,190 @@
+Clazz.declarePackage ("J.adapter.readers.aflow");
+Clazz.load (["J.adapter.readers.xtal.VaspPoscarReader", "java.util.Hashtable"], "J.adapter.readers.aflow.AFLOWReader", ["java.lang.Float", "java.util.Arrays", "JU.BS", "$.Lst", "$.PT", "$.SB", "JU.Logger"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.aabb = null;
+this.readPRE = false;
+this.fracB = NaN;
+this.compositions = null;
+this.getComposition = false;
+this.listKey = null;
+this.listKeyCase = null;
+this.fileModelNumber = 0;
+this.havePRE = false;
+this.titleMsg = null;
+this.keyMap = null;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.aflow, "AFLOWReader", J.adapter.readers.xtal.VaspPoscarReader);
+Clazz.prepareFields (c$, function () {
+this.keyMap =  new java.util.Hashtable ();
+});
+Clazz.overrideMethod (c$, "initializeReader", 
+function () {
+this.readPRE = this.checkFilterKey ("PRE");
+var s;
+s = this.getFilter ("CA=");
+if (s != null) this.fracB = (1 - this.parseFloatStr (s));
+s = this.getFilter ("CB=");
+if (s != null) this.fracB = this.parseFloatStr (s);
+s = this.getFilter ("LIST=");
+this.listKey = (s == null ? "HF" : s);
+this.listKeyCase = this.listKey;
+this.getComposition = !Float.isNaN (this.fracB);
+this.discardLinesUntilStartsWith ("[");
+this.aabb = this.line.substring (1, this.line.indexOf ("]"));
+var pt = (JU.PT.isUpperCase (this.aabb.charAt (1)) ? 1 : 2);
+this.defaultLabels =  Clazz.newArray (-1, [this.aabb.substring (0, pt), this.aabb.substring (pt)]);
+while (this.rd ().indexOf ("] REFERENCE:") >= 0) this.appendLoadNote (this.line);
+
+this.compositions =  new java.util.Hashtable ();
+this.quiet = true;
+this.asc.bsAtoms =  new JU.BS ();
+this.addJmolScript ("unitcell off;axes off;");
+this.havePRE = (this.line.indexOf ("Structure PRE") >= 0);
+});
+Clazz.overrideMethod (c$, "checkLine", 
+function () {
+if (!this.havePRE) this.discardLinesUntilContains ("Structure PRE");
+this.havePRE = false;
+if (this.line == null) return false;
+this.continuing = new Boolean (this.continuing & this.readPrePost ()).valueOf ();
+return this.continuing;
+});
+Clazz.defineMethod (c$, "readPrePost", 
+ function () {
+this.fileModelNumber++;
+this.titleMsg = "#" + (this.modelNumber + 1) + (this.getComposition ? "," + this.fileModelNumber + ", Cb=" + this.fracB : "");
+this.elementLabel = null;
+var n0 = this.asc.bsAtoms.cardinality ();
+if (this.readPRE) {
+this.readStructure (this.titleMsg);
+} else {
+this.readElementLabelsOnly ();
+this.discardLinesUntilContains ("Structure POST");
+this.readStructure (this.titleMsg);
+}if (this.getData ()) {
+this.applySymmetryAndSetTrajectory ();
+} else {
+this.asc.bsAtoms.clearBits (this.asc.getLastAtomSetAtomIndex (), this.asc.ac);
+this.doCheckUnitCell = false;
+}this.finalizeModel ();
+if (n0 != this.asc.bsAtoms.cardinality ()) JU.Logger.info ("AFLOW: file#, saved#, atoms: " + this.fileModelNumber + " " + this.modelNumber + " " + (this.asc.bsAtoms.cardinality () - n0));
+return !this.haveModel || this.modelNumber != this.desiredModelNumber;
+});
+Clazz.defineMethod (c$, "finalizeModel", 
+ function () {
+this.asc.removeLastUnselectedAtoms ();
+});
+Clazz.defineMethod (c$, "readElementLabelsOnly", 
+ function () {
+this.readLines (5);
+this.rdline ();
+var n = this.getTokens ().length;
+this.elementLabel =  new Array (n);
+this.rdline ();
+this.line = "";
+var s = null;
+var last = null;
+for (var i = 0; i < n; i++) {
+while (s == null || s.equals (last)) {
+this.rdline ();
+var tokens = this.getTokens ();
+if (tokens.length != 4 || (s = this.elementLabel[i] = this.getElement (tokens[3])) == null) {
+i = n + 1;
+break;
+}}
+last = s;
+}
+if (s == null) this.elementLabel = this.defaultLabels;
+});
+Clazz.defineMethod (c$, "getData", 
+ function () {
+this.discardLinesUntilContains ("- DATA -");
+var htAFLOW =  new java.util.Hashtable ();
+htAFLOW.put ("fileModelNumber", Integer.$valueOf (this.fileModelNumber));
+htAFLOW.put ("modelNumber", Integer.$valueOf (this.modelNumber + 1));
+htAFLOW.put ("AaBb", this.aabb);
+var pt = 0;
+var sb =  new JU.SB ();
+var listVal = 3.4028235E38;
+var strcb = "?";
+var listValStr = null;
+var cb = 0;
+while (this.rdline () != null && (pt = this.line.indexOf (" # ")) >= 0) {
+var key = this.line.substring (pt + 3).trim ();
+var val = this.line.substring (0, pt).trim ();
+sb.append (key).append ("=").append (val).append (" | ");
+if (key.toUpperCase ().startsWith (this.listKey)) {
+this.listKey = key.toUpperCase ();
+this.listKeyCase = key;
+listValStr = val;
+listVal = this.parseFloatStr (val);
+}if (key.equals ("Ca")) {
+var ca = this.parseFloatStr (val);
+if (this.getComposition && Math.abs ((1 - ca) - this.fracB) > 0.01) return false;
+} else if (key.equals ("Cb")) {
+cb = this.parseFloatStr (strcb = val);
+if (this.getComposition && Math.abs (cb - this.fracB) > 0.01) return false;
+} else if (key.equals ("Hf_atom [eV] (VASP)")) {
+var e = this.parseFloatStr (val);
+this.asc.setAtomSetEnergy (val, e);
+}}
+this.asc.setAtomSetName (this.titleMsg + (this.getComposition ? "" : " Cb=" + cb) + " " + this.listKey + "=" + listValStr);
+var count_min = this.compositions.get (strcb);
+if (!this.doGetModel (++this.modelNumber, null)) return false;
+if (count_min == null) this.compositions.put (strcb, count_min =  Clazz.newFloatArray (-1, [0, 3.4028235E38, 0]));
+count_min[0]++;
+if (listVal < count_min[1]) {
+count_min[1] = listVal;
+count_min[2] = this.fileModelNumber;
+}while (this.line.indexOf ("- URL -") < 0) this.rdline ();
+
+sb.append ("URL=" + this.rdline () + "|");
+while (this.line.indexOf ("aurl=") < 0) this.rdline ();
+
+sb.append (this.line);
+var pairs = JU.PT.split (sb.toString (), " | ");
+for (var i = pairs.length; --i >= 0; ) {
+var kv = pairs[i].$plit ("=");
+if (kv.length < 2) continue;
+var f = this.parseFloatStr (kv[1]);
+var o = Float.isNaN (f) ? kv[1] : Float.$valueOf (f);
+htAFLOW.put (kv[0], o);
+var kvclean = this.cleanKey (kv[0]);
+if (kvclean !== kv[0]) htAFLOW.put (kvclean, o);
+}
+this.asc.setCurrentModelInfo ("aflowInfo", htAFLOW);
+return true;
+});
+Clazz.defineMethod (c$, "cleanKey", 
+ function (key) {
+var kclean = this.keyMap.get (key);
+if (kclean != null) return kclean;
+var chars = key.toCharArray ();
+for (var i = chars.length; --i >= 0; ) if (!JU.PT.isLetterOrDigit (chars[i])) chars[i] = '_';
+
+this.keyMap.put (key, kclean = JU.PT.trim (JU.PT.rep ( String.instantialize (chars), "__", "_"), "_"));
+return kclean;
+}, "~S");
+Clazz.overrideMethod (c$, "finalizeSubclassReader", 
+function () {
+this.alignUnitCells ();
+this.listCompositions ();
+this.finalizeReaderASCR ();
+});
+Clazz.defineMethod (c$, "listCompositions", 
+ function () {
+var list =  new JU.Lst ();
+for (var e, $e = this.compositions.entrySet ().iterator (); $e.hasNext () && ((e = $e.next ()) || true);) {
+var count_min = e.getValue ();
+list.addLast (e.getKey () + "\t" + (Clazz.floatToInt (count_min[0])) + "\t" + Clazz.floatToInt (count_min[2]) + "\t" + this.listKeyCase + "\t" + count_min[1]);
+}
+var a =  new Array (list.size ());
+list.toArray (a);
+java.util.Arrays.sort (a);
+for (var i = 0, n = a.length; i < n; i++) this.appendLoadNote (this.aabb + "\t" + a[i]);
+
+});
+Clazz.defineMethod (c$, "alignUnitCells", 
+ function () {
+});
+});

static/j2s/J/adapter/readers/cif/Cif2DataParser.js

@@ -0,0 +1,207 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (["JU.CifDataParser"], "J.adapter.readers.cif.Cif2DataParser", ["java.lang.Float", "java.util.Hashtable", "JU.Lst", "$.PT"], function () {
+c$ = Clazz.declareType (J.adapter.readers.cif, "Cif2DataParser", JU.CifDataParser);
+Clazz.overrideMethod (c$, "getVersion", 
+function () {
+return 2;
+});
+Clazz.overrideMethod (c$, "toUnicode", 
+function (data) {
+return data;
+}, "~S");
+Clazz.overrideMethod (c$, "isQuote", 
+function (ch) {
+switch (ch) {
+case '\1':
+case '\'':
+case '\"':
+case '[':
+case ']':
+case '{':
+case '}':
+case ';':
+return true;
+}
+return false;
+}, "~S");
+Clazz.overrideMethod (c$, "getQuotedStringOrObject", 
+function (ch) {
+return this.processQuotedString ();
+}, "~S");
+Clazz.overrideMethod (c$, "preprocessString", 
+function () {
+this.line = (this.ich == 0 ? this.str : this.str.substring (this.ich));
+return this.setString (this.processSemiString ());
+});
+Clazz.defineMethod (c$, "processQuotedString", 
+ function () {
+var str = null;
+var quoteChar = this.str.charAt (this.ich);
+var tripleChar = null;
+try {
+switch (quoteChar) {
+case '\1':
+str = this.str.substring (1, (this.ich = this.str.indexOf ("\1", this.ich + 1)));
+this.ich++;
+break;
+case '[':
+return this.readList ();
+case ']':
+this.ich++;
+return "]";
+case '{':
+return this.readTable ();
+case '}':
+this.ich++;
+return "}";
+case '\'':
+case '"':
+if (this.str.indexOf ("'''") == this.ich) tripleChar = "'''";
+ else if (this.str.indexOf ("\"\"\"") == this.ich) tripleChar = "\"\"\"";
+var nchar = (tripleChar == null ? 1 : 3);
+var pt = this.ich + nchar;
+var pt1 = 0;
+while ((pt1 = (tripleChar == null ? this.str.indexOf (quoteChar, pt) : this.str.indexOf (tripleChar, pt))) < 0) {
+if (this.readLine () == null) break;
+this.str += this.line;
+}
+this.ich = pt1 + nchar;
+this.cch = this.str.length;
+str = this.str.substring (pt, pt1);
+break;
+}
+} catch (e) {
+if (Clazz.exceptionOf (e, Exception)) {
+System.out.println ("exception in Cif2DataParser ; " + e);
+} else {
+throw e;
+}
+}
+return (this.cterm == '\0' || this.asObject ? str : JU.PT.esc (str));
+});
+Clazz.defineMethod (c$, "processSemiString", 
+function () {
+var pt1;
+var pt2;
+var str = this.preprocessSemiString ();
+if (str.indexOf (';') != 1 && (pt1 = str.indexOf ('\\')) > 1 && ((pt2 = str.indexOf ('\n')) > pt1 || pt2 < 0)) {
+var prefix = str.substring (1, pt1);
+str = JU.PT.rep (str, "\n" + prefix, "\n");
+str = "\1" + str.substring (str.charAt (pt1 + 1) == '\\' ? pt1 + 1 : pt2 < 0 ? str.length - 1 : pt2 + 1);
+}this.ich = 0;
+return this.fixLineFolding (str);
+});
+Clazz.defineMethod (c$, "readList", 
+function () {
+this.ich++;
+var cterm0 = this.cterm;
+this.cterm = ']';
+var ns = this.nullString;
+this.nullString = null;
+var lst = (this.asObject ?  new JU.Lst () : null);
+var n = 0;
+var str = "";
+while (true) {
+var value = (this.asObject ? this.getNextTokenObject () : this.getNextToken ());
+if (value == null || value.equals ("]")) break;
+if (this.asObject) {
+lst.addLast (value);
+} else {
+if (n++ > 0) str += ",";
+str += value;
+}}
+this.cterm = cterm0;
+this.nullString = ns;
+return (this.asObject ? lst : "[" + str + "]");
+});
+Clazz.defineMethod (c$, "readTable", 
+function () {
+this.ich++;
+var cterm0 = this.cterm;
+this.cterm = '}';
+var ns = this.nullString;
+this.nullString = null;
+var map = (this.asObject ?  new java.util.Hashtable () : null);
+var n = 0;
+var str = "";
+while (true) {
+var key = this.getNextToken ();
+if (key == null || key.equals ("}")) break;
+while (this.isSpaceOrColon (this.ich)) this.ich++;
+
+if (this.asObject) {
+map.put (key, this.getNextTokenObject ());
+} else {
+if (n++ > 0) str += ",";
+str += key + " : " + this.getNextToken ();
+}}
+this.cterm = cterm0;
+this.nullString = ns;
+return (this.asObject ? map : "{" + str + "}");
+});
+Clazz.defineMethod (c$, "isSpaceOrColon", 
+ function (ich) {
+if (ich < this.cch) switch (this.line.charAt (ich)) {
+case ' ':
+case '\t':
+case '\n':
+case ':':
+return true;
+}
+return false;
+}, "~N");
+Clazz.overrideMethod (c$, "unquoted", 
+function (s) {
+if (this.cterm == '\0' && !this.asObject) return s;
+var n = s.length;
+if (n > 0) {
+var c = s.charAt (0);
+if (JU.PT.isDigit (c) || c == '-' || c == '.' && n > 1) {
+var pt = s.indexOf ('(');
+var isFloat = (s.indexOf (".") >= 0);
+if (n > 1 && pt > 0 && s.indexOf (')', pt + 1) == n - 1) s = s.substring (0, pt);
+try {
+if (isFloat) {
+var f = Float.parseFloat (s);
+if (this.asObject) return Float.$valueOf (f);
+s = "" + f;
+if (s.indexOf (".") < 0 && s.indexOf ("E") < 0) s += ".0";
+return s;
+}var i = Integer.parseInt (s);
+return (this.asObject ? Integer.$valueOf (i) : "" + i);
+} catch (e) {
+}
+}}return (this.asObject ? s : JU.PT.esc (s));
+}, "~S");
+Clazz.defineMethod (c$, "fixLineFolding", 
+ function (str) {
+if (str.indexOf ('\\') < 0) return str;
+var n = str.length;
+if (str.endsWith ("\\\1")) str = str.substring (0, n - 1) + "\n\1";
+var pt = 0;
+while ((pt = str.indexOf ('\\', pt + 1)) >= 0) {
+var eol = str.indexOf ('\n', pt);
+if (eol < 0) break;
+for (var i = eol; --i > pt; ) {
+var ch = str.charAt (i);
+if (!JU.PT.isWhitespace (ch)) {
+if (ch == '\\') {
+pt = i;
+break;
+}pt = eol;
+break;
+}}
+if (pt < eol) str = str.substring (0, pt) + str.substring (eol + 1);
+}
+return str;
+}, "~S");
+c$.getArrayFromStringList = Clazz.defineMethod (c$, "getArrayFromStringList", 
+function (s, n) {
+var f =  Clazz.newFloatArray (n, 0);
+JU.PT.parseFloatArrayInfested (JU.PT.getTokens (s.$replace (',', ' ').$replace ('[', ' ')), f);
+var d =  Clazz.newDoubleArray (n, 0);
+for (var i = 0; i < n; i++) d[i] = f[i];
+
+return d;
+}, "~S,~N");
+});

static/j2s/J/adapter/readers/cif/Cif2Reader.js

@@ -0,0 +1,8 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (["J.adapter.readers.cif.CifReader"], "J.adapter.readers.cif.Cif2Reader", ["J.adapter.readers.cif.Cif2DataParser"], function () {
+c$ = Clazz.declareType (J.adapter.readers.cif, "Cif2Reader", J.adapter.readers.cif.CifReader);
+Clazz.overrideMethod (c$, "getCifDataParser", 
+function () {
+return  new J.adapter.readers.cif.Cif2DataParser ().set (this, null, this.debugging);
+});
+});

static/j2s/J/adapter/readers/cif/CifReader.js

@@ -0,0 +1,1226 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (["J.adapter.smarter.AtomSetCollectionReader", "JU.Lst", "$.P3"], "J.adapter.readers.cif.CifReader", ["java.lang.Boolean", "$.Character", "$.Float", "java.util.Hashtable", "JU.BS", "$.CifDataParser", "$.PT", "$.Rdr", "$.V3", "J.adapter.smarter.Atom", "J.api.JmolAdapter", "JU.Logger", "$.Vibration"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.modr = null;
+this.parser = null;
+this.isAFLOW = false;
+this.filterAssembly = false;
+this.allowRotations = true;
+this.readIdeal = true;
+this.configurationPtr = -2147483648;
+this.useAuthorChainID = true;
+this.thisDataSetName = "";
+this.lastDataSetName = null;
+this.chemicalName = "";
+this.thisStructuralFormula = "";
+this.thisFormula = "";
+this.iHaveDesiredModel = false;
+this.isMMCIF = false;
+this.isLigand = false;
+this.isMagCIF = false;
+this.haveHAtoms = false;
+this.molecularType = "GEOM_BOND default";
+this.lastAltLoc = '\0';
+this.haveAromatic = false;
+this.conformationIndex = 0;
+this.nMolecular = 0;
+this.appendedData = null;
+this.skipping = false;
+this.nAtoms = 0;
+this.ac = 0;
+this.auditBlockCode = null;
+this.lastSpaceGroupName = null;
+this.modulated = false;
+this.isCourseGrained = false;
+this.haveCellWaveVector = false;
+this.$latticeType = null;
+this.htGroup1 = null;
+this.nAtoms0 = 0;
+this.titleAtomSet = 1;
+this.intTableNo = 0;
+this.htCellTypes = null;
+this.modelMap = null;
+this.htAudit = null;
+this.symops = null;
+this.pdbID = null;
+this.key = null;
+this.key0 = null;
+this.data = null;
+this.isLoop = false;
+this.col2key = null;
+this.key2col = null;
+this.field = null;
+this.firstChar = '\0';
+this.htOxStates = null;
+this.bondTypes = null;
+this.disorderAssembly = ".";
+this.lastDisorderAssembly = null;
+this.lattvecs = null;
+this.magCenterings = null;
+this.maxSerial = 0;
+this.atomRadius = null;
+this.bsConnected = null;
+this.bsSets = null;
+this.ptOffset = null;
+this.bsMolecule = null;
+this.bsExclude = null;
+this.firstAtom = 0;
+this.atoms = null;
+this.bsBondDuplicates = null;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.cif, "CifReader", J.adapter.smarter.AtomSetCollectionReader);
+Clazz.prepareFields (c$, function () {
+this.col2key =  Clazz.newIntArray (100, 0);
+this.key2col =  Clazz.newIntArray (100, 0);
+this.bondTypes =  new JU.Lst ();
+this.ptOffset =  new JU.P3 ();
+});
+Clazz.overrideMethod (c$, "initializeReader", 
+function () {
+this.initSubclass ();
+this.allowPDBFilter = true;
+this.appendedData = this.htParams.get ("appendedData");
+var conf = this.getFilter ("CONF ");
+if (conf != null) this.configurationPtr = this.parseIntStr (conf);
+this.isMolecular = this.checkFilterKey ("MOLECUL") && !this.checkFilterKey ("BIOMOLECULE");
+this.isPrimitive = this.checkFilterKey ("PRIMITIVE");
+this.readIdeal = !this.checkFilterKey ("NOIDEAL");
+this.filterAssembly = this.checkFilterKey ("$");
+this.useAuthorChainID = !this.checkFilterKey ("NOAUTHORCHAINS");
+if (this.isMolecular) {
+this.forceSymmetry (false);
+this.molecularType = "filter \"MOLECULAR\"";
+}this.asc.checkSpecial = !this.checkFilterKey ("NOSPECIAL");
+this.allowRotations = !this.checkFilterKey ("NOSYM");
+if (this.strSupercell != null && this.strSupercell.indexOf (",") >= 0) this.addCellType ("conventional", this.strSupercell, true);
+if (this.binaryDoc != null) return;
+this.readCifData ();
+this.continuing = false;
+});
+Clazz.defineMethod (c$, "initSubclass", 
+function () {
+});
+Clazz.defineMethod (c$, "readCifData", 
+ function () {
+this.parser = this.getCifDataParser ();
+this.line = "";
+while ((this.key = this.parser.peekToken ()) != null) if (!this.readAllData ()) break;
+
+if (this.appendedData != null) {
+this.parser = (this.getInterface ("JU.CifDataParser")).set (null, JU.Rdr.getBR (this.appendedData), this.debugging);
+while ((this.key = this.parser.peekToken ()) != null) if (!this.readAllData ()) break;
+
+}});
+Clazz.defineMethod (c$, "getCifDataParser", 
+function () {
+return  new JU.CifDataParser ().set (this, null, this.debugging);
+});
+Clazz.defineMethod (c$, "readAllData", 
+ function () {
+if (this.key.startsWith ("data_")) {
+this.isLigand = false;
+if (this.asc.atomSetCount == 0) this.iHaveDesiredModel = false;
+if (this.iHaveDesiredModel) return false;
+if (this.desiredModelNumber != -2147483648) this.appendLoadNote (null);
+this.newModel (++this.modelNumber);
+this.haveCellWaveVector = false;
+if (this.auditBlockCode == null) this.modulated = false;
+if (!this.skipping) {
+this.nAtoms0 = this.asc.ac;
+this.processDataParameter ();
+this.nAtoms = this.asc.ac;
+}return true;
+}if (this.skipping && this.key.equals ("_audit_block_code")) {
+this.iHaveDesiredModel = false;
+this.skipping = false;
+}this.isLoop = this.key.startsWith ("loop_");
+if (this.isLoop) {
+if (this.skipping && !this.isMMCIF) {
+this.parser.getTokenPeeked ();
+this.parser.skipLoop (false);
+} else {
+this.processLoopBlock ();
+}return true;
+}if (this.key.indexOf ("_") != 0) {
+JU.Logger.warn (this.key.startsWith ("save_") ? "CIF reader ignoring save_" : "CIF ERROR ? should be an underscore: " + this.key);
+this.parser.getTokenPeeked ();
+} else if (!this.getData ()) {
+return true;
+}if (!this.skipping) {
+this.key = this.parser.fixKey (this.key0 = this.key);
+if (this.key.startsWith ("_chemical_name") || this.key.equals ("_chem_comp_name")) {
+this.processChemicalInfo ("name");
+} else if (this.key.startsWith ("_chemical_formula_structural")) {
+this.processChemicalInfo ("structuralFormula");
+} else if (this.key.startsWith ("_chemical_formula_sum") || this.key.equals ("_chem_comp_formula")) {
+this.processChemicalInfo ("formula");
+} else if (this.key.equals ("_cell_modulation_dimension")) {
+this.modDim = this.parseIntStr (this.data);
+} else if (this.key.startsWith ("_cell_") && this.key.indexOf ("_commen_") < 0) {
+this.processCellParameter ();
+} else if (this.key.startsWith ("_atom_sites_fract_tran")) {
+this.processUnitCellTransformMatrix ();
+} else if (this.key.startsWith ("_audit")) {
+if (this.key.equals ("_audit_block_code")) {
+this.auditBlockCode = this.parser.fullTrim (this.data).toUpperCase ();
+this.appendLoadNote (this.auditBlockCode);
+if (this.htAudit != null && this.auditBlockCode.contains ("_MOD_")) {
+var key = JU.PT.rep (this.auditBlockCode, "_MOD_", "_REFRNCE_");
+if (this.asc.setSymmetry (this.htAudit.get (key)) != null) {
+this.unitCellParams = this.asc.getSymmetry ().getUnitCellParams ();
+this.iHaveUnitCell = true;
+}} else if (this.htAudit != null) {
+if (this.symops != null) for (var i = 0; i < this.symops.size (); i++) this.setSymmetryOperator (this.symops.get (i));
+
+}if (this.lastSpaceGroupName != null) this.setSpaceGroupName (this.lastSpaceGroupName);
+} else if (this.key.equals ("_audit_creation_date")) {
+this.symmetry = null;
+}} else if (this.key.equals (J.adapter.readers.cif.CifReader.singleAtomID)) {
+this.readSingleAtom ();
+} else if (this.key.startsWith ("_symmetry_space_group_name_h-m") || this.key.startsWith ("_symmetry_space_group_name_hall") || this.key.startsWith ("_space_group_name") || this.key.contains ("_ssg_name") || this.key.contains ("_magn_name") || this.key.contains ("_bns_name")) {
+this.processSymmetrySpaceGroupName ();
+} else if (this.key.startsWith ("_space_group_transform") || this.key.startsWith ("_parent_space_group") || this.key.startsWith ("_space_group_magn_transform")) {
+this.processUnitCellTransform ();
+} else if (this.key.contains ("_database_code")) {
+this.addModelTitle ("ID");
+} else if ("__citation_title__publ_section_title__active_magnetic_irreps_details__".contains ("_" + this.key + "__")) {
+this.addModelTitle ("TITLE");
+} else if (this.key.startsWith ("_aflow_")) {
+this.isAFLOW = true;
+} else if (this.key.equals ("_symmetry_int_tables_number")) {
+this.intTableNo = this.parseIntStr (this.data);
+this.rotateHexCell = (this.isAFLOW && (this.intTableNo >= 143 && this.intTableNo <= 194));
+} else if (this.key.equals ("_entry_id")) {
+this.pdbID = this.data;
+} else {
+this.processSubclassEntry ();
+}}return true;
+});
+Clazz.defineMethod (c$, "addModelTitle", 
+ function (key) {
+if (this.asc.atomSetCount > this.titleAtomSet) this.appendLoadNote ("\nMODEL: " + (this.titleAtomSet = this.asc.atomSetCount));
+this.appendLoadNote (key + ": " + this.parser.fullTrim (this.data));
+}, "~S");
+Clazz.defineMethod (c$, "processSubclassEntry", 
+function () {
+if (this.modDim > 0) this.getModulationReader ().processEntry ();
+});
+Clazz.defineMethod (c$, "processUnitCellTransform", 
+ function () {
+this.data = JU.PT.replaceAllCharacters (this.data, " ", "");
+if (this.key.contains ("_from_parent") || this.key.contains ("child_transform")) this.addCellType ("parent", this.data, true);
+ else if (this.key.contains ("_to_standard") || this.key.contains ("transform_bns_pp_abc")) this.addCellType ("standard", this.data, false);
+this.appendLoadNote (this.key + ": " + this.data);
+});
+Clazz.defineMethod (c$, "addCellType", 
+ function (type, data, isFrom) {
+if (this.htCellTypes == null) this.htCellTypes =  new java.util.Hashtable ();
+if (data.startsWith ("!")) {
+data = data.substring (1);
+isFrom = !isFrom;
+}var cell = (isFrom ? "!" : "") + data;
+this.htCellTypes.put (type, cell);
+if (type.equalsIgnoreCase (this.strSupercell)) {
+this.strSupercell = cell;
+this.htCellTypes.put ("conventional", (isFrom ? "" : "!") + data);
+}}, "~S,~S,~B");
+Clazz.defineMethod (c$, "readSingleAtom", 
+ function () {
+var atom =  new J.adapter.smarter.Atom ();
+atom.set (0, 0, 0);
+atom.atomName = this.parser.fullTrim (this.data);
+atom.getElementSymbol ();
+this.asc.addAtom (atom);
+});
+Clazz.defineMethod (c$, "getModulationReader", 
+ function () {
+return (this.modr == null ? this.initializeMSCIF () : this.modr);
+});
+Clazz.defineMethod (c$, "initializeMSCIF", 
+ function () {
+if (this.modr == null) this.ms = this.modr = this.getInterface ("J.adapter.readers.cif.MSCifParser");
+this.modulated = (this.modr.initialize (this, this.modDim) > 0);
+return this.modr;
+});
+Clazz.defineMethod (c$, "newModel", 
+function (modelNo) {
+this.skipping = !this.doGetModel (this.modelNumber = modelNo, null);
+if (this.skipping) {
+if (!this.isMMCIF) this.parser.getTokenPeeked ();
+return;
+}this.chemicalName = "";
+this.thisStructuralFormula = "";
+this.thisFormula = "";
+this.iHaveDesiredModel = this.isLastModel (this.modelNumber);
+if (this.isCourseGrained) this.asc.setCurrentModelInfo ("courseGrained", Boolean.TRUE);
+if (this.nAtoms0 == this.asc.ac) {
+this.modelNumber--;
+this.haveModel = false;
+this.asc.removeCurrentAtomSet ();
+} else {
+this.applySymmetryAndSetTrajectory ();
+}}, "~N");
+Clazz.overrideMethod (c$, "finalizeSubclassReader", 
+function () {
+if (this.asc.iSet > 0 && this.asc.getAtomSetAtomCount (this.asc.iSet) == 0) this.asc.atomSetCount--;
+ else if (!this.isMMCIF || !this.finalizeSubclass ()) this.applySymmetryAndSetTrajectory ();
+var n = this.asc.atomSetCount;
+if (n > 1) this.asc.setCollectionName ("<collection of " + n + " models>");
+if (this.pdbID != null) this.asc.setCurrentModelInfo ("pdbID", this.pdbID);
+this.finalizeReaderASCR ();
+this.addHeader ();
+if (this.haveAromatic) this.addJmolScript ("calculate aromatic");
+});
+Clazz.defineMethod (c$, "addHeader", 
+function () {
+var header = this.parser.getFileHeader ();
+if (header.length > 0) {
+var s = this.setLoadNote ();
+this.appendLoadNote (null);
+this.appendLoadNote (header);
+this.appendLoadNote (s);
+this.setLoadNote ();
+this.asc.setInfo ("fileHeader", header);
+}});
+Clazz.defineMethod (c$, "finalizeSubclass", 
+function () {
+return false;
+});
+Clazz.overrideMethod (c$, "doPreSymmetry", 
+function () {
+if (this.magCenterings != null) this.addLatticeVectors ();
+if (this.modDim > 0) this.getModulationReader ().setModulation (false, null);
+if (this.isMagCIF) {
+this.asc.getXSymmetry ().scaleFractionalVibs ();
+this.vibsFractional = true;
+}});
+Clazz.overrideMethod (c$, "applySymmetryAndSetTrajectory", 
+function () {
+if (this.isMMCIF) this.asc.checkSpecial = false;
+var doCheckBonding = this.doCheckUnitCell && !this.isMMCIF;
+if (this.isMMCIF) {
+var modelIndex = this.asc.iSet;
+this.asc.setCurrentModelInfo ("PDB_CONECT_firstAtom_count_max",  Clazz.newIntArray (-1, [this.asc.getAtomSetAtomIndex (modelIndex), this.asc.getAtomSetAtomCount (modelIndex), this.maxSerial]));
+}if (this.htCellTypes != null) {
+for (var e, $e = this.htCellTypes.entrySet ().iterator (); $e.hasNext () && ((e = $e.next ()) || true);) this.asc.setCurrentModelInfo ("unitcell_" + e.getKey (), e.getValue ());
+
+this.htCellTypes = null;
+}if (!this.haveCellWaveVector) this.modDim = 0;
+if (this.doApplySymmetry && !this.iHaveFractionalCoordinates) this.fractionalizeCoordinates (true);
+this.applySymTrajASCR ();
+if (doCheckBonding && (this.bondTypes.size () > 0 || this.isMolecular)) this.setBondingAndMolecules ();
+this.asc.setCurrentModelInfo ("fileHasUnitCell", Boolean.TRUE);
+this.asc.xtalSymmetry = null;
+});
+Clazz.overrideMethod (c$, "finalizeSubclassSymmetry", 
+function (haveSymmetry) {
+var sym = (haveSymmetry ? this.asc.getXSymmetry ().getBaseSymmetry () : null);
+if (sym != null && sym.getSpaceGroup () == null) {
+this.appendLoadNote ("Invalid or missing space group operations!");
+sym = null;
+}if (this.modDim > 0 && sym != null) {
+this.addLatticeVectors ();
+this.asc.setTensors ();
+this.getModulationReader ().setModulation (true, sym);
+this.modr.finalizeModulation ();
+}if (this.isMagCIF) {
+this.asc.setNoAutoBond ();
+if (sym != null) {
+this.addJmolScript ("vectors on;vectors 0.15;");
+var n = this.asc.getXSymmetry ().setSpinVectors ();
+this.appendLoadNote (n + " magnetic moments - use VECTORS ON/OFF or VECTOR MAX x.x or SELECT VXYZ>0");
+}}if (sym != null && this.auditBlockCode != null && this.auditBlockCode.contains ("REFRNCE")) {
+if (this.htAudit == null) this.htAudit =  new java.util.Hashtable ();
+this.htAudit.put (this.auditBlockCode, sym);
+}}, "~B");
+Clazz.defineMethod (c$, "processDataParameter", 
+ function () {
+this.bondTypes.clear ();
+this.parser.getTokenPeeked ();
+this.thisDataSetName = (this.key.length < 6 ? "" : this.key.substring (5));
+if (this.thisDataSetName.length > 0) this.nextAtomSet ();
+if (this.debugging) JU.Logger.debug (this.key);
+});
+Clazz.defineMethod (c$, "nextAtomSet", 
+function () {
+this.asc.setCurrentModelInfo ("isCIF", Boolean.TRUE);
+if (this.asc.iSet >= 0) {
+if (this.isMMCIF) {
+this.setModelPDB (true);
+if (this.pdbID != null) this.asc.setCurrentModelInfo ("pdbID", this.pdbID);
+}this.asc.newAtomSet ();
+if (this.isMMCIF) {
+this.setModelPDB (true);
+if (this.pdbID != null) this.asc.setCurrentModelInfo ("pdbID", this.pdbID);
+}} else {
+this.asc.setCollectionName (this.thisDataSetName);
+}});
+Clazz.defineMethod (c$, "processChemicalInfo", 
+ function (type) {
+if (type.equals ("name")) {
+this.chemicalName = this.data = this.parser.fullTrim (this.data);
+this.appendLoadNote (this.chemicalName);
+if (!this.data.equals ("?")) this.asc.setInfo ("modelLoadNote", this.data);
+} else if (type.equals ("structuralFormula")) {
+this.thisStructuralFormula = this.data = this.parser.fullTrim (this.data);
+} else if (type.equals ("formula")) {
+this.thisFormula = this.data = this.parser.fullTrim (this.data);
+if (this.thisFormula.length > 1) this.appendLoadNote (this.thisFormula);
+}if (this.debugging) {
+JU.Logger.debug (type + " = " + this.data);
+}return this.data;
+}, "~S");
+Clazz.defineMethod (c$, "processSymmetrySpaceGroupName", 
+ function () {
+if (this.key.indexOf ("_ssg_name") >= 0) {
+this.modulated = true;
+this.$latticeType = this.data.substring (0, 1);
+} else if (this.modulated) {
+return;
+}this.data = this.parser.toUnicode (this.data);
+this.setSpaceGroupName (this.lastSpaceGroupName = (this.key.indexOf ("h-m") > 0 ? "HM:" : this.modulated ? "SSG:" : this.key.indexOf ("bns") >= 0 ? "BNS:" : this.key.indexOf ("hall") >= 0 ? "Hall:" : "") + this.data);
+});
+Clazz.defineMethod (c$, "addLatticeVectors", 
+ function () {
+this.lattvecs = null;
+if (this.magCenterings != null) {
+this.$latticeType = "Magnetic";
+this.lattvecs =  new JU.Lst ();
+for (var i = 0; i < this.magCenterings.size (); i++) {
+var s = this.magCenterings.get (i);
+var f =  Clazz.newFloatArray (this.modDim + 4, 0);
+if (s.indexOf ("x1") >= 0) for (var j = 1; j <= this.modDim + 3; j++) s = JU.PT.rep (s, "x" + j, "");
+
+var tokens = JU.PT.split (JU.PT.replaceAllCharacters (s, "xyz+", ""), ",");
+var n = 0;
+for (var j = 0; j < tokens.length; j++) {
+s = tokens[j].trim ();
+if (s.length == 0) continue;
+if ((f[j] = JU.PT.parseFloatFraction (s)) != 0) n++;
+}
+if (n >= 2) this.lattvecs.addLast (f);
+}
+this.magCenterings = null;
+} else if (this.$latticeType != null && "ABCFI".indexOf (this.$latticeType) >= 0) {
+this.lattvecs =  new JU.Lst ();
+try {
+this.ms.addLatticeVector (this.lattvecs, this.$latticeType);
+} catch (e) {
+if (Clazz.exceptionOf (e, Exception)) {
+} else {
+throw e;
+}
+}
+}if (this.lattvecs != null && this.lattvecs.size () > 0 && this.asc.getSymmetry ().addLatticeVectors (this.lattvecs)) {
+this.appendLoadNote ("Note! " + this.lattvecs.size () + " symmetry operators added for lattice centering " + this.$latticeType);
+for (var i = 0; i < this.lattvecs.size (); i++) this.appendLoadNote (JU.PT.toJSON (null, this.lattvecs.get (i)));
+
+}this.$latticeType = null;
+});
+Clazz.defineMethod (c$, "processCellParameter", 
+ function () {
+for (var i = J.api.JmolAdapter.cellParamNames.length; --i >= 0; ) if (this.key.equals (J.api.JmolAdapter.cellParamNames[i])) {
+var p = this.parseFloatStr (this.data);
+if (this.rotateHexCell && i == 5 && p == 120) p = -1;
+this.setUnitCellItem (i, p);
+return;
+}
+});
+Clazz.defineMethod (c$, "processUnitCellTransformMatrix", 
+ function () {
+var v = this.parseFloatStr (this.data);
+if (Float.isNaN (v)) return;
+for (var i = 0; i < J.adapter.readers.cif.CifReader.TransformFields.length; i++) {
+if (this.key.indexOf (J.adapter.readers.cif.CifReader.TransformFields[i]) >= 0) {
+this.setUnitCellItem (6 + i, v);
+return;
+}}
+});
+Clazz.defineMethod (c$, "getData", 
+ function () {
+this.key = this.parser.getTokenPeeked ();
+this.data = this.parser.getNextToken ();
+if (this.debugging && this.data != null && this.data.length > 0 && this.data.charAt (0) != '\0') JU.Logger.debug (">> " + this.key + " " + this.data);
+if (this.data == null) {
+JU.Logger.warn ("CIF ERROR ? end of file; data missing: " + this.key);
+return false;
+}return (this.data.length == 0 || this.data.charAt (0) != '\0');
+});
+Clazz.defineMethod (c$, "processLoopBlock", 
+function () {
+this.parser.getTokenPeeked ();
+this.key = this.parser.peekToken ();
+if (this.key == null) return;
+this.key = this.parser.fixKey (this.key0 = this.key);
+if (this.modDim > 0) switch (this.getModulationReader ().processLoopBlock ()) {
+case 0:
+break;
+case -1:
+this.parser.skipLoop (false);
+case 1:
+return;
+}
+var isLigand = false;
+if (this.key.startsWith ("_atom_site") || (isLigand = this.key.equals ("_chem_comp_atom_comp_id"))) {
+if (!this.processAtomSiteLoopBlock (isLigand)) return;
+if (this.thisDataSetName.equals ("global")) this.asc.setCollectionName (this.thisDataSetName = this.chemicalName);
+if (!this.thisDataSetName.equals (this.lastDataSetName)) {
+this.asc.setAtomSetName (this.thisDataSetName);
+this.lastDataSetName = this.thisDataSetName;
+}this.asc.setCurrentModelInfo ("chemicalName", this.chemicalName);
+this.asc.setCurrentModelInfo ("structuralFormula", this.thisStructuralFormula);
+this.asc.setCurrentModelInfo ("formula", this.thisFormula);
+return;
+}if (this.key.startsWith ("_space_group_symop") || this.key.startsWith ("_symmetry_equiv_pos") || this.key.startsWith ("_symmetry_ssg_equiv")) {
+if (this.ignoreFileSymmetryOperators) {
+JU.Logger.warn ("ignoring file-based symmetry operators");
+this.parser.skipLoop (false);
+} else {
+this.processSymmetryOperationsLoopBlock ();
+}return;
+}if (this.key.startsWith ("_citation")) {
+this.processCitationListBlock ();
+return;
+}if (this.key.startsWith ("_atom_type")) {
+this.processAtomTypeLoopBlock ();
+return;
+}if ((this.isMolecular || !this.doApplySymmetry) && this.key.startsWith ("_geom_bond")) {
+this.processGeomBondLoopBlock ();
+return;
+}if (this.processSubclassLoopBlock ()) return;
+if (this.key.equals ("_propagation_vector_seq_id")) {
+this.addMore ();
+return;
+}this.parser.skipLoop (false);
+});
+Clazz.defineMethod (c$, "processSubclassLoopBlock", 
+function () {
+return false;
+});
+Clazz.defineMethod (c$, "addMore", 
+ function () {
+var str;
+var n = 0;
+try {
+while ((str = this.parser.peekToken ()) != null && str.charAt (0) == '_') {
+this.parser.getTokenPeeked ();
+n++;
+}
+var m = 0;
+var s = "";
+while ((str = this.parser.getNextDataToken ()) != null) {
+s += str + (m % n == 0 ? "=" : " ");
+if (++m % n == 0) {
+this.appendUunitCellInfo (s.trim ());
+s = "";
+}}
+} catch (e) {
+if (Clazz.exceptionOf (e, Exception)) {
+} else {
+throw e;
+}
+}
+});
+Clazz.defineMethod (c$, "fieldProperty", 
+function (i) {
+return (i >= 0 && (this.field = this.parser.getColumnData (i)).length > 0 && (this.firstChar = this.field.charAt (0)) != '\0' ? this.col2key[i] : -1);
+}, "~N");
+Clazz.defineMethod (c$, "parseLoopParameters", 
+function (fields) {
+this.parser.parseDataBlockParameters (fields, this.isLoop ? null : this.key0, this.data, this.key2col, this.col2key);
+}, "~A");
+Clazz.defineMethod (c$, "parseLoopParametersFor", 
+function (key, fields) {
+if (fields[0].charAt (0) == '*') for (var i = fields.length; --i >= 0; ) if (fields[i].charAt (0) == '*') fields[i] = key + fields[i].substring (1);
+
+this.parseLoopParameters (fields);
+}, "~S,~A");
+Clazz.defineMethod (c$, "disableField", 
+ function (fieldIndex) {
+var i = this.key2col[fieldIndex];
+if (i != -1) this.col2key[i] = -1;
+}, "~N");
+Clazz.defineMethod (c$, "processAtomTypeLoopBlock", 
+ function () {
+this.parseLoopParameters (J.adapter.readers.cif.CifReader.atomTypeFields);
+if (!this.checkAllFieldsPresent (J.adapter.readers.cif.CifReader.atomTypeFields, -1, false)) {
+this.parser.skipLoop (false);
+return;
+}var atomTypeSymbol;
+var oxidationNumber = 0;
+while (this.parser.getData ()) {
+if (this.isNull (atomTypeSymbol = this.getField (0)) || Float.isNaN (oxidationNumber = this.parseFloatStr (this.getField (1)))) continue;
+if (this.htOxStates == null) this.htOxStates =  new java.util.Hashtable ();
+this.htOxStates.put (atomTypeSymbol, Float.$valueOf (oxidationNumber));
+}
+});
+Clazz.defineMethod (c$, "processAtomSiteLoopBlock", 
+function (isLigand) {
+this.isLigand = isLigand;
+var pdbModelNo = -1;
+var haveCoord = true;
+this.parseLoopParametersFor ("_atom_site", J.adapter.readers.cif.CifReader.atomFields);
+if (this.key2col[55] != -1) {
+this.setFractionalCoordinates (false);
+} else if (this.key2col[6] != -1 || this.key2col[52] != -1) {
+this.setFractionalCoordinates (false);
+this.disableField (3);
+this.disableField (4);
+this.disableField (5);
+if (this.key2col[16] != -1 && !this.isMMCIF) {
+this.setIsPDB ();
+this.isMMCIF = true;
+}} else if (this.key2col[3] != -1) {
+this.setFractionalCoordinates (true);
+this.disableField (6);
+this.disableField (7);
+this.disableField (8);
+} else if (this.key2col[20] != -1 || this.key2col[21] != -1 || this.key2col[63] != -1) {
+haveCoord = false;
+} else {
+this.parser.skipLoop (false);
+return false;
+}var modelField = this.key2col[17];
+var siteMult = 0;
+while (this.parser.getData ()) {
+if (modelField >= 0) {
+pdbModelNo = this.checkPDBModelField (modelField, pdbModelNo);
+if (pdbModelNo < 0) break;
+if (this.skipping) continue;
+}var atom = null;
+if (haveCoord) {
+atom =  new J.adapter.smarter.Atom ();
+} else {
+if (this.fieldProperty (this.key2col[20]) != -1 || this.fieldProperty (this.key2col[21]) != -1 || this.fieldProperty (this.key2col[63]) != -1) {
+if ((atom = this.asc.getAtomFromName (this.field)) == null) continue;
+} else {
+continue;
+}}var componentId = null;
+var strChain = null;
+var id = null;
+var seqID = 0;
+var n = this.parser.getColumnCount ();
+for (var i = 0; i < n; ++i) {
+var tok = this.fieldProperty (i);
+switch (tok) {
+case -1:
+break;
+case 71:
+seqID = this.parseIntStr (this.field);
+break;
+case 70:
+id = this.field;
+break;
+case 50:
+case 0:
+var elementSymbol;
+if (this.field.length < 2) {
+elementSymbol = this.field;
+} else {
+var ch1 = Character.toLowerCase (this.field.charAt (1));
+if (J.adapter.smarter.Atom.isValidSym2 (this.firstChar, ch1)) {
+elementSymbol = "" + this.firstChar + ch1;
+} else {
+elementSymbol = "" + this.firstChar;
+if (!this.haveHAtoms && this.firstChar == 'H') this.haveHAtoms = true;
+}}atom.elementSymbol = elementSymbol;
+if (this.htOxStates != null && this.htOxStates.containsKey (this.field)) {
+var charge = this.htOxStates.get (this.field).floatValue ();
+atom.formalCharge = Math.round (charge);
+if (Math.abs (atom.formalCharge - charge) > 0.1) if (this.debugging) {
+JU.Logger.debug ("CIF charge on " + this.field + " was " + charge + "; rounded to " + atom.formalCharge);
+}}break;
+case 49:
+case 1:
+case 2:
+atom.atomName = this.field;
+break;
+case 55:
+var x = this.parseFloatStr (this.field);
+if (this.readIdeal && !Float.isNaN (x)) atom.x = x;
+break;
+case 56:
+var y = this.parseFloatStr (this.field);
+if (this.readIdeal && !Float.isNaN (y)) atom.y = y;
+break;
+case 57:
+var z = this.parseFloatStr (this.field);
+if (this.readIdeal && !Float.isNaN (z)) atom.z = z;
+break;
+case 52:
+case 6:
+case 3:
+atom.x = this.parseFloatStr (this.field);
+break;
+case 53:
+case 7:
+case 4:
+atom.y = this.parseFloatStr (this.field);
+break;
+case 54:
+case 8:
+case 5:
+atom.z = this.parseFloatStr (this.field);
+break;
+case 51:
+atom.formalCharge = this.parseIntStr (this.field);
+break;
+case 9:
+var floatOccupancy = this.parseFloatStr (this.field);
+if (!Float.isNaN (floatOccupancy)) atom.foccupancy = floatOccupancy;
+break;
+case 10:
+atom.bfactor = this.parseFloatStr (this.field) * (this.isMMCIF ? 1 : 100);
+break;
+case 48:
+case 11:
+atom.group3 = this.field;
+break;
+case 59:
+componentId = this.field;
+if (!this.useAuthorChainID) this.setChainID (atom, strChain = this.field);
+break;
+case 12:
+if (this.useAuthorChainID) this.setChainID (atom, strChain = this.field);
+break;
+case 13:
+this.maxSerial = Math.max (this.maxSerial, atom.sequenceNumber = this.parseIntStr (this.field));
+break;
+case 14:
+atom.insertionCode = this.firstChar;
+break;
+case 15:
+case 60:
+atom.altLoc = this.firstChar;
+break;
+case 58:
+this.disorderAssembly = this.field;
+break;
+case 19:
+if (this.firstChar == '-' && this.field.length > 1) {
+atom.altLoc = this.field.charAt (1);
+atom.ignoreSymmetry = true;
+} else {
+atom.altLoc = this.firstChar;
+}break;
+case 16:
+if ("HETATM".equals (this.field)) atom.isHetero = true;
+break;
+case 18:
+if ("dum".equals (this.field)) {
+atom.x = NaN;
+continue;
+}break;
+case 61:
+if (this.modulated) siteMult = this.parseIntStr (this.field);
+break;
+case 62:
+case 47:
+if (this.field.equalsIgnoreCase ("Uiso")) {
+var j = this.key2col[34];
+if (j != -1) this.asc.setU (atom, 7, this.parseFloatStr (this.parser.getColumnData (j)));
+}break;
+case 22:
+case 23:
+case 24:
+case 25:
+case 26:
+case 27:
+case 28:
+case 29:
+case 30:
+case 31:
+case 32:
+case 33:
+this.asc.setU (atom, (this.col2key[i] - 22) % 6, this.parseFloatStr (this.field));
+break;
+case 35:
+case 36:
+case 37:
+case 38:
+case 39:
+case 40:
+this.asc.setU (atom, 6, 4);
+this.asc.setU (atom, (this.col2key[i] - 35) % 6, this.parseFloatStr (this.field));
+break;
+case 41:
+case 42:
+case 43:
+case 44:
+case 45:
+case 46:
+this.asc.setU (atom, 6, 0);
+this.asc.setU (atom, (this.col2key[i] - 41) % 6, this.parseFloatStr (this.field));
+break;
+case 64:
+case 65:
+case 66:
+case 67:
+case 68:
+case 69:
+this.isMagCIF = true;
+var pt = atom.vib;
+if (pt == null) atom.vib = pt =  new JU.Vibration ().setType (-2);
+var v = this.parseFloatStr (this.field);
+switch (tok) {
+case 64:
+case 67:
+pt.x = v;
+this.appendLoadNote ("magnetic moment: " + this.line);
+break;
+case 65:
+case 68:
+pt.y = v;
+break;
+case 66:
+case 69:
+pt.z = v;
+break;
+}
+break;
+}
+}
+if (!haveCoord) continue;
+if (Float.isNaN (atom.x) || Float.isNaN (atom.y) || Float.isNaN (atom.z)) {
+JU.Logger.warn ("atom " + atom.atomName + " has invalid/unknown coordinates");
+continue;
+}if (atom.elementSymbol == null && atom.atomName != null) atom.getElementSymbol ();
+if (!this.filterCIFAtom (atom, componentId)) continue;
+this.setAtomCoord (atom);
+if (this.isMMCIF && !this.processSubclassAtom (atom, componentId, strChain)) continue;
+if (this.asc.iSet < 0) this.nextAtomSet ();
+this.asc.addAtomWithMappedName (atom);
+if (id != null) {
+this.asc.atomSymbolicMap.put (id, atom);
+if (seqID > 0) {
+var pt = atom.vib;
+if (pt == null) pt = this.asc.addVibrationVector (atom.index, 0, NaN, 1094713365);
+pt.x = seqID;
+}}this.ac++;
+if (this.modDim > 0 && siteMult != 0) atom.vib = JU.V3.new3 (siteMult, 0, NaN);
+}
+this.asc.setCurrentModelInfo ("isCIF", Boolean.TRUE);
+if (this.isMMCIF) this.setModelPDB (true);
+if (this.isMMCIF && this.skipping) this.skipping = false;
+return true;
+}, "~B");
+Clazz.defineMethod (c$, "checkPDBModelField", 
+function (modelField, currentModelNo) {
+return 0;
+}, "~N,~N");
+Clazz.defineMethod (c$, "processSubclassAtom", 
+function (atom, assemblyId, strChain) {
+return true;
+}, "J.adapter.smarter.Atom,~S,~S");
+Clazz.defineMethod (c$, "filterCIFAtom", 
+function (atom, componentId) {
+if (!this.filterAtom (atom, -1)) return false;
+if (this.filterAssembly && this.filterReject (this.filter, "$", componentId)) return false;
+if (this.configurationPtr > 0) {
+if (!this.disorderAssembly.equals (this.lastDisorderAssembly)) {
+this.lastDisorderAssembly = this.disorderAssembly;
+this.lastAltLoc = '\0';
+this.conformationIndex = this.configurationPtr;
+}if (atom.altLoc != '\0') {
+if (this.conformationIndex >= 0 && atom.altLoc != this.lastAltLoc) {
+this.lastAltLoc = atom.altLoc;
+this.conformationIndex--;
+}if (this.conformationIndex != 0) {
+JU.Logger.info ("ignoring " + atom.atomName);
+return false;
+}}}return true;
+}, "J.adapter.smarter.Atom,~S");
+Clazz.defineMethod (c$, "processCitationListBlock", 
+ function () {
+this.parseLoopParameters (J.adapter.readers.cif.CifReader.citationFields);
+while (this.parser.getData ()) {
+var title = this.getField (0);
+if (!this.isNull (title)) this.appendLoadNote ("TITLE: " + this.parser.toUnicode (title));
+}
+});
+Clazz.defineMethod (c$, "processSymmetryOperationsLoopBlock", 
+ function () {
+this.parseLoopParametersFor ("_space_group_symop", J.adapter.readers.cif.CifReader.symmetryOperationsFields);
+var n;
+this.symops =  new JU.Lst ();
+for (n = J.adapter.readers.cif.CifReader.symmetryOperationsFields.length; --n >= 0; ) if (this.key2col[n] != -1) break;
+
+if (n < 0) {
+JU.Logger.warn ("required _space_group_symop key not found");
+this.parser.skipLoop (false);
+return;
+}n = 0;
+var isMag = false;
+while (this.parser.getData ()) {
+var ssgop = false;
+var nn = this.parser.getColumnCount ();
+var timeRev = (this.fieldProperty (this.key2col[7]) == -1 && this.fieldProperty (this.key2col[8]) == -1 && this.fieldProperty (this.key2col[6]) == -1 ? 0 : this.field.equals ("-1") ? -1 : 1);
+for (var i = 0, tok; i < nn; ++i) {
+switch (tok = this.fieldProperty (i)) {
+case 5:
+if (this.field.indexOf ('~') >= 0) this.field = JU.PT.rep (this.field, "~", "");
+case 2:
+case 3:
+this.modulated = true;
+ssgop = true;
+case 0:
+case 4:
+case 1:
+if (this.allowRotations || timeRev != 0 || ++n == 1) if (!this.modulated || ssgop) {
+if (tok == 1 || tok == 3) {
+isMag = true;
+timeRev = (this.field.endsWith (",+1") || this.field.endsWith (",1") ? 1 : this.field.endsWith (",-1") ? -1 : 0);
+if (timeRev != 0) this.field = this.field.substring (0, this.field.lastIndexOf (','));
+}if (timeRev != 0) this.field += "," + (timeRev == 1 ? "m" : "-m");
+this.field = this.field.$replace (';', ' ');
+this.symops.addLast (this.field);
+this.setSymmetryOperator (this.field);
+}break;
+case 9:
+case 10:
+case 11:
+isMag = true;
+if (this.magCenterings == null) this.magCenterings =  new JU.Lst ();
+this.magCenterings.addLast (this.field);
+break;
+}
+}
+}
+if (this.ms != null && !isMag) this.addLatticeVectors ();
+});
+Clazz.defineMethod (c$, "getBondOrder", 
+function (field) {
+switch (field.toUpperCase ().charAt (0)) {
+default:
+JU.Logger.warn ("unknown CIF bond order: " + field);
+case '\0':
+case 'S':
+return 1;
+case 'D':
+return 2;
+case 'T':
+return 3;
+case 'Q':
+return 4;
+case 'A':
+this.haveAromatic = true;
+return 515;
+}
+}, "~S");
+Clazz.defineMethod (c$, "processGeomBondLoopBlock", 
+ function () {
+var bondLoopBug = (this.stateScriptVersionInt >= 130304 && this.stateScriptVersionInt < 140403 || this.stateScriptVersionInt >= 150000 && this.stateScriptVersionInt < 150403);
+this.parseLoopParameters (J.adapter.readers.cif.CifReader.geomBondFields);
+if (bondLoopBug || !this.checkAllFieldsPresent (J.adapter.readers.cif.CifReader.geomBondFields, 2, true)) {
+this.parser.skipLoop (false);
+return;
+}var bondCount = 0;
+var name1;
+var name2 = null;
+while (this.parser.getData ()) {
+name2 = null;
+if (this.asc.getAtomIndex (name1 = this.getField (0)) < 0 || this.asc.getAtomIndex (name2 = this.getField (1)) < 0) {
+if (name2 == null && this.asc.getAtomIndex (name1 = name1.toUpperCase ()) < 0 || this.asc.getAtomIndex (name2 = name2.toUpperCase ()) < 0) continue;
+}var order = this.getBondOrder (this.getField (3));
+var sdist = this.getField (2);
+var distance = this.parseFloatStr (sdist);
+if (distance == 0 || Float.isNaN (distance)) {
+if (!this.iHaveFractionalCoordinates) {
+var a = this.asc.getAtomFromName (name1);
+var b = this.asc.getAtomFromName (name2);
+if (a != null && b != null) this.asc.addNewBondWithOrder (a.index, b.index, order);
+}continue;
+}var dx = 0;
+var pt = sdist.indexOf ('(');
+if (pt >= 0) {
+var data = sdist.toCharArray ();
+var sdx = sdist.substring (pt + 1, sdist.length - 1);
+var n = sdx.length;
+for (var j = pt; --j >= 0; ) {
+if (data[j] == '.' && --j < 0) break;
+data[j] = (--n < 0 ? '0' : sdx.charAt (n));
+}
+dx = this.parseFloatStr (String.valueOf (data));
+if (Float.isNaN (dx)) {
+JU.Logger.info ("error reading uncertainty for " + this.line);
+dx = 0.015;
+}} else {
+dx = 0.015;
+}bondCount++;
+this.bondTypes.addLast ( Clazz.newArray (-1, [name1, name2, Float.$valueOf (distance), Float.$valueOf (dx), Integer.$valueOf (order)]));
+}
+if (bondCount > 0) {
+JU.Logger.info (bondCount + " bonds read");
+if (!this.doApplySymmetry) {
+this.isMolecular = true;
+this.forceSymmetry (false);
+}}});
+Clazz.defineMethod (c$, "setBondingAndMolecules", 
+ function () {
+this.atoms = this.asc.atoms;
+this.firstAtom = this.asc.getLastAtomSetAtomIndex ();
+var nat = this.asc.getLastAtomSetAtomCount ();
+this.ac = this.firstAtom + nat;
+JU.Logger.info ("CIF creating molecule for " + nat + " atoms " + (this.bondTypes.size () > 0 ? " using GEOM_BOND records" : ""));
+this.bsSets =  new Array (nat);
+this.symmetry = this.asc.getSymmetry ();
+for (var i = this.firstAtom; i < this.ac; i++) {
+var ipt = this.asc.getAtomFromName (this.atoms[i].atomName).index - this.firstAtom;
+if (this.bsSets[ipt] == null) this.bsSets[ipt] =  new JU.BS ();
+this.bsSets[ipt].set (i - this.firstAtom);
+}
+if (this.isMolecular) {
+this.atomRadius =  Clazz.newFloatArray (this.ac, 0);
+for (var i = this.firstAtom; i < this.ac; i++) {
+var elemnoWithIsotope = J.api.JmolAdapter.getElementNumber (this.atoms[i].getElementSymbol ());
+this.atoms[i].elementNumber = elemnoWithIsotope;
+var charge = (this.atoms[i].formalCharge == -2147483648 ? 0 : this.atoms[i].formalCharge);
+if (elemnoWithIsotope > 0) this.atomRadius[i] = J.api.JmolAdapter.getBondingRadius (elemnoWithIsotope, charge);
+}
+this.bsConnected =  new Array (this.ac);
+for (var i = this.firstAtom; i < this.ac; i++) this.bsConnected[i] =  new JU.BS ();
+
+this.bsMolecule =  new JU.BS ();
+this.bsExclude =  new JU.BS ();
+}var isFirst = true;
+this.bsBondDuplicates =  new JU.BS ();
+while (this.createBonds (isFirst)) {
+isFirst = false;
+}
+if (this.isMolecular && this.iHaveFractionalCoordinates && !this.bsMolecule.isEmpty ()) {
+if (this.asc.bsAtoms == null) this.asc.bsAtoms =  new JU.BS ();
+this.asc.bsAtoms.clearBits (this.firstAtom, this.ac);
+this.asc.bsAtoms.or (this.bsMolecule);
+this.asc.bsAtoms.andNot (this.bsExclude);
+for (var i = this.firstAtom; i < this.ac; i++) {
+if (this.asc.bsAtoms.get (i)) this.symmetry.toCartesian (this.atoms[i], true);
+ else if (this.debugging) JU.Logger.debug (this.molecularType + " removing " + i + " " + this.atoms[i].atomName + " " + this.atoms[i]);
+}
+this.asc.setCurrentModelInfo ("unitCellParams", null);
+if (this.nMolecular++ == this.asc.iSet) {
+this.asc.clearGlobalBoolean (0);
+this.asc.clearGlobalBoolean (1);
+this.asc.clearGlobalBoolean (2);
+}}if (this.bondTypes.size () > 0) this.asc.setCurrentModelInfo ("hasBonds", Boolean.TRUE);
+this.bondTypes.clear ();
+this.atomRadius = null;
+this.bsSets = null;
+this.bsConnected = null;
+this.bsMolecule = null;
+this.bsExclude = null;
+});
+Clazz.defineMethod (c$, "fixAtomForBonding", 
+ function (pt, i) {
+pt.setT (this.atoms[i]);
+if (this.iHaveFractionalCoordinates) this.symmetry.toCartesian (pt, true);
+}, "JU.P3,~N");
+Clazz.defineMethod (c$, "createBonds", 
+ function (doInit) {
+var list = "";
+var haveH = false;
+for (var i = this.bondTypes.size (); --i >= 0; ) {
+if (this.bsBondDuplicates.get (i)) continue;
+var o = this.bondTypes.get (i);
+var distance = (o[2]).floatValue ();
+var dx = (o[3]).floatValue ();
+var order = (o[4]).intValue ();
+var iatom1 = this.asc.getAtomIndex (o[0]);
+var iatom2 = this.asc.getAtomIndex (o[1]);
+if (doInit) {
+var key = ";" + iatom1 + ";" + iatom2 + ";" + distance;
+if (list.indexOf (key) >= 0) {
+this.bsBondDuplicates.set (i);
+continue;
+}list += key;
+}var bs1 = this.bsSets[iatom1 - this.firstAtom];
+var bs2 = this.bsSets[iatom2 - this.firstAtom];
+if (bs1 == null || bs2 == null) continue;
+if (this.atoms[iatom1].elementNumber == 1 || this.atoms[iatom2].elementNumber == 1) haveH = true;
+for (var j = bs1.nextSetBit (0); j >= 0; j = bs1.nextSetBit (j + 1)) {
+for (var k = bs2.nextSetBit (0); k >= 0; k = bs2.nextSetBit (k + 1)) {
+if ((!this.isMolecular || !this.bsConnected[j + this.firstAtom].get (k)) && this.checkBondDistance (this.atoms[j + this.firstAtom], this.atoms[k + this.firstAtom], distance, dx)) this.addNewBond (j + this.firstAtom, k + this.firstAtom, order);
+}
+}
+}
+if (!this.iHaveFractionalCoordinates) return false;
+if (this.bondTypes.size () > 0 && !haveH) for (var i = this.firstAtom; i < this.ac; i++) if (this.atoms[i].elementNumber == 1) {
+var checkAltLoc = (this.atoms[i].altLoc != '\0');
+for (var k = this.firstAtom; k < this.ac; k++) if (k != i && this.atoms[k].elementNumber != 1 && (!checkAltLoc || this.atoms[k].altLoc == '\0' || this.atoms[k].altLoc == this.atoms[i].altLoc)) {
+if (!this.bsConnected[i].get (k) && this.checkBondDistance (this.atoms[i], this.atoms[k], 1.1, 0)) this.addNewBond (i, k, 1);
+}
+}
+if (!this.isMolecular) return false;
+if (doInit) for (var i = this.firstAtom; i < this.ac; i++) if (this.atoms[i].atomSite + this.firstAtom == i && !this.bsMolecule.get (i)) this.setBs (this.atoms, i, this.bsConnected, this.bsMolecule);
+
+var bondTolerance = this.vwr.getFloat (570425348);
+var bsBranch =  new JU.BS ();
+var cart1 =  new JU.P3 ();
+var cart2 =  new JU.P3 ();
+var nFactor = 2;
+for (var i = this.firstAtom; i < this.ac; i++) if (!this.bsMolecule.get (i) && !this.bsExclude.get (i)) for (var j = this.bsMolecule.nextSetBit (0); j >= 0; j = this.bsMolecule.nextSetBit (j + 1)) if (this.symmetry.checkDistance (this.atoms[j], this.atoms[i], this.atomRadius[i] + this.atomRadius[j] + bondTolerance, 0, nFactor, nFactor, nFactor, this.ptOffset)) {
+this.setBs (this.atoms, i, this.bsConnected, bsBranch);
+for (var k = bsBranch.nextSetBit (0); k >= 0; k = bsBranch.nextSetBit (k + 1)) {
+this.atoms[k].add (this.ptOffset);
+this.fixAtomForBonding (cart1, k);
+var bs = this.bsSets[this.asc.getAtomIndex (this.atoms[k].atomName) - this.firstAtom];
+if (bs != null) for (var ii = bs.nextSetBit (0); ii >= 0; ii = bs.nextSetBit (ii + 1)) {
+if (ii + this.firstAtom == k) continue;
+this.fixAtomForBonding (cart2, ii + this.firstAtom);
+if (cart2.distance (cart1) < 0.1) {
+this.bsExclude.set (k);
+break;
+}}
+this.bsMolecule.set (k);
+}
+return true;
+}
+
+return false;
+}, "~B");
+Clazz.defineMethod (c$, "checkBondDistance", 
+ function (a, b, distance, dx) {
+if (this.iHaveFractionalCoordinates) return this.symmetry.checkDistance (a, b, distance, dx, 0, 0, 0, this.ptOffset);
+var d = a.distance (b);
+return (dx > 0 ? Math.abs (d - distance) <= dx : d <= distance && d > 0.1);
+}, "J.adapter.smarter.Atom,J.adapter.smarter.Atom,~N,~N");
+Clazz.defineMethod (c$, "addNewBond", 
+ function (i, j, order) {
+this.asc.addNewBondWithOrder (i, j, order);
+if (!this.isMolecular) return;
+this.bsConnected[i].set (j);
+this.bsConnected[j].set (i);
+}, "~N,~N,~N");
+Clazz.defineMethod (c$, "setBs", 
+ function (atoms, iatom, bsBonds, bs) {
+var bsBond = bsBonds[iatom];
+bs.set (iatom);
+for (var i = bsBond.nextSetBit (0); i >= 0; i = bsBond.nextSetBit (i + 1)) {
+if (!bs.get (i)) this.setBs (atoms, i, bsBonds, bs);
+}
+}, "~A,~N,~A,JU.BS");
+Clazz.defineMethod (c$, "checkSubclassSymmetry", 
+function () {
+return this.doCheckUnitCell;
+});
+Clazz.defineMethod (c$, "checkAllFieldsPresent", 
+function (keys, lastKey, critical) {
+for (var i = (lastKey < 0 ? keys.length : lastKey); --i >= 0; ) if (this.key2col[i] == -1) {
+if (critical) JU.Logger.warn ("CIF reader missing property: " + keys[i]);
+return false;
+}
+return true;
+}, "~A,~N,~B");
+Clazz.defineMethod (c$, "getField", 
+function (type) {
+var i = this.key2col[type];
+return (i == -1 ? "\0" : this.parser.getColumnData (i));
+}, "~N");
+Clazz.defineMethod (c$, "isNull", 
+function (key) {
+return key.equals ("\0");
+}, "~S");
+Clazz.defineStatics (c$,
+"titleRecords", "__citation_title__publ_section_title__active_magnetic_irreps_details__",
+"TransformFields",  Clazz.newArray (-1, ["x[1][1]", "x[1][2]", "x[1][3]", "r[1]", "x[2][1]", "x[2][2]", "x[2][3]", "r[2]", "x[3][1]", "x[3][2]", "x[3][3]", "r[3]"]),
+"ATOM_TYPE_SYMBOL", 0,
+"ATOM_TYPE_OXIDATION_NUMBER", 1,
+"atomTypeFields",  Clazz.newArray (-1, ["_atom_type_symbol", "_atom_type_oxidation_number"]),
+"NONE", -1,
+"TYPE_SYMBOL", 0,
+"LABEL", 1,
+"AUTH_ATOM", 2,
+"FRACT_X", 3,
+"FRACT_Y", 4,
+"FRACT_Z", 5,
+"CARTN_X", 6,
+"CARTN_Y", 7,
+"CARTN_Z", 8,
+"OCCUPANCY", 9,
+"B_ISO", 10,
+"COMP_ID", 11,
+"AUTH_ASYM_ID", 12,
+"AUTH_SEQ_ID", 13,
+"INS_CODE", 14,
+"ALT_ID", 15,
+"GROUP_PDB", 16,
+"MODEL_NO", 17,
+"DUMMY_ATOM", 18,
+"DISORDER_GROUP", 19,
+"ANISO_LABEL", 20,
+"ANISO_MMCIF_ID", 21,
+"ANISO_U11", 22,
+"ANISO_U22", 23,
+"ANISO_U33", 24,
+"ANISO_U12", 25,
+"ANISO_U13", 26,
+"ANISO_U23", 27,
+"ANISO_MMCIF_U11", 28,
+"ANISO_MMCIF_U22", 29,
+"ANISO_MMCIF_U33", 30,
+"ANISO_MMCIF_U12", 31,
+"ANISO_MMCIF_U13", 32,
+"ANISO_MMCIF_U23", 33,
+"U_ISO_OR_EQUIV", 34,
+"ANISO_B11", 35,
+"ANISO_B22", 36,
+"ANISO_B33", 37,
+"ANISO_B12", 38,
+"ANISO_B13", 39,
+"ANISO_B23", 40,
+"ANISO_BETA_11", 41,
+"ANISO_BETA_22", 42,
+"ANISO_BETA_33", 43,
+"ANISO_BETA_12", 44,
+"ANISO_BETA_13", 45,
+"ANISO_BETA_23", 46,
+"ADP_TYPE", 47,
+"CC_COMP_ID", 48,
+"CC_ATOM_ID", 49,
+"CC_ATOM_SYM", 50,
+"CC_ATOM_CHARGE", 51,
+"CC_ATOM_X", 52,
+"CC_ATOM_Y", 53,
+"CC_ATOM_Z", 54,
+"CC_ATOM_X_IDEAL", 55,
+"CC_ATOM_Y_IDEAL", 56,
+"CC_ATOM_Z_IDEAL", 57,
+"DISORDER_ASSEMBLY", 58,
+"ASYM_ID", 59,
+"SUBSYS_ID", 60,
+"SITE_MULT", 61,
+"THERMAL_TYPE", 62,
+"MOMENT_LABEL", 63,
+"MOMENT_PRELIM_X", 64,
+"MOMENT_PRELIM_Y", 65,
+"MOMENT_PRELIM_Z", 66,
+"MOMENT_X", 67,
+"MOMENT_Y", 68,
+"MOMENT_Z", 69,
+"ATOM_ID", 70,
+"SEQ_ID", 71,
+"FAMILY_ATOM", "_atom_site",
+"atomFields",  Clazz.newArray (-1, ["*_type_symbol", "*_label", "*_auth_atom_id", "*_fract_x", "*_fract_y", "*_fract_z", "*_cartn_x", "*_cartn_y", "*_cartn_z", "*_occupancy", "*_b_iso_or_equiv", "*_auth_comp_id", "*_auth_asym_id", "*_auth_seq_id", "*_pdbx_pdb_ins_code", "*_label_alt_id", "*_group_pdb", "*_pdbx_pdb_model_num", "*_calc_flag", "*_disorder_group", "*_aniso_label", "*_anisotrop_id", "*_aniso_u_11", "*_aniso_u_22", "*_aniso_u_33", "*_aniso_u_12", "*_aniso_u_13", "*_aniso_u_23", "*_anisotrop_u[1][1]", "*_anisotrop_u[2][2]", "*_anisotrop_u[3][3]", "*_anisotrop_u[1][2]", "*_anisotrop_u[1][3]", "*_anisotrop_u[2][3]", "*_u_iso_or_equiv", "*_aniso_b_11", "*_aniso_b_22", "*_aniso_b_33", "*_aniso_b_12", "*_aniso_b_13", "*_aniso_b_23", "*_aniso_beta_11", "*_aniso_beta_22", "*_aniso_beta_33", "*_aniso_beta_12", "*_aniso_beta_13", "*_aniso_beta_23", "*_adp_type", "_chem_comp_atom_comp_id", "_chem_comp_atom_atom_id", "_chem_comp_atom_type_symbol", "_chem_comp_atom_charge", "_chem_comp_atom_model_cartn_x", "_chem_comp_atom_model_cartn_y", "_chem_comp_atom_model_cartn_z", "_chem_comp_atom_pdbx_model_cartn_x_ideal", "_chem_comp_atom_pdbx_model_cartn_y_ideal", "_chem_comp_atom_pdbx_model_cartn_z_ideal", "*_disorder_assembly", "*_label_asym_id", "*_subsystem_code", "*_symmetry_multiplicity", "*_thermal_displace_type", "*_moment_label", "*_moment_crystalaxis_mx", "*_moment_crystalaxis_my", "*_moment_crystalaxis_mz", "*_moment_crystalaxis_x", "*_moment_crystalaxis_y", "*_moment_crystalaxis_z", "*_id", "*_label_seq_id"]));
+c$.singleAtomID = c$.prototype.singleAtomID = J.adapter.readers.cif.CifReader.atomFields[48];
+Clazz.defineStatics (c$,
+"CITATION_TITLE", 0,
+"citationFields",  Clazz.newArray (-1, ["_citation_title"]),
+"SYM_XYZ", 0,
+"SYM_MAGN_XYZ", 1,
+"SYM_SSG_ALG", 2,
+"SYM_MAGN_SSG_ALG", 3,
+"SYM_EQ_XYZ", 4,
+"SYM_SSG_EQ_XYZ", 5,
+"SYM_MAGN_REV", 7,
+"SYM_MAGN_SSG_REV", 8,
+"SYM_MAGN_REV_PRELIM", 6,
+"SYM_MAGN_CENTERING", 9,
+"SYM_MAGN_SSG_CENTERING", 10,
+"SYM_MAGN_SSG_CENT_XYZ", 11,
+"FAMILY_SGOP", "_space_group_symop",
+"symmetryOperationsFields",  Clazz.newArray (-1, ["*_operation_xyz", "*_magn_operation_xyz", "*_ssg_operation_algebraic", "*_magn_ssg_operation_algebraic", "_symmetry_equiv_pos_as_xyz", "_symmetry_ssg_equiv_pos_as_xyz", "*_magn_operation_timereversal", "*_magn_ssg_operation_timereversal", "*_operation_timereversal", "*_magn_centering_xyz", "*_magn_ssg_centering_algebraic", "*_magn_ssg_centering_xyz"]),
+"GEOM_BOND_ATOM_SITE_LABEL_1", 0,
+"GEOM_BOND_ATOM_SITE_LABEL_2", 1,
+"GEOM_BOND_DISTANCE", 2,
+"CCDC_GEOM_BOND_TYPE", 3,
+"geomBondFields",  Clazz.newArray (-1, ["_geom_bond_atom_site_label_1", "_geom_bond_atom_site_label_2", "_geom_bond_distance", "_ccdc_geom_bond_type"]));
+});

static/j2s/J/adapter/readers/cif/MMCifReader.js

@@ -0,0 +1,709 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (["J.adapter.readers.cif.CifReader"], "J.adapter.readers.cif.MMCifReader", ["java.lang.Boolean", "java.util.Hashtable", "JU.BS", "$.Lst", "$.M4", "$.P3", "$.PT", "$.SB", "J.adapter.smarter.Atom", "$.Structure", "J.c.STR", "JU.BSUtil", "$.Logger"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.isBiomolecule = false;
+this.byChain = false;
+this.bySymop = false;
+this.chainAtomMap = null;
+this.chainAtomCounts = null;
+this.vBiomolecules = null;
+this.htBiomts = null;
+this.htSites = null;
+this.htHetero = null;
+this.htBondMap = null;
+this.assemblyIdAtoms = null;
+this.thisChain = -1;
+this.modelIndex = 0;
+this.chainSum = null;
+this.chainAtomCount = null;
+this.isLigandBondBug = false;
+this.mident = null;
+this.requiresSorting = false;
+this.structConnMap = null;
+this.structConnList = "";
+this.doSetBonds = false;
+this.modelStrings = "";
+this.done = false;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.cif, "MMCifReader", J.adapter.readers.cif.CifReader);
+Clazz.overrideMethod (c$, "initSubclass", 
+function () {
+this.setIsPDB ();
+this.mident = JU.M4.newM4 (null);
+this.isMMCIF = true;
+if (this.isDSSP1) this.asc.setInfo ("isDSSP1", Boolean.TRUE);
+if (this.htParams.containsKey ("isMutate")) this.asc.setInfo ("isMutate", Boolean.TRUE);
+this.doSetBonds = this.checkFilterKey ("ADDBONDS");
+this.byChain = this.checkFilterKey ("BYCHAIN");
+if (this.checkFilterKey ("BIOMOLECULE")) this.filter = JU.PT.rep (this.filter, "BIOMOLECULE", "ASSEMBLY");
+this.isBiomolecule = this.checkFilterKey ("ASSEMBLY");
+if (this.isBiomolecule) {
+this.filter = this.filter.$replace (':', ' ');
+this.bySymop = this.checkFilterKey ("BYSYMOP");
+}this.isCourseGrained = this.byChain || this.bySymop;
+if (this.isCourseGrained) {
+this.chainAtomMap =  new java.util.Hashtable ();
+this.chainAtomCounts =  new java.util.Hashtable ();
+}this.isLigandBondBug = (this.stateScriptVersionInt >= 140204 && this.stateScriptVersionInt <= 140208 || this.stateScriptVersionInt >= 140304 && this.stateScriptVersionInt <= 140308);
+});
+Clazz.overrideMethod (c$, "processSubclassEntry", 
+function () {
+if (this.key0.startsWith ("_pdbx_struct_assembly_gen.") || this.key0.startsWith ("_struct_conn.") || this.key0.startsWith ("_struct_ref_seq_dif.") || this.key0.startsWith ("_struct_conf.") || this.key0.startsWith ("_struct_sheet_range.")) this.processSubclassLoopBlock ();
+ else if (this.key.equals ("_rna3d")) {
+this.addedData = this.data;
+this.addedDataKey = this.key;
+} else if (this.key.equals ("_dssr")) {
+this.dssr = this.vwr.parseJSONMap (this.reader.readLine ());
+this.reader.readLine ();
+}});
+Clazz.overrideMethod (c$, "processSubclassLoopBlock", 
+function () {
+if (this.key0.startsWith ("_struct_ncs_oper.")) return this.processStructOperListBlock (true);
+if (this.key0.startsWith ("_pdbx_struct_oper_list.")) return this.processStructOperListBlock (false);
+if (this.key0.startsWith ("_pdbx_struct_assembly_gen.")) return this.processAssemblyGenBlock ();
+if (this.key0.startsWith ("_struct_ref_seq_dif.")) return this.processSequence ();
+if (this.isCourseGrained) return false;
+if (this.key0.startsWith ("_struct_site_gen.")) return this.processStructSiteBlock ();
+if (this.key0.startsWith ("_chem_comp.")) return this.processChemCompLoopBlock ();
+if (this.key0.startsWith ("_struct_conf.")) return this.processStructConfLoopBlock ();
+if (this.key0.startsWith ("_struct_sheet_range.")) return this.processStructSheetRangeLoopBlock ();
+if (this.isLigandBondBug) return false;
+if (this.key0.startsWith ("_chem_comp_bond.")) return this.processCompBondLoopBlock ();
+if (this.key0.startsWith ("_struct_conn.")) return this.processStructConnLoopBlock ();
+return false;
+});
+Clazz.defineMethod (c$, "sortAssemblyModels", 
+function () {
+var natoms = this.asc.ac;
+var lastSet = -1;
+var atoms = this.asc.atoms;
+var newAtoms =  new Array (natoms);
+var ids = JU.PT.split ("," + this.modelStrings + ",", ",,");
+var bsAtomsNew = (this.asc.bsAtoms == null ? null : JU.BS.newN (this.asc.bsAtoms.size ()));
+for (var im = 1, n = 0; im < ids.length; im++) {
+var sModel = ids[im];
+var modelIndex = -1;
+for (var is = 0; is < this.asc.atomSetCount; is++) {
+var ia0 = this.asc.getAtomSetAtomIndex (is);
+var ia1 = ia0 + this.asc.getAtomSetAtomCount (is);
+var am = "" + this.modelMap.get ("_" + is);
+if (am.equals (sModel)) {
+if (modelIndex < 0 && (modelIndex = is) > lastSet) lastSet = is;
+for (var i = ia0; i < ia1; i++) {
+if (bsAtomsNew == null || this.asc.bsAtoms.get (i)) {
+if (bsAtomsNew != null) bsAtomsNew.set (n);
+atoms[i].atomSetIndex = modelIndex;
+newAtoms[n++] = atoms[i];
+}}
+}}
+}
+this.asc.atoms = newAtoms;
+this.asc.bsAtoms = bsAtomsNew;
+if (++lastSet < this.asc.atomSetCount) this.asc.atomSetCount = lastSet;
+});
+Clazz.overrideMethod (c$, "finalizeSubclass", 
+function () {
+if (this.byChain && !this.isBiomolecule) for (var id, $id = this.chainAtomMap.keySet ().iterator (); $id.hasNext () && ((id = $id.next ()) || true);) this.createParticle (id);
+
+var haveBiomolecule = (this.isBiomolecule && this.vBiomolecules != null && this.vBiomolecules.size () > 0);
+if (!this.isCourseGrained && this.asc.ac == this.nAtoms) {
+this.asc.removeCurrentAtomSet ();
+} else {
+if ((this.dssr != null || this.validation != null || this.addedData != null) && !this.isCourseGrained && !this.requiresSorting) {
+var vs = (this.getInterface ("J.adapter.readers.cif.MMCifValidationParser")).set (this);
+var note = null;
+if (this.addedData == null) {
+if (this.validation != null || this.dssr != null) note = vs.finalizeValidations (this.vwr, this.modelMap);
+} else if (this.addedDataKey.equals ("_rna3d")) {
+note = vs.finalizeRna3d (this.modelMap);
+}if (note != null) this.appendLoadNote (note);
+}this.setHetero ();
+if (this.doSetBonds) this.setBonds ();
+}if (this.asc.ac == 0 && !this.isCourseGrained) return false;
+var spaceGroup = this.sgName;
+if (this.htSites != null) this.addSites (this.htSites);
+if (haveBiomolecule) {
+this.asc.setCurrentModelInfo ("biomolecules", this.vBiomolecules);
+this.setBiomolecules ();
+if (this.thisBiomolecule != null) {
+if (this.iHaveFractionalCoordinates) this.fractionalizeCoordinates (false);
+this.asc.getXSymmetry ().applySymmetryBio (this.thisBiomolecule, this.applySymmetryToBonds, this.filter);
+this.asc.xtalSymmetry = null;
+}this.doCheckUnitCell = new Boolean (this.doCheckUnitCell & (this.iHaveUnitCell && this.doApplySymmetry)).valueOf ();
+if (this.doCheckUnitCell) {
+this.ignoreFileSpaceGroupName = true;
+this.sgName = spaceGroup;
+this.fractionalizeCoordinates (true);
+this.asc.setCurrentModelInfo ("biosymmetry", null);
+this.asc.setCurrentModelInfo ("biosymmetryCount", null);
+this.asc.checkSpecial = false;
+if (this.byChain) return true;
+}}if (this.latticeCells != null && this.latticeCells[0] != 0) this.addJmolScript ("unitcell;axes on;axes unitcell;");
+if (this.requiresSorting) this.sortAssemblyModels ();
+return true;
+});
+Clazz.overrideMethod (c$, "checkSubclassSymmetry", 
+function () {
+this.asc.checkSpecial = false;
+var modelIndex = this.asc.iSet;
+this.asc.setCurrentModelInfo ("PDB_CONECT_firstAtom_count_max",  Clazz.newIntArray (-1, [this.asc.getAtomSetAtomIndex (modelIndex), this.asc.getAtomSetAtomCount (modelIndex), this.maxSerial]));
+return false;
+});
+Clazz.defineMethod (c$, "setBonds", 
+ function () {
+if (this.htBondMap == null) return;
+var bsAtoms = this.asc.bsAtoms;
+if (bsAtoms == null) bsAtoms = JU.BSUtil.newBitSet2 (0, this.asc.ac);
+var atoms = this.asc.atoms;
+var seqid = -1;
+var comp = null;
+var map = null;
+for (var i = bsAtoms.nextSetBit (0); i >= 0; i = bsAtoms.nextSetBit (i + 1)) {
+var a = atoms[i];
+var pt = (a.vib == null ? a.sequenceNumber : a.vib.x);
+if (pt != seqid) {
+seqid = pt;
+if (comp != null) this.processBonds (this.htBondMap.get (comp), map, false);
+map =  new java.util.Hashtable ();
+comp = atoms[i].group3;
+if (!this.htBondMap.containsKey (comp)) {
+comp = null;
+continue;
+}}if (comp == null) continue;
+map.put (a.atomName, Integer.$valueOf (a.index));
+}
+if (comp != null) this.processBonds (this.htBondMap.get (comp), map, false);
+if (this.structConnMap != null) {
+map =  new java.util.Hashtable ();
+seqid = -1;
+comp = null;
+for (var i = bsAtoms.nextSetBit (0); i >= 0; i = bsAtoms.nextSetBit (i + 1)) {
+var a = atoms[i];
+var pt = (a.vib == null ? a.sequenceNumber : a.vib.x);
+if (pt != seqid) {
+seqid = pt;
+var ckey = a.chainID + a.group3 + seqid;
+if (this.structConnList.indexOf (ckey) < 0) {
+comp = null;
+continue;
+}comp = ckey;
+}if (comp == null) continue;
+map.put (comp + a.atomName + a.altLoc, Integer.$valueOf (a.index));
+}
+this.processBonds (this.structConnMap, map, true);
+}this.appendLoadNote (this.asc.bondCount + " bonds added");
+});
+Clazz.defineMethod (c$, "processBonds", 
+ function (cmap, map, isStructConn) {
+var i1;
+var i2;
+for (var i = 0, n = cmap.size (); i < n; i++) {
+var o = cmap.get (i);
+if ((i1 = map.get (o[0])) == null || (i2 = map.get (o[1])) == null) continue;
+if (this.debugging) JU.Logger.debug ((isStructConn ? "_struct_conn" : "_comp_bond") + " adding bond " + i1 + " " + i2 + " order=" + o[2]);
+this.asc.addNewBondWithOrder (i1.intValue (), i2.intValue (), (o[2]).intValue ());
+}
+}, "JU.Lst,java.util.Map,~B");
+Clazz.defineMethod (c$, "processSequence", 
+ function () {
+this.parseLoopParameters (J.adapter.readers.cif.MMCifReader.structRefFields);
+var g1;
+var g3;
+while (this.parser.getData ()) {
+if (this.isNull (g1 = this.getField (1).toLowerCase ()) || g1.length != 1 || this.isNull (g3 = this.getField (0))) continue;
+if (this.htGroup1 == null) this.asc.setInfo ("htGroup1", this.htGroup1 =  new java.util.Hashtable ());
+this.htGroup1.put (g3, g1);
+}
+return true;
+});
+Clazz.defineMethod (c$, "processAssemblyGenBlock", 
+ function () {
+this.parseLoopParameters (J.adapter.readers.cif.MMCifReader.assemblyFields);
+while (this.parser.getData ()) {
+var assem =  new Array (3);
+var count = 0;
+var p;
+var n = this.parser.getColumnCount ();
+for (var i = 0; i < n; ++i) {
+switch (p = this.fieldProperty (i)) {
+case 0:
+case 1:
+case 2:
+count++;
+assem[p] = this.field;
+break;
+}
+}
+if (count == 3) this.addAssembly (assem);
+}
+return true;
+});
+Clazz.defineMethod (c$, "addAssembly", 
+function (assem) {
+var id = assem[0];
+var list = assem[2];
+var operators = assem[1];
+var name = "biomolecule " + id;
+JU.Logger.info (name + " operators " + operators + " ASYM_IDs " + list);
+this.appendLoadNote ("found " + name + ": " + list);
+if (this.vBiomolecules == null) this.vBiomolecules =  new JU.Lst ();
+var info = null;
+for (var i = this.vBiomolecules.size (); --i >= 0; ) if (this.vBiomolecules.get (i).get ("name").equals (name)) {
+info = this.vBiomolecules.get (i);
+break;
+}
+if (info == null) {
+info =  new java.util.Hashtable ();
+info.put ("name", name);
+var iMolecule = this.parseIntStr (id);
+info.put ("molecule", iMolecule == -2147483648 ? id : Integer.$valueOf (iMolecule));
+info.put ("biomts",  new JU.Lst ());
+info.put ("chains",  new JU.Lst ());
+info.put ("assemblies",  new JU.Lst ());
+info.put ("operators",  new JU.Lst ());
+this.vBiomolecules.addLast (info);
+}(info.get ("assemblies")).addLast ("$" + list.$replace (',', '$'));
+(info.get ("operators")).addLast (this.decodeAssemblyOperators (operators));
+this.checkFilterAssembly (id, info);
+}, "~A");
+Clazz.defineMethod (c$, "checkFilterAssembly", 
+function (id, info) {
+if (this.checkFilterKey ("ASSEMBLY " + id + ";") || this.checkFilterKey ("ASSEMBLY=" + id + ";")) this.thisBiomolecule = info;
+}, "~S,java.util.Map");
+Clazz.defineMethod (c$, "decodeAssemblyOperators", 
+ function (ops) {
+var pt = ops.indexOf (")(");
+if (pt >= 0) return this.crossBinary (this.decodeAssemblyOperators (ops.substring (0, pt + 1)), this.decodeAssemblyOperators (ops.substring (pt + 1)));
+if (ops.startsWith ("(")) {
+if (ops.indexOf ("-") >= 0) ops = JU.BS.unescape ("({" + ops.substring (1, ops.length - 1).$replace ('-', ':').$replace (',', ' ') + "})").toJSON ();
+ops = JU.PT.rep (ops, " ", "");
+ops = ops.substring (1, ops.length - 1);
+}return ops;
+}, "~S");
+Clazz.defineMethod (c$, "crossBinary", 
+ function (ops1, ops2) {
+var sb =  new JU.SB ();
+var opsLeft = JU.PT.split (ops1, ",");
+var opsRight = JU.PT.split (ops2, ",");
+for (var i = 0; i < opsLeft.length; i++) for (var j = 0; j < opsRight.length; j++) sb.append (",").append (opsLeft[i]).append ("|").append (opsRight[j]);
+
+
+return sb.toString ().substring (1);
+}, "~S,~S");
+Clazz.defineMethod (c$, "processStructOperListBlock", 
+ function (isNCS) {
+this.parseLoopParametersFor ((isNCS ? "_struct_ncs_oper" : "_pdbx_struct_oper_list"), isNCS ? J.adapter.readers.cif.MMCifReader.ncsoperFields : J.adapter.readers.cif.MMCifReader.operFields);
+var m =  Clazz.newFloatArray (16, 0);
+m[15] = 1;
+while (this.parser.getData ()) {
+var count = 0;
+var id = null;
+var xyz = null;
+var n = this.parser.getColumnCount ();
+for (var i = 0; i < n; ++i) {
+var p = this.fieldProperty (i);
+switch (p) {
+case -1:
+break;
+case 12:
+id = this.field;
+break;
+case 13:
+xyz = this.field;
+break;
+default:
+m[p] = this.parseFloatStr (this.field);
+++count;
+}
+}
+if (id != null && (count == 12 || xyz != null && this.symmetry != null)) {
+JU.Logger.info ((isNCS ? "noncrystallographic symmetry operator " : "assembly operator ") + id + " " + xyz);
+var m4 =  new JU.M4 ();
+if (count != 12) {
+this.symmetry.getMatrixFromString (xyz, m, false, 0);
+m[3] *= this.symmetry.getUnitCellInfoType (0) / 12;
+m[7] *= this.symmetry.getUnitCellInfoType (1) / 12;
+m[11] *= this.symmetry.getUnitCellInfoType (2) / 12;
+}m4.setA (m);
+this.addMatrix (id, m4, isNCS);
+}}
+return true;
+}, "~B");
+Clazz.defineMethod (c$, "addMatrix", 
+function (id, m4, isNCS) {
+if (isNCS) {
+if (m4.equals (this.mident)) return;
+m4.m33 = 0;
+if (this.lstNCS == null) this.lstNCS =  new JU.Lst ();
+this.lstNCS.addLast (m4);
+} else {
+if (this.htBiomts == null) this.htBiomts =  new java.util.Hashtable ();
+this.htBiomts.put (id, m4);
+}}, "~S,JU.M4,~B");
+Clazz.defineMethod (c$, "processChemCompLoopBlock", 
+ function () {
+this.parseLoopParameters (J.adapter.readers.cif.MMCifReader.chemCompFields);
+var groupName;
+var hetName;
+while (this.parser.getData ()) if (!this.isNull (groupName = this.getField (0)) && !this.isNull (hetName = this.getField (1))) this.addHetero (groupName, hetName, true, true);
+
+return true;
+});
+Clazz.defineMethod (c$, "addHetero", 
+function (groupName, hetName, doCheck, addNote) {
+if (doCheck && !this.vwr.getJBR ().isHetero (groupName)) return;
+if (this.htHetero == null) this.htHetero =  new java.util.Hashtable ();
+if (doCheck && this.htHetero.containsKey (groupName)) return;
+this.htHetero.put (groupName, hetName);
+if (addNote) this.appendLoadNote (groupName + " = " + hetName);
+}, "~S,~S,~B,~B");
+Clazz.defineMethod (c$, "processStructConfLoopBlock", 
+ function () {
+if (this.ignoreStructure) {
+this.parser.skipLoop (false);
+return false;
+}this.parseLoopParametersFor ("_struct_conf", J.adapter.readers.cif.MMCifReader.structConfFields);
+if (!this.checkAllFieldsPresent (J.adapter.readers.cif.MMCifReader.structConfFields, -1, true)) {
+this.parser.skipLoop (true);
+return false;
+}while (this.parser.getData ()) {
+var structure =  new J.adapter.smarter.Structure (-1, J.c.STR.HELIX, J.c.STR.HELIX, null, 0, 0, null);
+var type = this.getField (0);
+if (type.startsWith ("TURN")) structure.structureType = structure.substructureType = J.c.STR.TURN;
+ else if (!type.startsWith ("HELX")) structure.structureType = structure.substructureType = J.c.STR.NONE;
+ else structure.substructureType = J.adapter.smarter.Structure.getHelixType (this.parseIntStr (this.getField (9)));
+structure.serialID = this.parseIntStr (this.getField (8));
+structure.structureID = this.getField (7);
+this.addStructure (structure);
+}
+return true;
+});
+Clazz.defineMethod (c$, "addStructure", 
+ function (structure) {
+structure.startChainID = this.vwr.getChainID (structure.startChainStr = this.getField (1), true);
+structure.startSequenceNumber = this.parseIntStr (this.getField (2));
+structure.startInsertionCode = this.getField (3).charAt (0);
+structure.endChainID = this.vwr.getChainID (structure.endChainStr = this.getField (4), true);
+structure.endSequenceNumber = this.parseIntStr (this.getField (5));
+structure.endInsertionCode = this.getField (6).charAt (0);
+this.asc.addStructure (structure);
+}, "J.adapter.smarter.Structure");
+Clazz.defineMethod (c$, "processStructSheetRangeLoopBlock", 
+ function () {
+if (this.ignoreStructure) {
+this.parser.skipLoop (false);
+return false;
+}this.parseLoopParametersFor ("_struct_sheet_range", J.adapter.readers.cif.MMCifReader.structSheetRangeFields);
+if (!this.checkAllFieldsPresent (J.adapter.readers.cif.MMCifReader.structSheetRangeFields, -1, true)) {
+this.parser.skipLoop (true);
+return false;
+}while (this.parser.getData ()) this.addStructure ( new J.adapter.smarter.Structure (-1, J.c.STR.SHEET, J.c.STR.SHEET, this.getField (0), this.parseIntStr (this.getField (7)), 1, null));
+
+return true;
+});
+Clazz.defineMethod (c$, "processStructSiteBlock", 
+ function () {
+this.parseLoopParametersFor ("_struct_site_gen", J.adapter.readers.cif.MMCifReader.structSiteFields);
+var htSite = null;
+this.htSites =  new java.util.Hashtable ();
+var seqNum;
+var resID;
+while (this.parser.getData ()) {
+if (this.isNull (seqNum = this.getField (3)) || this.isNull (resID = this.getField (1))) continue;
+var siteID = this.getField (0);
+htSite = this.htSites.get (siteID);
+if (htSite == null) {
+htSite =  new java.util.Hashtable ();
+htSite.put ("groups", "");
+this.htSites.put (siteID, htSite);
+}var insCode = this.getField (4);
+var chainID = this.getField (2);
+var group = "[" + resID + "]" + seqNum + (this.isNull (insCode) ? "" : "^" + insCode) + (this.isNull (chainID) ? "" : ":" + chainID);
+var groups = htSite.get ("groups");
+groups += (groups.length == 0 ? "" : ",") + group;
+htSite.put ("groups", groups);
+}
+return true;
+});
+Clazz.defineMethod (c$, "setBiomolecules", 
+ function () {
+if (this.assemblyIdAtoms == null && this.chainAtomCounts == null) return;
+var bsAll =  new JU.BS ();
+for (var i = this.vBiomolecules.size (); --i >= 0; ) {
+var biomolecule = this.vBiomolecules.get (i);
+this.setBiomolecule (biomolecule, (biomolecule === this.thisBiomolecule ? bsAll : null));
+}
+if (this.isBiomolecule && bsAll.cardinality () < this.asc.ac) {
+if (this.asc.bsAtoms != null) this.asc.bsAtoms.and (bsAll);
+ else if (!this.isCourseGrained) this.asc.bsAtoms = bsAll;
+}});
+Clazz.defineMethod (c$, "setBiomolecule", 
+ function (biomolecule, bsAll) {
+var biomtchains = biomolecule.get ("chains");
+var biomts = biomolecule.get ("biomts");
+var operators = biomolecule.get ("operators");
+var assemblies = biomolecule.get ("assemblies");
+var sum =  new JU.P3 ();
+var count = 0;
+var bsAtoms =  new JU.BS ();
+var nAtomsTotal = 0;
+var isBioCourse = (this.isBiomolecule && this.isCourseGrained);
+for (var i = operators.size (); --i >= 0; ) {
+var ops = JU.PT.split (operators.get (i), ",");
+var ids = JU.PT.split (assemblies.get (i), "$");
+var chainlist = "";
+var nAtoms = 0;
+for (var j = 1; j < ids.length; j++) {
+var id = ids[j];
+chainlist += ":" + id + ";";
+if (this.assemblyIdAtoms != null) {
+biomolecule.put ("asemblyIdAtoms", this.assemblyIdAtoms);
+var bs = this.assemblyIdAtoms.get (id);
+if (bs != null) {
+bsAtoms.or (bs);
+if (bsAll != null) bsAll.or (bs);
+nAtoms += bs.cardinality ();
+}} else if (isBioCourse) {
+var asum = this.chainAtomMap.get (id);
+if (asum != null) {
+if (this.bySymop) {
+sum.add (asum);
+count += this.chainAtomCounts.get (id)[0];
+} else {
+this.createParticle (id);
+nAtoms++;
+}}}}
+if (!this.isBiomolecule) continue;
+for (var j = 0; j < ops.length; j++) {
+var m = this.getOpMatrix (ops[j]);
+if (m == null) return 0;
+if (m.equals (this.mident)) {
+biomts.add (0, this.mident);
+biomtchains.add (0, chainlist);
+} else {
+biomts.addLast (m);
+biomtchains.addLast (chainlist);
+}}
+if (this.bySymop && bsAll != null) {
+nAtoms = 1;
+var a1 =  new J.adapter.smarter.Atom ();
+a1.setT (sum);
+a1.scale (1 / count);
+a1.radius = 16;
+this.asc.addAtom (a1);
+}nAtoms *= ops.length;
+nAtomsTotal += nAtoms;
+}
+biomolecule.put ("atomCount", Integer.$valueOf (nAtomsTotal));
+return nAtomsTotal;
+}, "java.util.Map,JU.BS");
+Clazz.defineMethod (c$, "createParticle", 
+ function (id) {
+var asum = this.chainAtomMap.get (id);
+var c = this.chainAtomCounts.get (id)[0];
+var a =  new J.adapter.smarter.Atom ();
+a.setT (asum);
+a.scale (1 / c);
+a.elementSymbol = "Pt";
+this.setChainID (a, id);
+a.radius = 16;
+this.asc.addAtom (a);
+}, "~S");
+Clazz.defineMethod (c$, "getOpMatrix", 
+ function (ops) {
+if (this.htBiomts == null) return JU.M4.newM4 (null);
+var pt = ops.indexOf ("|");
+if (pt >= 0) {
+var m = JU.M4.newM4 (this.htBiomts.get (ops.substring (0, pt)));
+m.mul (this.htBiomts.get (ops.substring (pt + 1)));
+return m;
+}return this.htBiomts.get (ops);
+}, "~S");
+Clazz.defineMethod (c$, "processStructConnLoopBlock", 
+ function () {
+this.parseLoopParametersFor ("_struct_conn", J.adapter.readers.cif.MMCifReader.structConnFields);
+while (this.parser.getData ()) {
+var sym1 = this.getField (5);
+var sym2 = this.getField (11);
+if (!sym1.equals (sym2) || !this.isNull (sym1) && !sym1.equals ("1_555")) continue;
+var type = this.getField (12);
+if (!type.startsWith ("covale") && !type.equals ("disulf") && !type.equals ("metalc")) continue;
+if (this.htBondMap == null) this.htBondMap =  new java.util.Hashtable ();
+var key1 = this.vwr.getChainID (this.getField (0), true) + this.getField (2) + this.parseFloatStr (this.getField (1)) + this.getField (3) + this.getField (4);
+var key2 = this.vwr.getChainID (this.getField (6), true) + this.getField (8) + this.parseFloatStr (this.getField (7)) + this.getField (9) + this.getField (10);
+var order = this.getBondOrder (this.getField (13));
+if (this.structConnMap == null) this.structConnMap =  new JU.Lst ();
+this.structConnMap.addLast ( Clazz.newArray (-1, [key1, key2, Integer.$valueOf (order)]));
+if (this.structConnList.indexOf (key1) < 0) this.structConnList += key1;
+if (this.structConnList.indexOf (key2) < 0) this.structConnList += key2;
+}
+return true;
+});
+Clazz.defineMethod (c$, "processCompBondLoopBlock", 
+ function () {
+this.doSetBonds = true;
+this.parseLoopParametersFor ("_chem_comp_bond", J.adapter.readers.cif.MMCifReader.chemCompBondFields);
+while (this.parser.getData ()) {
+var comp = this.getField (0);
+var atom1 = this.getField (1);
+var atom2 = this.getField (2);
+var order = this.getBondOrder (this.getField (3));
+if ((this.getField (4).charAt (0) == 'Y')) switch (order) {
+case 1:
+order = 513;
+break;
+case 2:
+order = 514;
+break;
+}
+if (this.isLigand) {
+this.asc.addNewBondWithOrderA (this.asc.getAtomFromName (atom1), this.asc.getAtomFromName (atom2), order);
+} else if (this.haveHAtoms || this.htHetero != null && this.htHetero.containsKey (comp)) {
+if (this.htBondMap == null) this.htBondMap =  new java.util.Hashtable ();
+var cmap = this.htBondMap.get (comp);
+if (cmap == null) this.htBondMap.put (comp, cmap =  new JU.Lst ());
+cmap.addLast ( Clazz.newArray (-1, [atom1, atom2, Integer.$valueOf (this.haveHAtoms ? order : 1)]));
+}}
+return true;
+});
+Clazz.overrideMethod (c$, "processSubclassAtom", 
+function (atom, assemblyId, strChain) {
+if (this.isBiomolecule) {
+if (this.isCourseGrained) {
+var sum = this.chainAtomMap.get (assemblyId);
+if (sum == null) {
+this.chainAtomMap.put (assemblyId, sum =  new JU.P3 ());
+this.chainAtomCounts.put (assemblyId,  Clazz.newIntArray (1, 0));
+}this.chainAtomCounts.get (assemblyId)[0]++;
+sum.add (atom);
+return false;
+}} else if (this.byChain) {
+if (this.thisChain != atom.chainID) {
+this.thisChain = atom.chainID;
+this.chainSum = this.chainAtomMap.get (strChain);
+if (this.chainSum == null) {
+this.chainAtomMap.put (strChain, this.chainSum =  new JU.P3 ());
+this.chainAtomCounts.put (strChain, this.chainAtomCount =  Clazz.newIntArray (1, 0));
+}}this.chainSum.add (atom);
+this.chainAtomCount[0]++;
+return false;
+}if (assemblyId != null) {
+if (this.assemblyIdAtoms == null) this.assemblyIdAtoms =  new java.util.Hashtable ();
+var bs = this.assemblyIdAtoms.get (assemblyId);
+if (bs == null) this.assemblyIdAtoms.put (assemblyId, bs =  new JU.BS ());
+bs.set (this.ac);
+}return true;
+}, "J.adapter.smarter.Atom,~S,~S");
+Clazz.overrideMethod (c$, "checkPDBModelField", 
+function (modelField, currentModelNo) {
+this.fieldProperty (modelField);
+var modelNo = this.parseIntStr (this.field);
+return (modelNo == currentModelNo ? modelNo : this.incrementModel (modelNo));
+}, "~N,~N");
+Clazz.defineMethod (c$, "incrementModel", 
+function (modelNo) {
+var isAssembly = (this.thisDataSetName != null && this.thisDataSetName.indexOf ("-assembly-") >= 0);
+if (isAssembly) {
+this.useFileModelNumbers = true;
+var key = "," + modelNo + ",";
+if (this.modelStrings.indexOf (key) >= 0) {
+this.requiresSorting = true;
+} else {
+this.modelStrings += key;
+}}if (this.iHaveDesiredModel && this.asc.atomSetCount > 0 && !isAssembly) {
+this.done = true;
+if (this.parser != null) {
+this.parser.skipLoop (false);
+this.skipping = false;
+}this.continuing = true;
+return -2147483648;
+}var modelNumberToUse = (this.useFileModelNumbers ? modelNo : ++this.modelIndex);
+this.setHetero ();
+this.newModel (modelNumberToUse);
+if (!this.skipping) {
+this.nextAtomSet ();
+if (this.modelMap == null || this.asc.ac == 0) this.modelMap =  new java.util.Hashtable ();
+this.modelMap.put ("" + modelNo, Integer.$valueOf (Math.max (0, this.asc.iSet)));
+this.modelMap.put ("_" + Math.max (0, this.asc.iSet), Integer.$valueOf (modelNo));
+}return modelNo;
+}, "~N");
+Clazz.defineMethod (c$, "setHetero", 
+ function () {
+if (this.htHetero != null) {
+this.asc.setCurrentModelInfo ("hetNames", this.htHetero);
+this.asc.setInfo ("hetNames", this.htHetero);
+}});
+Clazz.defineStatics (c$,
+"OPER_ID", 12,
+"OPER_XYZ", 13,
+"FAMILY_NCS_CAT", "_struct_ncs_oper.",
+"FAMILY_NCS", "_struct_ncs_oper",
+"ncsoperFields",  Clazz.newArray (-1, ["*_matrix[1][1]", "*_matrix[1][2]", "*_matrix[1][3]", "*_vector[1]", "*_matrix[2][1]", "*_matrix[2][2]", "*_matrix[2][3]", "*_vector[2]", "*_matrix[3][1]", "*_matrix[3][2]", "*_matrix[3][3]", "*_vector[3]", "*_id", "*_symmetry_operation"]),
+"FAMILY_OPER_CAT", "_pdbx_struct_oper_list.",
+"FAMILY_OPER", "_pdbx_struct_oper_list",
+"operFields",  Clazz.newArray (-1, ["*_matrix[1][1]", "*_matrix[1][2]", "*_matrix[1][3]", "*_vector[1]", "*_matrix[2][1]", "*_matrix[2][2]", "*_matrix[2][3]", "*_vector[2]", "*_matrix[3][1]", "*_matrix[3][2]", "*_matrix[3][3]", "*_vector[3]", "*_id", "*_symmetry_operation"]),
+"ASSEM_ID", 0,
+"ASSEM_OPERS", 1,
+"ASSEM_LIST", 2,
+"FAMILY_ASSEM_CAT", "_pdbx_struct_assembly_gen.",
+"assemblyFields",  Clazz.newArray (-1, ["_pdbx_struct_assembly_gen_assembly_id", "_pdbx_struct_assembly_gen_oper_expression", "_pdbx_struct_assembly_gen_asym_id_list"]),
+"FAMILY_SEQUENCEDIF_CAT", "_struct_ref_seq_dif.",
+"STRUCT_REF_G3", 0,
+"STRUCT_REF_G1", 1,
+"structRefFields",  Clazz.newArray (-1, ["_struct_ref_seq_dif_mon_id", "_struct_ref_seq_dif_db_mon_id"]),
+"CHEM_COMP_ID", 0,
+"CHEM_COMP_NAME", 1,
+"FAMILY_CHEMCOMP_CAT", "_chem_comp.",
+"chemCompFields",  Clazz.newArray (-1, ["_chem_comp_id", "_chem_comp_name"]),
+"CONF_TYPE_ID", 0,
+"BEG_ASYM_ID", 1,
+"BEG_SEQ_ID", 2,
+"BEG_INS_CODE", 3,
+"END_ASYM_ID", 4,
+"END_SEQ_ID", 5,
+"END_INS_CODE", 6,
+"STRUCT_ID", 7,
+"SERIAL_NO", 8,
+"HELIX_CLASS", 9,
+"FAMILY_STRUCTCONF_CAT", "_struct_conf.",
+"FAMILY_STRUCTCONF", "_struct_conf",
+"structConfFields",  Clazz.newArray (-1, ["*_conf_type_id", "*_beg_auth_asym_id", "*_beg_auth_seq_id", "*_pdbx_beg_pdb_ins_code", "*_end_auth_asym_id", "*_end_auth_seq_id", "*_pdbx_end_pdb_ins_code", "*_id", "*_pdbx_pdb_helix_id", "*_pdbx_pdb_helix_class"]),
+"SHEET_ID", 0,
+"STRAND_ID", 7,
+"FAMILY_SHEET_CAT", "_struct_sheet_range.",
+"FAMILY_SHEET", "_struct_sheet_range",
+"structSheetRangeFields",  Clazz.newArray (-1, ["*_sheet_id", "*_beg_auth_asym_id", "*_beg_auth_seq_id", "*_pdbx_beg_pdb_ins_code", "*_end_auth_asym_id", "*_end_auth_seq_id", "*_pdbx_end_pdb_ins_code", "*_id"]),
+"SITE_ID", 0,
+"SITE_COMP_ID", 1,
+"SITE_ASYM_ID", 2,
+"SITE_SEQ_ID", 3,
+"SITE_INS_CODE", 4,
+"FAMILY_STRUCSITE_CAT", "_struct_site_gen.",
+"FAMILY_STRUCSITE", "_struct_site_gen",
+"structSiteFields",  Clazz.newArray (-1, ["*_site_id", "*_auth_comp_id", "*_auth_asym_id", "*_auth_seq_id", "*_label_alt_id"]),
+"STRUCT_CONN_ASYM1", 0,
+"STRUCT_CONN_SEQ1", 1,
+"STRUCT_CONN_COMP1", 2,
+"STRUCT_CONN_ATOM1", 3,
+"STRUCT_CONN_ALT1", 4,
+"STRUCT_CONN_SYMM1", 5,
+"STRUCT_CONN_ASYM2", 6,
+"STRUCT_CONN_SEQ2", 7,
+"STRUCT_CONN_COMP2", 8,
+"STRUCT_CONN_ATOM2", 9,
+"STRUCT_CONN_ALT2", 10,
+"STRUCT_CONN_SYMM2", 11,
+"STRUCT_CONN_TYPE", 12,
+"STRUCT_CONN_ORDER", 13,
+"FAMILY_STRUCTCONN_CAT", "_struct_conn.",
+"FAMILY_STRUCTCONN", "_struct_conn",
+"structConnFields",  Clazz.newArray (-1, ["*_ptnr1_auth_asym_id", "*_ptnr1_auth_seq_id", "*_ptnr1_auth_comp_id", "*_ptnr1_label_atom_id", "*_pdbx_ptnr1_label_alt_id", "*_ptnr1_symmetry", "*_ptnr2_auth_asym_id", "*_ptnr2_auth_seq_id", "*_ptnr2_auth_comp_id", "*_ptnr2_label_atom_id", "*_pdbx_ptnr2_label_alt_id", "*_ptnr2_symmetry", "*_conn_type_id", "*_pdbx_value_order"]),
+"CHEM_COMP_BOND_ID", 0,
+"CHEM_COMP_BOND_ATOM_ID_1", 1,
+"CHEM_COMP_BOND_ATOM_ID_2", 2,
+"CHEM_COMP_BOND_VALUE_ORDER", 3,
+"CHEM_COMP_BOND_AROMATIC_FLAG", 4,
+"FAMILY_COMPBOND_CAT", "_chem_comp_bond.",
+"FAMILY_COMPBOND", "_chem_comp_bond",
+"chemCompBondFields",  Clazz.newArray (-1, ["*_comp_id", "*_atom_id_1", "*_atom_id_2", "*_value_order", "*_pdbx_aromatic_flag"]));
+});

static/j2s/J/adapter/readers/cif/MMCifValidationParser.js

@@ -0,0 +1,132 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (null, "J.adapter.readers.cif.MMCifValidationParser", ["java.lang.Character", "java.util.Hashtable", "JU.Lst", "$.PT", "JS.SV", "JU.Logger"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.asResidues = false;
+this.reader = null;
+this.resMap = null;
+this.atomMap = null;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.cif, "MMCifValidationParser");
+Clazz.makeConstructor (c$, 
+function () {
+});
+Clazz.defineMethod (c$, "set", 
+function (reader) {
+this.reader = reader;
+this.asResidues = reader.checkFilterKey ("ASRES");
+return this;
+}, "J.adapter.smarter.AtomSetCollectionReader");
+Clazz.defineMethod (c$, "finalizeValidations", 
+function (vwr, modelMap) {
+var map = this.reader.dssr;
+if (map != null) return vwr.getAnnotationParser (true).fixDSSRJSONMap (map);
+this.mapAtomResIDs (modelMap);
+var svMap = this.reader.validation;
+var retProps = this.reader.vwr.getAnnotationParser (false).catalogValidations (this.reader.vwr, svMap, this.getModelAtomIndices (), this.resMap, (this.asResidues ? null : this.atomMap), modelMap);
+var note = (retProps == null || retProps.size () == 0 ? null : this.setProperties (retProps));
+svMap.mapPut ("_note", JS.SV.newS (note));
+return note;
+}, "JV.Viewer,java.util.Map");
+Clazz.defineMethod (c$, "finalizeRna3d", 
+function (modelMap) {
+this.mapAtomResIDs (modelMap);
+var svMap = this.getRna3dMap (this.reader.addedData);
+var note = this.reader.vwr.getAnnotationParser (false).catalogStructureUnits (this.reader.vwr, svMap, this.getModelAtomIndices (), this.resMap, null, modelMap);
+svMap.mapPut ("_note", JS.SV.newS (note));
+for (var i = this.reader.asc.atomSetCount; --i >= 0; ) {
+var info = this.reader.asc.getAtomSetAuxiliaryInfo (i);
+info.put ("rna3d", svMap);
+}
+return note;
+}, "java.util.Map");
+Clazz.defineMethod (c$, "getRna3dMap", 
+ function (addedData) {
+var map =  new java.util.Hashtable ();
+var next =  Clazz.newIntArray (1, 0);
+var id = "";
+while ((id = JU.PT.getQuotedStringNext (addedData, next)).length > 0) {
+var units = JU.PT.getQuotedStringNext (addedData, next);
+var type = "?";
+switch (id.charAt (0)) {
+case 'H':
+type = "hairpinLoops";
+break;
+case 'I':
+type = "internalLoops";
+break;
+case 'J':
+type = "junctions";
+break;
+default:
+JU.Logger.error ("MMCif could not read: " + id + " " + units);
+continue;
+}
+var list = map.get (type);
+if (list == null) map.put (type, list =  new JU.Lst ());
+var m =  new java.util.Hashtable ();
+m.put ("index", Integer.$valueOf (JU.PT.parseInt (id.substring (id.lastIndexOf ("_") + 1))));
+m.put ("units", units);
+list.addLast (m);
+}
+return JS.SV.getVariableMap (map);
+}, "~S");
+Clazz.defineMethod (c$, "mapAtomResIDs", 
+ function (modelMap) {
+var atoms = this.reader.asc.atoms;
+this.resMap =  new java.util.Hashtable ();
+this.atomMap =  new java.util.Hashtable ();
+var iresLast = -1;
+var resLast = null;
+var smodel = "" + modelMap.get ("_0");
+for (var i = 0, model = 1, i0 = 0, n = this.reader.asc.getAtomSetAtomCount (0); i < n; i++) {
+var a = atoms[i];
+var ires = a.sequenceNumber;
+var res = smodel + "_" + a.chainID + "_" + ires + "_" + (a.insertionCode == '\0' ? "" : "" + a.insertionCode);
+var atom = res + "_" + a.atomName.toUpperCase () + "_" + (a.altLoc == '\0' ? "" : "" + Character.toLowerCase (a.altLoc));
+var ia = Integer.$valueOf (i - i0);
+if (ires != iresLast) {
+iresLast = ires;
+if (resLast != null) resLast[1] = i - i0;
+this.resMap.put (res, resLast =  Clazz.newIntArray (-1, [i - i0, n]));
+}this.atomMap.put (atom, ia);
+if (i == n - 1) {
+i0 += n;
+n = this.reader.asc.getAtomSetAtomCount (model++);
+}}
+}, "java.util.Map");
+Clazz.defineMethod (c$, "getModelAtomIndices", 
+ function () {
+var indices =  Clazz.newIntArray (this.reader.asc.atomSetCount + 1, 0);
+for (var m = indices.length - 1; --m >= 0; ) indices[m] = this.reader.baseAtomIndex + this.reader.asc.getAtomSetAtomIndex (m);
+
+indices[indices.length - 1] = this.reader.asc.ac;
+return indices;
+});
+Clazz.defineMethod (c$, "setProperties", 
+ function (propList) {
+var note = "Validations loaded:";
+for (var i = 0, n = propList.size (); i < n; ) {
+var key = propList.get (i++);
+var f = propList.get (i++);
+var model = (propList.get (i++)).intValue ();
+var isGroup = (propList.get (i++)).booleanValue ();
+var count = 0;
+var max = 0;
+var reslast = -1;
+var i0 = this.reader.asc.getAtomSetAtomIndex (model);
+for (var j = f.length; --j >= 0; ) if (f[j] != 0) {
+if (isGroup) {
+var res = this.reader.asc.atoms[i0 + j].sequenceNumber;
+if (res != reslast) {
+reslast = res;
+count++;
+}} else {
+count++;
+}max = Math.max (f[j], max);
+}
+note += "\n  property_" + key + " (" + (isGroup ? "residues: " : "atoms: ") + count + (max == 1 ? "" : ", max: " + (Clazz.floatToInt (max * 100)) / 100) + ")";
+this.reader.asc.setAtomProperties (key, f, model, isGroup);
+}
+return note;
+}, "JU.Lst");
+});

static/j2s/J/adapter/readers/cif/MMTFReader.js

@@ -0,0 +1,286 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (["J.adapter.readers.cif.MMCifReader"], "J.adapter.readers.cif.MMTFReader", ["java.lang.Boolean", "java.util.Hashtable", "JU.BS", "$.Lst", "$.M4", "$.MessagePackReader", "$.PT", "$.SB", "J.adapter.smarter.Atom", "$.Bond", "$.Structure", "JS.SV", "JU.Logger"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.haveStructure = false;
+this.$pdbID = null;
+this.map = null;
+this.fileAtomCount = 0;
+this.opCount = 0;
+this.groupModels = null;
+this.groupMap = null;
+this.groupDSSP = null;
+this.atomGroup = null;
+this.labelAsymList = null;
+this.atomMap = null;
+this.entities = null;
+this.groupCount = 0;
+this.ac0 = 0;
+this.bsStructures = null;
+this.lastGroup = 0;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.cif, "MMTFReader", J.adapter.readers.cif.MMCifReader);
+Clazz.overrideMethod (c$, "addHeader", 
+function () {
+});
+Clazz.overrideMethod (c$, "setup", 
+function (fullPath, htParams, reader) {
+this.isBinary = true;
+this.isMMCIF = true;
+this.iHaveFractionalCoordinates = false;
+this.setupASCR (fullPath, htParams, reader);
+}, "~S,java.util.Map,~O");
+Clazz.overrideMethod (c$, "processBinaryDocument", 
+function () {
+var doDoubleBonds = (!this.isCourseGrained && !this.checkFilterKey ("NODOUBLE"));
+this.isDSSP1 = !this.checkFilterKey ("DSSP2");
+var mmtfImplementsDSSP2 = false;
+this.applySymmetryToBonds = true;
+this.map = ( new JU.MessagePackReader (this.binaryDoc, true)).readMap ();
+this.entities = this.map.get ("entityList");
+if (JU.Logger.debugging) {
+for (var s, $s = this.map.keySet ().iterator (); $s.hasNext () && ((s = $s.next ()) || true);) JU.Logger.info (s);
+
+}this.asc.setInfo ("noAutoBond", Boolean.TRUE);
+JU.Logger.info ("MMTF version " + this.map.get ("mmtfVersion"));
+JU.Logger.info ("MMTF Producer " + this.map.get ("mmtfProducer"));
+var title = this.map.get ("title");
+if (title != null) this.appendLoadNote (title);
+this.$pdbID = this.map.get ("structureId");
+if (this.$pdbID == null) this.$pdbID = this.map.get ("pdbId");
+this.fileAtomCount = (this.map.get ("numAtoms")).intValue ();
+var nBonds = (this.map.get ("numBonds")).intValue ();
+this.groupCount = (this.map.get ("numGroups")).intValue ();
+this.groupModels =  Clazz.newIntArray (this.groupCount, 0);
+this.groupDSSP =  Clazz.newIntArray (this.groupCount, 0);
+this.groupMap =  Clazz.newIntArray (this.groupCount, 0);
+var modelCount = (this.map.get ("numModels")).intValue ();
+this.appendLoadNote ("id=" + this.$pdbID + " numAtoms=" + this.fileAtomCount + " numBonds=" + nBonds + " numGroups=" + this.groupCount + " numModels=" + modelCount);
+this.getMMTFAtoms (doDoubleBonds);
+if (!this.isCourseGrained) {
+var bo = this.decode ("bondOrderList");
+var bi = this.decode ("bondAtomList");
+this.addMMTFBonds (bo, bi, 0, doDoubleBonds, true);
+if (this.isDSSP1 || mmtfImplementsDSSP2) this.getStructure ();
+}this.setMMTFSymmetry ();
+this.getMMTFBioAssembly ();
+this.setModelPDB (true);
+if (JU.Logger.debuggingHigh) JU.Logger.info (JS.SV.getVariable (this.map).asString ());
+});
+Clazz.defineMethod (c$, "applySymmetryAndSetTrajectory", 
+function () {
+this.ac0 = this.ac;
+Clazz.superCall (this, J.adapter.readers.cif.MMTFReader, "applySymmetryAndSetTrajectory", []);
+if (this.haveStructure) this.addStructureSymmetry ();
+});
+Clazz.defineMethod (c$, "getMMTFAtoms", 
+ function (doMulti) {
+var chainsPerModel = this.map.get ("chainsPerModel");
+var groupsPerChain = this.map.get ("groupsPerChain");
+this.labelAsymList = this.decode ("chainIdList");
+var authAsymList = this.decode ("chainNameList");
+var groupTypeList = this.decode ("groupTypeList");
+var groupIdList = this.decode ("groupIdList");
+var groupList = this.map.get ("groupList");
+var insCodes = this.decode ("insCodeList");
+var atomId = this.decode ("atomIdList");
+var haveSerial = (atomId != null);
+var altloc = this.decode ("altLocList");
+var occ = this.decode ("occupancyList");
+var x = this.decode ("xCoordList");
+var y = this.decode ("yCoordList");
+var z = this.decode ("zCoordList");
+var bf = this.decode ("bFactorList");
+var nameList = (this.useAuthorChainID ? authAsymList : this.labelAsymList);
+var iModel = -1;
+var iChain = 0;
+var nChain = 0;
+var iGroup = 0;
+var nGroup = 0;
+var chainpt = 0;
+var seqNo = 0;
+var iatom = 0;
+var chainID = "";
+var authAsym = "";
+var labelAsym = "";
+var insCode = '\u0000';
+this.atomMap =  new Array (this.fileAtomCount);
+this.atomGroup =  Clazz.newIntArray (this.fileAtomCount, 0);
+for (var j = 0, thisGroup = -1; j < this.groupCount; j++) {
+if (++iGroup >= nGroup) {
+chainID = nameList[chainpt];
+authAsym = authAsymList[chainpt];
+labelAsym = this.labelAsymList[chainpt];
+nGroup = groupsPerChain[chainpt++];
+iGroup = 0;
+if (++iChain >= nChain) {
+this.groupModels[j] = ++iModel;
+nChain = chainsPerModel[iModel];
+iChain = 0;
+this.setModelPDB (true);
+this.incrementModel (iModel + 1);
+this.asc.setCurrentModelInfo ("pdbID", this.$pdbID);
+this.nAtoms0 = this.asc.ac;
+if (this.done) return;
+}}var g = groupList[groupTypeList[j]];
+var atomNameList = g.get ("atomNameList");
+var len = atomNameList.length;
+if (this.skipping) {
+iatom += len;
+continue;
+}var a0 = iatom;
+if (insCodes != null) insCode = insCodes[j];
+seqNo = groupIdList[j];
+var group3 = g.get ("groupName");
+var isHetero = this.vwr.getJBR ().isHetero (group3);
+if (isHetero) {
+var hetName = "" + g.get ("chemCompType");
+if (this.htHetero == null || !this.htHetero.containsKey (group3)) {
+if (this.entities != null && hetName.equals ("NON-POLYMER")) out : for (var i = this.entities.length; --i >= 0; ) {
+var entity = this.entities[i];
+var chainList = entity.get ("chainIndexList");
+for (var k = chainList.length; --k >= 0; ) if (chainList[k] == iChain) {
+hetName = "a component of the entity \"" + entity.get ("description") + "\"";
+break out;
+}
+}
+this.addHetero (group3, hetName, false, true);
+}}var elementList = g.get ("elementList");
+var haveAtom = false;
+for (var ia = 0, pt = 0; ia < len; ia++, iatom++) {
+var a =  new J.adapter.smarter.Atom ();
+a.isHetero = isHetero;
+if (insCode.charCodeAt (0) != 0) a.insertionCode = insCode;
+this.setAtomCoordXYZ (a, x[iatom], y[iatom], z[iatom]);
+a.elementSymbol = elementList[pt];
+a.atomName = atomNameList[pt++];
+if (seqNo >= 0) this.maxSerial = Math.max (this.maxSerial, a.sequenceNumber = seqNo);
+a.group3 = group3;
+this.setChainID (a, chainID);
+if (bf != null) a.bfactor = bf[iatom];
+if (altloc != null) a.altLoc = altloc[iatom];
+if (occ != null) a.foccupancy = occ[iatom];
+if (haveSerial) a.atomSerial = atomId[iatom];
+if (!this.filterAtom (a, -1) || !this.processSubclassAtom (a, labelAsym, authAsym)) continue;
+if (!haveAtom) {
+thisGroup++;
+haveAtom = true;
+}if (haveSerial) {
+this.asc.addAtomWithMappedSerialNumber (a);
+} else {
+this.asc.addAtom (a);
+}this.atomMap[iatom] = a;
+this.atomGroup[this.ac] = j;
+this.groupMap[j] = this.lastGroup = thisGroup;
+this.ac++;
+}
+if (!this.isCourseGrained) {
+var bo = g.get ("bondOrderList");
+var bi = g.get ("bondAtomList");
+this.addMMTFBonds (bo, bi, a0, doMulti, false);
+}}
+this.asc.setCurrentModelInfo ("pdbID", this.$pdbID);
+}, "~B");
+Clazz.defineMethod (c$, "addMMTFBonds", 
+ function (bo, bi, a0, doMulti, isInter) {
+if (bi == null) return;
+doMulti = new Boolean (doMulti & (bo != null)).valueOf ();
+for (var bj = 0, pt = 0, nj = Clazz.doubleToInt (bi.length / 2); bj < nj; bj++) {
+var a1 = this.atomMap[bi[pt++] + a0];
+var a2 = this.atomMap[bi[pt++] + a0];
+if (a1 != null && a2 != null) {
+var bond =  new J.adapter.smarter.Bond (a1.index, a2.index, doMulti ? bo[bj] : 1);
+this.asc.addBond (bond);
+if (JU.Logger.debugging && isInter) {
+JU.Logger.info ("inter-group (" + (a1.atomSetIndex + 1) + "." + a1.index + "/" + (a2.atomSetIndex + 1) + "." + a2.index + ") bond " + a1.group3 + a1.sequenceNumber + "." + a1.atomName + " - " + a2.group3 + a2.sequenceNumber + "." + a2.atomName + " " + bond.order);
+}}}
+}, "~A,~A,~N,~B,~B");
+Clazz.defineMethod (c$, "setMMTFSymmetry", 
+ function () {
+this.setSpaceGroupName (this.map.get ("spaceGroup"));
+var o = this.map.get ("unitCell");
+if (o != null) for (var i = 0; i < 6; i++) this.setUnitCellItem (i, o[i]);
+
+});
+Clazz.defineMethod (c$, "getMMTFBioAssembly", 
+ function () {
+var o = this.map.get ("bioAssemblyList");
+if (o == null) return;
+if (this.vBiomolecules == null) this.vBiomolecules =  new JU.Lst ();
+for (var i = o.length; --i >= 0; ) {
+var info =  new java.util.Hashtable ();
+this.vBiomolecules.addLast (info);
+var iMolecule = i + 1;
+this.checkFilterAssembly ("" + iMolecule, info);
+info.put ("name", "biomolecule " + iMolecule);
+info.put ("molecule", Integer.$valueOf (iMolecule));
+var assemb =  new JU.Lst ();
+var ops =  new JU.Lst ();
+info.put ("biomts",  new JU.Lst ());
+info.put ("chains",  new JU.Lst ());
+info.put ("assemblies", assemb);
+info.put ("operators", ops);
+var m = o[i];
+var tlist = m.get ("transformList");
+var chlist =  new JU.SB ();
+for (var j = 0, n = tlist.length; j < n; j++) {
+var t = tlist[j];
+chlist.setLength (0);
+var chainList = t.get ("chainIndexList");
+for (var k = 0, kn = chainList.length; k < kn; k++) chlist.append ("$").append (this.labelAsymList[chainList[k]]);
+
+assemb.addLast (chlist.append ("$").toString ());
+var id = "" + (++this.opCount);
+this.addMatrix (id, JU.M4.newA16 (t.get ("matrix")), false);
+ops.addLast (id);
+}
+}
+});
+Clazz.defineMethod (c$, "getStructure", 
+ function () {
+var a = this.decode ("secStructList");
+if (JU.Logger.debugging) JU.Logger.info (JU.PT.toJSON ("secStructList", a));
+this.bsStructures =  Clazz.newArray (-1, [ new JU.BS (), null,  new JU.BS (),  new JU.BS (),  new JU.BS (), null,  new JU.BS ()]);
+var lastGroup = -1;
+for (var j = 0; j < a.length; j++) {
+var type = a[j];
+switch (type) {
+case 0:
+case 2:
+case 3:
+case 4:
+case 6:
+var igroup = this.groupMap[j];
+this.bsStructures[type].set (igroup);
+this.groupDSSP[igroup] = type + 1;
+lastGroup = j;
+}
+}
+var n = (this.isDSSP1 ? this.asc.iSet : this.groupModels[lastGroup]);
+if (lastGroup >= 0) {
+this.haveStructure = true;
+this.asc.addStructure ( new J.adapter.smarter.Structure (n, null, null, null, 0, 0, this.bsStructures));
+}});
+Clazz.defineMethod (c$, "addStructureSymmetry", 
+ function () {
+if (this.asc.ac == 0) return;
+var atoms = this.asc.atoms;
+var bsAtoms = this.asc.bsAtoms;
+var ptGroup = this.lastGroup;
+var mygroup = -1;
+for (var i = this.ac0, n = this.asc.ac; i < n; i++) {
+if (bsAtoms == null || bsAtoms.get (i)) {
+var a = atoms[i];
+var igroup = this.atomGroup[a.atomSite];
+if (igroup != mygroup) {
+mygroup = igroup;
+ptGroup++;
+}var dssp = this.groupDSSP[igroup];
+if (dssp > 0) {
+this.bsStructures[dssp - 1].set (ptGroup);
+}}}
+});
+Clazz.defineMethod (c$, "decode", 
+ function (key) {
+return JU.MessagePackReader.decode (this.map.get (key));
+}, "~S");
+});

static/j2s/J/adapter/readers/cif/MSCifParser.js

@@ -0,0 +1,383 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (["J.adapter.readers.cif.MSRdr"], "J.adapter.readers.cif.MSCifParser", ["java.lang.Character", "$.Double", "JU.M3", "$.Matrix", "$.PT", "J.adapter.readers.cif.Cif2DataParser"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.field = null;
+this.comSSMat = null;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.cif, "MSCifParser", J.adapter.readers.cif.MSRdr);
+Clazz.makeConstructor (c$, 
+function () {
+Clazz.superConstructor (this, J.adapter.readers.cif.MSCifParser, []);
+});
+Clazz.defineMethod (c$, "processEntry", 
+function () {
+var cr = this.cr;
+if (cr.key.equals ("_cell_commen_t_section_1")) {
+this.isCommensurate = true;
+this.commensurateSection1 = cr.parseIntStr (cr.data);
+}if (cr.key.startsWith ("_cell_commen_supercell_matrix")) {
+this.isCommensurate = true;
+if (this.comSSMat == null) this.comSSMat = JU.M3.newM3 (null);
+var tokens = JU.PT.split (cr.key, "_");
+var r = cr.parseIntStr (tokens[tokens.length - 2]);
+var c = cr.parseIntStr (tokens[tokens.length - 1]);
+if (r > 0 && c > 0) this.comSSMat.setElement (r - 1, c - 1, cr.parseFloatStr (cr.data));
+}});
+Clazz.defineMethod (c$, "processLoopBlock", 
+function () {
+var cr = this.cr;
+var key = cr.key;
+if (key.equals ("_cell_subsystem_code")) return this.processSubsystemLoopBlock ();
+if (!key.startsWith ("_cell_wave") && !key.contains ("fourier") && !key.contains ("legendre") && !key.contains ("_special_func")) {
+if (key.contains ("crenel_ortho")) cr.appendLoadNote ("WARNING: Orthogonalized non-Legendre functions not supported.\nThe following block has been ignored. Use Legendre functions instead.\n\n" + cr.parser.skipLoop (true) + "=================================\n");
+return 0;
+}if (cr.asc.iSet < 0) cr.asc.newAtomSet ();
+cr.parseLoopParametersFor ("_atom_site", J.adapter.readers.cif.MSCifParser.modulationFields);
+var tok;
+while (cr.parser.getData ()) {
+var ignore = false;
+var type_id = null;
+var atomLabel = null;
+var axis = null;
+var pt =  Clazz.newDoubleArray (-1, [NaN, NaN, NaN]);
+var q = null;
+var c = NaN;
+var w = NaN;
+var fid = null;
+var n = cr.parser.getColumnCount ();
+for (var i = 0; i < n; ++i) {
+switch (tok = this.fieldProperty (cr, i)) {
+case 1:
+cr.haveCellWaveVector = true;
+case 0:
+case 41:
+case 42:
+case 43:
+pt[0] = pt[1] = pt[2] = 0;
+case 14:
+case 26:
+case 51:
+case 36:
+case 44:
+case 45:
+case 46:
+switch (tok) {
+case 1:
+type_id = "W_";
+break;
+case 0:
+type_id = "F_";
+fid = this.field;
+break;
+case 41:
+case 42:
+case 43:
+fid = "?" + this.field;
+pt[2] = 1;
+continue;
+case 44:
+case 45:
+case 46:
+atomLabel = axis = "*";
+case 14:
+case 26:
+case 51:
+case 36:
+type_id = Character.toUpperCase (J.adapter.readers.cif.MSCifParser.modulationFields[tok].charAt (11)) + "_";
+break;
+}
+type_id += this.field;
+break;
+case 47:
+type_id = "J_O";
+pt[0] = pt[2] = 1;
+axis = "0";
+atomLabel = this.field;
+break;
+case 31:
+type_id = "O_0";
+axis = "0";
+atomLabel = this.field;
+break;
+case 19:
+type_id = "D_S";
+axis = "0";
+atomLabel = this.field;
+break;
+case 56:
+type_id = "M_T";
+axis = "0";
+atomLabel = this.field;
+break;
+case 62:
+type_id = "D_L";
+atomLabel = this.field;
+break;
+case 66:
+type_id = "U_L";
+atomLabel = this.field;
+break;
+case 70:
+type_id = "O_L";
+atomLabel = this.field;
+break;
+case 12:
+case 25:
+case 49:
+case 34:
+atomLabel = this.field;
+break;
+case 13:
+case 50:
+case 63:
+axis = this.field;
+if (this.modAxes != null && this.modAxes.indexOf (axis.toUpperCase ()) < 0) ignore = true;
+break;
+case 67:
+case 35:
+axis = this.field.toUpperCase ();
+break;
+case 8:
+q = J.adapter.readers.cif.Cif2DataParser.getArrayFromStringList (this.field, this.modDim);
+break;
+default:
+var f = cr.parseFloatStr (this.field);
+switch (tok) {
+case 65:
+case 72:
+case 69:
+pt[0] = f;
+if (f != 0) pt[2] = 0;
+break;
+case 28:
+case 32:
+case 16:
+case 53:
+case 38:
+case 78:
+case 74:
+case 76:
+pt[2] = 0;
+case 2:
+case 5:
+case 20:
+case 57:
+pt[0] = f;
+break;
+case 9:
+if (q == null) q =  Clazz.newDoubleArray (this.modDim, 0);
+q[0] = f;
+break;
+case 10:
+if (q == null) q =  Clazz.newDoubleArray (this.modDim, 0);
+q[1] = f;
+break;
+case 11:
+if (q == null) q =  Clazz.newDoubleArray (this.modDim, 0);
+q[2] = f;
+break;
+case 17:
+case 29:
+case 54:
+case 39:
+pt[0] = f;
+pt[2] = 1;
+break;
+case 71:
+case 75:
+case 27:
+axis = "0";
+case 64:
+case 68:
+case 3:
+case 6:
+case 18:
+case 30:
+case 55:
+case 40:
+case 33:
+case 21:
+case 58:
+case 48:
+case 15:
+case 52:
+case 37:
+case 73:
+case 77:
+pt[1] = f;
+break;
+case 4:
+case 7:
+case 22:
+case 59:
+pt[2] = f;
+break;
+case 23:
+case 60:
+c = f;
+break;
+case 24:
+case 61:
+w = f;
+break;
+}
+break;
+}
+}
+if (ignore || type_id == null && q == null || atomLabel != null && !atomLabel.equals ("*") && cr.rejectAtomName (atomLabel)) continue;
+var ok = true;
+if (q != null) {
+for (var j = 0, nzero = q.length; j < q.length; j++) if (Double.isNaN (q[j]) || q[j] > 1e100 || q[j] == 0 && --nzero == 0) {
+ok = false;
+}
+if (!ok) continue;
+this.addMod ("F_coefs_", fid, q);
+pt[0] = NaN;
+}for (var j = 0, nzero = pt.length; j < pt.length; j++) if (Double.isNaN (pt[j]) || pt[j] > 1e100 || pt[j] == 0 && --nzero == 0) {
+ok = false;
+break;
+}
+if (!ok) continue;
+switch (type_id.charAt (0)) {
+case 'C':
+case 'D':
+case 'O':
+case 'M':
+case 'U':
+case 'J':
+if (atomLabel == null || axis == null) continue;
+if (type_id.equals ("D_S") || type_id.equals ("M_T")) {
+if (Double.isNaN (c) || Double.isNaN (w)) continue;
+if (pt[0] != 0) this.addMod (type_id + "#x;" + atomLabel, fid,  Clazz.newDoubleArray (-1, [c, w, pt[0]]));
+if (pt[1] != 0) this.addMod (type_id + "#y;" + atomLabel, fid,  Clazz.newDoubleArray (-1, [c, w, pt[1]]));
+if (pt[2] != 0) this.addMod (type_id + "#z;" + atomLabel, fid,  Clazz.newDoubleArray (-1, [c, w, pt[2]]));
+continue;
+}if (type_id.indexOf ("_L") == 1) {
+if (type_id.startsWith ("U")) type_id += Clazz.doubleToInt (pt[1]);
+ else axis += Clazz.doubleToInt (pt[1]);
+}type_id += "#" + axis + ";" + atomLabel;
+break;
+}
+this.addMod (type_id, fid, pt);
+}
+return 1;
+});
+Clazz.defineMethod (c$, "addMod", 
+ function (id, fid, params) {
+if (fid != null) id += fid;
+this.addModulation (null, id, params, -1);
+}, "~S,~S,~A");
+Clazz.defineMethod (c$, "processSubsystemLoopBlock", 
+ function () {
+var cr = this.cr;
+cr.parseLoopParameters (null);
+while (cr.parser.getData ()) {
+this.fieldProperty (cr, 0);
+var id = this.field;
+this.addSubsystem (id, this.getSparseMatrix (cr, "_w_", 1, 3 + this.modDim));
+}
+return 1;
+});
+Clazz.defineMethod (c$, "getSparseMatrix", 
+ function (cr, term, i, dim) {
+var m =  new JU.Matrix (null, dim, dim);
+var a = m.getArray ();
+var key;
+var p;
+var n = cr.parser.getColumnCount ();
+for (; i < n; ++i) {
+if ((p = this.fieldProperty (cr, i)) < 0 || !(key = cr.parser.getColumnName (p)).contains (term)) continue;
+var tokens = JU.PT.split (key, "_");
+var r = cr.parseIntStr (tokens[tokens.length - 2]);
+var c = cr.parseIntStr (tokens[tokens.length - 1]);
+if (r > 0 && c > 0) a[r - 1][c - 1] = cr.parseFloatStr (this.field);
+}
+return m;
+}, "J.adapter.readers.cif.CifReader,~S,~N,~N");
+Clazz.defineMethod (c$, "fieldProperty", 
+ function (cr, i) {
+return ((this.field = cr.parser.getColumnData (i)).length > 0 && this.field.charAt (0) != '\0' ? cr.col2key[i] : -1);
+}, "J.adapter.readers.cif.CifReader,~N");
+Clazz.defineStatics (c$,
+"FWV_ID", 0,
+"WV_ID", 1,
+"WV_X", 2,
+"WV_Y", 3,
+"WV_Z", 4,
+"FWV_X", 5,
+"FWV_Y", 6,
+"FWV_Z", 7,
+"FWV_Q_COEF", 8,
+"JANA_FWV_Q1_COEF", 9,
+"JANA_FWV_Q2_COEF", 10,
+"JANA_FWV_Q3_COEF", 11,
+"FWV_DISP_LABEL", 12,
+"FWV_DISP_AXIS", 13,
+"FWV_DISP_SEQ_ID", 14,
+"FWV_DISP_COS", 15,
+"FWV_DISP_SIN", 16,
+"FWV_DISP_MODULUS", 17,
+"FWV_DISP_PHASE", 18,
+"DISP_SPEC_LABEL", 19,
+"DISP_SAW_AX", 20,
+"DISP_SAW_AY", 21,
+"DISP_SAW_AZ", 22,
+"DISP_SAW_C", 23,
+"DISP_SAW_W", 24,
+"FWV_OCC_LABEL", 25,
+"FWV_OCC_SEQ_ID", 26,
+"FWV_OCC_COS", 27,
+"FWV_OCC_SIN", 28,
+"FWV_OCC_MODULUS", 29,
+"FWV_OCC_PHASE", 30,
+"OCC_SPECIAL_LABEL", 31,
+"OCC_CRENEL_C", 32,
+"OCC_CRENEL_W", 33,
+"FWV_U_LABEL", 34,
+"FWV_U_TENS", 35,
+"FWV_U_SEQ_ID", 36,
+"FWV_U_COS", 37,
+"FWV_U_SIN", 38,
+"FWV_U_MODULUS", 39,
+"FWV_U_PHASE", 40,
+"FD_ID", 41,
+"FO_ID", 42,
+"FU_ID", 43,
+"FDP_ID", 44,
+"FOP_ID", 45,
+"FUP_ID", 46,
+"JANA_OCC_ABS_LABEL", 47,
+"JANA_OCC_ABS_O_0", 48,
+"FWV_SPIN_LABEL", 49,
+"FWV_SPIN_AXIS", 50,
+"FWV_SPIN_SEQ_ID", 51,
+"FWV_SPIN_COS", 52,
+"FWV_SPIN_SIN", 53,
+"FWV_SPIN_MODULUS", 54,
+"FWV_SPIN_PHASE", 55,
+"SPIN_SPEC_LABEL", 56,
+"SPIN_SAW_AX", 57,
+"SPIN_SAW_AY", 58,
+"SPIN_SAW_AZ", 59,
+"SPIN_SAW_C", 60,
+"SPIN_SAW_W", 61,
+"LEG_DISP_LABEL", 62,
+"LEG_DISP_AXIS", 63,
+"LEG_DISP_ORDER", 64,
+"LEG_DISP_COEF", 65,
+"LEG_U_LABEL", 66,
+"LEG_U_TENS", 67,
+"LEG_U_ORDER", 68,
+"LEG_U_COEF", 69,
+"LEG_OCC_LABEL", 70,
+"LEG_OCC_ORDER", 71,
+"LEG_OCC_COEF", 72,
+"DEPR_FD_COS", 73,
+"DEPR_FD_SIN", 74,
+"DEPR_FO_COS", 75,
+"DEPR_FO_SIN", 76,
+"DEPR_FU_COS", 77,
+"DEPR_FU_SIN", 78,
+"modulationFields",  Clazz.newArray (-1, ["*_fourier_wave_vector_seq_id", "_cell_wave_vector_seq_id", "_cell_wave_vector_x", "_cell_wave_vector_y", "_cell_wave_vector_z", "*_fourier_wave_vector_x", "*_fourier_wave_vector_y", "*_fourier_wave_vector_z", "*_fourier_wave_vector_q_coeff", "*_fourier_wave_vector_q1_coeff", "*_fourier_wave_vector_q2_coeff", "*_fourier_wave_vector_q3_coeff", "*_displace_fourier_atom_site_label", "*_displace_fourier_axis", "*_displace_fourier_wave_vector_seq_id", "*_displace_fourier_param_cos", "*_displace_fourier_param_sin", "*_displace_fourier_param_modulus", "*_displace_fourier_param_phase", "*_displace_special_func_atom_site_label", "*_displace_special_func_sawtooth_ax", "*_displace_special_func_sawtooth_ay", "*_displace_special_func_sawtooth_az", "*_displace_special_func_sawtooth_c", "*_displace_special_func_sawtooth_w", "*_occ_fourier_atom_site_label", "*_occ_fourier_wave_vector_seq_id", "*_occ_fourier_param_cos", "*_occ_fourier_param_sin", "*_occ_fourier_param_modulus", "*_occ_fourier_param_phase", "*_occ_special_func_atom_site_label", "*_occ_special_func_crenel_c", "*_occ_special_func_crenel_w", "*_u_fourier_atom_site_label", "*_u_fourier_tens_elem", "*_u_fourier_wave_vector_seq_id", "*_u_fourier_param_cos", "*_u_fourier_param_sin", "*_u_fourier_param_modulus", "*_u_fourier_param_phase", "*_displace_fourier_id", "*_occ_fourier_id", "*_u_fourier_id", "*_displace_fourier_param_id", "*_occ_fourier_param_id", "*_u_fourier_param_id", "*_occ_fourier_absolute_site_label", "*_occ_fourier_absolute", "*_moment_fourier_atom_site_label", "*_moment_fourier_axis", "*_moment_fourier_wave_vector_seq_id", "*_moment_fourier_param_cos", "*_moment_fourier_param_sin", "*_moment_fourier_param_modulus", "*_moment_fourier_param_phase", "*_moment_special_func_atom_site_label", "*_moment_special_func_sawtooth_ax", "*_moment_special_func_sawtooth_ay", "*_moment_special_func_sawtooth_az", "*_moment_special_func_sawtooth_c", "*_moment_special_func_sawtooth_w", "*_displace_legendre_atom_site_label", "*_displace_legendre_axis", "*_displace_legendre_param_order", "*_displace_legendre_param_coeff", "*_u_legendre_atom_site_label", "*_u_legendre_tens_elem", "*_u_legendre_param_order", "*_u_legendre_param_coeff", "*_occ_legendre_atom_site_label", "*_occ_legendre_param_order", "*_occ_legendre_param_coeff", "*_displace_fourier_cos", "*_displace_fourier_sin", "*_occ_fourier_cos", "*_occ_fourier_sin", "*_u_fourier_cos", "*_u_fourier_sin"]),
+"NONE", -1);
+});

static/j2s/J/adapter/readers/cif/MSRdr.js

@@ -0,0 +1,593 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (["J.adapter.smarter.MSInterface"], "J.adapter.readers.cif.MSRdr", ["java.lang.Boolean", "$.Exception", "$.Float", "java.util.Hashtable", "JU.Lst", "$.M3", "$.Matrix", "$.P3", "$.PT", "J.adapter.readers.cif.Subsystem", "J.adapter.smarter.AtomSetCollectionReader", "JU.BSUtil", "$.BoxInfo", "$.Escape", "$.Logger", "$.Modulation", "$.ModulationSet", "$.Vibration"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.cr = null;
+this.modDim = 0;
+this.modAxes = null;
+this.modAverage = false;
+this.isCommensurate = false;
+this.commensurateSection1 = 0;
+this.modPack = false;
+this.modVib = false;
+this.modType = null;
+this.modCell = null;
+this.modDebug = false;
+this.modSelected = -1;
+this.modLast = false;
+this.sigma = null;
+this.htModulation = null;
+this.htAtomMods = null;
+this.iopLast = -1;
+this.gammaE = null;
+this.nOps = 0;
+this.haveOccupancy = false;
+this.atoms = null;
+this.ac = 0;
+this.haveAtomMods = false;
+this.modCoord = false;
+this.finalized = false;
+this.symmetry = null;
+this.supercellSymmetry = null;
+this.legendres = null;
+this.atModel = "@0";
+this.modMatrices = null;
+this.qlist100 = null;
+this.qs = null;
+this.modCount = 0;
+this.modTUV = null;
+this.htSubsystems = null;
+this.minXYZ0 = null;
+this.maxXYZ0 = null;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.cif, "MSRdr", null, J.adapter.smarter.MSInterface);
+Clazz.defineMethod (c$, "getSigma", 
+function () {
+return this.sigma;
+});
+Clazz.makeConstructor (c$, 
+function () {
+});
+Clazz.overrideMethod (c$, "initialize", 
+function (r, modDim) {
+this.cr = r;
+this.modCoord = r.checkFilterKey ("MODCOORD");
+this.modDebug = r.checkFilterKey ("MODDEBUG");
+this.modPack = !r.checkFilterKey ("MODNOPACK");
+this.modLast = r.checkFilterKey ("MODLAST");
+this.modAxes = r.getFilter ("MODAXES=");
+this.modType = r.getFilter ("MODTYPE=");
+this.modCell = r.getFilter ("MODCELL=");
+this.modSelected = r.parseIntStr ("" + r.getFilter ("MOD="));
+this.modVib = r.checkFilterKey ("MODVIB");
+this.modAverage = r.checkFilterKey ("MODAVE");
+var smodTUV = r.getFilter ("MODT=");
+if (smodTUV != null || (smodTUV = r.getFilter ("MODTUV=")) != null) {
+this.modTUV =  new JU.P3 ();
+var tuv = (JU.PT.replaceAllCharacters (smodTUV, "{}()", "") + ",0,0,0").$plit (",");
+this.modTUV.x = JU.PT.parseFloatFraction (tuv[0]);
+this.modTUV.y = JU.PT.parseFloatFraction (tuv[1]);
+this.modTUV.z = JU.PT.parseFloatFraction (tuv[2]);
+if (Float.isNaN (this.modTUV.lengthSquared ())) {
+JU.Logger.error ("MSRdr cannot read modTUV=" + smodTUV);
+this.modTUV = null;
+}}this.setModDim (modDim);
+return modDim;
+}, "J.adapter.smarter.AtomSetCollectionReader,~N");
+Clazz.defineMethod (c$, "setSubsystemOptions", 
+ function () {
+this.cr.forceSymmetry (this.modPack);
+if (this.modCell != null) this.cr.addJmolScript ("unitcell {%" + this.modCell + "}");
+});
+Clazz.defineMethod (c$, "setModDim", 
+function (ndim) {
+this.htModulation =  new java.util.Hashtable ();
+this.modDim = ndim;
+this.cr.appendLoadNote ("Modulation dimension = " + this.modDim);
+}, "~N");
+Clazz.overrideMethod (c$, "addModulation", 
+function (map, id, pt, iModel) {
+var ch = id.charAt (0);
+switch (ch) {
+case 'O':
+case 'D':
+case 'M':
+case 'U':
+if (this.modType != null && this.modType.indexOf (ch) < 0 || this.modSelected > 0 && this.modSelected != 1) return;
+break;
+}
+var isOK = false;
+for (var i = pt.length; --i >= 0; ) {
+if (this.modSelected > 0 && i + 1 != this.modSelected && id.contains ("_coefs_")) {
+pt[i] = 0;
+} else if (pt[i] != 0) {
+isOK = true;
+break;
+}}
+if (!isOK) return;
+if (map == null) map = this.htModulation;
+if (id.indexOf ("@") < 0) id += "@" + (iModel >= 0 ? iModel : this.cr.asc.iSet);
+if (id.startsWith ("D_L#") || id.startsWith ("U_L")) {
+if (this.legendres == null) this.legendres =  new JU.Lst ();
+this.legendres.addLast (id);
+}JU.Logger.info ("Adding " + id + " " + JU.Escape.e (pt));
+map.put (id, pt);
+}, "java.util.Map,~S,~A,~N");
+Clazz.overrideMethod (c$, "setModulation", 
+function (isPost, symmetry) {
+if (this.modDim == 0 || this.htModulation == null) return;
+if (this.modDebug) JU.Logger.debugging = JU.Logger.debuggingHigh = true;
+this.cr.asc.setInfo ("someModelsAreModulated", Boolean.TRUE);
+this.symmetry = symmetry;
+this.setModulationForStructure (this.cr.asc.iSet, isPost);
+if (this.modDebug) JU.Logger.debugging = JU.Logger.debuggingHigh = false;
+}, "~B,J.api.SymmetryInterface");
+Clazz.overrideMethod (c$, "finalizeModulation", 
+function () {
+if (!this.finalized && this.modDim > 0 && !this.modVib) {
+if (this.modTUV != null) this.cr.appendLoadNote ("modTUV=" + this.modTUV);
+this.cr.asc.setInfo ("modulationOn", this.modTUV == null ? Boolean.TRUE : this.modTUV);
+this.cr.addJmolScript ("set modulateOccupancy " + (this.haveOccupancy && !this.isCommensurate ? true : false));
+}this.finalized = true;
+});
+Clazz.defineMethod (c$, "checkKey", 
+ function (key, checkQ) {
+var pt = key.indexOf (this.atModel);
+return (pt < 0 || key.indexOf ("_pos#") >= 0 || key.indexOf ("*;*") >= 0 || checkQ && key.indexOf ("?") >= 0 ? null : key.substring (0, pt));
+}, "~S,~B");
+Clazz.overrideMethod (c$, "getMod", 
+function (key) {
+return this.htModulation.get (key + this.atModel);
+}, "~S");
+Clazz.overrideMethod (c$, "getModulationMap", 
+function () {
+return this.htModulation;
+});
+Clazz.defineMethod (c$, "setModulationForStructure", 
+ function (iModel, isPost) {
+this.atModel = "@" + iModel;
+if (this.htModulation.containsKey ("X_" + this.atModel)) return;
+if (!isPost) {
+this.initModForStructure (iModel);
+return;
+}this.htModulation.put ("X_" + this.atModel,  Clazz.newDoubleArray (0, 0));
+this.cr.appendLoadNote (this.modCount + " modulations for " + this.ac + " modulated atoms");
+if (!this.haveAtomMods) return;
+var n = this.cr.asc.ac;
+this.atoms = this.cr.asc.atoms;
+if (this.symmetry != null) this.nOps = this.symmetry.getSpaceGroupOperationCount ();
+this.supercellSymmetry = this.cr.asc.getXSymmetry ().symmetry;
+if (this.supercellSymmetry === this.symmetry) this.supercellSymmetry = null;
+this.iopLast = -1;
+var i0 = this.cr.asc.getLastAtomSetAtomIndex ();
+for (var i = i0; i < n; i++) this.modulateAtom (this.atoms[i]);
+
+this.htAtomMods = null;
+if (this.minXYZ0 != null) this.trimAtomSet ();
+this.htSubsystems = null;
+}, "~N,~B");
+Clazz.defineMethod (c$, "initModForStructure", 
+ function (iModel) {
+var key;
+if (this.legendres != null) this.fixLegendre ();
+this.sigma =  new JU.Matrix (null, this.modDim, 3);
+this.qs = null;
+this.modMatrices =  Clazz.newArray (-1, [this.sigma, null]);
+var pt;
+for (var i = 0; i < this.modDim; i++) {
+pt = this.getMod ("W_" + (i + 1));
+if (pt == null) {
+this.cr.appendLoadNote ("NOTE!: Not enough cell wave vectors for d=" + this.modDim);
+return;
+}this.fixDouble (pt);
+this.cr.appendLoadNote ("W_" + (i + 1) + " = " + JU.Escape.e (pt));
+this.cr.appendUunitCellInfo ("q" + (i + 1) + "=" + pt[0] + " " + pt[1] + " " + pt[2]);
+this.sigma.getArray ()[i] =  Clazz.newDoubleArray (-1, [pt[0], pt[1], pt[2]]);
+}
+var map =  new java.util.Hashtable ();
+for (var e, $e = this.htModulation.entrySet ().iterator (); $e.hasNext () && ((e = $e.next ()) || true);) {
+if ((key = this.checkKey (e.getKey (), false)) == null) continue;
+pt = e.getValue ();
+switch (key.charAt (0)) {
+case 'O':
+this.haveOccupancy = true;
+case 'D':
+case 'M':
+case 'U':
+if (pt[2] == 1 && key.charAt (2) != 'S' && key.charAt (2) != 'T' && key.charAt (2) != 'L') {
+var ipt = key.indexOf ("?");
+if (ipt >= 0) {
+var s = key.substring (ipt + 1);
+pt = this.getMod (key.substring (0, 2) + s + "#*;*");
+if (pt != null) this.addModulation (map, key = key.substring (0, ipt), pt, iModel);
+} else {
+var a = pt[0];
+var d = 2 * 3.141592653589793 * pt[1];
+pt[0] = (a * Math.cos (d));
+pt[1] = (a * Math.sin (-d));
+pt[2] = 0;
+JU.Logger.info ("msCIF setting " + key + " " + JU.Escape.e (pt));
+}}break;
+case 'W':
+if (this.modDim > 1) {
+continue;
+}case 'F':
+if (key.indexOf ("_coefs_") >= 0) {
+this.cr.appendLoadNote ("Wave vector " + key + "=" + JU.Escape.eAD (pt));
+} else {
+var ptHarmonic = this.calculateQCoefs (pt);
+if (ptHarmonic == null) {
+this.cr.appendLoadNote ("Cannot match atom wave vector " + key + " " + JU.Escape.eAD (pt) + " to a cell wave vector or its harmonic");
+} else {
+var k2 = key + "_coefs_";
+if (!this.htModulation.containsKey (k2 + this.atModel)) {
+this.addModulation (map, k2, ptHarmonic, iModel);
+if (key.startsWith ("F_")) this.cr.appendLoadNote ("atom wave vector " + key + " = " + JU.Escape.e (pt) + " fn = " + JU.Escape.e (ptHarmonic));
+}}}break;
+}
+}
+if (!map.isEmpty ()) this.htModulation.putAll (map);
+if (this.htSubsystems == null) {
+this.haveAtomMods = false;
+} else {
+this.cr.strSupercell = null;
+this.haveAtomMods = true;
+this.htAtomMods =  new java.util.Hashtable ();
+}for (var e, $e = this.htModulation.entrySet ().iterator (); $e.hasNext () && ((e = $e.next ()) || true);) {
+if ((key = this.checkKey (e.getKey (), true)) == null) continue;
+var params = e.getValue ();
+var atomName = key.substring (key.indexOf (";") + 1);
+var pt_ = atomName.indexOf ("#=");
+if (pt_ >= 0) {
+params = this.getMod (atomName.substring (pt_ + 2));
+atomName = atomName.substring (0, pt_);
+}if (JU.Logger.debuggingHigh) JU.Logger.debug ("SetModulation: " + key + " " + JU.Escape.e (params));
+var type = key.charAt (0);
+pt_ = key.indexOf ("#") + 1;
+var utens = null;
+switch (type) {
+case 'U':
+utens = key.substring (pt_, key.indexOf (";"));
+case 'O':
+case 'D':
+case 'M':
+if (this.modAverage) break;
+var axis = key.charAt (pt_);
+type = this.getModType (key);
+if (this.htAtomMods == null) this.htAtomMods =  new java.util.Hashtable ();
+var p =  Clazz.newDoubleArray (params.length, 0);
+for (var i = p.length; --i >= 0; ) p[i] = params[i];
+
+var qcoefs = this.getQCoefs (key);
+if (qcoefs == null) throw  new Exception ("Missing cell wave vector for atom wave vector for " + key + " " + JU.Escape.e (params));
+this.addAtomModulation (atomName, axis, type, p, utens, qcoefs);
+this.haveAtomMods = true;
+break;
+}
+}
+}, "~N");
+Clazz.defineMethod (c$, "fixLegendre", 
+ function () {
+for (var i = this.legendres.size (); --i >= 0; ) {
+var key = this.legendres.get (i);
+var pt = this.htModulation.get (key);
+if (pt != null) {
+var key1 = "O_0#0" + key.substring (key.indexOf (";"));
+var pt1 = this.htModulation.get (key1);
+if (pt1 == null) {
+JU.Logger.error ("Crenel " + key1 + " not found for legendre modulation " + key);
+pt[2] = NaN;
+} else {
+this.htModulation.put (key,  Clazz.newDoubleArray (-1, [pt1[0], pt1[1], pt[0], pt[1]]));
+}}}
+});
+Clazz.defineMethod (c$, "fixDouble", 
+ function (pt) {
+if (this.cr.fixJavaFloat) for (var i = pt.length; --i >= 0; ) pt[i] = JU.PT.fixDouble (pt[i], 100000.0);
+
+}, "~A");
+Clazz.overrideMethod (c$, "getQCoefs", 
+function (key) {
+var fn = Math.max (0, this.cr.parseIntAt (key, 2));
+if (fn == 0) {
+if (this.qlist100 == null) {
+this.qlist100 =  Clazz.newDoubleArray (this.modDim, 0);
+this.qlist100[0] = 1;
+}return this.qlist100;
+}return this.getMod ("F_coefs_" + fn);
+}, "~S");
+Clazz.overrideMethod (c$, "getModType", 
+function (key) {
+var type = key.charAt (0);
+var id = key.charAt (2);
+return (id == 'S' ? 's' : id == 'T' ? 't' : id == 'L' ? (type == 'D' ? 'l' : 'L') : id == '0' ? 'c' : type == 'D' ? 'f' : type == 'O' ? 'o' : type == 'M' ? 'm' : type == 'U' ? 'u' : '?');
+}, "~S");
+Clazz.defineMethod (c$, "calculateQCoefs", 
+ function (p) {
+if (this.qs == null) {
+this.qs =  new Array (this.modDim);
+for (var i = 0; i < this.modDim; i++) {
+this.qs[i] = this.toP3 (this.getMod ("W_" + (i + 1)));
+}
+}var pt = this.toP3 (p);
+for (var i = 0; i < this.modDim; i++) if (this.qs[i] != null) {
+var ifn = this.approxInt (pt.dot (this.qs[i]) / this.qs[i].dot (this.qs[i]));
+if (ifn != 0) {
+p =  Clazz.newDoubleArray (this.modDim, 0);
+p[i] = ifn;
+return p;
+}}
+var p3 = this.toP3 (p);
+var jmin = (this.modDim < 2 ? 0 : -3);
+var jmax = (this.modDim < 2 ? 0 : 3);
+var kmin = (this.modDim < 3 ? 0 : -3);
+var kmax = (this.modDim < 3 ? 0 : 3);
+for (var i = -3; i <= 3; i++) for (var j = jmin; j <= jmax; j++) for (var k = kmin; k <= kmax; k++) {
+pt.setT (this.qs[0]);
+pt.scale (i);
+if (this.modDim > 1 && this.qs[1] != null) pt.scaleAdd2 (j, this.qs[1], pt);
+if (this.modDim > 2 && this.qs[2] != null) pt.scaleAdd2 (k, this.qs[2], pt);
+if (pt.distanceSquared (p3) < 0.0001) {
+p =  Clazz.newDoubleArray (this.modDim, 0);
+switch (this.modDim) {
+default:
+p[2] = k;
+case 2:
+p[1] = j;
+case 1:
+p[0] = i;
+break;
+}
+return p;
+}pt.setT (this.qs[0]);
+pt.scale (1 / i);
+if (this.modDim > 1 && this.qs[1] != null) pt.scaleAdd2 (1 / j, this.qs[1], pt);
+if (this.modDim > 2 && this.qs[2] != null) pt.scaleAdd2 (1 / k, this.qs[2], pt);
+if (pt.distanceSquared (p3) < 0.0001) {
+p =  Clazz.newDoubleArray (this.modDim, 0);
+switch (this.modDim) {
+default:
+p[2] = 1 / k;
+case 2:
+p[1] = 1 / j;
+case 1:
+p[0] = 1 / i;
+break;
+}
+return p;
+}}
+
+
+pt = this.toP3 (p);
+for (var i = 0; i < this.modDim; i++) if (this.qs[i] != null) {
+p3 = this.qs[i];
+var ifn = 0;
+if (pt.x != 0) ifn = this.approxInt (pt.x / p3.x);
+if (pt.y != 0) ifn = Math.max (this.approxInt (pt.y / p3.y), ifn);
+if (ifn == 0 && pt.z != 0) ifn = Math.max (this.approxInt (pt.z / p3.z), ifn);
+if (ifn == 0) continue;
+if (p3.x != 0 && this.approxInt (10 + p3.x * ifn - pt.x) == 0 || p3.y != 0 && this.approxInt (10 + p3.y * ifn - pt.y) == 0 || p3.z != 0 && this.approxInt (10 + p3.z * ifn - pt.z) == 0) continue;
+p =  Clazz.newDoubleArray (this.modDim, 0);
+p[i] = ifn;
+return p;
+}
+return null;
+}, "~A");
+Clazz.defineMethod (c$, "approxInt", 
+ function (fn) {
+var ifn = Math.round (fn);
+return (Math.abs (fn - ifn) < 0.001 ? ifn : 0);
+}, "~N");
+Clazz.defineMethod (c$, "toP3", 
+ function (x) {
+return JU.P3.new3 (x[0], x[1], x[2]);
+}, "~A");
+Clazz.defineMethod (c$, "addAtomModulation", 
+ function (atomName, axis, type, params, utens, qcoefs) {
+var list = this.htAtomMods.get (atomName);
+if (list == null) {
+this.ac++;
+this.htAtomMods.put (atomName, list =  new JU.Lst ());
+}list.addLast ( new JU.Modulation (axis, type, params, utens, qcoefs));
+this.modCount++;
+}, "~S,~S,~S,~A,~S,~A");
+Clazz.overrideMethod (c$, "addSubsystem", 
+function (code, w) {
+if (code == null) return;
+var ss =  new J.adapter.readers.cif.Subsystem (this, code, w);
+this.cr.appendLoadNote ("subsystem " + code + "\n" + w);
+this.setSubsystem (code, ss);
+}, "~S,JU.Matrix");
+Clazz.defineMethod (c$, "addUStr", 
+ function (atom, id, val) {
+var i = Clazz.doubleToInt ("U11U22U33U12U13U23UISO".indexOf (id) / 3);
+if (JU.Logger.debuggingHigh) JU.Logger.debug ("MOD RDR adding " + id + " " + i + " " + val + " to " + atom.anisoBorU[i]);
+this.cr.asc.setU (atom, i, val + atom.anisoBorU[i]);
+}, "J.adapter.smarter.Atom,~S,~N");
+Clazz.defineMethod (c$, "modulateAtom", 
+ function (a) {
+if (this.modCoord && this.htSubsystems != null) {
+var ptc = JU.P3.newP (a);
+var spt = this.getSymmetry (a);
+spt.toCartesian (ptc, true);
+}var list = this.htAtomMods.get (a.atomName);
+if (list == null && a.altLoc != '\0' && this.htSubsystems != null) {
+list =  new JU.Lst ();
+}if (list == null || this.symmetry == null || a.bsSymmetry == null) return;
+var iop = Math.max (a.bsSymmetry.nextSetBit (0), 0);
+if (this.modLast) iop = Math.max ((a.bsSymmetry.length () - 1) % this.nOps, iop);
+if (JU.Logger.debuggingHigh) JU.Logger.debug ("\nsetModulation: i=" + a.index + " " + a.atomName + " xyz=" + a + " occ=" + a.foccupancy);
+if (iop != this.iopLast) {
+this.iopLast = iop;
+this.gammaE =  new JU.M3 ();
+this.getSymmetry (a).getSpaceGroupOperation (iop).getRotationScale (this.gammaE);
+}if (JU.Logger.debugging) {
+JU.Logger.debug ("setModulation iop = " + iop + " " + this.symmetry.getSpaceGroupXyz (iop, false) + " " + a.bsSymmetry);
+}var ms =  new JU.ModulationSet ().setMod (a.index + " " + a.atomName, this.getAtomR0 (this.cr.asc.atoms[a.atomSite]), this.getAtomR0 (a), this.modDim, list, this.gammaE, this.getMatrices (a), this.getSymmetry (a), this.nOps, iop, Clazz.instanceOf (a.vib, JU.Vibration) ? a.vib : null, this.isCommensurate);
+ms.calculate (this.modTUV, false);
+if (!Float.isNaN (ms.vOcc)) {
+a.foccupancy = ms.setOccupancy (this.getMod ("J_O#0;" + a.atomName), a.foccupancy, (a.vib == null ? 0 : a.vib.x));
+}if (ms.htUij != null) {
+var t = (a.tensors == null ? null : a.tensors.get (0));
+if (t != null && t.parBorU != null) {
+a.anisoBorU =  Clazz.newFloatArray (8, 0);
+for (var i = 0; i < 8; i++) a.anisoBorU[i] = t.parBorU[i];
+
+t.isUnmodulated = true;
+}if (a.anisoBorU == null) {
+JU.Logger.error ("MOD RDR cannot modulate nonexistent atom anisoBorU for atom " + a.atomName);
+} else {
+if (JU.Logger.debuggingHigh) {
+JU.Logger.debug ("setModulation Uij(initial)=" + JU.Escape.eAF (a.anisoBorU));
+JU.Logger.debug ("setModulation tensor=" + JU.Escape.e ((a.tensors.get (0)).getInfo ("all")));
+}for (var e, $e = ms.htUij.entrySet ().iterator (); $e.hasNext () && ((e = $e.next ()) || true);) this.addUStr (a, e.getKey (), e.getValue ().floatValue ());
+
+var sym = this.getAtomSymmetry (a, this.symmetry);
+t = this.cr.asc.getXSymmetry ().addRotatedTensor (a, sym.getTensor (this.cr.vwr, a.anisoBorU), iop, false, sym);
+t.isModulated = true;
+t.id = JU.Escape.e (a.anisoBorU);
+a.bfactor = a.anisoBorU[7] * 100;
+a.anisoBorU = null;
+if (JU.Logger.debuggingHigh) {
+JU.Logger.debug ("setModulation Uij(final)=" + JU.Escape.eAF (a.anisoBorU) + "\n");
+JU.Logger.debug ("setModulation tensor=" + JU.Escape.e ((a.tensors.get (1)).getInfo ("all")));
+}}}if (Float.isNaN (ms.x)) ms.set (0, 0, 0);
+a.vib = ms;
+}, "J.adapter.smarter.Atom");
+Clazz.defineMethod (c$, "getAtomR0", 
+ function (atom) {
+var r0 = JU.P3.newP (atom);
+if (this.supercellSymmetry != null) {
+this.supercellSymmetry.toCartesian (r0, true);
+this.symmetry.toFractional (r0, true);
+}return r0;
+}, "J.adapter.smarter.Atom");
+Clazz.overrideMethod (c$, "getAtomSymmetry", 
+function (a, defaultSymmetry) {
+var ss;
+return (this.htSubsystems == null || (ss = this.getSubsystem (a)) == null ? defaultSymmetry : ss.getSymmetry ());
+}, "J.adapter.smarter.Atom,J.api.SymmetryInterface");
+Clazz.defineMethod (c$, "setSubsystem", 
+ function (code, system) {
+if (this.htSubsystems == null) this.htSubsystems =  new java.util.Hashtable ();
+this.htSubsystems.put (code, system);
+this.setSubsystemOptions ();
+}, "~S,J.adapter.readers.cif.Subsystem");
+Clazz.defineMethod (c$, "getMatrices", 
+ function (a) {
+var ss = this.getSubsystem (a);
+return (ss == null ? this.modMatrices : ss.getModMatrices ());
+}, "J.adapter.smarter.Atom");
+Clazz.defineMethod (c$, "getSymmetry", 
+ function (a) {
+var ss = this.getSubsystem (a);
+return (ss == null ? this.symmetry : ss.getSymmetry ());
+}, "J.adapter.smarter.Atom");
+Clazz.defineMethod (c$, "getSubsystem", 
+ function (a) {
+return (this.htSubsystems == null ? null : this.htSubsystems.get ("" + a.altLoc));
+}, "J.adapter.smarter.Atom");
+Clazz.overrideMethod (c$, "setMinMax0", 
+function (minXYZ, maxXYZ) {
+if (this.htSubsystems == null) return;
+var symmetry = this.getDefaultUnitCell ();
+this.minXYZ0 = JU.P3.newP (minXYZ);
+this.maxXYZ0 = JU.P3.newP (maxXYZ);
+var pt0 = JU.P3.newP (minXYZ);
+var pt1 = JU.P3.newP (maxXYZ);
+var pt =  new JU.P3 ();
+symmetry.toCartesian (pt0, true);
+symmetry.toCartesian (pt1, true);
+var pts = JU.BoxInfo.unitCubePoints;
+if (this.sigma == null) {
+JU.Logger.error ("Why are we in MSRdr.setMinMax0 without modulation init?");
+return;
+}for (var e, $e = this.htSubsystems.entrySet ().iterator (); $e.hasNext () && ((e = $e.next ()) || true);) {
+var sym = e.getValue ().getSymmetry ();
+for (var i = 8; --i >= 0; ) {
+pt.x = (pts[i].x == 0 ? pt0.x : pt1.x);
+pt.y = (pts[i].y == 0 ? pt0.y : pt1.y);
+pt.z = (pts[i].z == 0 ? pt0.z : pt1.z);
+this.expandMinMax (pt, sym, minXYZ, maxXYZ);
+}
+}
+}, "JU.P3,JU.P3");
+Clazz.defineMethod (c$, "expandMinMax", 
+ function (pt, sym, minXYZ, maxXYZ) {
+var pt2 = JU.P3.newP (pt);
+var slop = 0.0001;
+sym.toFractional (pt2, false);
+if (minXYZ.x > pt2.x + slop) minXYZ.x = Clazz.doubleToInt (Math.floor (pt2.x)) - 1;
+if (minXYZ.y > pt2.y + slop) minXYZ.y = Clazz.doubleToInt (Math.floor (pt2.y)) - 1;
+if (minXYZ.z > pt2.z + slop) minXYZ.z = Clazz.doubleToInt (Math.floor (pt2.z)) - 1;
+if (maxXYZ.x < pt2.x - slop) maxXYZ.x = Clazz.doubleToInt (Math.ceil (pt2.x)) + 1;
+if (maxXYZ.y < pt2.y - slop) maxXYZ.y = Clazz.doubleToInt (Math.ceil (pt2.y)) + 1;
+if (maxXYZ.z < pt2.z - slop) maxXYZ.z = Clazz.doubleToInt (Math.ceil (pt2.z)) + 1;
+}, "JU.P3,J.api.SymmetryInterface,JU.P3,JU.P3");
+Clazz.defineMethod (c$, "trimAtomSet", 
+ function () {
+if (!this.cr.doApplySymmetry) return;
+var asc = this.cr.asc;
+var bs = asc.bsAtoms;
+var sym = this.getDefaultUnitCell ();
+var atoms = asc.atoms;
+var pt =  new JU.P3 ();
+if (bs == null) bs = asc.bsAtoms = JU.BSUtil.newBitSet2 (0, asc.ac);
+for (var i = bs.nextSetBit (0); i >= 0; i = bs.nextSetBit (i + 1)) {
+var a = atoms[i];
+var isOK = (!this.isCommensurate || this.modAverage || a.foccupancy >= 0.5);
+if (isOK) {
+pt.setT (a);
+if (a.vib != null) pt.add (a.vib);
+this.getSymmetry (a).toCartesian (pt, false);
+sym.toFractional (pt, false);
+if (this.cr.fixJavaFloat) JU.PT.fixPtFloats (pt, 100000.0);
+isOK = asc.xtalSymmetry.isWithinCell (3, pt, this.minXYZ0.x, this.maxXYZ0.x, this.minXYZ0.y, this.maxXYZ0.y, this.minXYZ0.z, this.maxXYZ0.z, 0.001);
+}if (isOK) {
+if (this.cr.fixJavaFloat) JU.PT.fixPtFloats (a, 100000.0);
+} else {
+bs.clear (i);
+}}
+});
+Clazz.defineMethod (c$, "getDefaultUnitCell", 
+ function () {
+return (this.modCell != null && this.htSubsystems.containsKey (this.modCell) ? this.htSubsystems.get (this.modCell).getSymmetry () : this.cr.asc.getSymmetry ());
+});
+Clazz.overrideMethod (c$, "getSymmetryFromCode", 
+function (code) {
+return this.htSubsystems.get (code).getSymmetry ();
+}, "~S");
+Clazz.overrideMethod (c$, "addLatticeVector", 
+function (lattvecs, data) {
+var a = null;
+var c = data.charAt (0);
+var dim = this.modDim + 3;
+switch (c) {
+case 'P':
+case 'X':
+break;
+case 'A':
+case 'B':
+case 'C':
+case 'I':
+a =  Clazz.newFloatArray (-1, [0.5, 0.5, 0.5]);
+if (c != 'I') a[c.charCodeAt (0) - 65] = 0;
+break;
+case 'F':
+this.addLatticeVector (lattvecs, "A");
+this.addLatticeVector (lattvecs, "B");
+this.addLatticeVector (lattvecs, "C");
+break;
+case 'M':
+dim++;
+case '0':
+if (data.indexOf (".") >= 0) a = J.adapter.smarter.AtomSetCollectionReader.getTokensFloat (data, null, dim);
+break;
+default:
+return false;
+}
+if (a != null) lattvecs.addLast (a);
+return true;
+}, "JU.Lst,~S");
+Clazz.defineStatics (c$,
+"U_LIST", "U11U22U33U12U13U23UISO");
+});

static/j2s/J/adapter/readers/cif/Subsystem.js

@@ -0,0 +1,105 @@
+Clazz.declarePackage ("J.adapter.readers.cif");
+Clazz.load (null, "J.adapter.readers.cif.Subsystem", ["JU.Lst", "$.Matrix", "$.V3", "JU.Logger", "$.SimpleUnitCell"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.msRdr = null;
+this.code = null;
+this.d = 0;
+this.w = null;
+this.symmetry = null;
+this.modMatrices = null;
+this.isFinalized = false;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.cif, "Subsystem");
+Clazz.makeConstructor (c$, 
+function (msRdr, code, w) {
+this.msRdr = msRdr;
+this.code = code;
+this.w = w;
+this.d = w.getArray ().length - 3;
+}, "J.adapter.readers.cif.MSRdr,~S,JU.Matrix");
+Clazz.defineMethod (c$, "getSymmetry", 
+function () {
+if (!this.isFinalized) this.setSymmetry (true);
+return this.symmetry;
+});
+Clazz.defineMethod (c$, "getModMatrices", 
+function () {
+if (!this.isFinalized) this.setSymmetry (true);
+return this.modMatrices;
+});
+Clazz.defineMethod (c$, "setSymmetry", 
+ function (setOperators) {
+var a;
+JU.Logger.info ("[subsystem " + this.code + "]");
+var winv = this.w.inverse ();
+JU.Logger.info ("w=" + this.w);
+JU.Logger.info ("w_inv=" + winv);
+var w33 = this.w.getSubmatrix (0, 0, 3, 3);
+var wd3 = this.w.getSubmatrix (3, 0, this.d, 3);
+var w3d = this.w.getSubmatrix (0, 3, 3, this.d);
+var wdd = this.w.getSubmatrix (3, 3, this.d, this.d);
+var sigma = this.msRdr.getSigma ();
+var sigma_nu = wdd.mul (sigma).add (wd3).mul (w3d.mul (sigma).add (w33).inverse ());
+var tFactor = wdd.sub (sigma_nu.mul (w3d));
+JU.Logger.info ("sigma_nu = " + sigma_nu);
+var s0 = this.msRdr.cr.asc.getSymmetry ();
+var vu43 = s0.getUnitCellVectors ();
+var vr43 = JU.SimpleUnitCell.getReciprocal (vu43, null, 1);
+var mard3 =  new JU.Matrix (null, 3 + this.d, 3);
+var mar3 =  new JU.Matrix (null, 3, 3);
+var mard3a = mard3.getArray ();
+var mar3a = mar3.getArray ();
+for (var i = 0; i < 3; i++) mard3a[i] = mar3a[i] =  Clazz.newDoubleArray (-1, [vr43[i + 1].x, vr43[i + 1].y, vr43[i + 1].z]);
+
+var sx = sigma.mul (mar3);
+a = sx.getArray ();
+for (var i = 0; i < this.d; i++) mard3a[i + 3] = a[i];
+
+a = this.w.mul (mard3).getArray ();
+var uc_nu =  new Array (4);
+uc_nu[0] = vu43[0];
+for (var i = 0; i < 3; i++) uc_nu[i + 1] = JU.V3.new3 (a[i][0], a[i][1], a[i][2]);
+
+uc_nu = JU.SimpleUnitCell.getReciprocal (uc_nu, null, 1);
+this.symmetry = (this.msRdr.cr.getInterface ("JS.Symmetry")).getUnitCell (uc_nu, false, null);
+this.modMatrices =  Clazz.newArray (-1, [sigma_nu, tFactor]);
+if (!setOperators) return;
+this.isFinalized = true;
+JU.Logger.info ("unit cell parameters: " + this.symmetry.getUnitCellInfo ());
+this.symmetry.createSpaceGroup (-1, "[subsystem " + this.code + "]",  new JU.Lst (), this.d);
+var nOps = s0.getSpaceGroupOperationCount ();
+for (var iop = 0; iop < nOps; iop++) {
+var rv = s0.getOperationRsVs (iop);
+var r0 = rv.getRotation ();
+var v0 = rv.getTranslation ();
+var r = this.w.mul (r0).mul (winv);
+var v = this.w.mul (v0);
+var code = this.code;
+if (this.isMixed (r)) {
+for (var e, $e = this.msRdr.htSubsystems.entrySet ().iterator (); $e.hasNext () && ((e = $e.next ()) || true);) {
+var ss = e.getValue ();
+if (ss === this) continue;
+var rj = ss.w.mul (r0).mul (winv);
+if (!this.isMixed (rj)) {
+r = rj;
+v = ss.w.mul (v0);
+code = ss.code;
+break;
+}}
+}var jf = this.symmetry.addOp (code, r, v, sigma_nu);
+JU.Logger.info (this.code + "." + (iop + 1) + (this.code.equals (code) ? "   " : ">" + code + " ") + jf);
+}
+}, "~B");
+Clazz.defineMethod (c$, "isMixed", 
+ function (r) {
+var a = r.getArray ();
+for (var i = 3; --i >= 0; ) for (var j = 3 + this.d; --j >= 3; ) if (a[i][j] != 0) return true;
+
+
+return false;
+}, "JU.Matrix");
+Clazz.overrideMethod (c$, "toString", 
+function () {
+return "Subsystem " + this.code + "\n" + this.w;
+});
+});

static/j2s/J/adapter/readers/molxyz/Mol3DReader.js

@@ -0,0 +1,8 @@
+Clazz.declarePackage ("J.adapter.readers.molxyz");
+Clazz.load (["J.adapter.readers.molxyz.MolReader"], "J.adapter.readers.molxyz.Mol3DReader", null, function () {
+c$ = Clazz.declareType (J.adapter.readers.molxyz, "Mol3DReader", J.adapter.readers.molxyz.MolReader);
+Clazz.overrideMethod (c$, "initializeReader", 
+function () {
+this.allow2D = false;
+});
+});

static/j2s/J/adapter/readers/molxyz/MolReader.js

@@ -0,0 +1,306 @@
+Clazz.declarePackage ("J.adapter.readers.molxyz");
+Clazz.load (["J.adapter.smarter.AtomSetCollectionReader"], "J.adapter.readers.molxyz.MolReader", ["java.lang.Exception", "java.util.Hashtable", "JU.PT", "J.adapter.smarter.Atom", "J.api.JmolAdapter", "JU.Logger"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.optimize2D = false;
+this.haveAtomSerials = false;
+this.dimension = null;
+this.allow2D = true;
+this.iatom0 = 0;
+this.vr = null;
+this.atomCount = 0;
+this.atomData = null;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.molxyz, "MolReader", J.adapter.smarter.AtomSetCollectionReader);
+Clazz.overrideMethod (c$, "initializeReader", 
+function () {
+this.optimize2D = this.checkFilterKey ("2D");
+});
+Clazz.overrideMethod (c$, "checkLine", 
+function () {
+var isMDL = (this.line.startsWith ("$MDL"));
+if (isMDL) {
+this.discardLinesUntilStartsWith ("$HDR");
+this.rd ();
+if (this.line == null) {
+JU.Logger.warn ("$HDR not found in MDL RG file");
+this.continuing = false;
+return false;
+}} else if (this.line.equals ("M  END")) {
+return true;
+}if (this.doGetModel (++this.modelNumber, null)) {
+this.iatom0 = this.asc.ac;
+this.processMolSdHeader ();
+this.processCtab (isMDL);
+this.vr = null;
+if (this.isLastModel (this.modelNumber)) {
+this.continuing = false;
+return false;
+}}if (this.line != null && this.line.indexOf ("$$$$") < 0) this.discardLinesUntilStartsWith ("$$$$");
+return true;
+});
+Clazz.overrideMethod (c$, "finalizeSubclassReader", 
+function () {
+this.finalizeReaderMR ();
+});
+Clazz.defineMethod (c$, "finalizeReaderMR", 
+function () {
+if (this.optimize2D) this.set2D ();
+this.isTrajectory = false;
+this.finalizeReaderASCR ();
+});
+Clazz.defineMethod (c$, "processMolSdHeader", 
+ function () {
+var header = "";
+var thisDataSetName = this.line.trim ();
+this.asc.setCollectionName (thisDataSetName);
+header += this.line + "\n";
+this.rd ();
+if (this.line == null) return;
+header += this.line + "\n";
+this.dimension = (this.line.length < 22 ? "3D" : this.line.substring (20, 22));
+if (this.dimension.equals ("2D")) {
+if (!this.allow2D) throw  new Exception ("File is 2D, not 3D");
+this.appendLoadNote ("This model is 2D. Its 3D structure has not been generated.");
+}this.asc.setInfo ("dimension", this.dimension);
+this.rd ();
+if (this.line == null) return;
+this.line = this.line.trim ();
+header += this.line + "\n";
+JU.Logger.info (header);
+this.checkCurrentLineForScript ();
+this.asc.setInfo ("fileHeader", header);
+this.newAtomSet (thisDataSetName);
+});
+Clazz.defineMethod (c$, "processCtab", 
+ function (isMDL) {
+if (isMDL) this.discardLinesUntilStartsWith ("$CTAB");
+if (this.rd () == null) return;
+if (this.line.indexOf ("V3000") >= 0) {
+this.optimize2D = (this.dimension.equals ("2D"));
+this.vr = (this.getInterface ("J.adapter.readers.molxyz.V3000Rdr")).set (this);
+this.discardLinesUntilContains ("COUNTS");
+this.vr.readAtomsAndBonds (this.getTokens ());
+} else {
+this.readAtomsAndBonds (this.parseIntRange (this.line, 0, 3), this.parseIntRange (this.line, 3, 6));
+}this.applySymmetryAndSetTrajectory ();
+}, "~B");
+Clazz.defineMethod (c$, "readAtomsAndBonds", 
+ function (ac, bc) {
+this.atomCount = ac;
+for (var i = 0; i < ac; ++i) {
+this.rd ();
+var len = this.line.length;
+var elementSymbol;
+var x;
+var y;
+var z;
+var charge = 0;
+var isotope = 0;
+var iAtom = -2147483648;
+x = this.parseFloatRange (this.line, 0, 10);
+y = this.parseFloatRange (this.line, 10, 20);
+z = this.parseFloatRange (this.line, 20, 30);
+if (len < 34) {
+elementSymbol = this.line.substring (31).trim ();
+} else {
+elementSymbol = this.line.substring (31, 34).trim ();
+if (elementSymbol.equals ("H1")) {
+elementSymbol = "H";
+isotope = 1;
+}if (len >= 39) {
+var code = this.parseIntRange (this.line, 36, 39);
+if (code >= 1 && code <= 7) charge = 4 - code;
+code = this.parseIntRange (this.line, 34, 36);
+if (code != 0 && code >= -3 && code <= 4) {
+isotope = J.api.JmolAdapter.getNaturalIsotope (J.api.JmolAdapter.getElementNumber (elementSymbol)) + code;
+}if (iAtom == -2147483648 && this.haveAtomSerials) iAtom = i + 1;
+}}this.addMolAtom (iAtom, isotope, elementSymbol, charge, x, y, z);
+}
+this.rd ();
+if (this.line.startsWith ("V  ")) {
+this.readAtomValues ();
+}if (bc == 0) this.asc.setNoAutoBond ();
+for (var i = 0; i < bc; ++i) {
+if (i > 0) this.rd ();
+var iAtom1;
+var iAtom2;
+var stereo = 0;
+iAtom1 = this.line.substring (0, 3).trim ();
+iAtom2 = this.line.substring (3, 6).trim ();
+var order = this.parseIntRange (this.line, 6, 9);
+if (this.optimize2D && order == 1 && this.line.length >= 12) stereo = this.parseIntRange (this.line, 9, 12);
+order = this.fixOrder (order, stereo);
+if (this.haveAtomSerials) this.asc.addNewBondFromNames (iAtom1, iAtom2, order);
+ else this.asc.addNewBondWithOrder (this.iatom0 + this.parseIntStr (iAtom1) - 1, this.iatom0 + this.parseIntStr (iAtom2) - 1, order);
+}
+var molData =  new java.util.Hashtable ();
+this.rd ();
+while (this.line != null && this.line.indexOf ("$$$$") != 0) {
+if (this.line.indexOf (">") == 0) {
+this.readMolData (molData);
+continue;
+}if (this.line.startsWith ("M  ISO")) {
+this.readIsotopes ();
+continue;
+}this.rd ();
+}
+if (this.atomData != null) {
+var atomValueName = molData.get ("atom_value_name");
+molData.put (atomValueName == null ? "atom_values" : atomValueName.toString (), this.atomData);
+}if (!molData.isEmpty ()) this.asc.setCurrentModelInfo ("molData", molData);
+}, "~N,~N");
+Clazz.defineMethod (c$, "readAtomValues", 
+ function () {
+this.atomData =  new Array (this.atomCount);
+for (var i = this.atomData.length; --i >= 0; ) this.atomData[i] = "";
+
+while (this.line.indexOf ("V  ") == 0) {
+var iAtom = this.parseIntAt (this.line, 3);
+if (iAtom < 1 || iAtom > this.atomCount) {
+JU.Logger.error ("V  nnn does not evalute to a valid atom number: " + iAtom);
+return;
+}var s = this.line.substring (6).trim ();
+this.atomData[iAtom - 1] = s;
+this.rd ();
+}
+});
+Clazz.defineMethod (c$, "readIsotopes", 
+ function () {
+var n = this.parseIntAt (this.line, 6);
+try {
+var i0 = this.asc.getLastAtomSetAtomIndex ();
+for (var i = 0, pt = 9; i < n; i++) {
+var ipt = this.parseIntAt (this.line, pt);
+var atom = this.asc.atoms[ipt + i0 - 1];
+var iso = this.parseIntAt (this.line, pt + 4);
+pt += 8;
+atom.elementSymbol = "" + iso + JU.PT.replaceAllCharacters (atom.elementSymbol, "0123456789", "");
+}
+} catch (e) {
+}
+this.rd ();
+});
+Clazz.defineMethod (c$, "readMolData", 
+ function (molData) {
+var atoms = this.asc.atoms;
+var dataName = JU.PT.trim (this.line, "> <").toLowerCase ();
+var data = "";
+var fdata = null;
+while (this.rd () != null && !this.line.equals ("$$$$") && this.line.length > 0) data += (this.line.length == 81 && this.line.charAt (80) == '+' ? this.line.substring (0, 80) : this.line + "\n");
+
+data = JU.PT.trim (data, "\n");
+JU.Logger.info (dataName + ":" + JU.PT.esc (data));
+molData.put (dataName, data);
+var ndata = 0;
+if (dataName.toUpperCase ().contains ("_PARTIAL_CHARGES")) {
+try {
+fdata = JU.PT.parseFloatArray (data);
+for (var i = this.asc.getLastAtomSetAtomIndex (), n = this.asc.ac; i < n; i++) atoms[i].partialCharge = 0;
+
+var pt = 0;
+for (var i = Clazz.floatToInt (fdata[pt++]); --i >= 0; ) {
+var atomIndex = Clazz.floatToInt (fdata[pt++]) + this.iatom0 - 1;
+var partialCharge = fdata[pt++];
+atoms[atomIndex].partialCharge = partialCharge;
+ndata++;
+}
+} catch (e) {
+for (var i = this.asc.getLastAtomSetAtomIndex (), n = this.asc.ac; i < n; i++) atoms[i].partialCharge = 0;
+
+JU.Logger.error ("error reading " + dataName + " field -- partial charges cleared");
+}
+JU.Logger.info (ndata + " partial charges read");
+} else if (dataName.toUpperCase ().contains ("ATOM_NAMES")) {
+ndata = 0;
+try {
+var tokens = JU.PT.getTokens (data);
+var pt = 0;
+for (var i = this.parseIntStr (tokens[pt++]); --i >= 0; ) {
+var iatom;
+while ((iatom = this.parseIntStr (tokens[pt++])) == -2147483648) {
+}
+var atomIndex = iatom + this.iatom0 - 1;
+var name = tokens[pt++];
+if (!name.equals (".")) atoms[atomIndex].atomName = name;
+ndata++;
+}
+} catch (e) {
+JU.Logger.error ("error reading " + dataName + " field");
+}
+JU.Logger.info (ndata + " atom names read");
+}}, "java.util.Map");
+Clazz.defineMethod (c$, "addMolAtom", 
+function (iAtom, isotope, elementSymbol, charge, x, y, z) {
+switch (isotope) {
+case 0:
+break;
+case 1:
+elementSymbol = "1H";
+break;
+case 2:
+elementSymbol = "2H";
+break;
+case 3:
+elementSymbol = "3H";
+break;
+default:
+elementSymbol = isotope + elementSymbol;
+}
+if (this.optimize2D && z != 0) this.optimize2D = false;
+var atom =  new J.adapter.smarter.Atom ();
+atom.elementSymbol = elementSymbol;
+atom.formalCharge = charge;
+this.setAtomCoordXYZ (atom, x, y, z);
+if (iAtom == -2147483648) {
+this.asc.addAtom (atom);
+} else {
+this.haveAtomSerials = true;
+atom.atomSerial = iAtom;
+this.asc.addAtomWithMappedSerialNumber (atom);
+}}, "~N,~N,~S,~N,~N,~N,~N");
+Clazz.defineMethod (c$, "fixOrder", 
+function (order, stereo) {
+switch (order) {
+default:
+case 0:
+case -10:
+return 1;
+case 1:
+switch (stereo) {
+case 1:
+return 1025;
+case 3:
+case 6:
+return 1041;
+}
+break;
+case 2:
+case 3:
+break;
+case 4:
+return 515;
+case 5:
+return 66;
+case 6:
+return 513;
+case 7:
+return 514;
+case 8:
+case 9:
+return 33;
+case 14:
+return 4;
+case 15:
+return 5;
+case 16:
+return 6;
+}
+return order;
+}, "~N,~N");
+Clazz.defineMethod (c$, "addMolBond", 
+function (iAtom1, iAtom2, order, stereo) {
+order = this.fixOrder (order, stereo);
+if (this.haveAtomSerials) this.asc.addNewBondFromNames (iAtom1, iAtom2, order);
+ else this.asc.addNewBondWithOrder (this.iatom0 + this.parseIntStr (iAtom1) - 1, this.iatom0 + this.parseIntStr (iAtom2) - 1, order);
+}, "~S,~S,~N,~N");
+});

static/j2s/J/adapter/readers/molxyz/V3000Rdr.js

@@ -0,0 +1,158 @@
+Clazz.declarePackage ("J.adapter.readers.molxyz");
+Clazz.load (null, "J.adapter.readers.molxyz.V3000Rdr", ["java.util.Hashtable", "JU.PT"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.mr = null;
+this.line = null;
+this.userData = null;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.molxyz, "V3000Rdr");
+Clazz.makeConstructor (c$, 
+function () {
+});
+Clazz.defineMethod (c$, "set", 
+function (mr) {
+this.mr = mr;
+return this;
+}, "J.adapter.smarter.AtomSetCollectionReader");
+Clazz.defineMethod (c$, "readAtomsAndBonds", 
+function (tokens) {
+var ac = this.mr.parseIntStr (tokens[3]);
+this.readAtoms (ac);
+this.readBonds (this.mr.parseIntStr (tokens[4]));
+this.readUserData (ac);
+}, "~A");
+Clazz.defineMethod (c$, "readAtoms", 
+ function (ac) {
+this.mr.discardLinesUntilContains ("BEGIN ATOM");
+for (var i = 0; i < ac; ++i) {
+this.rd ();
+this.checkLineContinuation ();
+var tokens = this.mr.getTokens ();
+var iAtom = this.mr.parseIntStr (tokens[2]);
+var elementSymbol = tokens[3];
+if (elementSymbol.equals ("*")) continue;
+var x = this.mr.parseFloatStr (tokens[4]);
+var y = this.mr.parseFloatStr (tokens[5]);
+var z = this.mr.parseFloatStr (tokens[6]);
+var charge = 0;
+var isotope = 0;
+for (var j = 7; j < tokens.length; j++) {
+var s = tokens[j].toUpperCase ();
+if (s.startsWith ("CHG=")) charge = this.mr.parseIntAt (tokens[j], 4);
+ else if (s.startsWith ("MASS=")) isotope = this.mr.parseIntAt (tokens[j], 5);
+}
+if (isotope > 1 && elementSymbol.equals ("H")) isotope = 1 - isotope;
+this.mr.addMolAtom (iAtom, isotope, elementSymbol, charge, x, y, z);
+}
+this.mr.discardLinesUntilContains ("END ATOM");
+}, "~N");
+Clazz.defineMethod (c$, "readBonds", 
+ function (bondCount) {
+this.mr.discardLinesUntilContains ("BEGIN BOND");
+if (bondCount == 0) this.mr.asc.setNoAutoBond ();
+for (var i = 0; i < bondCount; ++i) {
+this.rd ();
+var stereo = 0;
+this.checkLineContinuation ();
+var tokens = this.mr.getTokens ();
+var order = this.mr.parseIntStr (tokens[3]);
+var iAtom1 = tokens[4];
+var iAtom2 = tokens[5];
+var cfg = this.getField ("CFG");
+if (cfg == null) {
+var endpts = this.getField ("ENDPTS");
+if (endpts != null && this.line.indexOf ("ATTACH=ALL") >= 0) {
+tokens = JU.PT.getTokens (endpts);
+var n = this.mr.parseIntStr (tokens[0]);
+var o = this.mr.fixOrder (order, 0);
+for (var k = 1; k <= n; k++) this.mr.asc.addNewBondFromNames (iAtom1, tokens[k], o);
+
+}} else {
+stereo = this.mr.parseIntStr (cfg);
+}this.mr.addMolBond (iAtom1, iAtom2, order, stereo);
+}
+this.mr.discardLinesUntilContains ("END BOND");
+}, "~N");
+Clazz.defineMethod (c$, "readUserData", 
+ function (ac) {
+this.userData = null;
+var pc = null;
+while (!this.rd ().contains ("END CTAB")) {
+if (!this.line.contains ("BEGIN SGROUP")) continue;
+var atoms;
+var name;
+var data;
+while (!this.rd ().contains ("END SGROUP")) {
+if (this.userData == null) this.userData =  new java.util.Hashtable ();
+if ((atoms = this.getField ("ATOMS")) == null || (name = this.getField ("FIELDNAME")) == null || (data = this.getField ("FIELDDATA")) == null) continue;
+name = name.toLowerCase ();
+var isPartial = (name.indexOf ("partial") >= 0);
+if (isPartial) {
+if (pc == null) pc = name;
+ else if (!pc.equals (name)) isPartial = false;
+}if (isPartial) {
+var at = this.mr.asc.atoms;
+for (var i = this.mr.asc.getLastAtomSetAtomIndex (), n = this.mr.asc.ac; i < n; i++) at[i].partialCharge = 0;
+
+}var a = null;
+var f = 0;
+if (isPartial) f = this.mr.parseFloatStr (data);
+ else if ((a = this.userData.get (name)) == null) this.userData.put (name, a =  new Array (ac));
+try {
+var tokens = JU.PT.getTokens (atoms);
+for (var i = tokens.length; --i >= 1; ) {
+var atom = tokens[i];
+if (isPartial) this.mr.asc.getAtomFromName (atom).partialCharge = f;
+ else a[this.mr.parseIntStr (atom) - 1] = data;
+}
+} catch (e) {
+if (Clazz.exceptionOf (e, Exception)) {
+} else {
+throw e;
+}
+}
+}
+}
+if (this.userData == null) return;
+for (var key, $key = this.userData.keySet ().iterator (); $key.hasNext () && ((key = $key.next ()) || true);) {
+var a = this.userData.get (key);
+var f =  Clazz.newFloatArray (a.length, 0);
+for (var i = 0; i < a.length; i++) f[i] = (a[i] == null ? 0 : this.mr.parseFloatStr (a[i]));
+
+this.mr.asc.setAtomProperties (key, f, -1, false);
+}
+}, "~N");
+Clazz.defineMethod (c$, "getField", 
+ function (key) {
+var pt = this.line.indexOf (key + "=");
+if (pt < 0) return null;
+pt += key.length + 1;
+var term = ' ';
+switch (this.line.charAt (pt)) {
+case '"':
+term = '"';
+break;
+case '(':
+term = ')';
+break;
+case '+':
+break;
+default:
+pt--;
+break;
+}
+return this.line.substring (pt + 1, (this.line + term).indexOf (term, pt + 1));
+}, "~S");
+Clazz.defineMethod (c$, "rd", 
+ function () {
+return (this.line = this.mr.rd ());
+});
+Clazz.defineMethod (c$, "checkLineContinuation", 
+ function () {
+while (this.line.endsWith ("-")) {
+var s = this.line;
+this.rd ();
+this.line = s + this.line;
+}
+});
+});

static/j2s/J/adapter/readers/molxyz/XyzReader.js

@@ -0,0 +1,80 @@
+Clazz.declarePackage ("J.adapter.readers.molxyz");
+Clazz.load (["J.adapter.smarter.AtomSetCollectionReader"], "J.adapter.readers.molxyz.XyzReader", ["java.lang.Float", "JU.Logger"], function () {
+c$ = Clazz.declareType (J.adapter.readers.molxyz, "XyzReader", J.adapter.smarter.AtomSetCollectionReader);
+Clazz.overrideMethod (c$, "checkLine", 
+function () {
+var modelAtomCount = this.parseIntStr (this.line);
+if (modelAtomCount == -2147483648) {
+this.continuing = false;
+return false;
+}this.vibrationNumber = ++this.modelNumber;
+if (this.desiredVibrationNumber <= 0 ? this.doGetModel (this.modelNumber, null) : this.doGetVibration (this.vibrationNumber)) {
+this.rd ();
+this.checkCurrentLineForScript ();
+this.asc.newAtomSet ();
+var name = this.line.trim ();
+this.readAtoms (modelAtomCount);
+this.applySymmetryAndSetTrajectory ();
+this.asc.setAtomSetName (name);
+if (this.isLastModel (this.modelNumber)) {
+this.continuing = false;
+return false;
+}} else {
+this.skipAtomSet (modelAtomCount);
+}this.discardLinesUntilNonBlank ();
+return false;
+});
+Clazz.overrideMethod (c$, "finalizeSubclassReader", 
+function () {
+this.isTrajectory = false;
+this.finalizeReaderASCR ();
+});
+Clazz.defineMethod (c$, "skipAtomSet", 
+ function (modelAtomCount) {
+this.rd ();
+for (var i = modelAtomCount; --i >= 0; ) this.rd ();
+
+}, "~N");
+Clazz.defineMethod (c$, "readAtoms", 
+ function (modelAtomCount) {
+for (var i = 0; i < modelAtomCount; ++i) {
+this.rd ();
+var tokens = this.getTokens ();
+if (tokens.length < 4) {
+JU.Logger.warn ("line cannot be read for XYZ atom data: " + this.line);
+continue;
+}var atom = this.addAtomXYZSymName (tokens, 1, null, null);
+this.setElementAndIsotope (atom, tokens[0]);
+var vpt = 4;
+switch (tokens.length) {
+case 4:
+continue;
+case 5:
+case 6:
+case 8:
+case 9:
+if (tokens[4].indexOf (".") >= 0) {
+atom.partialCharge = this.parseFloatStr (tokens[4]);
+} else {
+var charge = this.parseIntStr (tokens[4]);
+if (charge != -2147483648) atom.formalCharge = charge;
+}switch (tokens.length) {
+case 5:
+continue;
+case 6:
+atom.radius = this.parseFloatStr (tokens[5]);
+continue;
+case 9:
+atom.atomSerial = this.parseIntStr (tokens[8]);
+}
+vpt++;
+default:
+var vx = this.parseFloatStr (tokens[vpt++]);
+var vy = this.parseFloatStr (tokens[vpt++]);
+var vz = this.parseFloatStr (tokens[vpt++]);
+if (Float.isNaN (vx) || Float.isNaN (vy) || Float.isNaN (vz)) continue;
+this.asc.addVibrationVector (atom.index, vx, vy, vz);
+}
+}
+}, "~N");
+});

static/j2s/J/adapter/readers/more/BinaryDcdReader.js

@@ -0,0 +1,152 @@
+Clazz.declarePackage ("J.adapter.readers.more");
+Clazz.load (["J.adapter.smarter.AtomSetCollectionReader"], "J.adapter.readers.more.BinaryDcdReader", ["java.lang.Boolean", "JU.BS", "$.P3", "$.SB", "JU.Escape", "$.Logger"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.nModels = 0;
+this.nAtoms = 0;
+this.nFree = 0;
+this.bsFree = null;
+this.xAll = null;
+this.yAll = null;
+this.zAll = null;
+this.crystGroup = 0;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.more, "BinaryDcdReader", J.adapter.smarter.AtomSetCollectionReader);
+Clazz.overrideMethod (c$, "setup", 
+function (fullPath, htParams, reader) {
+this.isBinary = true;
+this.requiresBSFilter = true;
+this.setupASCR (fullPath, htParams, reader);
+}, "~S,java.util.Map,~O");
+Clazz.overrideMethod (c$, "initializeReader", 
+function () {
+this.initializeTrajectoryFile ();
+this.asc.setInfo ("ignoreUnitCell", Boolean.TRUE);
+});
+Clazz.overrideMethod (c$, "processBinaryDocument", 
+function () {
+var bytes =  Clazz.newByteArray (40, 0);
+this.binaryDoc.setStream (null, this.binaryDoc.readInt () == 0x54);
+this.binaryDoc.readInt ();
+this.nModels = this.binaryDoc.readInt ();
+var nPriv = this.binaryDoc.readInt ();
+var nSaveC = this.binaryDoc.readInt ();
+var nStep = this.binaryDoc.readInt ();
+this.binaryDoc.readInt ();
+this.binaryDoc.readInt ();
+this.binaryDoc.readInt ();
+var ndegf = this.binaryDoc.readInt ();
+this.nFree = Clazz.doubleToInt (ndegf / 3);
+var nFixed = this.binaryDoc.readInt ();
+var delta4 = this.binaryDoc.readInt ();
+this.crystGroup = this.binaryDoc.readInt ();
+this.binaryDoc.readByteArray (bytes, 0, 32);
+this.binaryDoc.readInt ();
+this.binaryDoc.readInt ();
+this.binaryDoc.readInt ();
+var sb =  new JU.SB ();
+for (var i = 0, n = this.binaryDoc.readInt (); i < n; i++) sb.append (this.trimString (this.binaryDoc.readString (80))).appendC ('\n');
+
+this.binaryDoc.readInt ();
+JU.Logger.info ("BinaryDcdReadaer:\n" + sb);
+this.binaryDoc.readInt ();
+this.nAtoms = this.binaryDoc.readInt ();
+this.binaryDoc.readInt ();
+this.nFree = this.nAtoms - nFixed;
+if (nFixed != 0) {
+this.binaryDoc.readInt ();
+this.bsFree = JU.BS.newN (this.nFree);
+for (var i = 0; i < this.nFree; i++) this.bsFree.set (this.binaryDoc.readInt () - 1);
+
+this.binaryDoc.readInt ();
+JU.Logger.info ("free: " + this.bsFree.cardinality () + " " + JU.Escape.eBS (this.bsFree));
+}this.readCoordinates ();
+JU.Logger.info ("Total number of trajectory steps=" + this.trajectorySteps.size ());
+});
+Clazz.defineMethod (c$, "trimString", 
+ function (s) {
+var pt = s.indexOf ('\0');
+if (pt >= 0) s = s.substring (0, pt);
+return s.trim ();
+}, "~S");
+Clazz.defineMethod (c$, "readFloatArray", 
+ function () {
+var n = Clazz.doubleToInt (this.binaryDoc.readInt () / 4);
+var data =  Clazz.newFloatArray (n, 0);
+for (var i = 0; i < n; i++) data[i] = this.binaryDoc.readFloat ();
+
+this.binaryDoc.readInt ();
+return data;
+});
+Clazz.defineMethod (c$, "readDoubleArray", 
+ function () {
+var n = Clazz.doubleToInt (this.binaryDoc.readInt () / 8);
+var data =  Clazz.newDoubleArray (n, 0);
+for (var i = 0; i < n; i++) data[i] = this.binaryDoc.readDouble ();
+
+this.binaryDoc.readInt ();
+return data;
+});
+Clazz.defineMethod (c$, "readCoordinates", 
+ function () {
+var ac = (this.bsFilter == null ? this.templateAtomCount : (this.htParams.get ("filteredAtomCount")).intValue ());
+for (var i = 0; i < this.nModels; i++) if (this.doGetModel (++this.modelNumber, null)) {
+var trajectoryStep =  new Array (ac);
+if (!this.getTrajectoryStep (trajectoryStep)) return;
+this.trajectorySteps.addLast (trajectoryStep);
+if (this.isLastModel (this.modelNumber)) return;
+} else {
+if (this.crystGroup > 0) this.readDoubleArray ();
+this.readFloatArray ();
+this.readFloatArray ();
+this.readFloatArray ();
+}
+});
+Clazz.defineMethod (c$, "getTrajectoryStep", 
+ function (trajectoryStep) {
+try {
+var ac = trajectoryStep.length;
+var n = -1;
+if (this.crystGroup > 0) this.calcUnitCell (this.readDoubleArray ());
+var x = this.readFloatArray ();
+var y = this.readFloatArray ();
+var z = this.readFloatArray ();
+var bs = (this.xAll == null ? null : this.bsFree);
+if (bs == null) {
+this.xAll = x;
+this.yAll = y;
+this.zAll = z;
+}for (var i = 0, vpt = 0; i < this.nAtoms; i++) {
+var pt =  new JU.P3 ();
+if (bs == null || bs.get (i)) {
+pt.set (x[vpt], y[vpt], z[vpt]);
+vpt++;
+} else {
+pt.set (this.xAll[i], this.yAll[i], this.zAll[i]);
+}if (this.bsFilter == null || this.bsFilter.get (i)) {
+if (++n == ac) return true;
+trajectoryStep[n] = pt;
+}}
+return true;
+} catch (e) {
+if (Clazz.exceptionOf (e, Exception)) {
+return false;
+} else {
+throw e;
+}
+}
+}, "~A");
+Clazz.defineMethod (c$, "calcUnitCell", 
+ function (abc) {
+var a = abc[0];
+var angle1 = abc[1];
+var b = abc[2];
+var angle2 = abc[3];
+var angle3 = abc[4];
+var c = abc[5];
+var alpha = (1.5707963267948966 - Math.asin (angle3)) * 180 / 3.141592653589793;
+var beta = (1.5707963267948966 - Math.asin (angle2)) * 180 / 3.141592653589793;
+var gamma = (1.5707963267948966 - Math.asin (angle1)) * 180 / 3.141592653589793;
+System.out.println ("unitcell:[" + a + " " + b + " " + c + " " + alpha + " " + beta + " " + gamma + "]");
+return  Clazz.newFloatArray (-1, [a, b, c, alpha, beta, gamma]);
+}, "~A");
+});

static/j2s/J/adapter/readers/more/ForceFieldReader.js

@@ -0,0 +1,93 @@
+Clazz.declarePackage ("J.adapter.readers.more");
+Clazz.load (["J.adapter.smarter.AtomSetCollectionReader", "java.util.Properties"], "J.adapter.readers.more.ForceFieldReader", ["java.lang.Character", "JU.PT", "J.adapter.smarter.Atom"], function () {
+c$ = Clazz.decorateAsClass (function () {
+this.userAtomTypes = null;
+this.atomTypes = null;
+Clazz.instantialize (this, arguments);
+}, J.adapter.readers.more, "ForceFieldReader", J.adapter.smarter.AtomSetCollectionReader);
+Clazz.prepareFields (c$, function () {
+this.atomTypes =  new java.util.Properties ();
+});
+Clazz.defineMethod (c$, "setUserAtomTypes", 
+function () {
+this.userAtomTypes = this.htParams.get ("atomTypes");
+if (this.userAtomTypes != null) this.userAtomTypes = ";" + this.userAtomTypes + ";";
+});
+Clazz.defineMethod (c$, "getElementSymbol", 
+function (atom, atomType) {
+var elementSymbol = this.atomTypes.get (atomType);
+if (elementSymbol != null) {
+atom.elementSymbol = elementSymbol;
+return true;
+}var nChar = atomType.length;
+var haveSymbol = (nChar < 2);
+var ptType;
+if (this.userAtomTypes != null && (ptType = this.userAtomTypes.indexOf (";" + atomType + "=>")) >= 0) {
+ptType += nChar + 3;
+elementSymbol = this.userAtomTypes.substring (ptType, this.userAtomTypes.indexOf (";", ptType)).trim ();
+haveSymbol = true;
+} else if (nChar == 1) {
+elementSymbol = atomType.toUpperCase ();
+haveSymbol = true;
+} else {
+var ch0 = atomType.charAt (0);
+var ch1 = atomType.charAt (1);
+var isXx = (JU.PT.isUpperCase (ch0) && JU.PT.isLowerCase (ch1));
+if (" IM IP sz az sy ay ayt ".indexOf (atomType) >= 0) {
+if (ch0 == 'I') {
+elementSymbol = atom.atomName.substring (0, 2);
+if (!JU.PT.isLowerCase (elementSymbol.charAt (1))) elementSymbol = elementSymbol.substring (0, 1);
+} else {
+elementSymbol = (ch0 == 's' ? "Si" : "Al");
+}} else if (nChar == 2 && isXx) {
+} else if (JU.PT.isLetter (ch0) && !JU.PT.isLetter (ch1)) {
+elementSymbol = "" + Character.toUpperCase (ch0);
+} else if (nChar > 2 && isXx && !JU.PT.isLetter (atomType.charAt (2))) {
+elementSymbol = "" + ch0 + ch1;
+} else {
+ch0 = Character.toUpperCase (ch0);
+var check = " " + atomType + " ";
+if (" CA CB CC CD CE CF CG CH CI CJ CK CM CN CP CQ CR CT CV CW HA HP HC HO HS HW LP NA NB NC NT OH OS OW SH AH BH HT HY AC BC CS OA OB OE OT  dw hc hi hn ho hp hs hw hscp htip ca cg ci cn co coh cp cr cs ct c3h c3m c4h c4m na nb nh nho nh+ ni nn np npc nr nt nz oc oe oh op oscp otip sc sh sp br cl ca+ ar si lp nu sz oz az pz ga ge tioc titd li+ na+ rb+ cs+ mg2+ ca2+ ba2+ cu2+ cl- br- so4 sy oy ay ayt nac+ mg2c fe2c mn4c mn3c co2c ni2c lic+ pd2+ ti4c sr2c ca2c cly- hocl py vy nh4+ so4y lioh naoh koh foh cloh beoh al  CE1 CF1 CF2 CF3 CG CD2 CH1E CH2E CH3E CM CP3 CPH1 CPH2 CQ66 CR55 CR56 CR66 CS66 CT CT3 CT4 CUA1 CUA2 CUA3 CUY1 CUY2 HA HC HMU HO HT LP NC NC2 NO2 NP NR1 NR2 NR3 NR55 NR56 NR66 NT NX OA OAC OC OE OH2 OK OM OS OSH OSI OT OW PO3 PO4 PT PUA1 PUY1 SE SH1E SK SO1 SO2 SO3 SO4 ST ST2 ST2  br br- br1 cl cl- cl1 cl12 cl13 cl14 cl1p ca+ cu+2 fe+2 mg+2 zn+2 cs+ li+ na+ rb+ al4z si si4 si4c si4z ar he kr ne xe  dw hi hw ca cg ci co coh cp cr cs ct ct3 na nb nh nho ni no np nt nt2 nz oa oc oh op os ot sp bt cl\' si4l si5l si5t si6 si6o si\'  br ca cc cd ce cf cl cp cq cu cv cx cy ha hc hn ho hp hs na nb nc nd nh oh os pb pc pd pe pf px py sh ss sx sy  hn2 ho2 cz oo oz si sio hsi osi ".indexOf (check) < 0) {
+} else if (" AH BH AC BC ".indexOf (check) >= 0) {
+elementSymbol = "" + ch1;
+} else if (" al al4z ar ba2+ beoh br br- br1 ca+ ca2+ ca2c cl cl\' cl- cl1 cl12 cl13 cl14 cl1p cloh cly- co2c cs+ cu+2 cu2+ fe+2 fe2c ga ge he kr li+ lic+ lioh lp LP mg+2 mg2+ mg2c mn3c mn4c na+ nac+ naoh ne ni2c nu pd2+ rb+ si si\' si4 si4c si4l si4z si5l si5t si6 si6o sio sr2c ti4c tioc titd xe zn+2 ".indexOf (check) >= 0) {
+elementSymbol = "" + ch0 + ch1;
+} else {
+elementSymbol = "" + ch0;
+}}if (elementSymbol == null) {
+elementSymbol = "" + ch0 + Character.toLowerCase (ch1);
+} else {
+haveSymbol = true;
+}}atom.elementSymbol = elementSymbol;
+if (haveSymbol) this.atomTypes.put (atomType, elementSymbol);
+return haveSymbol;
+}, "J.adapter.smarter.Atom,~S");
+c$.deducePdbElementSymbol = Clazz.defineMethod (c$, "deducePdbElementSymbol", 
+function (isHetero, XX, group3) {
+var i = XX.indexOf ('\0');
+var atomType = null;
+if (i >= 0) {
+atomType = XX.substring (i + 1);
+XX = XX.substring (0, i);
+if (atomType != null && atomType.length == 1) return atomType;
+}if (XX.equalsIgnoreCase (group3)) return XX;
+var len = XX.length;
+var ch1 = ' ';
+i = 0;
+while (i < len && (ch1 = XX.charAt (i++)) <= '9') {
+}
+var ch2 = (i < len ? XX.charAt (i) : ' ');
+var full = group3 + "." + ch1 + ch2;
+if (("OEC.CA ICA.CA OC1.CA OC2.CA OC4.CA").indexOf (full) >= 0) return "Ca";
+if (XX.indexOf ("'") > 0 || XX.indexOf ("*") >= 0 || "HCNO".indexOf (ch1) >= 0 && ch2 <= 'H' || XX.startsWith ("CM")) return "" + ch1;
+if (isHetero && J.adapter.smarter.Atom.isValidSymNoCase (ch1, ch2)) return ("" + ch1 + ch2).trim ();
+if (J.adapter.smarter.Atom.isValidSym1 (ch1)) return "" + ch1;
+if (J.adapter.smarter.Atom.isValidSym1 (ch2)) return "" + ch2;
+return "Xx";
+}, "~B,~S,~S");
+Clazz.defineStatics (c$,
+"ffTypes", " CA CB CC CD CE CF CG CH CI CJ CK CM CN CP CQ CR CT CV CW HA HP HC HO HS HW LP NA NB NC NT OH OS OW SH AH BH HT HY AC BC CS OA OB OE OT  dw hc hi hn ho hp hs hw hscp htip ca cg ci cn co coh cp cr cs ct c3h c3m c4h c4m na nb nh nho nh+ ni nn np npc nr nt nz oc oe oh op oscp otip sc sh sp br cl ca+ ar si lp nu sz oz az pz ga ge tioc titd li+ na+ rb+ cs+ mg2+ ca2+ ba2+ cu2+ cl- br- so4 sy oy ay ayt nac+ mg2c fe2c mn4c mn3c co2c ni2c lic+ pd2+ ti4c sr2c ca2c cly- hocl py vy nh4+ so4y lioh naoh koh foh cloh beoh al  CE1 CF1 CF2 CF3 CG CD2 CH1E CH2E CH3E CM CP3 CPH1 CPH2 CQ66 CR55 CR56 CR66 CS66 CT CT3 CT4 CUA1 CUA2 CUA3 CUY1 CUY2 HA HC HMU HO HT LP NC NC2 NO2 NP NR1 NR2 NR3 NR55 NR56 NR66 NT NX OA OAC OC OE OH2 OK OM OS OSH OSI OT OW PO3 PO4 PT PUA1 PUY1 SE SH1E SK SO1 SO2 SO3 SO4 ST ST2 ST2  br br- br1 cl cl- cl1 cl12 cl13 cl14 cl1p ca+ cu+2 fe+2 mg+2 zn+2 cs+ li+ na+ rb+ al4z si si4 si4c si4z ar he kr ne xe  dw hi hw ca cg ci co coh cp cr cs ct ct3 na nb nh nho ni no np nt nt2 nz oa oc oh op os ot sp bt cl\' si4l si5l si5t si6 si6o si\'  br ca cc cd ce cf cl cp cq cu cv cx cy ha hc hn ho hp hs na nb nc nd nh oh os pb pc pd pe pf px py sh ss sx sy  hn2 ho2 cz oo oz si sio hsi osi ",
+"twoChar", " al al4z ar ba2+ beoh br br- br1 ca+ ca2+ ca2c cl cl' cl- cl1 cl12 cl13 cl14 cl1p cloh cly- co2c cs+ cu+2 cu2+ fe+2 fe2c ga ge he kr li+ lic+ lioh lp LP mg+2 mg2+ mg2c mn3c mn4c na+ nac+ naoh ne ni2c nu pd2+ rb+ si si' si4 si4c si4l si4z si5l si5t si6 si6o sio sr2c ti4c tioc titd xe zn+2 ",
+"specialTypes", " IM IP sz az sy ay ayt ",
+"secondCharOnly", " AH BH AC BC ");
+});