160 intermediary checkpoints from the tr1-13B training these models have a bug in them. While we are fixing things if you try to use any of these please run it through this script: ``` python -c ' import sys, torch f=sys.argv[1] sd=torch.load(f) d=2048 for k in sd.keys(): if k.endswith(".attn.bias"): sd[k] = torch.tril(torch.ones((d, d), dtype=torch.float16)).view(1, 1, d, d) torch.save(sd, f) ' global_step594/pytorch_model.bin ```