polyguard-rl/docs/submission_artifacts.md

Submission Artifact Index

This page points reviewers to the shared environment, training scripts, and training logs/results. It is intentionally path-based so the artifacts can be found from a fresh clone without relying on local outputs/ or checkpoints/ folders.

Environment And Runtime

Core OpenEnv/runtime files:

• openenv.yaml - OpenEnv package entrypoint and deployment metadata.
• server/app.py - ASGI/FastAPI bridge used by OpenEnv validation and Space deployment.
• app/env/env_core.py - canonical PolyGuardEnv reset/step/state implementation.
• app/env/fastapi_app.py - HTTP API, catalog, reset, step, and candidate-step routes.
• app/env/reward_router.py - verifier-backed reward routing.
• app/env/reward_scaling.py - reward clamping/rounding to [0.001, 0.999].
• app/env/anti_cheat.py - anti-hacking and invalid-action checks.
• app/env/catalog.py - task preset and sub-environment catalog.

Dependency and container files:

• pyproject.toml and uv.lock - local Python environment lock.
• requirements.txt - local/runtime pip dependency export.
• requirements-space.txt - Hugging Face Space dependency export.
• .env.example - non-secret environment variable template.
• Dockerfile - local/container runtime.
• Dockerfile.space - product HF Space runtime.
• app/hf_space/Dockerfile - HF training/evidence Space runtime.
• configs/sft.yaml and configs/grpo.yaml - train-loop defaults.
• configs/rewards.yaml, configs/curriculum.yaml, and configs/env_*.yaml - environment/reward/curriculum configuration.

Secrets are not committed. Hugging Face access is supplied through HF_TOKEN as an environment variable or notebook/Space secret.

Training Scripts And Notebooks

End-to-end runner notebooks:

• PolyGuard_SFT_GRPO_One_Run_Runner.ipynb - one-run data build, SFT, GRPO, artifact pull, inference validation, chart generation, and Space deployment.
• notebooks/09_training_loop.ipynb - modular walkthrough of the same loop.

Dataset and corpus scripts:

• scripts/bootstrap_data.py
• scripts/build_training_corpus.py
• scripts/generate_sft_data.py

SFT/GRPO training scripts:

• scripts/train_sft_trl.py - TRL SFT baseline.
• scripts/train_grpo_trl.py - TRL GRPO with environment-backed reward.
• scripts/train_grpo_policy.py
• scripts/train_grpo_planner.py
• scripts/train_grpo_supervisor.py
• scripts/train_grpo_dosing.py
• app/training/sft_trl.py
• app/training/grpo_trl.py
• app/training/openenv_wrapper.py
• app/training/reward_functions.py
• app/training/callbacks.py
• app/training/checkpointing.py

Hugging Face training/evidence scripts:

• scripts/deploy_training_space.py - creates/runs the GPU training Space.
• app/hf_space/training_runner.py - Space-side training orchestrator.
• scripts/monitor_training_space_status.py - Space status/log monitor.
• scripts/pull_training_artifacts.py - artifact puller from the HF model repo.
• scripts/deploy_evidence_space.py and app/hf_space/evidence_runner.py - evaluation-only evidence Space.
• scripts/generate_hf_training_report.py - training/sweep chart generation.
• scripts/generate_submission_evidence.py - evidence bundle generation without retraining.
• scripts/deploy_final_artifact_space.py - packages final public evidence/model artifacts into the final HF Space.

Post-training and inference scripts:

• scripts/merge_adapters_safe.py
• scripts/test_inference_postsave.py
• scripts/benchmark_inference.py
• scripts/activate_sweep_model.py
• scripts/install_hf_active_bundle.py

Training Logs And Result Evidence

Final curated evidence:

• docs/results/final_submission_evidence/README.md - final evidence overview.
• docs/results/final_submission_evidence/manifest.json - artifact availability and final HF Space manifest.
• docs/results/final_submission_evidence/reports/submission_summary.json - final three-model summary.
• docs/results/final_submission_evidence/reports/grpo_trl_run.json - Qwen 3B GRPO training run report.
• docs/results/final_submission_evidence/reports/postsave_inference_grpo.json - post-save GRPO inference check.
• docs/results/final_submission_evidence/reports/grpo_ablation_report.json - GRPO/policy ablation report.
• docs/results/final_submission_evidence/reports/basic_llm_vs_polyguard_report.json - baseline LLM-style policy vs full PolyGuard pipeline.
• docs/results/final_submission_evidence/reports/action_traces.jsonl - matched action traces with verifier output.
• docs/results/final_submission_evidence/charts/curated/README.md - visually reviewed chart index.

Per-model sweep histories:

• docs/results/sweeps/qwen-qwen2-5-0-5b-instruct/sft_history.json
• docs/results/sweeps/qwen-qwen2-5-0-5b-instruct/sft_trl_run.json
• docs/results/sweeps/qwen-qwen2-5-1-5b-instruct/sft_history.json
• docs/results/sweeps/qwen-qwen2-5-1-5b-instruct/sft_trl_run.json
• docs/results/sweeps/qwen-qwen2-5-3b-instruct/sft_history.json
• docs/results/sweeps/qwen-qwen2-5-3b-instruct/sft_trl_run.json
• docs/results/sweeps/qwen-qwen2-5-3b-instruct/grpo_history.json
• docs/results/sweeps/qwen-qwen2-5-3b-instruct/grpo_trl_run.json

Three-model submission evidence:

• docs/results/submission_evidence_qwen_0_5b_1_5b_3b/reports/runs/qwen-qwen2-5-0-5b-instruct/sft_history.json
• docs/results/submission_evidence_qwen_0_5b_1_5b_3b/reports/runs/qwen-qwen2-5-1-5b-instruct/sft_history.json
• docs/results/submission_evidence_qwen_0_5b_1_5b_3b/reports/runs/qwen-qwen2-5-3b-instruct/sft_history.json
• docs/results/submission_evidence_qwen_0_5b_1_5b_3b/reports/runs/qwen-qwen2-5-3b-instruct/grpo_history.json
• docs/results/submission_evidence_qwen_0_5b_1_5b_3b/reports/runs/qwen-qwen2-5-3b-instruct/grpo_reward_components.jsonl
• docs/results/submission_evidence_qwen_0_5b_1_5b_3b/reports/remote_stage_records.json

Completed-run status snapshots:

• docs/results/qwen_completed_runs/reports/remote_status/live_hf_status_snapshot.json
• docs/results/qwen_completed_runs/reports/remote_status/qwen_0_5b_completed_commands.json
• docs/results/qwen_completed_runs/reports/remote_status/qwen_1_5b_completed_commands.json
• docs/results/qwen_completed_runs/reports/remote_status/qwen_0_5b_1_5b_remote_stage_durations.json
• docs/results/submission_evidence/qwen_3b_continuation/training_space_runtime_status.json

Legacy/local smoke logs are retained under docs/results/active_model/, docs/results/grpo_training_cycle/, and submission_bundle/ for auditability.

Model Artifacts

The public final artifact/evidence Space is:

• https://huggingface.co/spaces/adithya9903/polyguard-openenv-final-artifacts

The tracked local manifest is:

• docs/results/final_submission_evidence/manifest.json

At packaging time, Qwen 3B had SFT and GRPO adapter directories plus checkpoint metadata in the final Space. Qwen 0.5B and 1.5B have reports/histories in this repo, but their adapter directories were not present in the checked artifact mirrors and are labeled reports_only_or_partial.

The final artifact Space and this checked-in evidence mirror are the public review paths. Authenticated downloads, when needed by maintainers, are operational details rather than part of the public submission narrative.

Reproduction Paths

Local smoke path: build the small corpus, run a short SFT pass, run a short GRPO pass, validate post-save inference, and generate local reports.

Full HF Space path: use the one-run notebook or training Space runner when you control the required Hugging Face credentials and hardware. The public evidence for review is the final curated bundle, not private training commands.