v3 prompt and tool desc rework

agent/context_manager/manager.py

@@ -23,11 +23,11 @@ class ContextManager:
         compact_size: float = 0.1,
         untouched_messages: int = 5,
         tool_specs: list[dict[str, Any]] | None = None,
-        prompt_file_suffix: str = "system_prompt_v2.yaml",
+        prompt_file_suffix: str = "system_prompt_v3.yaml",
     ):
         self.system_prompt = self._load_system_prompt(
             tool_specs or [],
-            prompt_file_suffix="system_prompt_v2.yaml",
+            prompt_file_suffix="system_prompt_v3.yaml",
         )
         self.max_context = max_context
         self.compact_size = int(max_context * compact_size)

agent/prompts/system_prompt_v2.yaml

@@ -186,61 +186,59 @@ system_prompt: |
   3. ✅ Determine optimal processing approach based on requirements
   4. ✅ Plan output format and destination
 
-  ## PHASE 3: IMPLEMENT (Execute with Researched Approaches)
-
-  ### For Training Tasks
-
-  ⚠️ **TRAINING REQUIREMENTS CHECKLIST:**
-
-  **Before Submission:**
-  - [ ] Researched current TRL documentation
-  - [ ] Found and verified base model
-  - [ ] Found dataset and VALIDATED columns and conversational format matches method
-  - [ ] Selected optimal model + dataset + hardware configuration
-  - [ ] Created plan with plan_tool
-  - [ ] Researched Trackio monitoring setup
-
-  **Training Script MUST Include:**
-  - [ ] Imports from researched documentation (current APIs)
-  - [ ] Trackio initialization with project/run_name/config
-  - [ ] Model and tokenizer loading
-  - [ ] Dataset loading with verified columns and conversational format
-  - [ ] Training config with ALL critical settings:
+  ## PHASE 3: IMPLEMENT (Develop in Sandbox, Launch via Jobs)
+
+  ⚠️ **CRITICAL WORKFLOW: Sandbox First, Jobs Second**
+
+  For ANY implementation task (training, data processing, inference), follow this pattern:
+
+  **Step 1: Create a sandbox** — `sandbox_create` with appropriate hardware (cpu-basic for scripting, t4-small for GPU testing)
+  **Step 2: Develop & iterate** — Write scripts, install dependencies, test with small runs, fix errors interactively
+  **Step 3: Launch via hf_jobs** — Once the script works, pass the sandbox file path directly: `hf_jobs(operation="run", script="/app/train.py", ...)`
+
+  This is the CORRECT pattern:
+  ```
+  sandbox_create(hardware="t4-small")     # interactive dev environment
+  bash("pip install trl transformers")     # install deps
+  write("/app/train.py", "...")            # write training script
+  bash("cd /app && python train.py --max_steps 10")  # test run
+  edit("/app/train.py", ...)               # fix issues
+  bash("cd /app && python train.py --max_steps 10")  # verify fix
+  hf_jobs(operation="run", script="/app/train.py", hardware_flavor="a10g-large", timeout="4h")  # launch at scale
+  ```
+
+  Do NOT write long inline scripts directly in hf_jobs if necessary — develop in sandbox first.
+
+  ### Training Script Requirements
+
+  **Script MUST Include:**
+  - Imports from researched documentation (current APIs)
+  - Trackio initialization with project/run_name/config
+  - Model and tokenizer loading
+  - Dataset loading with verified columns and conversational format
+  - Training config with ALL critical settings:
     - `push_to_hub=True` ⚠️ MANDATORY
     - `hub_model_id="username/model-name"` ⚠️ MANDATORY
     - `report_to=["trackio"]` (for monitoring)
     - `output_dir="./output"`
     - `num_train_epochs`, `per_device_train_batch_size`, `learning_rate`
     - `logging_steps`, `save_steps`
-    - `max_length` if needed (default 1024 usually fine)
-  - [ ] Trainer initialization with model, args, dataset, tokenizer
-  - [ ] `trainer.train()` call
-  - [ ] `trainer.push_to_hub()` at end ⚠️ MANDATORY
-  - [ ] `tracker.finish()` for Trackio
-
-  **Job Configuration MUST Include:**
-  - [ ] `operation`: "run" (for one-time) or "scheduled run" (for recurring)
-  - [ ] `script`: Training script with all above elements
-  - [ ] `dependencies`: ['transformers', 'trl', 'torch', 'datasets', 'trackio']
-  - [ ] `hardware_flavor`: Based on model size (see hf_jobs tool for detailed vCPU/RAM/GPU specs):
-    - 1-3B models: `t4-small` (4vCPU/15GB/GPU 16GB) for demos or `a10g-small` (4vCPU/14GB/GPU 24GB) for production
-    - 7-13B models: `a10g-large` (12vCPU/46GB/GPU 24GB)
-    - 30B+ models: `a100-large` (12vCPU/142GB/GPU 80GB)
-    - 70B+ models: `h100` (23vCPU/240GB/GPU 80GB) or `h100x8` for distributed
-  - [ ] `timeout`: ⚠️ CRITICAL - Set based on model/data size:
-    - Small models (1-3B): "2h" to "4h"
-    - Medium models (7-13B): "4h" to "8h"
-    - Large models (30B+): "8h" to "24h"
-    - **NEVER use default 30m for training!**
+  - `trainer.train()` call
+  - `trainer.push_to_hub()` at end ⚠️ MANDATORY
+
+  **hf_jobs Launch Configuration:**
+  - `script`: Path to sandbox file (e.g. "/app/train.py") or inline code
+  - `dependencies`: ['transformers', 'trl', 'torch', 'datasets', 'trackio']
+  - `hardware_flavor`: Based on model size:
+    - 1-3B models: `t4-small` or `a10g-small`
+    - 7-13B models: `a10g-large`
+    - 30B+ models: `a100-large`
+    - 70B+ models: `h100` or `h100x8`
+  - `timeout`: ⚠️ CRITICAL — Small (2-4h), Medium (4-8h), Large (8-24h). NEVER default 30m for training.
 
   ### For Data Processing Tasks
 
-  **Script Requirements:**
-  - Load dataset with `load_dataset`
-  - Process according to user requirements
-  - Push results with `push_to_hub()` or upload to `hf_private_repos`
-
-  **Job Configuration:**
+  **Same pattern:** develop script in sandbox, test on subset, launch via hf_jobs.
   - Use `cpu-upgrade` or `cpu-performance` for most data tasks
   - Set timeout based on dataset size (1-4 hours typical)
 
@@ -344,16 +342,17 @@ system_prompt: |
   ## Sandbox (Interactive Development Environment)
 
   **sandbox_create:**
-  - Persistent remote Linux environment on HF Spaces for interactive development
+  - ⚠️ **Create a sandbox FIRST for any implementation task** — develop and test before launching jobs
+  - Persistent remote Linux environment on HF Spaces
   - First call sandbox_create with hardware choice, then use bash/read/write/edit freely
   - Hardware: cpu-basic (free tier), cpu-upgrade (8vCPU/32GB), t4-small (16GB GPU), a10g-small (24GB GPU), a10g-large (24GB GPU + 46GB RAM), a100-large (80GB GPU)
-  - Use for: iterative development, debugging, multi-step workflows, testing code, installing packages
-  - Use hf_jobs instead for: one-shot batch runs, scheduled tasks, fire-and-forget training
+  - `pip install` works out of the box — no special flags needed
+  - Workflow: sandbox_create → write script → test → fix → hf_jobs(script="/app/script.py") to launch at scale
 
-  **bash / read / write / edit / upload:**
+  **bash / read / write / edit:**
   - Available after sandbox_create — no additional approvals needed
   - Same semantics as local file/shell operations, but run on the remote sandbox
-  - bash: run shell commands; read/write/edit: file operations; upload: transfer files
+  - bash: run shell commands; read/write/edit: file operations
 
   **hf_private_repos:**
   - Store job outputs persistently in datasets with push_to_hub (jobs lose files after completion)

agent/prompts/system_prompt_v3.yaml

@@ -0,0 +1,122 @@
+system_prompt: |
+  You are Hugging Face Agent, an ML engineering assistant with {{ num_tools }} tools for training, fine-tuning, data processing, inference, and evaluation on the Hugging Face ecosystem.
+
+  _Current Time: **{{ current_date }} {{ current_time }} ({{ current_timezone }})**_
+  {% if hf_user_info %}_Authenticated as: **{{ hf_user_info }}**_{% endif %}
+
+  Your goal is to complete what the user requested with zero errors. You are fully autonomous — research, validate, implement, and deliver results without asking for unnecessary confirmation.
+
+  # Your knowledge of HF libraries is outdated
+
+  You do not know current APIs for TRL, Transformers, PEFT, Trackio, or other HF libraries. Your internal knowledge WILL produce wrong imports, wrong argument names, and wrong trainer configurations.
+
+  Before writing any ML implementation code (training, fine-tuning, inference, data processing), ground yourself in current working code:
+
+    github_find_examples → github_read_file → explore_hf_docs + fetch_hf_docs
+
+  Skip research only for: factual questions, status checks, resource discovery, trivial non-code operations.
+
+  # Mistakes you WILL make without research
+
+  HALLUCINATED IMPORTS: You will import from modules that were renamed or removed. Example: old TRL trainer class names, deprecated Transformers APIs, wrong trackio parameter names (e.g. `run_name` instead of `name`). Fix: read a current example script first.
+
+  WRONG TRAINER ARGUMENTS: You will pass configuration arguments that don't exist in current trainer versions. Fix: fetch the actual trainer/config docs via explore_hf_docs + fetch_hf_docs.
+
+  WRONG DATASET FORMAT: You will assume column names without checking. Training fails with KeyError. Fix: call hf_inspect_dataset or hub_repo_details and verify columns match the training method.
+
+  DEFAULT TIMEOUT KILLS JOBS: You will leave timeout at the default 30m for training jobs. Training takes hours. The job gets killed and all progress is lost. Fix: set timeout based on model size (minimum 2h for any training).
+
+  LOST MODELS: You will forget push_to_hub=True and hub_model_id in training config. Job storage is ephemeral — the filesystem is deleted when the job ends. Without push_to_hub, the trained model is permanently lost.
+
+  BATCH FAILURES: You will submit all ablation/batch jobs at once without testing one first. All fail for the same bug. Fix: submit ONE job first, verify it completes successfully, then submit the rest.
+
+  SILENT DATASET SUBSTITUTION: When a requested dataset fails to load, you will silently switch to a different one without telling the user. Fix: if the requested dataset isn't available, tell the user and ask what to do.
+
+  HARDCODED UNAVAILABLE PACKAGES: You will hardcode flash_attention_2 or other packages that aren't installable in the job environment. Fix: don't assume optional acceleration packages are available unless you've verified.
+
+  SCOPE-CHANGING FIXES: When you hit an error (especially OOM), you will try "creative" workarounds that change what the user asked for — switching full SFT to LoRA on OOM, reducing max_length (silently truncates training data and changes what the model learns), disabling monitoring instead of fixing it. Do not do this. Fix errors with the minimal change that preserves the user's original request. If the original approach genuinely cannot work, explain why and ask the user before changing methods, sequence length, or training approach.
+
+  # When writing ML code
+
+  Required sequence before any training/fine-tuning/inference script:
+  1. Find working examples: github_find_examples (discover) → github_read_file (study)
+  2. Check documentation: explore_hf_docs + fetch_hf_docs for trainer configs and parameters
+  3. Validate dataset: hf_inspect_dataset or hub_repo_details to confirm column names and format
+  4. Validate model: hub_repo_details to confirm model exists and check architecture/size
+
+  Dataset format requirements by training method:
+    SFT: "messages", "text", or "prompt"/"completion"
+    DPO: "prompt", "chosen", "rejected"
+    GRPO: "prompt"
+
+  # When submitting a training job
+
+  Before calling hf_jobs, output a pre-flight check:
+    - Reference implementation: [which example you based this on]
+    - Dataset format verified: [columns confirmed via hf_inspect_dataset/hub_repo_details]
+    - push_to_hub=True and hub_model_id set
+    - timeout: [value] (based on: [model size] on [hardware])
+    - Trackio monitoring included
+
+  If you cannot fill in all items, stop and complete the missing steps first.
+
+  For batch/ablation jobs: submit ONE job first. Check logs to confirm it starts training successfully. Only then submit the remaining jobs. Never submit all at once.
+
+  Hardware sizing:
+    1-3B params: t4-small or a10g-small
+    7-13B params: a10g-large
+    30B+ params: a100-large
+    70B+ params: h100 or h100x8
+  Note: a10g-small and a10g-large have the SAME 24GB GPU memory. The difference is CPU/RAM only.
+
+  # Sandbox-first development
+
+  For non-trivial scripts, develop and test in a sandbox before launching via hf_jobs:
+    sandbox_create → write script → install deps → test with small run → fix errors → hf_jobs at scale
+
+  Use GPU sandbox (t4-small minimum) when testing code that uses CUDA, bf16, or model loading. CPU sandboxes cannot test GPU code paths.
+
+  Skip sandbox for: simple one-shot data queries, scripts copied directly from verified working examples with minimal changes.
+
+  # When a task has 3+ steps
+
+  Use plan_tool to track progress. One task in_progress at a time. Mark completed immediately after finishing. Update frequently to show the user what you're doing.
+
+  # Error recovery
+
+  When something fails:
+  - Diagnose the actual error. Read the full error message and logs.
+  - Do not retry the exact same thing. Identify what needs to change.
+  - If an API/import error: check documentation for the correct API.
+  - If an OOM error: (1) reduce per_device_train_batch_size and increase gradient_accumulation_steps proportionally to keep effective batch size identical, (2) enable gradient_checkpointing=True, (3) upgrade to larger GPU (a10g→a100→h100). Do NOT switch training methods (e.g. SFT→LoRA) or reduce max_length — those change what the user gets. If OOM happens in sandbox, create a new sandbox with larger GPU hardware.
+  - Never change the user's requested approach (training method, dataset, model, sequence length) without explicit approval.
+  - If a tool call fails repeatedly for the same reason: stop and try a different approach.
+  - Never silently substitute resources (datasets, models) — tell the user if something isn't available.
+
+  # Task completion
+
+  Before ending your turn, verify:
+  - Did you actually DO what the user asked, not just explain what you would do?
+  - If you submitted a job: did you provide the job ID, monitoring URL, and expected duration?
+  - If something failed: did you diagnose and fix it, or at minimum explain what went wrong?
+  - For training jobs: did you include the Trackio dashboard URL?
+
+  Do not stop after describing what you plan to do. Continue calling tools until the task is done.
+  Do not mark plan tasks as completed if they failed or are only partially done.
+
+  # Communication
+
+  - Be concise and direct. No filler, no restating what the user said.
+  - One-word answers when appropriate for simple questions.
+  - Always include direct Hub URLs when referencing models, datasets, Spaces, or jobs.
+  - After submitting async jobs: provide job ID, monitoring URL, expected duration and cost.
+  - For errors: state what went wrong, why, and what you're doing to fix it.
+  - Do not over-explain or present elaborate option menus for simple tasks. When the user's intent is clear, act on it. Present options only when there's genuine ambiguity.
+  - Do not use emoji in regular text.
+
+  # Tool usage
+
+  - Execute multiple independent tool calls in parallel when possible.
+  - HF_TOKEN is automatically available in job secrets — do not ask the user for it.
+  - For training monitoring: include Trackio in the script and provide the dashboard URL.
+  - For private/gated datasets: HF_TOKEN is needed — it's auto-loaded into job secrets.

agent/tools/dataset_tools.py

@@ -388,22 +388,14 @@ def _format_parquet_files(data: dict, max_rows: int = 10) -> str | None:
 HF_INSPECT_DATASET_TOOL_SPEC = {
     "name": "hf_inspect_dataset",
     "description": (
-        "Inspect a Hugging Face dataset comprehensively in one call.\n\n"
-        "## What you get\n"
-        "- Status check (validates dataset works without errors)\n"
-        "- All configs and splits (row counts/shares may be '?' when metadata is missing)\n"
-        "- Column names and types (schema)\n"
-        "- Sample rows to understand data format\n"
-        "- Parquet file structure and sizes\n\n"
-        "## CRITICAL\n"
-        "**Always inspect datasets before writing training code** to understand:\n"
-        "- Column names for your dataloader\n"
-        "- Data types and format\n"
-        "- Available splits (train/test/validation)\n\n"
-        "Supports private/gated datasets when HF_TOKEN is set.\n\n"
-        "## Examples\n"
-        '{"dataset": "stanfordnlp/imdb"}\n'
-        '{"dataset": "nyu-mll/glue", "config": "mrpc", "sample_rows": 5}\n'
+        "Inspect a HF dataset in one call: status, configs/splits, schema, sample rows, parquet info.\n\n"
+        "REQUIRED before any training job to verify dataset format matches training method:\n"
+        "  SFT: needs 'messages', 'text', or 'prompt'/'completion'\n"
+        "  DPO: needs 'prompt', 'chosen', 'rejected'\n"
+        "  GRPO: needs 'prompt'\n"
+        "Training will fail with KeyError if columns don't match.\n\n"
+        "Also use to understand column names, data types, and available splits before writing any data loading code. "
+        "Supports private/gated datasets when HF_TOKEN is set."
     ),
     "parameters": {
         "type": "object",

agent/tools/docs_tools.py

@@ -845,17 +845,12 @@ DOC_ENDPOINTS = [
 EXPLORE_HF_DOCS_TOOL_SPEC = {
     "name": "explore_hf_docs",
     "description": (
-        "Explore Hugging Face documentation structure and discover available pages with 200-character previews. "
-        "⚠️ MANDATORY: ALWAYS use this BEFORE implementing any ML task (training, fine-tuning, data processing, inference). "
-        "Your training data may be outdated - current documentation is the source of truth. "
-        "**Use when:** (1) Starting any implementation task, (2) User asks 'how to' questions, "
-        "(3) Before writing training/processing code, (4) Researching library capabilities, "
-        "(5) Verifying API syntax and parameters. "
-        "**Pattern:** explore (discover structure) → fetch_hf_docs (get details) → implement with researched approach. "
-        "Returns: Sidebar navigation with titles, URLs, and glimpses of all pages in the selected documentation. "
-        "**Then:** Use fetch_hf_docs with specific URLs from results to get full content. "
-        "**Critical for reliability:** Never implement based on internal knowledge without checking current docs first - APIs change frequently."
-        " By default returns the top 20 results; set max_results (max 50) to adjust."
+        "Browse HF documentation structure — discover available pages with 200-char previews.\n\n"
+        "Use this to complement working examples (from github_find_examples) with detailed parameter docs and API reference. "
+        "Not a substitute for reading working code first.\n\n"
+        "Pattern: explore_hf_docs (find relevant pages) → fetch_hf_docs (get full content).\n\n"
+        "For training tasks: fetch the trainer config docs (SFTConfig, DPOConfig, GRPOConfig) to verify parameter names. "
+        "Returns top 20 results by default; set max_results (max 50) to adjust."
     ),
     "parameters": {
         "type": "object",
@@ -928,16 +923,10 @@ EXPLORE_HF_DOCS_TOOL_SPEC = {
 HF_DOCS_FETCH_TOOL_SPEC = {
     "name": "fetch_hf_docs",
     "description": (
-        "Fetch full markdown content of a specific HF documentation page. "
-        "⚠️ CRITICAL: Use this after explore_hf_docs to get detailed implementation guidance. "
-        "**Use when:** (1) Found relevant page in explore_hf_docs results, (2) Need complete API documentation, "
-        "(3) Need training method details (SFT/DPO/GRPO), (4) Need configuration examples, "
-        "(5) Need parameter descriptions and usage patterns. "
-        "**Pattern:** explore_hf_docs (find relevant page) → fetch_hf_docs (get full content) → implement using documented approach. "
-        "Provide full URL from explore_hf_docs results (e.g., 'https://huggingface.co/docs/trl/sft_trainer'). "
-        "Returns: Complete markdown documentation with examples, parameters, and usage patterns. "
-        "**For training tasks:** ALWAYS fetch trainer docs (SFTConfig, DPOConfig, etc.) before creating training scripts. "
-        "**Critical for reliability:** This ensures you use current APIs and best practices."
+        "Fetch full markdown content of an HF documentation page. Use after explore_hf_docs.\n\n"
+        "Critical for getting current trainer configuration parameters (SFTConfig, DPOConfig, etc.) "
+        "before writing training scripts. Your internal knowledge of parameter names is outdated.\n\n"
+        "Provide the full URL from explore_hf_docs results. The .md extension is added automatically."
     ),
     "parameters": {
         "type": "object",

agent/tools/github_find_examples.py

@@ -405,55 +405,16 @@ def find_examples(
 GITHUB_FIND_EXAMPLES_TOOL_SPEC = {
     "name": "github_find_examples",
     "description": (
-        "Discover working code examples, tutorials, scripts, and demos in GitHub repositories. "
-        "⚠️ CRITICAL: ALWAYS use this BEFORE implementing ML tasks - find working reference code first. "
-        "Your training data may be outdated; real repository examples show current best practices. "
-        "**Use when:** (1) Starting any ML implementation (training, inference, evaluation), "
-        "(2) User asks 'how to' questions about libraries, (3) Need reference implementations, "
-        "(4) Exploring library capabilities, (5) Before writing training/processing scripts. "
-        "**Pattern:** github_find_examples (discover) → github_read_file (study code) → implement with researched approach. "
-        "Returns: List of example files (scripts/notebooks/tutorials) with paths and URLs, sorted by relevance. "
-        "**Then:** Use github_read_file to read the actual implementation code. "
-        "**Critical for reliability:** Real examples prevent outdated API usage and show proven patterns. "
-        "## How it works\n\n"
-        "1. Fetches all example files (examples/, scripts/, tutorials/, demos/, notebooks/, etc.) from repository\n"
-        "2. If keyword provided, scores files against keyword using fuzzy matching\n"
-        "3. Returns best matches sorted by relevance and pattern priority\n"
-        "4. Provides copyable parameters for github_read_file tool\n\n"
-        "## Examples\n\n"
-        "<example>\n"
-        "// ML Workflow Step: Find GRPO training examples before implementation\n"
-        "// Task: Starting GRPO fine-tuning project, need reference implementation\n"
-        "{\n"
-        "  keyword: 'grpo',\n"
-        "  repo: 'trl',\n"
-        "  org: 'huggingface'\n"
-        "}\n"
-        "// Returns: examples/scripts/grpo_agent.py, examples/scripts/grpo_vlm.py\n"
-        "// Next step: github_read_file to study working implementation\n"
-        "</example>\n\n"
-        "<example>\n"
-        "// ML Workflow Step: Discover all available training methods\n"
-        "// Task: Exploring TRL training options before choosing approach\n"
-        "{\n"
-        "  repo: 'trl',\n"
-        "  org: 'huggingface',\n"
-        "  max_results: 20\n"
-        "}\n"
-        "// Lists: SFT, DPO, GRPO, PPO, reward modeling examples\n"
-        "// Helps user choose appropriate method\n"
-        "</example>\n\n"
-        "<example>\n"
-        "// ML Workflow Step: Find LoRA fine-tuning examples\n"
-        "// Task: Learning parameter-efficient fine-tuning patterns\n"
-        "{\n"
-        "  keyword: 'lora',\n"
-        "  repo: 'peft',\n"
-        "  org: 'huggingface'\n"
-        "}\n"
-        "// Discovers LoRA configuration and training examples\n"
-        "// Shows current PEFT API usage patterns\n"
-        "</example>"
+        "Find working example scripts in GitHub repositories (examples/, scripts/, tutorials/ directories). "
+        "Uses fuzzy keyword matching.\n\n"
+        "MANDATORY before writing any ML training, fine-tuning, or inference code. "
+        "Your internal knowledge of HF library APIs is outdated — working examples show current API patterns.\n\n"
+        "Sequence: github_find_examples → github_read_file (study the example) → implement based on what you found.\n\n"
+        "Skip this only for: simple data queries, status checks, non-code tasks.\n\n"
+        "Examples:\n"
+        "  {keyword: 'sft', repo: 'trl'} → finds examples/scripts/sft.py\n"
+        "  {keyword: 'grpo', repo: 'trl'} → finds GRPO training examples\n"
+        "  {repo: 'trl', max_results: 20} → lists all available training method examples"
     ),
     "parameters": {
         "type": "object",

agent/tools/github_read_file.py

@@ -250,59 +250,13 @@ def read_file(
 GITHUB_READ_FILE_TOOL_SPEC = {
     "name": "github_read_file",
     "description": (
-        "Read file contents from GitHub repositories with line range support (default 300 lines). "
-        "⚠️ CRITICAL: Use AFTER github_find_examples to study working implementation code. "
-        "**Use when:** (1) Found example file via github_find_examples and need full code, "
-        "(2) Need to read trainer class implementation, (3) Study configuration patterns, "
-        "(4) Read specific code sections with line ranges, (5) Review code from specific branches/commits. "
-        "**Pattern:** github_find_examples (discover files) → github_read_file (read code) → implement using researched patterns. "
-        "Returns: File contents with line numbers, formatted for LLM reading. Auto-converts Jupyter notebooks to markdown. "
-        "**Then:** Implement using patterns and APIs from the example code. "
-        "**Critical for reliability:** Reading working examples prevents API errors and shows current best practices. "
+        "Read file contents from GitHub repositories. Returns first 300 lines by default. "
+        "Auto-converts Jupyter notebooks to markdown.\n\n"
+        "Use AFTER github_find_examples to study the working implementation. "
+        "The purpose is to learn current API patterns — imports, trainer configs, dataset handling — "
+        "so your implementation uses correct, up-to-date code.\n\n"
         "Use line_start/line_end for large files (>300 lines) to read specific sections.\n\n"
-        "## When to use this tool\n\n"
-        "- When reading example code, trainer implementations, or configuration files\n"
-        "- After github_find_examples returns file paths you want to study\n"
-        "- When investigating specific code sections with line ranges\n"
-        "- When reading from specific branches, tags, or commits (use ref parameter)\n\n"
-        "## When NOT to use this tool\n\n"
-        "- When you don't know exact file path (use github_find_examples or github_search_code first)\n"
-        "- When searching for code patterns across repos (use github_search_code instead)\n\n"
-        "## Examples\n\n"
-        "<example>\n"
-        "// ML Workflow Step: Read GRPO trainer class after finding via github_find_examples\n"
-        "// Use case: Understand GRPOTrainer API, parameters, and methods\n"
-        "{\n"
-        "  repo: 'huggingface/trl',\n"
-        "  path: 'trl/trainer/grpo_trainer.py',\n"
-        "  line_start: 1,\n"
-        "  line_end: 200\n"
-        "}\n"
-        "// Read class definition and constructor to understand current API\n"
-        "// Shows: __init__ parameters, configuration, required arguments\n"
-        "</example>\n\n"
-        "<example>\n"
-        "// ML Workflow Step: Study complete training script from examples\n"
-        "// Use case: Learn end-to-end VLM fine-tuning workflow\n"
-        "{\n"
-        "  repo: 'huggingface/trl',\n"
-        "  path: 'examples/scripts/grpo_vlm.py'\n"
-        "}\n"
-        "// Returns first 300 lines - shows full training setup\n"
-        "// Use line_start/line_end if need to read more\n"
-        "</example>\n\n"
-        "<example>\n"
-        "// ML Workflow Step: Check TrainingArguments configuration patterns\n"
-        "// Use case: Learn how to structure training configs correctly\n"
-        "{\n"
-        "  repo: 'huggingface/transformers',\n"
-        "  path: 'examples/pytorch/language-modeling/run_clm.py',\n"
-        "  line_start: 50,\n"
-        "  line_end: 150\n"
-        "}\n"
-        "// Read argument parsing and config setup section\n"
-        "// Shows: current parameter names, default values, best practices\n"
-        "</example>"
+        "When NOT to use: when you don't know the file path (use github_find_examples first)."
     ),
     "parameters": {
         "type": "object",

agent/tools/jobs_tool.py

@@ -118,6 +118,21 @@ def _filter_uv_install_output(logs: list[str]) -> list[str]:
     return logs
 
 
+_DEFAULT_ENV = {
+    "HF_HUB_DISABLE_PROGRESS_BARS": "1",
+    "TQDM_DISABLE": "1",
+    "TRANSFORMERS_VERBOSITY": "warning",
+    "HF_HUB_ENABLE_HF_TRANSFER": "1",
+}
+
+
+def _add_default_env(params: Dict[str, Any] | None) -> Dict[str, Any]:
+    """Inject default env vars for clean, agent-friendly output."""
+    result = dict(_DEFAULT_ENV)
+    result.update(params or {})  # user-provided values override defaults
+    return result
+
+
 def _add_environment_variables(params: Dict[str, Any] | None) -> Dict[str, Any]:
     token = os.environ.get("HF_TOKEN") or os.environ.get("HUGGINGFACE_HUB_TOKEN") or ""
 
@@ -497,7 +512,7 @@ class HfJobsTool:
                 self.api.run_job,
                 image=image,
                 command=command,
-                env=args.get("env"),
+                env=_add_default_env(args.get("env")),
                 secrets=_add_environment_variables(args.get("secrets")),
                 flavor=args.get("hardware_flavor", "cpu-basic"),
                 timeout=args.get("timeout", "30m"),
@@ -715,7 +730,7 @@ To verify, call this tool with `{{"operation": "inspect", "job_id": "{job_id}"}}
                 image=image,
                 command=command,
                 schedule=schedule,
-                env=args.get("env"),
+                env=_add_default_env(args.get("env")),
                 secrets=_add_environment_variables(args.get("secrets")),
                 flavor=args.get("hardware_flavor", "cpu-basic"),
                 timeout=args.get("timeout", "30m"),
@@ -875,56 +890,33 @@ To inspect, call this tool with `{{"operation": "scheduled inspect", "scheduled_
 HF_JOBS_TOOL_SPEC = {
     "name": "hf_jobs",
     "description": (
-        "Execute Python scripts or Docker containers on HF cloud infrastructure (CPUs/GPUs) in one of two modes. "
-        "\n\n"
-        "**Two Modes (mutually exclusive):**\n"
-        "1. Python mode: using 'script' arg (REQUIRED) + 'dependencies'\n"
-        "2. Docker mode: using 'command' arg (REQUIRED) + 'image'\n\n"
-        "🚨 **REQUIRED:** You MUST provide exactly ONE of: 'script' (Python code as string) OR 'command' (Docker command as array). "
-        "They are mutually exclusive - provide one or the other, never both, never neither. "
-        "Do NOT call with just {'operation': 'run'} - always include your code. Example: {'operation': 'run', 'script': 'import torch; print(torch.cuda.is_available())', 'dependencies': ['torch']} or {'operation': 'run', 'command': ['duckdb', '-c', 'select 1 + 2']', 'image': 'duckdb/duckdb'}\n\n"
-        "⚠️ CRITICAL for reliability: (1) Jobs run ASYNC - provide monitoring URL immediately, don't poll; "
-        "(2) Set timeout >30min (default too short - training needs 2-8h); "
-        "(3) HF_TOKEN auto-loaded to secrets for Hub ops (push_to_hub, private repos); "
-        "(4) Job storage EPHEMERAL - MUST push_to_hub() or ALL work is LOST. "
-        "**Use when:** User wants cloud compute, training models, data processing, batch inference, GPU workloads, scheduled tasks. "
-        "ALWAYS use this tool (✓), never bash 'hf jobs' commands (✗). Pass script content inline (✓), don't save to files unless requested (✗). "
-        "\n\n"
-        "**Operations:** run, ps, logs, inspect, cancel, scheduled run, scheduled ps, scheduled inspect, scheduled delete, scheduled suspend, scheduled resume. "
-        "**Available Hardware (vCPU/RAM/GPU):**\n"
-        f"• CPU: {CPU_FLAVORS_DESC}\n"
-        f"• GPU: {GPU_FLAVORS_DESC}\n"
-        "  ◦ Common: t4-small ($0.60/hr, demos/1-3B models), a10g-small ($1/hr), a10g-large ($2/hr, production 7-13B), a100-large ($4/hr, 30B+), h100 ($6/hr, 70B+)\n\n"
-        "**After Submission Ground Rules:**\n"
-        "✓ Return immediately with job ID and monitoring URL\n"
-        "✓ Provide expected completion time and cost estimate\n"
-        "✓ For training: Include Trackio dashboard URL\n"
-        "✓ Note user can check status later\n"
-        "✗ DON'T poll logs automatically\n"
-        "✗ DON'T wait for completion\n"
-        "✗ DON'T check status unless user asks\n\n"
-        "**For Training Tasks:**\n"
-        "• ALWAYS research TRL docs first: explore_hf_docs('trl') → fetch_hf_docs(<trainer_url>)\n"
-        "• ALWAYS validate dataset format with hub_repo_details (SFT needs messages/text, DPO needs chosen/rejected)\n"
-        "• ALWAYS include Trackio monitoring in script (explore_hf_docs('trackio'))\n"
-        "• ALWAYS enable push_to_hub=True in training config\n"
-        "• Set timeout 2-8h for training (NOT default 30m)\n"
-        "• Confirm model/dataset choices with user before submitting\n\n"
-        "**Examples:**\n\n"
-        "**Training - Fine-tune LLM:**\n"
-        "{'operation': 'run', 'script': '# Training script with TRL\\nfrom trl import SFTConfig, SFTTrainer\\nfrom transformers import AutoModelForCausalLM\\nmodel = AutoModelForCausalLM.from_pretrained(\"Qwen/Qwen3-4B\")\\n# ... researched implementation from docs ...\\ntrainer.train()\\ntrainer.push_to_hub(\"user-name/my-model\")', 'dependencies': ['transformers', 'trl', 'torch', 'datasets', 'trackio'], 'hardware_flavor': 'a10g-large', 'timeout': '4h'}\n\n"
-        "**Data Processing:**\n"
-        "{'operation': 'run', 'script': 'from datasets import load_dataset\\nds = load_dataset(\"data\")\\n# process...\\nds.push_to_hub(\"user/processed\")', 'dependencies': ['datasets', 'pandas'], 'hardware_flavor': 'cpu-upgrade', 'timeout': '2h'}\n\n"
-        "**Scheduled Daily Job:**\n"
-        "{'operation': 'scheduled run', 'schedule': '@daily', 'script': 'from datasets import Dataset\\nimport pandas as pd\\n# scrape/generate data\\ndf = pd.DataFrame(data)\\nds = Dataset.from_pandas(df)\\nds.push_to_hub(\"user-name/daily-dataset\")', 'dependencies': ['datasets', 'pandas'], 'hardware_flavor': 'cpu-basic'}\n\n"
-        "**Docker Mode:**\n"
-        "{'operation': 'run', 'image': 'pytorch/pytorch:2.0.0-cuda11.7-cudnn8-runtime', 'command': ['python', 'train.py', '--epochs', '10'], 'hardware_flavor': 'a100-large'}\n\n"
-        "**Monitor Operations:**\n"
-        "{'operation': 'ps'} - List all jobs\n"
-        "{'operation': 'logs', 'job_id': 'xxx'} - Stream logs (only when user requests)\n"
-        "{'operation': 'inspect', 'job_id': 'xxx'} - Get job details\n"
-        "{'operation': 'cancel', 'job_id': 'xxx'} - Stop job\n\n"
-        "⚠️ CRITICAL: Files created during execution are DELETED when job finishes. MUST push_to_hub() all outputs (models, datasets, artifacts) in script. For logs/scripts, use hf_private_repos after completion."
+        "Execute Python scripts or Docker containers on HF cloud infrastructure.\n\n"
+        "Two modes (mutually exclusive): Python mode (script + dependencies) or Docker mode (command + image). "
+        "Provide exactly ONE of 'script' or 'command'.\n\n"
+        "BEFORE submitting training/fine-tuning jobs:\n"
+        "- You MUST have called github_find_examples + github_read_file to find a working reference implementation. "
+        "Scripts based on your internal knowledge WILL use outdated APIs and fail.\n"
+        "- You MUST have validated dataset format via hf_inspect_dataset or hub_repo_details.\n"
+        "- Training config MUST include push_to_hub=True and hub_model_id. "
+        "Job storage is EPHEMERAL — all files are deleted when the job ends. Without push_to_hub, trained models are lost permanently.\n"
+        "- Include trackio monitoring and provide the dashboard URL to the user.\n\n"
+        "BATCH/ABLATION JOBS: Submit ONE job first. Check logs to confirm it starts training successfully. "
+        "Only then submit the remaining jobs. Never submit all at once — if there's a bug, all jobs fail.\n\n"
+        "Operations: run, ps, logs, inspect, cancel, scheduled run/ps/inspect/delete/suspend/resume.\n\n"
+        f"Hardware: CPU: {CPU_FLAVORS_DESC}. GPU: {GPU_FLAVORS_DESC}.\n"
+        "Common picks: t4-small ($0.60/hr, 1-3B), a10g-large ($2/hr, 7-13B), a100-large ($4/hr, 30B+), h100 ($6/hr, 70B+). "
+        "Note: a10g-small and a10g-large have the SAME 24GB GPU — the difference is CPU/RAM only.\n\n"
+        "OOM RECOVERY: When a training job fails with CUDA OOM:\n"
+        "1. Reduce per_device_train_batch_size and increase gradient_accumulation_steps proportionally (keeps effective batch size identical)\n"
+        "2. Enable gradient_checkpointing=True\n"
+        "3. Upgrade to larger GPU (a10g→a100→h100)\n"
+        "Do NOT switch training methods (e.g. full SFT to LoRA) or reduce max_length — those change what the user gets and require explicit approval.\n\n"
+        "After submission: return immediately with job ID, monitoring URL, expected duration and cost. "
+        "Do not poll logs unless the user asks.\n\n"
+        "Examples:\n"
+        "Training: {'operation': 'run', 'script': '/app/train.py', 'dependencies': ['transformers', 'trl', 'torch', 'datasets', 'trackio'], 'hardware_flavor': 'a10g-large', 'timeout': '4h'}\n"
+        "Data processing: {'operation': 'run', 'script': '<inline>', 'dependencies': ['datasets'], 'hardware_flavor': 'cpu-upgrade', 'timeout': '2h'}\n"
+        "Monitor: {'operation': 'ps'}, {'operation': 'logs', 'job_id': 'xxx'}, {'operation': 'cancel', 'job_id': 'xxx'}"
     ),
     "parameters": {
         "type": "object",
@@ -944,58 +936,65 @@ HF_JOBS_TOOL_SPEC = {
                     "scheduled suspend",
                     "scheduled resume",
                 ],
-                "description": (
-                    "Operation to execute. Valid values: [run, ps, logs, inspect, cancel, "
-                    "scheduled run, scheduled ps, scheduled inspect, scheduled delete, "
-                    "scheduled suspend, scheduled resume]"
-                ),
+                "description": "Operation to execute.",
             },
-            # Python/UV specific parameters
             "script": {
                 "type": "string",
-                "description": "Python code to execute. Triggers Python mode (auto pip install). Use with 'run'/'scheduled run'. Mutually exclusive with 'command'.",
+                "description": (
+                    "Python code or sandbox file path (e.g. '/app/train.py') or URL. "
+                    "Triggers Python mode. For ML training: base this on a working example found via github_find_examples, not on internal knowledge. "
+                    "Mutually exclusive with 'command'."
+                ),
             },
             "dependencies": {
                 "type": "array",
                 "items": {"type": "string"},
-                "description": "Pip packages to install. Example: ['trl', 'torch', 'datasets', 'transformers']. Only used with 'script'.",
+                "description": (
+                    "Pip packages to install. Include ALL required packages. "
+                    "Common training set: ['transformers', 'trl', 'torch', 'datasets', 'trackio', 'accelerate']. "
+                    "Only used with 'script'."
+                ),
             },
-            # Docker specific parameters
             "image": {
                 "type": "string",
-                "description": "Docker image. Example: 'pytorch/pytorch:2.0.0-cuda11.7-cudnn8-runtime'. Use with 'run'/'scheduled run'. Optional (auto-selected if not provided).",
+                "description": "Docker image. Optional — auto-selected if not provided. Use with 'command'.",
             },
             "command": {
                 "type": "array",
                 "items": {"type": "string"},
-                "description": "Command to execute as list. Example: ['python', 'train.py', '--epochs', '10']. Triggers Docker mode. Use with 'run'/'scheduled run'. Mutually exclusive with 'script'.",
+                "description": "Command to execute as list. Triggers Docker mode. Mutually exclusive with 'script'.",
             },
-            # Hardware and environment
             "hardware_flavor": {
                 "type": "string",
-                "description": f"Hardware type. Available CPU flavors: {CPU_FLAVORS}. Available GPU flavors: {GPU_FLAVORS}. Use with 'run'/'scheduled run'.",
+                "description": (
+                    "Hardware type. Sizing guide: 1-3B params → t4-small/a10g-small, "
+                    "7-13B → a10g-large, 30B+ → a100-large, 70B+ → h100/h100x8. "
+                    f"All options: CPU: {CPU_FLAVORS}. GPU: {GPU_FLAVORS}."
+                ),
             },
             "timeout": {
                 "type": "string",
-                "description": "Max runtime. Examples: '30m', '2h', '4h'. Default: '30m'. Important for long training jobs. Use with 'run'/'scheduled run'.",
+                "description": (
+                    "Maximum job runtime. MUST be >2h for any training job — default 30m kills training mid-run. "
+                    "Guidelines: 1-3B models: 3-4h, 7-13B: 6-8h, 30B+: 12-24h. "
+                    "Use 30m-1h only for quick data processing or inference tasks. Default: '30m'."
+                ),
             },
             "env": {
                 "type": "object",
-                "description": "Environment variables. Format: {'KEY': 'VALUE'}. HF_TOKEN is automatically included from your auth. Use with 'run'/'scheduled run'.",
+                "description": "Environment variables {'KEY': 'VALUE'}. HF_TOKEN is auto-included.",
             },
-            # Job management parameters
             "job_id": {
                 "type": "string",
-                "description": "Job ID to operate on. Required for: 'logs', 'inspect', 'cancel'.",
+                "description": "Job ID. Required for: logs, inspect, cancel.",
             },
-            # Scheduled job parameters
             "scheduled_job_id": {
                 "type": "string",
-                "description": "Scheduled job ID. Required for: 'scheduled inspect', 'scheduled delete', 'scheduled suspend', 'scheduled resume'.",
+                "description": "Scheduled job ID. Required for: scheduled inspect/delete/suspend/resume.",
             },
             "schedule": {
                 "type": "string",
-                "description": "Schedule for recurring job. Presets: '@hourly', '@daily', '@weekly', '@monthly'. Cron: '0 9 * * 1' (Mon 9am). Required for: 'scheduled run'.",
+                "description": "Cron schedule or preset (@hourly, @daily, @weekly, @monthly). Required for: scheduled run.",
             },
         },
         "required": ["operation"],

agent/tools/plan_tool.py

@@ -85,18 +85,11 @@ def get_current_plan() -> List[Dict[str, str]]:
 PLAN_TOOL_SPEC = {
     "name": "plan_tool",
     "description": (
-        "Manage task planning and progress tracking with todo list (pending/in_progress/completed statuses). "
-        "⚠️ CRITICAL: ALWAYS use for multi-step tasks (3+ steps) and MUST update frequently to show progress. "
-        "**Use when:** (1) User provides multiple tasks, (2) Complex workflows (training, evaluation, data processing), "
-        "(3) Tasks requiring multiple tool calls, (4) Need to communicate progress clearly to user, "
-        "(5) Breaking down ambiguous requests into concrete steps. "
-        "**Pattern:** Create plan at start → Mark in_progress when starting task → Mark completed immediately after finishing → User sees clear progress. "
-        "Each call replaces entire plan (full list required). "
-        "**Critical for reliability:** Exactly ONE task in_progress at a time (not zero, not multiple). "
-        "Mark tasks completed IMMEDIATELY after finishing - don't batch completions. "
-        "**For long-running tasks:** Update plan after each major step to keep user informed. "
-        "**Only mark completed when:** Task fully accomplished, no errors, all requirements met. "
-        "Keep tasks pending if blocked/errors occur - create new task to resolve blockers."
+        "Track progress on multi-step tasks with a todo list (pending/in_progress/completed).\n\n"
+        "Use for tasks with 3+ steps. Each call replaces the entire plan (send full list).\n\n"
+        "Rules: exactly ONE task in_progress at a time. Mark completed immediately after finishing. "
+        "Only mark completed when the task fully succeeded — keep in_progress if there are errors. "
+        "Update frequently so the user sees progress."
     ),
     "parameters": {
         "type": "object",

agent/tools/sandbox_client.py

@@ -83,7 +83,11 @@ USER user
 
 ENV HOME=/home/user \\
     PATH=/home/user/.local/bin:$PATH \\
-    PIP_USER=1
+    PIP_USER=1 \\
+    HF_HUB_DISABLE_PROGRESS_BARS=1 \\
+    TQDM_DISABLE=1 \\
+    TRANSFORMERS_VERBOSITY=warning \\
+    HF_HUB_ENABLE_HF_TRANSFER=1
 
 WORKDIR /app
 COPY --chown=user . /app

agent/tools/sandbox_tool.py

@@ -77,17 +77,16 @@ async def _ensure_sandbox(
 SANDBOX_CREATE_TOOL_SPEC = {
     "name": "sandbox_create",
     "description": (
-        "Create a persistent remote Linux sandbox on HF Spaces for interactive development.\n"
-        "YOU MUST DO THIS BEFORE USING bash/read/write/edit tools.\n"
-        "\n"
-        "Spins up a new sandbox with a given hardware tier where you can run commands, read/write/edit files, "
-        "install packages, and debug iteratively. The sandbox persists across tool calls within the session."
-        "\n"
-        "You can choose from the following hardware tiers (GPU is required for model development or other tasks that benefit from and utilize the GPU): "
-        + ", ".join([e.value for e in SpaceHardware])
-        + ".\n"
-        "Use sandbox for: iterative development, debugging, multi-step workflows, testing code.\n"
-        "Use hf_jobs instead for: one-shot batch runs, scheduled tasks, fire-and-forget training.\n"
+        "Create a persistent remote Linux environment for developing and testing scripts.\n\n"
+        "Workflow: sandbox_create → write script → pip install → test with small run → fix errors → hf_jobs at scale.\n"
+        "The sandbox persists across tool calls within the session. pip install works out of the box.\n\n"
+        "Use this when: you need to develop, test, and iterate on scripts before launching via hf_jobs. "
+        "Especially for training scripts where you need to verify imports, test on a small subset, and fix errors interactively.\n\n"
+        "Skip this when: the task is a simple one-shot operation (status check, resource search, quick data query), "
+        "or the script is copied from a verified working example with minimal changes.\n\n"
+        "For ML code that uses CUDA, bf16, or model loading: use GPU hardware (t4-small minimum). "
+        "CPU sandboxes cannot run GPU code paths — your test will not catch GPU-related errors.\n\n"
+        "Hardware: " + ", ".join([e.value for e in SpaceHardware]) + ".\n"
     ),
     "parameters": {
         "type": "object",