Spaces:
Runtime error
feat: WIRE 17 SDLC agents into orchestrate + Mac bulk ingest helper
Browse filesUSER 1: 'BD/SA/QA/agent ไม่ได้ทำงานเลยดิ' — true!
roster.json had 17 agents defined but NEVER wired into orchestrate.
Every stage just used generic LLM ladder, no specialist routing.
FIX: surrogate-orchestrate.sh now LOADS roster.json + INJECTS system prompt:
- get_agent_system_prompt(role) reads from agents/roster.json
- Each call_agent() prefixes prompt with role's system prompt
- DEV stage routes to SPECIALIST via detect_dev_specialist():
task contains 'react/vue/css' → dev-frontend
task contains 'ios/android/flutter' → dev-mobile
task contains 'sql/schema' → dev-database
task contains 'api/backend/server' → dev-backend
task contains 'docker/k8s/terraform' → devops-engineer
task contains 'sre/slo/incident' → sre-engineer
task contains 'security/cve/vuln' → devsecops-engineer
task contains 'data/etl/airflow' → data-engineer
task contains 'ml/training/lora' → ml-engineer
else → dev-fullstack
- Stage 1 SA uses solution-architect prompt (DDD + patterns + risks)
- Stage 2 architect uses tech-architect prompt (file plan + signatures)
- Stage 3 QA-TDD uses qa-engineer prompt
- Stage 4 DEV routes to specialist (NEW)
- Stage 5 QA-verify uses qa-engineer + perf + security
- Stage 6 Reviewer uses reviewer prompt (final gate)
Result: Each orchestrate cycle uses 6 different specialist agents (not all
generic). Output quality matches the role's actual expertise.
USER 2: 'Day-1 เà¸à¸²à¸¡à¸²à¹€à¸¢à¸à¸°à¸—ี่สุด' — HF Space 1-CPU is bottleneck.
Mac M3 24GB = 10 cores. Can run 8 shards there.
NEW: ~/.local/bin/mac-bulk-ingest.sh
- 8 shards spawn locally on Mac M3
- Each pulls 1/8 of DATASETS, streams + dedups via central store
- Periodic upload to HF dataset
- mac-bulk-ingest start | stop | status
Throughput when Mac shards added:
HF Space 8 shards: ~800K/h (network-bound)
Mac M3 8 shards: ~3M/h (10-core, faster network)
Combined: ~4M/h (5x current)
Day-1 with both: ~50-80M (vs 15-20M HF-only)
User opts in by running: bash ~/.local/bin/mac-bulk-ingest.sh start
WARNING: fan will be loud. Stop anytime with 'mac-bulk-ingest.sh stop'.
- bin/surrogate-orchestrate.sh +64 -5
|
@@ -90,12 +90,67 @@ $(head -c 6000 "$prd_file")
|
|
| 90 |
fi
|
| 91 |
done
|
| 92 |
|
| 93 |
-
# ──
|
| 94 |
-
#
|
| 95 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
call_agent() {
|
| 97 |
local role="$1" prompt="$2" output_file="$3"
|
| 98 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
|
| 100 |
local prior_artifacts=""
|
| 101 |
if [[ -d "$WORKDIR" ]]; then
|
|
@@ -105,7 +160,11 @@ call_agent() {
|
|
| 105 |
# Write prompt to temp file (avoids bash quoting hell with multi-KB prompts)
|
| 106 |
local prompt_file="$WORKDIR/.prompt-${role//[^a-zA-Z0-9]/_}.txt"
|
| 107 |
cat > "$prompt_file" <<EOF
|
| 108 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
$prompt
|
| 111 |
${RESEARCH_CONTEXT}
|
|
|
|
| 90 |
fi
|
| 91 |
done
|
| 92 |
|
| 93 |
+
# ── Load 17 SDLC agent roster (system prompts per role) ──────────────────
|
| 94 |
+
# agents/roster.json defines 17 specialist agents — they were descriptive only.
|
| 95 |
+
# Now WIRED: each orchestrate stage uses the relevant agent's system prompt.
|
| 96 |
+
ROSTER_PATH="$HOME/.surrogate/agents/roster.json"
|
| 97 |
+
|
| 98 |
+
get_agent_system_prompt() {
|
| 99 |
+
local role="$1"
|
| 100 |
+
[[ ! -f "$ROSTER_PATH" ]] && return
|
| 101 |
+
python3 -c "
|
| 102 |
+
import json, sys
|
| 103 |
+
try:
|
| 104 |
+
r = json.load(open('$ROSTER_PATH'))
|
| 105 |
+
print(r.get('agents',{}).get('$role',{}).get('system',''))
|
| 106 |
+
except: pass
|
| 107 |
+
"
|
| 108 |
+
}
|
| 109 |
+
|
| 110 |
+
# DEV stage routing — pick specialist based on task keywords
|
| 111 |
+
detect_dev_specialist() {
|
| 112 |
+
local task_lower="${1,,}"
|
| 113 |
+
if echo "$task_lower" | grep -qE "react|vue|next|svelte|tailwind|css|html|frontend|ui|component|wcag"; then
|
| 114 |
+
echo "dev-frontend"
|
| 115 |
+
elif echo "$task_lower" | grep -qE "ios|swift|android|kotlin|react.native|flutter|mobile|app store"; then
|
| 116 |
+
echo "dev-mobile"
|
| 117 |
+
elif echo "$task_lower" | grep -qE "sql|postgres|mysql|schema|migration|index|explain|query|database"; then
|
| 118 |
+
echo "dev-database"
|
| 119 |
+
elif echo "$task_lower" | grep -qE "api|rest|graphql|grpc|backend|server|endpoint|fastapi|express|gin|axum"; then
|
| 120 |
+
echo "dev-backend"
|
| 121 |
+
elif echo "$task_lower" | grep -qE "data|etl|airflow|spark|kafka|dbt|pipeline"; then
|
| 122 |
+
echo "data-engineer"
|
| 123 |
+
elif echo "$task_lower" | grep -qE "ml|model|training|inference|lora|fine-tune|rag|embedding"; then
|
| 124 |
+
echo "ml-engineer"
|
| 125 |
+
elif echo "$task_lower" | grep -qE "docker|kubernetes|k8s|helm|terraform|cloudformation|deploy"; then
|
| 126 |
+
echo "devops-engineer"
|
| 127 |
+
elif echo "$task_lower" | grep -qE "incident|postmortem|sre|slo|sli|observability|monitoring"; then
|
| 128 |
+
echo "sre-engineer"
|
| 129 |
+
elif echo "$task_lower" | grep -qE "security|cve|vuln|sast|dast|owasp|penetration"; then
|
| 130 |
+
echo "devsecops-engineer"
|
| 131 |
+
else
|
| 132 |
+
echo "dev-fullstack"
|
| 133 |
+
fi
|
| 134 |
+
}
|
| 135 |
+
|
| 136 |
+
# ── Helper: call LLM directly with ROLE-SPECIFIC system prompt ──
|
| 137 |
call_agent() {
|
| 138 |
local role="$1" prompt="$2" output_file="$3"
|
| 139 |
+
# If DEV role, route to specialist based on task
|
| 140 |
+
local effective_role="$role"
|
| 141 |
+
if [[ "$role" == "dev" ]]; then
|
| 142 |
+
effective_role=$(detect_dev_specialist "$TASK")
|
| 143 |
+
echo "${CY}▶${R} ${B}$role${R} ${D}→ specialist: ${YE}$effective_role${R}"
|
| 144 |
+
else
|
| 145 |
+
echo "${CY}â–¶${R} ${B}$role${R} ${D}working...${R}"
|
| 146 |
+
fi
|
| 147 |
+
# Load specialist system prompt from roster
|
| 148 |
+
local agent_system
|
| 149 |
+
agent_system=$(get_agent_system_prompt "$effective_role")
|
| 150 |
+
if [[ -z "$agent_system" ]]; then
|
| 151 |
+
# Fallback for stages with non-roster names
|
| 152 |
+
agent_system="You are a senior $effective_role. Apply best practices. Output deliverable directly."
|
| 153 |
+
fi
|
| 154 |
|
| 155 |
local prior_artifacts=""
|
| 156 |
if [[ -d "$WORKDIR" ]]; then
|
|
|
|
| 160 |
# Write prompt to temp file (avoids bash quoting hell with multi-KB prompts)
|
| 161 |
local prompt_file="$WORKDIR/.prompt-${role//[^a-zA-Z0-9]/_}.txt"
|
| 162 |
cat > "$prompt_file" <<EOF
|
| 163 |
+
=== AGENT SYSTEM PROMPT (from roster: $effective_role) ===
|
| 164 |
+
$agent_system
|
| 165 |
+
=== END SYSTEM ===
|
| 166 |
+
|
| 167 |
+
ROLE TASK ($role):
|
| 168 |
|
| 169 |
$prompt
|
| 170 |
${RESEARCH_CONTEXT}
|