scrapeRL / docs /test /input-dashboard-streaming-test-report.md
NeerajCodz's picture
docs: init proto
24f0bf0

input-dashboard-live-stream-endpoint-test-report

scope

  • Input-first 2-window UX (Input -> Dashboard) with required fields: assets, instructions, output instructions
  • Real-time scrape flow (SSE + websocket broadcast)
  • Session-based scrape lifecycle (/api/scrape/*)
  • Frontend/backend integration through single docker compose up
  • Full endpoint smoke through frontend proxy (http://localhost:3000/api/*)

environment

  • Runtime: docker compose up --build -d
  • Frontend: http://localhost:3000
  • Backend: http://localhost:8000
  • Health check: GET http://localhost:3000/api/health -> 200

regression-fixes-applied

Endpoint Previous issue Fix Result
POST /api/agents/plan 500 (PlannerAgent.create_plan missing) Replaced with deterministic valid plan generation in route 200
GET /api/tools/categories 500 response validation mismatch Updated return typing to match actual payload 200
GET /api/providers and GET /api/providers/google 500 (list_models missing on provider impls) Switched provider model retrieval to get_models() 200
GET /api/plugins/categories 404 due dynamic route capture Moved static /categories route before /{plugin_id} 200

10-manual-scrape-stream-scenarios-low-medium-high

Test Complexity Output Memory Plugins Status
low-json low json on none completed
medium-csv-plugins medium csv on mcp-html, skill-extractor completed
high-markdown high markdown on mcp-browser, proc-json completed
low-text-no-memory low text off none completed
medium-json-multi-assets medium json on mcp-search completed
high-csv-unavailable-plugin high csv on mcp-pdf partial (expected unavailable-plugin warning)
low-json-simple-query low json on none completed
medium-markdown-plugins medium markdown on skill-planner, proc-csv completed
high-text high text on mcp-browser completed
low-csv low csv on none completed

full-endpoint-smoke-test-frontend-proxy

  • Target: http://localhost:3000/api/*
  • Total calls: 60
  • Server errors (5xx): 0
  • Unexpected statuses: 0
  • Covered route groups: health, agents, tasks, episode, memory, providers, plugins, tools, settings, scrape

integration-checks

  • GET http://localhost:3000/favicon.ico -> 200 (favicon 404 resolved)
  • Frontend proxy to backend verified for all dashboard-critical endpoints:
    • /api/health
    • /api/agents/list
    • /api/plugins
    • /api/memory/stats/overview
    • /api/settings

outcome

  • Frontend and backend are now reliably connected via docker compose.
  • The previously failing 500/404 dashboard endpoints are fixed.
  • Input-first session-based scraper flow, live updates, plugins, memory, and scrape lifecycle endpoints are working end-to-end.

document-flow

flowchart TD
    A[document] --> B[key-sections]
    B --> C[implementation]
    B --> D[operations]
    B --> E[validation]

related-api-reference

item value
api-reference api-reference.md