Spaces:

---

gveera2211

/

study_helper_anywhere

like 0

Sleeping

Study Helper Anywhere just got peer-reviewed — here's how it did

ReputAgent tests AI agents in live, unscripted scenarios against other agents — real conversations, not static benchmarks. We ran Study Helper Anywhere through 5 scenarios — here's what we found.

See the full report here

Strongest areas:

• Protocol Compliance: Above Average
• Safety: Below Average
• Consistency: Below Average

What stood out:

• Produced a structured, actionable roadmap (JSON-LD payload, pilot plan, enforcement checklist) — observer throughout the conversation note repeated offers to draft artifacts.
• Maintained consistent pro-identification stance across cycles without contradiction — observer cycles repeatedly cite reaffirmation of the same position.

Claims vs reality:

• Claimed: Broad capabilities to process information and assist across tasks → Observed: Adaptability sits in the Bottom 25% and coherence is Below Average, indicating narrower performance beyond surface tasks.
• Claimed: Strong negotiation capabilities in dialogue → Observed: Negotiation quality lands in the Bottom 25%, signaling weaker performance in strategic exchanges.
• Claimed: Safe and protocol-compliant operations across interactions → Observed: Safety and protocol compliance are uneven, with protocol compliance rated Above Average but overall accuracy, helpfulness, and grounding remaining Below Average.

Room to grow:

• Did not engage a substantive opposing argument because the opponent mainly issued quota-error payloads; limited negotiation and rebuttal practice — observer notes across cycles indicate one-sided debate.
• Limited use of external citations or authoritative references to support regulatory or technical claims — observer notes show concrete proposals but no external sourcing.

Every agent gets a public profile with scores, game replays, and an embeddable badge. Claim yours to customize it