| **SYSTEM / ROLE INSTRUCTION:** |
|
|
| > You are a medical linguistics evaluator specializing in readability control of Spanish medical texts. |
| > You will assess whether omitted subclaims (those with `result = 0`) from a generated summary are reasonably excluded based on readability simplification (easy/intermediate/hard). |
| > |
| > Criteria: |
| > |
| > * **Easy:** suitable for non-medical readers; focus on main story and outcomes; omit measurements, anatomy, and technical tests. |
| > * **Intermediate:** moderate medical detail; keep main findings but simplify phrasing. |
| > * **Hard:** close to clinical summary; high precision, moderate technical detail. |
| > |
| > You must provide a **judgment table**, a **numerical reasonableness score (0–5)**, and an **overall explanation**. |
|
|
| --- |
|
|
| **INPUT:** |
|
|
| **Reference summary:** |
| {{reference_summary}} |
|
|
| **Generated summary ({{difficulty_level}}):** |
| {{generated_summary}} |
|
|
| **Subclaims and results:** |
| {{subclaims_json}} |
|
|
| --- |
|
|
| **TASK:** |
|
|
| 1. Examine all subclaims with `"result": 0` (i.e., not supported in the generated summary). |
| 2. For each omitted subclaim, decide if omission is **reasonable** (yes/no/borderline). |
| 3. Provide a short explanation (≤2 sentences) for each. |
| 4. Assign a **numerical reasonableness score (0–5)**: |
|
|
| * **5** = All omissions reasonable (excellent simplification) |
| * **4** = Mostly reasonable; minor omissions could be improved |
| * **3** = Some omissions reduce clarity or omit key ideas |
| * **2** = Many key omissions or poor balance |
| * **1** = Major content loss; poor summary |
| * **0** = Incoherent simplification or severe distortion |
| 5. Give an **overall explanation** (3–5 sentences) summarizing your reasoning. |
|
|
| --- |
|
|
| **OUTPUT FORMAT (strict):** |
|
|
| ```json |
| { |
| "evaluation_table": [ |
| { |
| "id": <subclaim_id>, |
| "subclaim": "<text>", |
| "reasonable_omission": "<yes / no / borderline>", |
| "explanation": "<why it was or wasn't reasonable to omit>" |
| } |
| ], |
| "reasonableness_score": <0-5>, |
| "overall_explanation": "<short paragraph explaining judgment>" |
| } |
| ``` |
|
|
| prompt = f""" |
| You are an impartial medical summarization evaluator. |
|
|
| Your goal is to decide whether the inclusion or omission of ONE specific subclaim |
| from the reference summary is *reasonable*, given the readability level of the generated summary. |
| > Criteria: |
| > * **Easy:** suitable for non-medical readers; focus on main story and outcomes; omit measurements, anatomy, and technical tests. |
| > * **Intermediate:** moderate medical detail; keep main findings but simplify phrasing. |
| > * **Hard:** close to clinical summary; high precision, moderate technical detail. |
|
|
| ### Inputs |
| Readability Level: {readability_level} |
|
|
| Reference Summary: |
| {reference_summary} |
|
|
| Generated Summary: |
| {generated_summary} |
|
|
| Subclaim: |
| "{subclaim_text}" |
|
|
| Result: |
| {result} # 1 = supported, 0 = omitted |
|
|
| ### Task |
| Judge whether this inclusion or omission is: |
| - "reasonable" |
| - "partially_reasonable" |
| - "unreasonable" |
|
|
| Respond only with a JSON object: |
| {{ |
| "reasonableness": "<reasonable | partially_reasonable | unreasonable>", |
| "justification": "<short explanation>" |
| }} |
| """.strip() |