Skip to content

Implement RoB 2 Domain 3 signaling-question rubrics #60

Description

@jd-coderepos

Scope

Implement the four Domain 3 rubrics under the RoB 2 pointwise package.

Domain 3: bias due to missing outcome data.

Target package

Create:

yescieval/rubric/pointwise/rob2/d3_missing_data/
├── __init__.py
├── sq_3_1_outcome_data_availability.py
├── sq_3_2_evidence_no_missing_data_bias.py
├── sq_3_3_missingness_could_depend.py
└── sq_3_4_missingness_likely_depended.py

Required rubric classes

ID Class
3.1 OutcomeDataAvailability
3.2 EvidenceNoMissingDataBias
3.3 MissingnessCouldDepend
3.4 MissingnessLikelyDepended

Each class must inherit RoB2PointwiseRubric and define:

rubric_id
rob2_domain_id = "D3"
name
question
guidance
allowed_responses

Criterion requirements

  • Use the current RoB 2 guidance for individually randomized parallel-group trials as the source of wording and question-specific guidance.
  • Keep each signaling question in its own module.
  • Do not encode conditional routing inside the rubric classes; routing is handled by Implement conditional RoB 2 rubric execution #52.
  • NA is workflow-generated for skipped questions and is not a free model answer for an executed rubric.
  • Export all four classes from d3_missing_data/__init__.py.

Tests

Create test/test_rob2_d3_rubrics.py and verify for every class:

  • correct ID and domain;
  • non-empty question and guidance;
  • correct response model and allowed response set;
  • successful prompt generation.

Definition of done

All four Domain 3 signaling questions are independently importable and executable through the shared RoB 2 rubric interface.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions