Skip to content

Implement RoB 2 Domain 4 signaling-question rubrics #61

Description

@jd-coderepos

Scope

Implement the five Domain 4 rubrics under the RoB 2 pointwise package.

Domain 4: bias in measurement of the outcome.

Target package

Create:

yescieval/rubric/pointwise/rob2/d4_measurement/
├── __init__.py
├── sq_4_1_measurement_method.py
├── sq_4_2_measurement_difference.py
├── sq_4_3_assessor_awareness.py
├── sq_4_4_assessment_could_be_influenced.py
└── sq_4_5_assessment_likely_influenced.py

Required rubric classes

ID Class
4.1 MeasurementMethod
4.2 MeasurementDifference
4.3 AssessorAwareness
4.4 AssessmentCouldBeInfluenced
4.5 AssessmentLikelyInfluenced

Each class must inherit RoB2PointwiseRubric and define:

rubric_id
rob2_domain_id = "D4"
name
question
guidance
allowed_responses

Criterion requirements

  • Use the current RoB 2 guidance for individually randomized parallel-group trials as the source of wording and question-specific guidance.
  • Keep each signaling question in its own module.
  • Do not encode conditional routing inside the rubric classes; routing is handled by Implement conditional RoB 2 rubric execution #52.
  • NA is workflow-generated for skipped questions and is not a free model answer for an executed rubric.
  • Export all five classes from d4_measurement/__init__.py.

Tests

Create test/test_rob2_d4_rubrics.py and verify for every class:

  • correct ID and domain;
  • non-empty question and guidance;
  • correct response model and allowed response set;
  • successful prompt generation.

Definition of done

All five Domain 4 signaling questions are independently importable and executable through the shared RoB 2 rubric interface.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions