‼️ If you want to work on this issue: please comment below and wait until a maintainer assigns this issue to you before opening a PR to avoid several contributions on the same issue. Thanks! 😊
✨ What You’ll Do
Right now, Pruna has several evaluation metrics, but we would need a simple MSE based metric.
You can do this by putting images on top of each other and literally looking at the pixel differences at the transitions.
Your task will be to:
- Add a new metric class
MSE under pruna/metrics/.
- Implement the standard metric interface (
update(...), compute(...)).
- Integrate this metric into Pruna’s registry so it can be selected in evaluation runs.
- Add tests that validate correctness on toy examples and edge cases.
- Document the new metric and provide a simple usage example.
🤖 Useful Resources
✅ Acceptance Criteria
- It follows the style guidelines.
- The new metric is documented in code and user docs.
- Tests demonstrate that the metric computes expected scores on simple controlled examples.
- The metric integrates properly into Pruna’s evaluation framework.
And don’t forget to give us a ⭐️!
❓ Questions?
Feel free to jump into the #contributing Discord channel if you hit any roadblocks. Can’t wait to see your contribution! 🚀
Share on Socials

✨ What You’ll Do
Right now, Pruna has several evaluation metrics, but we would need a simple MSE based metric.
You can do this by putting images on top of each other and literally looking at the pixel differences at the transitions.
Your task will be to:
MSEunderpruna/metrics/.update(...),compute(...)).🤖 Useful Resources
pruna/metricsas references for structure.✅ Acceptance Criteria
And don’t forget to give us a ⭐️!
❓ Questions?
Feel free to jump into the #contributing Discord channel if you hit any roadblocks. Can’t wait to see your contribution! 🚀
Share on Socials