Skip to content

Add quantile-quantile plot to wandb custom charts - #539

Draft
lm2612 wants to merge 6 commits into
mainfrom
add_wandb_quantiles
Draft

Add quantile-quantile plot to wandb custom charts#539
lm2612 wants to merge 6 commits into
mainfrom
add_wandb_quantiles

Conversation

@lm2612

@lm2612 lm2612 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Pull Request

Description

Addresses issue #538
It would be useful to compare p-levels as well as the median on W&B. We can do this using a quantile-quantile plot, which would show the true p-level on the x-axis against the fraction of validation data that falls below that p-level on the y-axis. The ideal model would line up along the y=x line.

To do this I have used the existing line plotting function that plots the MAE at different horizons. This involves storing the fraction of the validation data below each level in a wandb table. I've extended the function wandb_line_plot() to also include the option to show the identity line where y=x between 0 and 1.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.
Please also list any relevant details for your test configuration

  • Yes

Checklist:

  • My code follows OCF's coding style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked my code and corrected any misspellings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant