Glossary · Reliability, calibration, and statistics
Covariate-shift weightw(x)
The ratio of a task's probability under the target population to its probability under the eval sample, used to reweight results to deployment. Weighting cannot create evidence where the sample has no support.
Defined in
Handbook · ValidationEstimands, pass rates, and intervalsWrite the estimand first, then report n, an honest interval and the right functional, from Wilson to pass^k to reweighted estimates.Related terms
- Stratified estimate
- A pass rate combined from per-stratum estimates with the target population's weights, Σ w_h p̂_h. It lets a suite over-sample informative slices and still report the deployment mix.
Where it comes up
- Task distributions and dataset engineeringHandbook · Claim
- Notation and formula cardHandbook · Reference