Jeffreys Test

src.meliora.core.jeffreys_test(data, ratings, default_flag, predicted_pd, alpha_level=0.05)[source]

Calculate the Jeffrey’s test for a given probability of defaults buckets

Parameters

  • data (Pandas DataFrame with at least three columns) – ratings : PD rating class of obligor default_flag : 1 (or True) for defaulted and 0 (or False) for good obligors probs_default : predicted probability of default of an obligor

  • ratings (column label for ratings) –

  • default_flag (column label for default_flag) –

  • probs_default (column label for probs_default) –

  • alpha_level (false positive rate of hypothesis test (default .05)) –

Returns

Rating (Index) : Contains the ratings of each class/group PD : predicted default rate in each group N : number of obligors in each group D : number of defaults in each group Default Rate : realised default rate per each group p_value : Jeffreys p-value reject : whether to reject the null hypothesis at alpha_level

Return type

Pandas DataFrame with the following columns

Notes

The Jeffreys test compares forecasted defaults with observed defaults in a binomial model with independent observations under the null hypothesis that the PD applied in the portfolio/rating grade at the beginning of the relevant observation period is greater than the true one (one-sided hypothesis test). The test updates a Beta distribution (with Jeffrey’s prior) in light of the number of defaults and non-defaults, then reports the posterior probability of the null hypothesis.

1

“Instructions for reporting the validation results of internal models - IRB Pillar I models for credit risks,” ECB, pp. 20-21, 2019.

Examples

>>> import random, numpy as np
>>> buckets = ['A', 'B', 'C']
>>> ratings = random.choices(buckets,  [0.4, 0.5, 0.1], k=1000)
>>> bucket_pds = {'A': .1, 'B': .15, 'C': .2}
>>> probs_default = [bucket_pds[r] for r in ratings]
>>> default_flag = [random.uniform(0, 1) < bucket_pds[r] for r in ratings]
>>> test_data = pd.DataFrame({'ratings': ratings,
                              'default_flag': default_flag,
                              'predicted_pd' : probs_default})
>>> from meliora import jeffreys_test
>>> jeffreys_test(test_data, 'ratings', 'default_flag', 'probs_default')

PD N D Default Rate p_value reject

ratings A 0.10 401 36 0.089776 0.748739 False B 0.15 489 73 0.149284 0.511781 False C 0.20 110 23 0.209091 0.397158 False