Hoshmer Lemeshow Test

src.meliora.core.hosmer_test(data, ratings, default_flag, predicted_pd, alpha_level=0.05)[source]

Calculate the Hosmer-Lemeshow Chi-squared test for a given probability of defaults buckets

Parameters

  • data (Pandas DataFrame with at least three columns) – ratings : PD rating class of obligor default_flag : 1 (or True) for defaulted and 0 (or False) for good obligors probs_default : predicted probability of default of an obligor

  • ratings (column label for ratings) –

  • default_flag (column label for default_flag) –

  • probs_default (column label for probs_default) –

  • alpha_level (false positive rate of hypothesis test (default .05)) –

Returns

Rating (Index) : Contains the ratings of each class/group PD : predicted default rate in each group and total N : number of obligors in each group and total D : number of defaults in each group and total Default Rate : realised default rate per each group and total p_value : overall Hosmer-Lemeshow test p-value reject : whether to reject the null hypothesis at alpha_level

Return type

Pandas DataFrame with the following columns

Notes

The Hosmer-Lemeshow Chi-squared test calculates a standardized sum of squared differences between the number of defaults and the expected number of defaults within each rating group. Under the null hypothesis that the PDs applied in the portfolio/rating grade at the beginning of the relevant observation period are equal to the true ones, the test statistic has an approximate Chi-squared distribution. Large values of this test statistic provide evidence against the null hypothesis.

1

“Backtesting Framework for PD, EAD and LGD - Public Version,” Bauke Maarse, Rabobank International, p. 43, 2012.

Examples

>>> import random, numpy as np
>>> buckets = ['A', 'B', 'C']
>>> ratings = random.choices(buckets,  [0.4, 0.5, 0.1], k=1000)
>>> bucket_pds = {'A': .1, 'B': .15, 'C': .2}
>>> probs_default = [bucket_pds[r] for r in ratings]
>>> default_flag = [random.uniform(0, 1) < bucket_pds[r] for r in ratings]
>>> test_data = pd.DataFrame({'ratings': ratings,
                              'default_flag': default_flag,
                              'predicted_pd' : probs_default})
>>> from meliora import hosmer_test
>>> hosmer_test(test_data, 'ratings', 'default_flag', 'probs_default')

PD N D Default Rate p_value reject

ratings A 0.10000 401.0 36.0 0.089776 None None B 0.15000 489.0 73.0 0.149284 None None C 0.20000 110.0 23.0 0.209091 None None total 0.13545 1000.0 132.0 0.132000 0.468902 False