Spiegelhalter Test

src.meliora.core.spiegelhalter_test(data, ratings, default_flag, predicted_pd, alpha_level=0.05)[source]

Calculate the Spiegelhalter test for a given probability of defaults buckets

Parameters

  • data (Pandas DataFrame with at least three columns) – ratings : PD rating class of obligor default_flag : 1 (or True) for defaulted and 0 (or False) for good obligors probs_default : predicted probability of default of an obligor

  • ratings (column label for ratings) –

  • default_flag (column label for default_flag) –

  • probs_default (column label for probs_default) –

  • alpha_level (false positive rate of hypothesis test (default .05)) –

Returns

Rating (Index) : Contains the ratings of each class/group PD : predicted default rate in each group and total N : number of obligors in each group and total D : number of defaults in each group and total Default Rate : realised default rate per each group and total p_value : overall Spiegelhalter test p-value reject : whether to reject the null hypothesis at alpha_level

Return type

Pandas DataFrame with the following columns

Notes

The Spiegelhalter test compares forecasted defaults with observed defaults by analyzing the prediction errors. Under the null hypothesis that the PDs applied in the portfolio/rating grade at the beginning of the relevant observation period are equal to the true ones, the mean squared error can be standardized into an approximately standard Normal test statistic. Large values of this test statistic provide evidence against the null hypothesis.

1

“Backtesting Framework for PD, EAD and LGD - Public Version,” Bauke Maarse, Rabobank International, pp. 43-44, 2012.

Examples

>>> import random, numpy as np
>>> buckets = ['A', 'B', 'C']
>>> ratings = random.choices(buckets,  [0.4, 0.5, 0.1], k=1000)
>>> bucket_pds = {'A': .1, 'B': .15, 'C': .2}
>>> probs_default = [bucket_pds[r] for r in ratings]
>>> default_flag = [random.uniform(0, 1) < bucket_pds[r] for r in ratings]
>>> test_data = pd.DataFrame({'ratings': ratings,
                              'default_flag': default_flag,
                              'predicted_pd' : probs_default})
>>> from meliora import spiegelhalter_test
>>> spiegelhalter_test(test_data, 'ratings', 'default_flag', 'probs_default')

PD N D Default Rate p_value reject

ratings A 0.10000 401.0 36.0 0.089776 None None B 0.15000 489.0 73.0 0.149284 None None C 0.20000 110.0 23.0 0.209091 None None total 0.13545 1000.0 132.0 0.132000 0.647161 False