pitci.lightgbm.LGBMBoosterLeafNodeScaledConformalPredictor

class pitci.lightgbm.LGBMBoosterLeafNodeScaledConformalPredictor(model)

Bases: pitci.base.LeafNodeScaledConformalPredictor

Conformal interval predictor for an underlying lgb.Booster model using absolute error scaled by leaf node counts as the nonconformity measure.

Class implements inductive conformal intervals where a calibration dataset is used to learn the information that is used when generating intervals for new instances.

The predictor outputs varying width intervals for every new instance. This is done by multiplying the baseline_interval by a scaling factor that depends on the input data. The scaling function uses the reciporcal of the number of times that the leaf nodes used in making each prediction were visited on the calibration dataset, or when the underlying model was trained - see the train_data argument for the calibrate() method for more information.

The intuition behind this is that for rows that have higher leaf node counts from the calibration set - the model will be more ‘familiar’ with hence the interval for these rows should be smaller. The inverse is true for rows that have lower leaf node counts from the calibration set.

The currently supported lightgbm objective functions, given the nonconformity measure that is based on absolute error, are defined in the SUPPORTED_OBJECTIVES attribute.

Parameters

model (lgb.Booster) – Underlying lgb.Booster model to generate prediction intervals with.

__version__

The version of the pitci package that generated the object.

Type

str

model

The underlying lgb.Booster model passed in initialising the object.

Type

lgb.Booster

leaf_node_counts

The number of times each leaf node in each tree was visited when making predictions on the calibration dataset. Each item in the list is a dict giving a mapping between leaf node index and counts for a given tree. The length of the list corresponds to the number of trees in model.

Type

list

baseline_interval

The default or baseline conformal half interval width. Will be scaled for each prediction generated.

Type

float

alpha

The confidence level of the conformal intervals that will be produced. Attribute is set when the calibrate() method is run.

Type

int or float

SUPPORTED_OBJECTIVES

Booster supported objectives. If a lgb.Booster with a non-supported objective is passed when initialising the class object an error will be raised.

Type

list

__init__(model)

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__(model)	Initialize self.
calibrate(data, response[, alpha, train_data])	Calibrate conformal intervals to a given sample of data at a given confidence level, alpha, between 0 and 1.
predict_with_interval(data)	Generate predictions with conformal intervals using the underlying model.

calibrate(data, response, alpha=0.95, train_data=None)

Calibrate conformal intervals to a given sample of data at a given confidence level, alpha, between 0 and 1.

This method must be run before predict_with_interval() can be used to generate predictions.

There are 2 items to be calibrated; the leaf node counts stored in the leaf_node_counts attribute and the half interval width stored in the baseline_interval attribute.

The user has the option to specify the training sample that was used to buid the model in the train_data argument. This is to allow the leaf_node_counts to be calibrated on the same data the underlying model was built on, rather than a separate calibration set which is what will be passed in the data argument. The default interval width for a given alpha has to be set on a separate sample to what was used to build the model. If not, the errors will be smaller than they otherwise would be, on a sample the underlying model has not seen before. However for the leaf_node_counts, ideally we want counts from the train sample - we’re not ‘learning’ anything new here, just recreating stats from when the model was built originally.

Parameters

• data (np.ndarray or pd.DataFrame) – Dataset to use to set baselines.

• response (np.ndarray or pd.Series) – The response values for the records in data.

• alpha (int or float, default = 0.95) – Confidence level for the intervals.

• train_data (np.ndarray, pd.DataFrame or None, default = None) – Optional dataset that can be passed to set baseline leaf_node_counts from, separate to the data argument used to set baseline_interval width.

predict_with_interval(data)

Generate predictions with conformal intervals using the underlying model.

Parameters

data (np.ndarray or pd.DataFrame) – Dataset to generate predictions with intervals on.

Returns

predictions_with_interval – Array of predictions with intervals for each row in data. Output array will have 3 columns where the first is the lower interval, second are the predictions and the third is the upper interval.

Return type

np.ndarray