conformal.classification¶
Classification module contains methods for conformal classification.
Conformal classifiers predict a set of classes (not always a single class) under a given significance level (error rate). Every classifier works in combination with a nonconformity measure and on average predicts the correct class with the given error rate. Lower error rates result in smaller sets of predicted classes.
Structure:
 ConformalClassifier
 Transductive (
TransductiveClassifier
)  Inductive (
InductiveClassifier
)  Cross (
CrossClassifier
)
 Transductive (

class
conformal.classification.
PredictionClass
(p, eps)[source]¶ Bases:
object
Conformal classification prediction object, which is produced by the
ConformalClassifier.predict()
method.
p
¶ List of pairs (pvalue, class)
Type: List

eps
¶ Default significance level (error rate).
Type: float
Examples
>>> train, test = next(LOOSampler(Table('iris'))) >>> tcp = TransductiveClassifier(InverseProbability(NaiveBayesLearner()), train)
>>> prediction = tcp.predict(test[0], 0.1) >>> print(prediction.confidence(), prediction.credibility())
>>> prediction = tcp.predict(test[0]) >>> print(prediction.classes(0.1), prediction.classes(0.9))

__init__
(p, eps)[source]¶ Initialize the prediction.
Parameters:  p (List) – List of pairs (pvalue, class)
 eps (float) – Default significance level (error rate).

classes
(eps=None)[source]¶ Compute the set of classes under the default or given eps value.
Parameters: eps (float) – Significance level (error rate). Returns: List of predicted classes.

verdict
(ref, eps=None)[source]¶ Conformal classification prediction is correct when the actual class appears among the predicted classes.
Parameters:  ref – Reference/actual class
 eps (float) – Significance level (error rate).
Returns: True if the prediction with default or specified eps is correct.

confidence
()[source]¶ Confidence is an efficiency measure of a single prediction.
Computes minimum \(\mathit{eps}\) that would still result in a prediction of a single label. \(\mathit{eps} = \text{second\_largest}(p_i)\)
Returns: Confidence \(1\mathit{eps}\). Return type: float

credibility
()[source]¶ Credibility is an efficiency measure of a single prediction. Small credibility indicates an unusual example.
Computes minimum \(\mathit{eps}\) that would result in an empty prediction set. \(\mathit{eps} = \text{max}(p_i)\)
Returns: Credibility \(\mathit{eps}\). Return type: float


class
conformal.classification.
ConformalClassifier
(nc_measure, mondrian=False)[source]¶ Bases:
orangecontrib.conformal.base.ConformalPredictor
Base class for conformal classifiers.

__init__
(nc_measure, mondrian=False)[source]¶ Verify that the nonconformity measure can be used for classification.

p_values
(example)[source]¶ Extending classes should implement this method to return a list of pairs (pvalue, class) for a given example.
Conformal classifier assigns an assumed class value to the given example and computes its nonconformity. Pvalue is the ratio of more nonconformal (stranger) instances that the given example.

predict
(example, eps=None)[source]¶ Compute a classification prediction object from pvalues for a given example and significance level.
Parameters:  example (Instance) – Orange row instance.
 eps (float) – Default significance level (error rate).
Returns: Classification prediction object.
Return type:


class
conformal.classification.
TransductiveClassifier
(nc_measure, train=None, mondrian=False)[source]¶ Bases:
conformal.classification.ConformalClassifier
Transductive classification.
Examples
>>> train, test = next(LOOSampler(Table('iris'))) >>> tcp = TransductiveClassifier(ProbabilityMargin(NaiveBayesLearner()), train) >>> print(tcp(test[0], 0.1))

__init__
(nc_measure, train=None, mondrian=False)[source]¶ Initialize transductive classifier with a nonconformity measure and a training set.
Fit the conformal classifier to the training set if present.
Parameters:  nc_measure (ClassNC) – Classification nonconformity measure.
 train (Optional[Table]) – Table of examples used as a training set.
 mondrian (bool) – Use a mondrian setting for computing pvalues.

fit
(train)[source]¶ Fit the conformal classifier to the training set and store the domain.
Parameters: train (Optional[Table]) – Table of examples used as a training set.

p_values
(example)[source]¶ Compute pvalues for every possible class.
Transductive classifier appends the given example with an assumed class value to the training set and compares its nonconformity against all other instances.
Parameters: example (Instance) – Orange row instance. Returns: List of pairs (pvalue, class)


class
conformal.classification.
InductiveClassifier
(nc_measure, train=None, calibrate=None, mondrian=False)[source]¶ Bases:
conformal.classification.ConformalClassifier
Inductive classification.
Examples
>>> train, test = next(LOOSampler(Table('iris'))) >>> train, calibrate = next(RandomSampler(train, 2, 1)) >>> icp = InductiveClassifier(InverseProbability(LogisticRegressionLearner()), train, calibrate) >>> print(icp(test[0], 0.1))

__init__
(nc_measure, train=None, calibrate=None, mondrian=False)[source]¶ Initialize inductive classifier with a nonconformity measure, training set and calibration set. If present, fit the conformal classifier to the training set and compute the nonconformity scores of calibration set.
Parameters:  nc_measure (ClassNC) – Classification nonconformity measure.
 train (Optional[Table]) – Table of examples used as a training set.
 calibrate (Optional[Table]) – Table of examples used as a calibration set.
 mondrian (bool) – Use a mondrian setting for computing pvalues.

fit
(train, calibrate)[source]¶ Fit the conformal classifier to the training set, compute and store nonconformity scores (
alpha
) on the calibration set and store the domain.Parameters:  train (Optional[Table]) – Table of examples used as a training set.
 calibrate (Optional[Table]) – Table of examples used as a calibration set.

p_values
(example)[source]¶ Compute pvalues for every possible class.
Inductive classifier assigns an assumed class value to the given example and compares its nonconformity against all other instances in the calibration set.
Parameters: example (Instance) – Orange row instance. Returns: List of pairs (pvalue, class)


class
conformal.classification.
CrossClassifier
(nc_measure, k, train=None, mondrian=False)[source]¶ Bases:
conformal.classification.InductiveClassifier
Cross classification.
Examples
>>> train, test = next(LOOSampler(Table('iris'))) >>> ccp = CrossClassifier(InverseProbability(LogisticRegressionLearner()), 3, train) >>> print(ccp(test[0], 0.1))

__init__
(nc_measure, k, train=None, mondrian=False)[source]¶ Initialize cross classifier with a nonconformity measure, number of folds and training set. If present, fit the conformal classifier to the training set.
Parameters:  nc_measure (ClassNC) – Classification nonconformity measure.
 k (int) – Number of folds.
 train (Optional[Table]) – Table of examples used as a training set.
 mondrian (bool) – Use a mondrian setting for computing pvalues.

fit
(train)[source]¶ Fit the cross classifier to the training set. Split the training set into k folds for use as training and calibration set with an inductive classifier. Concatenate the computed nonconformity scores and store them (
InductiveClassifier.alpha
).Parameters: train (Table) – Table of examples used as a training set.


class
conformal.classification.
LOOClassifier
(nc_measure, train=None, mondrian=False)[source]¶ Bases:
conformal.classification.CrossClassifier
Leaveoneout classifier is a cross conformal classifier with the number of folds equal to the size of the training set.
Examples
>>> train, test = next(LOOSampler(Table('iris'))) >>> loocp = LOOClassifier(InverseProbability(LogisticRegressionLearner()), train) >>> print(loocp(test[0], 0.1))

__init__
(nc_measure, train=None, mondrian=False)[source]¶ Initialize cross classifier with a nonconformity measure, number of folds and training set. If present, fit the conformal classifier to the training set.
Parameters:  nc_measure (ClassNC) – Classification nonconformity measure.
 k (int) – Number of folds.
 train (Optional[Table]) – Table of examples used as a training set.
 mondrian (bool) – Use a mondrian setting for computing pvalues.

fit
(train)[source]¶ Fit the cross classifier to the training set. Split the training set into k folds for use as training and calibration set with an inductive classifier. Concatenate the computed nonconformity scores and store them (
InductiveClassifier.alpha
).Parameters: train (Table) – Table of examples used as a training set.
