hCzdZddlmZmZddlZddlmZmZddl m Z m Z ddl m Z ddlmZGd d e ZGd d ee ZGddeZGddeZGddeZGddeZGddee ZGddeZGddeZGddeZeeeededZdS)zZLosses and corresponding default initial estimators for gradient boosting decision trees. )ABCMetaabstractmethodN)expit logsumexp)DummyClassifierDummyRegressor) TREE_LEAF)_weighted_percentileceZdZdZdZdZedZed dZedZ dd Z ed Z ed Z dS) LossFunctionaMAbstract base class for various loss functions. Parameters ---------- n_classes : int Number of classes. Attributes ---------- K : int The number of regression trees to be induced; 1 for regression and binary classification; ``n_classes`` for multi-class classification. Fc||_dSN)K)self n_classess c/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/sklearn/ensemble/_gb_losses.py__init__zLossFunction.__init__!s cdS)z-Default ``init`` estimator for loss function.Nrs rinit_estimatorzLossFunction.init_estimator$rNcdS)adCompute the loss. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves). sample_weight : ndarray of shape (n_samples,), default=None Sample weights. Nrryraw_predictions sample_weights r__call__zLossFunction.__call__(rrc dS)NCompute the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. Nrrrrkargss rnegative_gradientzLossFunction.negative_gradient8rr皙?rc ~||} | } d| |<tj|jt kdD](} ||| | ||||dd| f|)|dd| fxx||jddddf| dzz cc<dS)azUpdate the terminal regions (=leaves) of the given tree and updates the current predictions of the model. Traverses tree and invokes template method `_update_terminal_region`. Parameters ---------- tree : tree.Tree The tree object. X : ndarray of shape (n_samples, n_features) The data array. y : ndarray of shape (n_samples,) The target labels. residual : ndarray of shape (n_samples,) The residuals (usually the negative gradient). raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. sample_weight : ndarray of shape (n_samples,) The weight of each sample. sample_mask : ndarray of shape (n_samples,) The sample mask to be used. learning_rate : float, default=0.1 Learning rate shrinks the contribution of each tree by ``learning_rate``. k : int, default=0 The index of the estimator being updated. rNaxis) applycopynpwhere children_leftr _update_terminal_regionvaluetake) rtreeXrresidualrr sample_mask learning_ratekterminal_regionsmasked_terminal_regionsleafs rupdate_terminal_regionsz$LossFunction.update_terminal_regionsFsR ::a==#3"7"7"9"902 -HT/9<==a@  D  ( ('1%     1AAAq!G1D1I1I 12J2 2 "  rc dS)z=Template method for updating terminal regions (i.e., leaves).Nr rr3r9r;r4rr5rrs rr0z$LossFunction._update_terminal_regionrrcdS)aLReturn the initial raw predictions. Parameters ---------- X : ndarray of shape (n_samples, n_features) The data array. estimator : object The estimator to use to compute the predictions. Returns ------- raw_predictions : ndarray of shape (n_samples, K) The initial raw predictions. K is equal to 1 for binary classification and regression, and equal to the number of classes for multiclass classification. ``raw_predictions`` is casted into float64. Nr)rr4 estimators rget_init_raw_predictionsz%LossFunction.get_init_raw_predictionss & rrr&r) __name__ __module__ __qualname____doc__is_multi_classrrrr r%r<r0rArrrr r s  N<<^<    ^     ^  , ? ? ? ? B L L^ L  ^   rr ) metaclassc.eZdZdZfdZdZdZxZS)RegressionLossFunctionz)Base class for regression loss functions.cLtddS)Nr)superr)r __class__s rrzRegressionLossFunction.__init__s$ 1%%%%%rcdt|drt|dstddS)zMake sure estimator has the required fit and predict methods. Parameters ---------- estimator : object The init estimator to check. fitpredictzNThe init parameter must be a valid estimator and support both fit and predict.Nhasattr ValueErrorrr@s rcheck_init_estimatorz+RegressionLossFunction.check_init_estimatorsH 5)) gi.K.K 0   rc||}|ddtjS)Nr(rL)rRreshapeastyper-float64)rr4r@ predictionss rrAz/RegressionLossFunction.get_init_raw_predictionss;''** ""2q))00<<> >##%%&&1/D/D/F/F+F1*LMNNO rc 0||z S)aTCompute half of the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples,) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. rgr#s rr%z#LeastSquaresError.negative_gradients?((****rr&rc ~|dd| fxx|||zz cc<dS)aLeast squares does not need to update terminal regions. But it has to update the predictions. Parameters ---------- tree : tree.Tree The tree object. X : ndarray of shape (n_samples, n_features) The data array. y : ndarray of shape (n_samples,) The target labels. residual : ndarray of shape (n_samples,) The residuals (usually the negative gradient). raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. sample_weight : ndarray of shape (n,) The weight of each sample. sample_mask : ndarray of shape (n,) The sample mask to be used. learning_rate : float, default=0.1 Learning rate shrinks the contribution of each tree by ``learning_rate``. k : int, default=0 The index of the estimator being updated. N)rRrg) rr3r4rr5rrr6r7r8s rr<z)LeastSquaresError.update_terminal_regionssJP 1a1F1F1H1H!HHrc dSrrr>s rr0z)LeastSquaresError._update_terminal_region s  rrrB) rCrDrErFrr r%r<r0rrrr`r`s///. + + +. (I(I(I(IT      rr`c,eZdZdZdZddZdZdZdS)LeastAbsoluteErrorzLoss function for least absolute deviation (LAD) regression. Parameters ---------- n_classes : int Number of classes c$tddSNquantile?rdrqrers rrz!LeastAbsoluteError.init_estimator7zC@@@@rNc (|;tj||z Sd|z tj|tj||z zzS)atCompute the least absolute error. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves). sample_weight : ndarray of shape (n_samples,), default=None Sample weights. NrL)r-absrgrbrhrs rr zLeastAbsoluteError.__call__:s  6!o3355566;;== =##%%&&O4I4I4K4K0K)L)LLMMN rc H|}d||z dkzdz S)aCompute the negative gradient. 1.0 if y - raw_predictions > 0.0 else -1.0 Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. rrrLrjr#s rr%z$LeastAbsoluteError.negative_gradientQs/*//11A'!+,q00rc tj||kd} || d}|| d|| dz } t| |d|j|ddf<dS)z1LAD updates terminal regions to median estimates.rr)2 percentileN)r-r.r2r r1) rr3r9r;r4rr5rrterminal_regiondiffs rr0z*LeastAbsoluteError._update_terminal_regionbs(#3t#;<eZdZdZd fd ZdZd dZdZdZxZ S) QuantileLossFunctionzLoss function for quantile regression. Quantile regression allows to estimate the percentiles of the conditional distribution of the target. Parameters ---------- alpha : float, default=0.9 The percentile. rcjt||_|dz|_dS)Nr)rNrrr{rs rrzQuantileLossFunction.__init__s/  #+rc.td|jS)Nrqrs)r rrs rrz#QuantileLossFunction.init_estimatorszDJGGGGrNc|}||z }|j}||k}|N|||zd|z ||zz |jdz }ng|t j||||zzd|z t j||||zzz |z }|S)aCompute the Quantile loss. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. NrLr)rgrrhrr-)rrrrr}rmaskrs rr zQuantileLossFunction.__call__s*//11?" ?"  T (((AIte9J9J+JJ DD }T2T$Z?@@@u9}dU';dD5k'I J JJK!!##$D rc b|j}|}||k}||zd|z |zz S)r"rL)rrg)rrrr$rrs rr%z&QuantileLossFunction.negative_gradient s@ )//11?" !e)u!455rc tj||kd} || d|| dz } || d}t| ||j} | |j|df<dS)Nrr))r-r.r2r r{r1) rr3r9r;r4rr5rrr|r}vals rr0z,QuantileLossFunction._update_terminal_region1s(#3t#;<666""""""""rrcDeZdZdZedZedZdZdS)ClassificationLossFunctionz-Base class for classification loss functions.cdS)aTemplate method to convert raw predictions into probabilities. Parameters ---------- raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. Returns ------- probas : ndarray of shape (n_samples, K) The predicted probabilities. Nrrrs r_raw_prediction_to_probaz3ClassificationLossFunction._raw_prediction_to_probaIrrcdS)aTemplate method to convert raw predictions to decisions. Parameters ---------- raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. Returns ------- encoded_predictions : ndarray of shape (n_samples, K) The predicted encoded labels. Nrrs r_raw_prediction_to_decisionz6ClassificationLossFunction._raw_prediction_to_decisionYrrcdt|drt|dstddS)zMake sure estimator has fit and predict_proba methods. Parameters ---------- estimator : object The init estimator to check. rQ predict_probazTThe init parameter must be a valid estimator and support both fit and predict_proba.NrSrVs rrWz/ClassificationLossFunction.check_init_estimatorisH 5)) gi.Q.Q :   rN)rCrDrErFrrrrWrrrrrFs]77   ^     ^       rrcNeZdZdZfdZdZd dZdZdZdZ d Z d Z xZ S) BinomialDevianceaBinomial deviance loss function for binary classification. Binary classification is a special case; here, we only need to fit one tree instead of ``n_classes`` trees. Parameters ---------- n_classes : int Number of classes. c|dkr-td|jj|t ddSNrz-{0:s} requires 2 classes; got {1:d} class(es)rLrMrUformatrOrCrNrrrrOs rrzBinomialDeviance.__init__^ >>?FFN+Y  1%%%%%rc"tdSNpriorrcrrs rrzBinomialDeviance.init_estimators0000rNc |}|0dtj||ztjd|z zSd|z tj|||ztjd|z zzS)aCompute the deviance (= 2 * negative log-likelihood). Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. Nr)rgr-rb logaddexprhrs rr zBinomialDeviance.__call__s *//11  _$ Q(H(HH  ##%%&&!O+r|A/O/OOQ rc J|t|z S)aPCompute half of the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. )rrgr#s rr%z"BinomialDeviance.negative_gradients#5..001111rc tj||kd} || d}|| d}|| d}tj||z} tj|||z zd|z |zz} t | dkrd|j|ddf<dS| | z |j|ddf<dS)zMake a single Newton-Raphson step. our node estimate is given by: sum(w * (y - prob)) / sum(w * prob * (1 - prob)) we take advantage that: y - prob = residual rr)rLu?j/ N)r-r.r2rhrvr1 rr3r9r;r4rr5rrr| numerator denominators rr0z(BinomialDeviance._update_terminal_regions&(#3t#;<>yQ''''rcV||}|dddf}tjtjj}tj||d|z }tj|d|z z }|ddtj S)NrLr( rr-finfofloat32epscliplogrYrZr[rr4r@probasproba_pos_classrrs rrAz)BinomialDeviance.get_init_raw_predictionss((++ A,hrz""&'/3C@@&A4G!HII&&r1--44RZ@@@rr rCrDrErFrrr r%r0rrrAr]r^s@rrrxs  &&&&&111 > 2 2 2===B (((AAAAAAArrcTeZdZdZdZfdZdZd dZddZd Z d Z d Z d Z xZ S)MultinomialDeviancezMultinomial deviance loss function for multi-class classification. For multi-class classification we need to fit ``n_classes`` trees at each stage. Parameters ---------- n_classes : int Number of classes. Tc|dkr,td|jjt |dS)Nz#{0:s} requires more than 2 classes.rrs rrzMultinomialDeviance.__init__sT q==5<rs('''''''********33333333""""""......Y Y Y Y Y WY Y Y Y x=====\W====2g g g g g .g g g TG G G G G /G G G Tr r r r r .r r r jV"V"V"V"V"1V"V"V"r/////////dzAzAzAzAzA1zAzAzAzppppp4pppfqAqAqAqAqA0qAqAqAj'( $" r