hEdZddlZddlZddlZddlZddlmZddlmZm Z ddl Z ddl m Z ddlmZddlmZmZmZmZdd lmZmZmZmZdd lmZmZmZmZdd lm Z m!Z!m"Z"m#Z#dd l$m%Z%m&Z&m'Z'dd l(m)Z)m*Z*ddl+m,Z,dZ-dddddddddd dZ.e#dgdgddgddge"hdge!eddddge!e ddddgdgddge!edddgedgdgdgdgddddddddddddddd d Z/dddddddddddd! d"Z0 d;d#Z1d$Z2dz Dictionary learning. N)ceil)IntegralReal)effective_n_jobs)linalg) BaseEstimatorClassNamePrefixFeaturesOutMixinTransformerMixin _fit_context)LarsLasso LassoLarsorthogonal_mp_gram) check_arraycheck_random_state gen_batchesgen_even_slices)HiddenInterval StrOptionsvalidate_params)randomized_svd row_normssvd_flip)Paralleldelayed)check_is_fittedcZ|r&|dvr$td|dSdS)N)omplarsz9Positive constraint not supported for '{}' coding method.) ValueErrorformatmethodpositives l/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/sklearn/decomposition/_dict_learning.py_check_positive_codingr(sG Fo-- G N Nv V V     -- lasso_larsTF gramcov algorithmregularizationcopy_covinitmax_iterverboser&c |j\} } |jd} |dkrt|| z } tjd}t |d| |d| |}||j|j||j}tjdi|n#tjdi|wxYw|dkr|t|| z }t|d||d | }|(|j d stj |}||_||j|jd |j}n/|dkr tjd}td| |t|d}||j|j||j}tjdi|n#tjdi|wxYw|dkr_tj |tjtj||z dzj}| rtj|dd |n<|dkr6t#||t|d t%|d |j}|| | S)a Generic sparse coding with precomputed Gram and/or covariance matrices. Each row of the result is the solution to a Lasso problem. Parameters ---------- X : ndarray of shape (n_samples, n_features) Data matrix. dictionary : ndarray of shape (n_components, n_features) The dictionary matrix against which to solve the sparse coding of the data. Some of the algorithms assume normalized rows. gram : ndarray of shape (n_components, n_components), default=None Precomputed Gram matrix, `dictionary * dictionary'` gram can be `None` if method is 'threshold'. cov : ndarray of shape (n_components, n_samples), default=None Precomputed covariance, `dictionary * X'`. algorithm : {'lasso_lars', 'lasso_cd', 'lars', 'omp', 'threshold'}, default='lasso_lars' The algorithm used: * `'lars'`: uses the least angle regression method (`linear_model.lars_path`); * `'lasso_lars'`: uses Lars to compute the Lasso solution; * `'lasso_cd'`: uses the coordinate descent method to compute the Lasso solution (`linear_model.Lasso`). lasso_lars will be faster if the estimated components are sparse; * `'omp'`: uses orthogonal matching pursuit to estimate the sparse solution; * `'threshold'`: squashes to zero all coefficients less than regularization from the projection `dictionary * data'`. regularization : int or float, default=None The regularization parameter. It corresponds to alpha when algorithm is `'lasso_lars'`, `'lasso_cd'` or `'threshold'`. Otherwise it corresponds to `n_nonzero_coefs`. init : ndarray of shape (n_samples, n_components), default=None Initialization value of the sparse code. Only used if `algorithm='lasso_cd'`. max_iter : int, default=1000 Maximum number of iterations to perform if `algorithm='lasso_cd'` or `'lasso_lars'`. copy_cov : bool, default=True Whether to copy the precomputed covariance matrix; if `False`, it may be overwritten. verbose : int, default=0 Controls the verbosity; the higher, the more messages. positive: bool, default=False Whether to enforce a positivity constraint on the sparse code. .. versionadded:: 0.20 Returns ------- code : ndarray of shape (n_components, n_features) The sparse codes. rr*ignore)allF)alpha fit_interceptr4 precomputefit_pathr&r3)Xylasso_cdT)r8r9r:r3 warm_startr&N WRITEABLE) check_inputr!)r9r4r:n_nonzero_coefsr; thresholdoutr )squared)Gramr<rAtol norms_squaredcopy_Xy)shapefloatnpseterrrfitTcoef_rflagsarrayr intsignmaximumabscliprrreshape)X dictionaryr-r.r/r0r1r2r3r4r& n_samples n_features n_componentsr8err_mgtr*new_codeclfr!s r'_sparse_encode_precomputedrb&s^GIz#A&LL  n%% 2 !iH---G##!!J NN:<N 5 5 5!'H I  BI  j n%% 2        :k* &x~~CI  acu5559 f   !iH---G# #N 3 3 D HHZ\133H / / /zH I  BI  k ! !GCLL2:bfSkkN.JA#N#NNQ  5 GHa8 4 4 4 4 e  %//#At444         I| 4 44sABB+7AF((F;z array-like>r r!r=rBr*leftclosedbooleanr4rZr[r-r.r/rAr8r1r2r3n_jobsr@r4r&prefer_skip_nested_validation) r-r.r/rAr8r1r2r3rir@r4r&c | ru|dkrQt|dtjtjg}t|dtjtjg}nt|}t|}|jd|jdkr-t d|j|jt|| t|||||||||| | | |  S)aFSparse coding. Each row of the result is the solution to a sparse coding problem. The goal is to find a sparse array `code` such that:: X ~= code * dictionary Read more in the :ref:`User Guide `. Parameters ---------- X : array-like of shape (n_samples, n_features) Data matrix. dictionary : array-like of shape (n_components, n_features) The dictionary matrix against which to solve the sparse coding of the data. Some of the algorithms assume normalized rows for meaningful output. gram : array-like of shape (n_components, n_components), default=None Precomputed Gram matrix, `dictionary * dictionary'`. cov : array-like of shape (n_components, n_samples), default=None Precomputed covariance, `dictionary' * X`. algorithm : {'lasso_lars', 'lasso_cd', 'lars', 'omp', 'threshold'}, default='lasso_lars' The algorithm used: * `'lars'`: uses the least angle regression method (`linear_model.lars_path`); * `'lasso_lars'`: uses Lars to compute the Lasso solution; * `'lasso_cd'`: uses the coordinate descent method to compute the Lasso solution (`linear_model.Lasso`). lasso_lars will be faster if the estimated components are sparse; * `'omp'`: uses orthogonal matching pursuit to estimate the sparse solution; * `'threshold'`: squashes to zero all coefficients less than regularization from the projection `dictionary * data'`. n_nonzero_coefs : int, default=None Number of nonzero coefficients to target in each column of the solution. This is only used by `algorithm='lars'` and `algorithm='omp'` and is overridden by `alpha` in the `omp` case. If `None`, then `n_nonzero_coefs=int(n_features / 10)`. alpha : float, default=None If `algorithm='lasso_lars'` or `algorithm='lasso_cd'`, `alpha` is the penalty applied to the L1 norm. If `algorithm='threshold'`, `alpha` is the absolute value of the threshold below which coefficients will be squashed to zero. If `algorithm='omp'`, `alpha` is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides `n_nonzero_coefs`. If `None`, default to 1. copy_cov : bool, default=True Whether to copy the precomputed covariance matrix; if `False`, it may be overwritten. init : ndarray of shape (n_samples, n_components), default=None Initialization value of the sparse codes. Only used if `algorithm='lasso_cd'`. max_iter : int, default=1000 Maximum number of iterations to perform if `algorithm='lasso_cd'` or `'lasso_lars'`. n_jobs : int, default=None Number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. check_input : bool, default=True If `False`, the input arrays X and dictionary will not be checked. verbose : int, default=0 Controls the verbosity; the higher, the more messages. positive : bool, default=False Whether to enforce positivity when finding the encoding. .. versionadded:: 0.20 Returns ------- code : ndarray of shape (n_samples, n_components) The sparse codes. See Also -------- sklearn.linear_model.lars_path : Compute Least Angle Regression or Lasso path using LARS algorithm. sklearn.linear_model.orthogonal_mp : Solves Orthogonal Matching Pursuit problems. sklearn.linear_model.Lasso : Train Linear Model with L1 prior as regularizer. SparseCoder : Find a sparse representation of data from a fixed precomputed dictionary. r=C)orderdtyperczRDictionary and X have different numbers of features:dictionary.shape: {} X.shape{} r-r.r/rAr8r1r2r3rir4r&) rrMfloat64float32rKr"r#r(_sparse_encoderhs r' sparse_encoderts T  " "$#bj"*-EJASRZ0HIIIAA$Z00JAAagaj(( --3VJ4Dag-N-N   9h///    '    r)rpc  j\} }jd}dvr&|!tt|dz d|n|d dkrtjj"dkrd tjjt | dksdkrt }|Sjd} jd}tj| |f}tt| t | }t|   f d |D}t||D] \}}|||< |S) z1Sparse coding without input/parameter validation.r)r!r N rcg?rBr=Fr,)rir4c3 K|]J}tt| dd|fnd |nd  VKdS)Nr,)rrb) .0 this_slicerZr/r1r.r[r-r2r3r&r0r4s r' z!_sparse_encode..s::  ,*++ jM &)oAAAzM""4)%)%5j!!4 ::::::r)) rKminmaxrMdotrPrrbemptylistrrzip)rZr[r-r.r/rAr8r1r2r3rir4r&r\r]r^codeslices code_viewsry this_viewr0s````` ``` `` @r'rsrszs$GIz#A&LO##(  ! Z"_a!8!8,GGN  ! N | [00vj*,// {yJ..fZ%%1$$ [(@(@) )      I#A&L 8Y - . .D /)-=f-E-EFF G GF9999::::::::::::::!:::J "%VZ!8!8%% I$Z Kr)c |j\}} t|}| |j|z}| |j|z}d} t| D]} || | fdkr2|| xx|dd| f|| |zz || | fz z cc<no|||} d| pdz} |d| t| }| |z|| <d|dd| f<| dz } |r$tj || dd|| || xxttj || dzcc<|r| dkrt| ddSdSdS) aUpdate the dense dictionary factor in place. Parameters ---------- dictionary : ndarray of shape (n_components, n_features) Value of the dictionary at the previous iteration. Y : ndarray of shape (n_samples, n_features) Data matrix. code : ndarray of shape (n_samples, n_components) Sparse coding of the data against which to optimize the dictionary. A : ndarray of shape (n_components, n_components), default=None Together with `B`, sufficient stats of the online model to update the dictionary. B : ndarray of shape (n_features, n_components), default=None Together with `A`, sufficient stats of the online model to update the dictionary. verbose: bool, default=False Degree of output the procedure will print. random_state : int, RandomState instance or None, default=None Used for randomly initializing the dictionary. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. positive : bool, default=False Whether to enforce positivity when finding the dictionary. .. versionadded:: 0.20 Nrgư>g{Gz?rc)sizerCz unused atoms resampled.)rKrrPrangechoicestdnormallenrMrXr|rnormprint)r[YrABr4 random_stater&r\r^n_unusedknewd noise_levelnoises r' _update_dictrsX#jI|%l33Ly FTMy C$JH < << QT7T>> qMMMa1g!z(99Qq!tWD DMMMM\((334D$((**/2K '';SYY'GGE 5LJqMDAJ MH  ? GJqM1d 1 > > > > 1 V[A77;;; 58a<< 3334444455<t jd t jn| rtd |||d z |fzt)|||||||||  }t+|||| | | dtj|||zz dzz|tjtj|zz}|||dkrL|d|d z }|||d zkr,| dkrtdn| rtd|zn$|dzdkr| | t3z| r ||||dzfS|||fS)z"Main dictionary learning algorithmNF)rnF) full_matricesrc[dict_learning] end.zCIteration % 3i (elapsed time: % 3is, % 4.1fmn, current cost % 7.3f)<)r/r8r2rir&r3r4r4rr&?rrz+--- Convergence reached after %d iterations)timerMrSrsvdrnewaxisrc_zerosr_rKasfortranarraynanrrsysstdoutwriteflushrtrsumrWappendlocals)rZr^r8r3rGr%ri dict_init code_initcallbackr4r return_n_iter positive_dict positive_codemethod_max_itert0rr[Srerrors current_costiidtdEs r'_dict_learningrs`( B!6x --- $j%@@@a#D*55jqqq"*}% 2  JAqAAA} }$%  qqq 01 uT28SYY q0@$ABBBCU ,"2J4DQ4G!HII I ":..J F6L!|| S)))) BHoo33 Y[[2  a<< J  S ! ! ! J         Ur27L12     "$       %"     RVQ ):%:q$@AAAEBF F4LLM M E    l### 66fRj(BC&*$$$a<<"IIIING"LMMM 6Q;;8/ HVXX   (Za//Z''r)cj|dkr,d|d}|r|d|z }tj|t|S|S)N deprecated'zB' is deprecated in version 1.1 and will be removed in version 1.4.r)warningswarn FutureWarning)paramnamedefaultadditional_messagemsgs r'_check_warn_deprecatedrsZ  X X X X   , +)++ +C c=))) r)rr!MbP?rv)r8n_iterr3 return_coderr batch_sizer4shufflerir% iter_offsetrreturn_inner_stats inner_statsrrrrrGmax_no_improvementc , ||| |f}|(td|Dstdt| dd} t|ddd }t|d d}t|d dd }|d| z}td;id|d|d|d| d| d|d| d|d|d|d|d|d|d|d| d|d|d ||}|s|jS||}||jfSt|dd!d" }| |jd#}| d$vrtd%t| |d| z} tj }|j\}}t|}t|}||}n/t|||&\}} }| ddtjf|z}t|}!||!kr|d|ddf}n=tj|tj||!z |jd#f|j'f}| d#krt'd(d)*| r*|}"||"n|}"t-|"d+tjtjgd,}"t-|d-|"jd,}tj|d./}t5||}#t7j|#}#|;tj||f|"j'}$tj||f|"j'}%n4|d}$|d#}%| d#z }&t;t=| | |z|#D]u\}&}'|"|'}(tj |z })| d#kr>t>j !d0t>j "n9| r7| d1ks|&tGd2| z zdkrt'd3|&|)|)d4z fztI|(|| || d||| 5 }*|&|d#z krt|&d#z|z}+nt|d6z|&zd#z|z }+|+d#z|z |+d#zz },|$|,z}$|$tj%|*j&|*z }$|%|,z}%|%tj%|(j&|*z }%tO||(|*|$|%| ||7||tQw|r|r ||$|%f|&| z d#zfS||$|%ffS|r| d#krt'd8d)*n| d#krt'd9d)*tI||| || d||| 5 }| d#kr-tj |z })t'd:|)|)d4z fz|r |||&| z d#zfS||fS|r ||&| z d#zfS|S)`. Parameters ---------- X : ndarray of shape (n_samples, n_features) Data matrix. n_components : int or None, default=2 Number of dictionary atoms to extract. If None, then ``n_components`` is set to ``n_features``. alpha : float, default=1 Sparsity controlling parameter. n_iter : int, default=100 Number of mini-batch iterations to perform. .. deprecated:: 1.1 `n_iter` is deprecated in 1.1 and will be removed in 1.4. Use `max_iter` instead. max_iter : int, default=None Maximum number of iterations over the complete dataset before stopping independently of any early stopping criterion heuristics. If ``max_iter`` is not None, ``n_iter`` is ignored. .. versionadded:: 1.1 return_code : bool, default=True Whether to also return the code U or just the dictionary `V`. dict_init : ndarray of shape (n_components, n_features), default=None Initial values for the dictionary for warm restart scenarios. If `None`, the initial values for the dictionary are created with an SVD decomposition of the data via :func:`~sklearn.utils.extmath.randomized_svd`. callback : callable, default=None A callable that gets invoked at the end of each iteration. batch_size : int, default=256 The number of samples to take in each batch. .. versionchanged:: 1.3 The default value of `batch_size` changed from 3 to 256 in version 1.3. verbose : bool, default=False To control the verbosity of the procedure. shuffle : bool, default=True Whether to shuffle the data before splitting it in batches. n_jobs : int, default=None Number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. method : {'lars', 'cd'}, default='lars' * `'lars'`: uses the least angle regression method to solve the lasso problem (`linear_model.lars_path`); * `'cd'`: uses the coordinate descent method to compute the Lasso solution (`linear_model.Lasso`). Lars will be faster if the estimated components are sparse. iter_offset : int, default=0 Number of previous iterations completed on the dictionary used for initialization. .. deprecated:: 1.1 `iter_offset` serves internal purpose only and will be removed in 1.4. random_state : int, RandomState instance or None, default=None Used for initializing the dictionary when ``dict_init`` is not specified, randomly shuffling the data when ``shuffle`` is set to ``True``, and updating the dictionary. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. return_inner_stats : bool, default=False Return the inner statistics A (dictionary covariance) and B (data approximation). Useful to restart the algorithm in an online setting. If `return_inner_stats` is `True`, `return_code` is ignored. .. deprecated:: 1.1 `return_inner_stats` serves internal purpose only and will be removed in 1.4. inner_stats : tuple of (A, B) ndarrays, default=None Inner sufficient statistics that are kept by the algorithm. Passing them at initialization is useful in online settings, to avoid losing the history of the evolution. `A` `(n_components, n_components)` is the dictionary covariance matrix. `B` `(n_features, n_components)` is the data approximation matrix. .. deprecated:: 1.1 `inner_stats` serves internal purpose only and will be removed in 1.4. return_n_iter : bool, default=False Whether or not to return the number of iterations. .. deprecated:: 1.1 `return_n_iter` will be removed in 1.4 and n_iter will never be returned. positive_dict : bool, default=False Whether to enforce positivity when finding the dictionary. .. versionadded:: 0.20 positive_code : bool, default=False Whether to enforce positivity when finding the code. .. versionadded:: 0.20 method_max_iter : int, default=1000 Maximum number of iterations to perform when solving the lasso problem. .. versionadded:: 0.22 tol : float, default=1e-3 Control early stopping based on the norm of the differences in the dictionary between 2 steps. Used only if `max_iter` is not None. To disable early stopping based on changes in the dictionary, set `tol` to 0.0. .. versionadded:: 1.1 max_no_improvement : int, default=10 Control early stopping based on the consecutive number of mini batches that does not yield an improvement on the smoothed cost function. Used only if `max_iter` is not None. To disable convergence detection based on cost function, set `max_no_improvement` to None. .. versionadded:: 1.1 Returns ------- code : ndarray of shape (n_samples, n_components), The sparse code (only returned if `return_code=True`). dictionary : ndarray of shape (n_components, n_features), The solutions to the dictionary learning problem. n_iter : int Number of iterations run. Returned only if `return_n_iter` is set to `True`. See Also -------- dict_learning : Solve a dictionary learning matrix factorization problem. DictionaryLearning : Find a dictionary that sparsely encodes data. MiniBatchDictionaryLearning : A faster, less accurate, version of the dictionary learning algorithm. SparsePCA : Sparse Principal Components Analysis. MiniBatchSparsePCA : Mini-batch Sparse Principal Components Analysis. Nc3"K|] }|dkV dS)rNrJ)rxargs r'rzz'dict_learning_online..as''L'L|(;'L'L'L'L'L'Lr)zuThe following arguments are incompatible with 'max_iter': return_n_iter, return_inner_stats, iter_offset, inner_statsrr)rrFz,From 1.4 inner_stats will never be returned.)rrrrzFrom 1.4 'n_iter' will never be returned. Refer to the 'n_iter_' and 'n_steps_' attributes of the MiniBatchDictionaryLearning object instead.lasso_r^r8rri fit_algorithmrrrrtransform_algorithmtransform_alpharrtransform_max_iterr4rrGrdzUse 'max_iter' instead.rc)r!cdz/Coding method not supported as a fit algorithm.rrorrrrmrnrocopyrW requirementsrrvY@z.Iteration % 3i (elapsed time: % 3is, % 4.1fmn)r)r/r8rir@r&r3r4rrzLearning code...|z"done (total time: % 3is, % 4.1fmn)rJ))r7r"rMiniBatchDictionaryLearningrO components_ transformrKr(rrLrrrMrrrrrorrrrrqrrrequirer itertoolscyclerrrrrrrrtr}rPrr)-rZr^r8rr3rrrrr4rrir%rrrrrrrrrGrdepsrestrrr\r]r[_rrX_trainbatchesrrrbatchthis_Xr this_codethetabetas- r'dict_learning_onlinersP -{K HDC'L'Lt'L'L'L$L$L J   )mQOOOK/I  )mTRRRK* W M&/)   % % 6 6  !&  "z  G  i & !4 3 "E (- (-  / G X! "# $ 21% & #a&&' * )? "==##D( ( $#:SFwqz  ^##JKKK6=111  F BGIz %LLE%l33L )!\ UUU1jqqq"*}% 2  JAq  qqq 01 U  HlQ& (8(;>DBPRUWPWGXX"  "$      Q  26Z/00EE*a-",q0:=>>E J&5195 T  RVIK + ++ T  RVFHi ( ((     %"    HVXX   &  &1vrK'7!';; ;1v% %$ Q;; $# . . . . . \\ #3     "$     Q;;r!B 6"b2gF G G G  $R+%5%99 9# #2 +a///r)r)rZr%rrr:0yE>) r3rGr%rirrrr4rrrrrct|||||||| || | || |}||}| r||j|j|jfS||j|jfS)a0Solve a dictionary learning matrix factorization problem. Finds the best dictionary and the corresponding sparse code for approximating the data matrix X by solving:: (U^*, V^*) = argmin 0.5 || X - U V ||_Fro^2 + alpha * || U ||_1,1 (U,V) with || V_k ||_2 = 1 for all 0 <= k < n_components where V is the dictionary and U is the sparse code. ||.||_Fro stands for the Frobenius norm and ||.||_1,1 stands for the entry-wise matrix norm which is the sum of the absolute values of all the entries in the matrix. Read more in the :ref:`User Guide `. Parameters ---------- X : array-like of shape (n_samples, n_features) Data matrix. n_components : int Number of dictionary atoms to extract. alpha : int or float Sparsity controlling parameter. max_iter : int, default=100 Maximum number of iterations to perform. tol : float, default=1e-8 Tolerance for the stopping condition. method : {'lars', 'cd'}, default='lars' The method used: * `'lars'`: uses the least angle regression method to solve the lasso problem (`linear_model.lars_path`); * `'cd'`: uses the coordinate descent method to compute the Lasso solution (`linear_model.Lasso`). Lars will be faster if the estimated components are sparse. n_jobs : int, default=None Number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. dict_init : ndarray of shape (n_components, n_features), default=None Initial value for the dictionary for warm restart scenarios. Only used if `code_init` and `dict_init` are not None. code_init : ndarray of shape (n_samples, n_components), default=None Initial value for the sparse code for warm restart scenarios. Only used if `code_init` and `dict_init` are not None. callback : callable, default=None Callable that gets invoked every five iterations. verbose : bool, default=False To control the verbosity of the procedure. random_state : int, RandomState instance or None, default=None Used for randomly initializing the dictionary. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. return_n_iter : bool, default=False Whether or not to return the number of iterations. positive_dict : bool, default=False Whether to enforce positivity when finding the dictionary. .. versionadded:: 0.20 positive_code : bool, default=False Whether to enforce positivity when finding the code. .. versionadded:: 0.20 method_max_iter : int, default=1000 Maximum number of iterations to perform. .. versionadded:: 0.22 Returns ------- code : ndarray of shape (n_samples, n_components) The sparse code factor in the matrix factorization. dictionary : ndarray of shape (n_components, n_features), The dictionary factor in the matrix factorization. errors : array Vector of errors at each iteration. n_iter : int Number of iterations run. Returned only if `return_n_iter` is set to True. See Also -------- dict_learning_online : Solve a dictionary learning matrix factorization problem online. DictionaryLearning : Find a dictionary that sparsely encodes data. MiniBatchDictionaryLearning : A faster, less accurate version of the dictionary learning algorithm. SparsePCA : Sparse Principal Components Analysis. MiniBatchSparsePCA : Mini-batch Sparse Principal Components Analysis. )r^r8r3rGrrirrrr4rrrr)DictionaryLearning fit_transformrerror_n_iter_)rZr^r8r3rGr%rirrrr4rrrrr estimatorrs r' dict_learningr6sR#! !##*I  " "1 % %D   !       & (8 88r)c$eZdZdZdZdZdZdS)_BaseSparseCodingz>Base class from SparseCoder and DictionaryLearning algorithms.ch||_||_||_||_||_||_||_dSN)rtransform_n_nonzero_coefsrr split_signrir)selfrr rr rirrs r'__init__z_BaseSparseCoding.__init__s@$7 )B&."4$ *r)c ||d}t|dr|j|j}n|j}t |||j|j||j|j|j }|j rb|j \}}tj |d|zf}tj|d|ddd|f<tj|d |dd|df<|}|S)zWPrivate method allowing to accommodate both DictionaryLearning and SparseCoder.F)resetr8N)r/rAr8r3rir&rr)_validate_datahasattrrr8rtrr rrirr rKrMr~rVminimum)rrZr[rrr\r] split_codes r' _transformz_BaseSparseCoding._transforms     / / 4 ! ! 3d&:&B"jOO"2O . :!,;'     ? $(J !Iz9a*n"=>>J)+D!)<)44444r)r ceZdZdZdgZddddddddfd Zdd Zdfd Zd Ze d Z e d Z e dZ xZ S) SparseCoderaSparse coding. Finds a sparse representation of data against a fixed, precomputed dictionary. Each row of the result is the solution to a sparse coding problem. The goal is to find a sparse array `code` such that:: X ~= code * dictionary Read more in the :ref:`User Guide `. Parameters ---------- dictionary : ndarray of shape (n_components, n_features) The dictionary atoms used for sparse coding. Lines are assumed to be normalized to unit norm. transform_algorithm : {'lasso_lars', 'lasso_cd', 'lars', 'omp', 'threshold'}, default='omp' Algorithm used to transform the data: - `'lars'`: uses the least angle regression method (`linear_model.lars_path`); - `'lasso_lars'`: uses Lars to compute the Lasso solution; - `'lasso_cd'`: uses the coordinate descent method to compute the Lasso solution (linear_model.Lasso). `'lasso_lars'` will be faster if the estimated components are sparse; - `'omp'`: uses orthogonal matching pursuit to estimate the sparse solution; - `'threshold'`: squashes to zero all coefficients less than alpha from the projection ``dictionary * X'``. transform_n_nonzero_coefs : int, default=None Number of nonzero coefficients to target in each column of the solution. This is only used by `algorithm='lars'` and `algorithm='omp'` and is overridden by `alpha` in the `omp` case. If `None`, then `transform_n_nonzero_coefs=int(n_features / 10)`. transform_alpha : float, default=None If `algorithm='lasso_lars'` or `algorithm='lasso_cd'`, `alpha` is the penalty applied to the L1 norm. If `algorithm='threshold'`, `alpha` is the absolute value of the threshold below which coefficients will be squashed to zero. If `algorithm='omp'`, `alpha` is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides `n_nonzero_coefs`. If `None`, default to 1. split_sign : bool, default=False Whether to split the sparse feature vector into the concatenation of its negative part and its positive part. This can improve the performance of downstream classifiers. n_jobs : int, default=None Number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. positive_code : bool, default=False Whether to enforce positivity when finding the code. .. versionadded:: 0.20 transform_max_iter : int, default=1000 Maximum number of iterations to perform if `algorithm='lasso_cd'` or `lasso_lars`. .. versionadded:: 0.22 Attributes ---------- n_components_ : int Number of atoms. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- DictionaryLearning : Find a dictionary that sparsely encodes data. MiniBatchDictionaryLearning : A faster, less accurate, version of the dictionary learning algorithm. MiniBatchSparsePCA : Mini-batch Sparse Principal Components Analysis. SparsePCA : Sparse Principal Components Analysis. sparse_encode : Sparse coding where each row of the result is the solution to a sparse coding problem. Examples -------- >>> import numpy as np >>> from sklearn.decomposition import SparseCoder >>> X = np.array([[-1, -1, -1], [0, 0, 3]]) >>> dictionary = np.array( ... [[0, 1, 0], ... [-1, -1, 2], ... [1, 1, 1], ... [0, 1, 1], ... [0, 2, 1]], ... dtype=np.float64 ... ) >>> coder = SparseCoder( ... dictionary=dictionary, transform_algorithm='lasso_lars', ... transform_alpha=1e-10, ... ) >>> coder.transform(X) array([[ 0., 0., -1., 0., 0.], [ 0., 1., 1., 0., 0.]]) r[r NFr+)rr rr rirrc dt|||||||||_dSr )superrr[) rr[rr rr rirr __class__s r'rzSparseCoder.__init__sC   %        %r)c|S)aDo nothing and return the estimator unchanged. This method is just there to implement the usual API and hence work in pipelines. Parameters ---------- X : Ignored Not used, present for API consistency by convention. y : Ignored Not used, present for API consistency by convention. Returns ------- self : object Returns the instance itself. rJrrZys r'rOzSparseCoder.fits & r)cRt||jS)aQEncode the data as a sparse combination of the dictionary atoms. Coding method is determined by the object parameter `transform_algorithm`. Parameters ---------- X : ndarray of shape (n_samples, n_features) Training vector, where `n_samples` is the number of samples and `n_features` is the number of features. y : Ignored Not used, present for API consistency by convention. Returns ------- X_new : ndarray of shape (n_samples, n_components) Transformed data. )r rr[)rrZr$r!s r'rzSparseCoder.transforms!(ww!!!T_555r)c8dtjtjgdS)NF) requires_fitpreserves_dtyperMrqrrrs r' _more_tagszSparseCoder._more_tagss ! " BJ7   r)c&|jjdS)zNumber of atoms.rr[rKr*s r' n_components_zSparseCoder.n_components_$Q''r)c&|jjdS)z%Number of features seen during `fit`.rcr-r*s r'n_features_in_zSparseCoder.n_features_in_r/r)c|jS)&Number of transformed output features.)r.r*s r'_n_features_outzSparseCoder._n_features_outs !!r)r )rrrr_required_parametersrrOrr+propertyr.r1r4 __classcell__r!s@r'rr#suun)> ""&%%%%%%%.*666666,   ((X(((X(""X"""""r)rcBeZdZUdZideeddddgdeedddgd eedddgd eedddgd ed d hgdehdgdeeddddgdeeddddgdedgdej dgdej dgde dgddgddgddgddgddgdeedddgiZ e e d< d*dddd d ddddddd!d!dd!d!dd"fd# Zd*d$Zed%&d*d'Zed(Zd)ZxZS)+raDictionary learning. Finds a dictionary (a set of atoms) that performs well at sparsely encoding the fitted data. Solves the optimization problem:: (U^*,V^*) = argmin 0.5 || X - U V ||_Fro^2 + alpha * || U ||_1,1 (U,V) with || V_k ||_2 <= 1 for all 0 <= k < n_components ||.||_Fro stands for the Frobenius norm and ||.||_1,1 stands for the entry-wise matrix norm which is the sum of the absolute values of all the entries in the matrix. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=None Number of dictionary elements to extract. If None, then ``n_components`` is set to ``n_features``. alpha : float, default=1.0 Sparsity controlling parameter. max_iter : int, default=1000 Maximum number of iterations to perform. tol : float, default=1e-8 Tolerance for numerical error. fit_algorithm : {'lars', 'cd'}, default='lars' * `'lars'`: uses the least angle regression method to solve the lasso problem (:func:`~sklearn.linear_model.lars_path`); * `'cd'`: uses the coordinate descent method to compute the Lasso solution (:class:`~sklearn.linear_model.Lasso`). Lars will be faster if the estimated components are sparse. .. versionadded:: 0.17 *cd* coordinate descent method to improve speed. transform_algorithm : {'lasso_lars', 'lasso_cd', 'lars', 'omp', 'threshold'}, default='omp' Algorithm used to transform the data: - `'lars'`: uses the least angle regression method (:func:`~sklearn.linear_model.lars_path`); - `'lasso_lars'`: uses Lars to compute the Lasso solution. - `'lasso_cd'`: uses the coordinate descent method to compute the Lasso solution (:class:`~sklearn.linear_model.Lasso`). `'lasso_lars'` will be faster if the estimated components are sparse. - `'omp'`: uses orthogonal matching pursuit to estimate the sparse solution. - `'threshold'`: squashes to zero all coefficients less than alpha from the projection ``dictionary * X'``. .. versionadded:: 0.17 *lasso_cd* coordinate descent method to improve speed. transform_n_nonzero_coefs : int, default=None Number of nonzero coefficients to target in each column of the solution. This is only used by `algorithm='lars'` and `algorithm='omp'`. If `None`, then `transform_n_nonzero_coefs=int(n_features / 10)`. transform_alpha : float, default=None If `algorithm='lasso_lars'` or `algorithm='lasso_cd'`, `alpha` is the penalty applied to the L1 norm. If `algorithm='threshold'`, `alpha` is the absolute value of the threshold below which coefficients will be squashed to zero. If `None`, defaults to `alpha`. .. versionchanged:: 1.2 When None, default value changed from 1.0 to `alpha`. n_jobs : int or None, default=None Number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. code_init : ndarray of shape (n_samples, n_components), default=None Initial value for the code, for warm restart. Only used if `code_init` and `dict_init` are not None. dict_init : ndarray of shape (n_components, n_features), default=None Initial values for the dictionary, for warm restart. Only used if `code_init` and `dict_init` are not None. callback : callable, default=None Callable that gets invoked every five iterations. .. versionadded:: 1.3 verbose : bool, default=False To control the verbosity of the procedure. split_sign : bool, default=False Whether to split the sparse feature vector into the concatenation of its negative part and its positive part. This can improve the performance of downstream classifiers. random_state : int, RandomState instance or None, default=None Used for initializing the dictionary when ``dict_init`` is not specified, randomly shuffling the data when ``shuffle`` is set to ``True``, and updating the dictionary. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. positive_code : bool, default=False Whether to enforce positivity when finding the code. .. versionadded:: 0.20 positive_dict : bool, default=False Whether to enforce positivity when finding the dictionary. .. versionadded:: 0.20 transform_max_iter : int, default=1000 Maximum number of iterations to perform if `algorithm='lasso_cd'` or `'lasso_lars'`. .. versionadded:: 0.22 Attributes ---------- components_ : ndarray of shape (n_components, n_features) dictionary atoms extracted from the data error_ : array vector of errors at each iteration n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_iter_ : int Number of iterations run. See Also -------- MiniBatchDictionaryLearning: A faster, less accurate, version of the dictionary learning algorithm. MiniBatchSparsePCA : Mini-batch Sparse Principal Components Analysis. SparseCoder : Find a sparse representation of data from a fixed, precomputed dictionary. SparsePCA : Sparse Principal Components Analysis. References ---------- J. Mairal, F. Bach, J. Ponce, G. Sapiro, 2009: Online dictionary learning for sparse coding (https://www.di.ens.fr/sierra/pdfs/icml09.pdf) Examples -------- >>> import numpy as np >>> from sklearn.datasets import make_sparse_coded_signal >>> from sklearn.decomposition import DictionaryLearning >>> X, dictionary, code = make_sparse_coded_signal( ... n_samples=30, n_components=15, n_features=20, n_nonzero_coefs=10, ... random_state=42, ... ) >>> dict_learner = DictionaryLearning( ... n_components=15, transform_algorithm='lasso_lars', transform_alpha=0.1, ... random_state=42, ... ) >>> X_transformed = dict_learner.fit(X).transform(X) We can check the level of sparsity of `X_transformed`: >>> np.mean(X_transformed == 0) 0.52... We can compare the average squared euclidean norm of the reconstruction error of the sparse coded signal relative to the squared euclidean norm of the original signal: >>> X_hat = X_transformed @ dict_learner.components_ >>> np.mean(np.sum((X_hat - X) ** 2, axis=1) / np.sum(X ** 2, axis=1)) 0.05... r^rcNrdrer8rr3rGrr!rr>r r!r=rBr*r rrirrrr4r rgrrrr_parameter_constraintsr+rr F)r8r3rGrrr rrirrrr4r rrrrc t||||| ||||_||_||_||_||_| |_| |_| |_ | |_ ||_ ||_ dSr ) r rr^r8r3rGrrrrr4rr)rr^r8r3rGrrr rrirrrr4r rrrrr!s r'rzDictionaryLearning.__init__s,   %        )   *""   (*r)c0|||S)Fit the model from data in X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vector, where `n_samples` is the number of samples and `n_features` is the number of features. y : Ignored Not used, present for API consistency by convention. Returns ------- self : object Returns the instance itself. )rr#s r'rOzDictionaryLearning.fits" 1 r)Trjct|j|jd|jz}t|j}||}|j|jd}n|j}t|||j |j |j ||j |j |j|j|j|j|d|j|j\}}}|_||_||_|S)aFit the model from data in X and return the transformed data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vector, where `n_samples` is the number of samples and `n_features` is the number of features. y : Ignored Not used, present for API consistency by convention. Returns ------- V : ndarray of shape (n_samples, n_components) Transformed data. r$rNrcT)r8rGr3r%rrirrrr4rrrr)r(rrrrrr^rKrr8rGr3rrirrrr4rrrr) rrZr$r%rr^VUEs r'rz DictionaryLearning.fit_transform s$ d&84CUVVVVD..)$*;<<    " "   $71:LL,L . *] 3;nn]L%,,!! ! ! 1a$ r)c&|jjdSr3rrrKr*s r'r4z"DictionaryLearning._n_features_outA%a((r)c6dtjtjgiSNr(r)r*s r'r+zDictionaryLearning._more_tagsF  BJ7  r)r )rrrrrrrrrMndarraycallabler:dict__annotations__rrOr rr6r4r+r7r8s@r'rrs~~@$(AtFCCCTJ$((4D8889$ XXh4???@$ q$v6667 $ **fd^445 $  JMMM N N $ $hhxD&P&P&PRV%W$ HHT1d6BBBDI$ 8T"$ bj$'$ bj$'$ Xt$$ I;$ yk!$" (#$$ )%$& )'$( xx!T&IIIJ)$$D2)+ !"&))+)+)+)+)+)+)+V(\555222652h))X)       r)rceZdZUdZideeddddgdeedddgd eedddeed hgd eeddddgd ed dhgddegdeedddgddgdde j gdehdgdeeddddgdeeddddgddgddgddgddgddgeedddgde geedddgeeddddgdZ e ed< d2dd ddddd dd!ddd"d"dd"d"d#dd$d%d&fd' Zd(Zd)Zd*Zd+Zd,Zed -d2d.Zed -d2d/Zed0Zd1ZxZS)3ra'Mini-batch dictionary learning. Finds a dictionary (a set of atoms) that performs well at sparsely encoding the fitted data. Solves the optimization problem:: (U^*,V^*) = argmin 0.5 || X - U V ||_Fro^2 + alpha * || U ||_1,1 (U,V) with || V_k ||_2 <= 1 for all 0 <= k < n_components ||.||_Fro stands for the Frobenius norm and ||.||_1,1 stands for the entry-wise matrix norm which is the sum of the absolute values of all the entries in the matrix. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=None Number of dictionary elements to extract. alpha : float, default=1 Sparsity controlling parameter. n_iter : int, default=1000 Total number of iterations over data batches to perform. .. deprecated:: 1.1 ``n_iter`` is deprecated in 1.1 and will be removed in 1.4. Use ``max_iter`` instead. max_iter : int, default=None Maximum number of iterations over the complete dataset before stopping independently of any early stopping criterion heuristics. If ``max_iter`` is not None, ``n_iter`` is ignored. .. versionadded:: 1.1 fit_algorithm : {'lars', 'cd'}, default='lars' The algorithm used: - `'lars'`: uses the least angle regression method to solve the lasso problem (`linear_model.lars_path`) - `'cd'`: uses the coordinate descent method to compute the Lasso solution (`linear_model.Lasso`). Lars will be faster if the estimated components are sparse. n_jobs : int, default=None Number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. batch_size : int, default=256 Number of samples in each mini-batch. .. versionchanged:: 1.3 The default value of `batch_size` changed from 3 to 256 in version 1.3. shuffle : bool, default=True Whether to shuffle the samples before forming batches. dict_init : ndarray of shape (n_components, n_features), default=None Initial value of the dictionary for warm restart scenarios. transform_algorithm : {'lasso_lars', 'lasso_cd', 'lars', 'omp', 'threshold'}, default='omp' Algorithm used to transform the data: - `'lars'`: uses the least angle regression method (`linear_model.lars_path`); - `'lasso_lars'`: uses Lars to compute the Lasso solution. - `'lasso_cd'`: uses the coordinate descent method to compute the Lasso solution (`linear_model.Lasso`). `'lasso_lars'` will be faster if the estimated components are sparse. - `'omp'`: uses orthogonal matching pursuit to estimate the sparse solution. - `'threshold'`: squashes to zero all coefficients less than alpha from the projection ``dictionary * X'``. transform_n_nonzero_coefs : int, default=None Number of nonzero coefficients to target in each column of the solution. This is only used by `algorithm='lars'` and `algorithm='omp'`. If `None`, then `transform_n_nonzero_coefs=int(n_features / 10)`. transform_alpha : float, default=None If `algorithm='lasso_lars'` or `algorithm='lasso_cd'`, `alpha` is the penalty applied to the L1 norm. If `algorithm='threshold'`, `alpha` is the absolute value of the threshold below which coefficients will be squashed to zero. If `None`, defaults to `alpha`. .. versionchanged:: 1.2 When None, default value changed from 1.0 to `alpha`. verbose : bool or int, default=False To control the verbosity of the procedure. split_sign : bool, default=False Whether to split the sparse feature vector into the concatenation of its negative part and its positive part. This can improve the performance of downstream classifiers. random_state : int, RandomState instance or None, default=None Used for initializing the dictionary when ``dict_init`` is not specified, randomly shuffling the data when ``shuffle`` is set to ``True``, and updating the dictionary. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. positive_code : bool, default=False Whether to enforce positivity when finding the code. .. versionadded:: 0.20 positive_dict : bool, default=False Whether to enforce positivity when finding the dictionary. .. versionadded:: 0.20 transform_max_iter : int, default=1000 Maximum number of iterations to perform if `algorithm='lasso_cd'` or `'lasso_lars'`. .. versionadded:: 0.22 callback : callable, default=None A callable that gets invoked at the end of each iteration. .. versionadded:: 1.1 tol : float, default=1e-3 Control early stopping based on the norm of the differences in the dictionary between 2 steps. Used only if `max_iter` is not None. To disable early stopping based on changes in the dictionary, set `tol` to 0.0. .. versionadded:: 1.1 max_no_improvement : int, default=10 Control early stopping based on the consecutive number of mini batches that does not yield an improvement on the smoothed cost function. Used only if `max_iter` is not None. To disable convergence detection based on cost function, set `max_no_improvement` to None. .. versionadded:: 1.1 Attributes ---------- components_ : ndarray of shape (n_components, n_features) Components extracted from the data. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_iter_ : int Number of iterations over the full dataset. n_steps_ : int Number of mini-batches processed. .. versionadded:: 1.1 See Also -------- DictionaryLearning : Find a dictionary that sparsely encodes data. MiniBatchSparsePCA : Mini-batch Sparse Principal Components Analysis. SparseCoder : Find a sparse representation of data from a fixed, precomputed dictionary. SparsePCA : Sparse Principal Components Analysis. References ---------- J. Mairal, F. Bach, J. Ponce, G. Sapiro, 2009: Online dictionary learning for sparse coding (https://www.di.ens.fr/sierra/pdfs/icml09.pdf) Examples -------- >>> import numpy as np >>> from sklearn.datasets import make_sparse_coded_signal >>> from sklearn.decomposition import MiniBatchDictionaryLearning >>> X, dictionary, code = make_sparse_coded_signal( ... n_samples=30, n_components=15, n_features=20, n_nonzero_coefs=10, ... random_state=42) >>> dict_learner = MiniBatchDictionaryLearning( ... n_components=15, batch_size=3, transform_algorithm='lasso_lars', ... transform_alpha=0.1, max_iter=20, random_state=42) >>> X_transformed = dict_learner.fit_transform(X) We can check the level of sparsity of `X_transformed`: >>> np.mean(X_transformed == 0) > 0.5 True We can compare the average squared euclidean norm of the reconstruction error of the sparse coded signal relative to the squared euclidean norm of the original signal: >>> X_hat = X_transformed @ dict_learner.components_ >>> np.mean(np.sum((X_hat - X) ** 2, axis=1) / np.sum(X ** 2, axis=1)) 0.052... r^rcNrdrer8rrrr3rrr!rirrrgrr>r r!r=rBr*r rr4r rrr)rrrGrr:rTr Fr+rrv)r8rr3rrirrrrr rr4r rrrrrrGrc (t| | | ||||||_||_||_||_||_| |_| |_||_ ||_ ||_ ||_ ||_ ||_||_||_dSr )r rr^r8rr3rrr4rrr rrrrrG)rr^r8rr3rrirrrrr rr4r rrrrrrGrr!s r'rz$MiniBatchDictionaryLearning.__init__Cs2   %        )    *"  $$(*  "4r)c|j|_|j|jd|_t|j|jd|jz|_t|j|jd|_ dS)Nrcrr) r^ _n_componentsrKr(rr_fit_algorithmr{r _batch_sizers r' _check_paramsz)MiniBatchDictionaryLearning._check_paramsusn!.   %!"D  t143EFFF&);;t ;;r)c |j|j}n4t||j|\}}}|ddtjf|z}|jt |kr|d|jddf}nQtj|tj|jt |z |jdf|j f}t|d|j d}tj |d }|S) z!Initialization of the dictionary.NrrcrrFrrr) rrrPrMrr concatenaterrKrorr)rrZrr[rrs r'_initialize_dictz,MiniBatchDictionaryLearning._initialize_dicts  > %JJ .4%L    Aq*111bj=)J6J  Z 0 0#$8d&8$8!!!$;Converged (lack of improvement in objective function) at step ) rKr{r4r _ewa_costrrrPrG _ewa_cost_min_no_improvementr) rrZr]new_dictold_dictr\rZn_stepsrr8 dict_diffs r'_check_convergencez.MiniBatchDictionaryLearning._check_convergencesWQZ ax 3sI 233 3 3| YWWWwWW:WWXXX5 > !'DNN)a-0EqMME!^q5y9J<<    K8 344t7II 8a<>Gog..G"279t7G+G#H#HII m&66GAg88 ) )5!%.!11Z);Q **ZXy!WE=,MM&((+++( EDM74=3C#CDDDLL"[L88TTdkF!)T-=>>Gog..Gf w77 , ,5$$WU^ZASUVWWW"&,"V1tEDL?qwOOODGG)J Q D,> NNN%   r)c&|jjdSrCrDr*s r'r4z+MiniBatchDictionaryLearning._n_features_out rEr)c6dtjtjgiSrGr)r*s r'r+z&MiniBatchDictionaryLearning._more_tags rHr)r )rrrrrrrrrrMrIrJr:rKrLrrSrVr[r^rhr rOrrr6r4r+r7r8s@r'rrLsWWr$(AtFCCCTJ$((4D8889$  HXq$v 6 6 6 F::|n-- . . $ XXh4???F$ **dF^445$ 4"$ xx!T&AAAB$ I;$ dBJ'$  JMMM N N $ $hhxD&P&P&PRV%W!$" HHT1d6BBBDI#$$ I;%$& yk'$( ()$* )+$, )-$. (x!T&IIIJ8$q$v6667'x!T&III4P5$$$D>0!"& /0000000d < < <: + + +$$$LBBBH\555qqq65qf\555***65*X))X)       r)r)NNFNFr )r):rrrrrmathrnumbersrrnumpyrMjoblibrscipyrbaser r r r linear_modelr rrrutilsrrrrutils._param_validationrrrr utils.extmathrrrutils.parallelrrutils.validationrr(rbrtrsrrrrrr rrrrJr)r'rs|  """"""""###### FEEEEEEEEEEEQQQQQQQQQQQQSSSSSSSSSSSS??????????..............        d5d5d5d5d5N^#nt$d# JMMM N N %HXq$vFFFM(4D888$?Kt$Xh4???@T"!{;K!$#''4       TTTT-,Tv       MMMMMh   M5M5M5M5`n(n(n(b    [       # 1[[[[[| ^:vtn--.#$HXq$vFFFG  #(    #X9X9X9X9X9vF4F4F4F4F479IF4F4F4RO"O"O"O"O"#]O"O"O"dT T T T T *MT T T n s  s  s  s  s  "3]s  s  s  s  s  r)