rehline¶
Overview¶
Composite Quantile Regressor (CQR) with a ridge penalty. |
|
ReHLine Minimization [1]. |
|
Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a elastic net penalty. |
|
Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty. |
|
Matrix Factorization (MF) with a piecewise linear-quadratic objective and ridge penalty. |
|
Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss |
|
Empirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss |
|
Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss |
|
Empirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss |
|
- |
|
Generate synthetic rating data using matrix factorization model. |
|
Compute the regularization path for Composite Quantile Regression (CQR) with ridge penalty. |
|
Compute the PLQ Empirical Risk Minimization (ERM) path over a range of regularization parameters. |
Classes¶
- class rehline.CQR_Ridge(quantiles, C=1.0, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100)¶
Bases:
rehline._base._BaseReHLine,sklearn.base.BaseEstimatorComposite Quantile Regressor (CQR) with a ridge penalty.
It allows for the fitting of a linear regression model that minimizes a composite quantile loss function.
\[\min_{\mathbf{\beta} \in \mathbb{R}^d, \mathbf{\beta_0} \in \mathbb{R}^K} \sum_{k=1}^K \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta} + \mathbf{\beta_0k}) + \frac{1}{2} \| \mathbf{\beta} \|_2^2.\]Parameters¶
- quantileslist of float (n_quantiles,)
The quantiles to be estimated.
- Cfloat, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. C will be absorbed by the ReHLine parameters when self.make_ReLHLoss is conducted.
- verboseint, default=0
Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in liblinear that, if enabled, may not work properly in a multithreaded context.
- max_iterint, default=1000
The maximum number of iterations to be run.
- tolfloat, default=1e-4
The tolerance for the stopping criterion.
- shrinkfloat, default=1
The shrinkage of dual variables for the ReHLine algorithm.
- warm_startbool, default=False
Whether to use the given dual params as an initial guess for the optimization algorithm.
- trace_freqint, default=100
The frequency at which to print the optimization trace.
Attributes¶
- coef_array-like
The optimized model coefficients.
- intercept_array-like
The optimized model intercepts.
- quantiles_: array-like
The quantiles to be estimated.
- n_iter_int
The number of iterations performed by the ReHLine solver.
- opt_result_object
The optimization result object.
- dual_obj_array-like
The dual objective function values.
- primal_obj_array-like
The primal objective function values.
Methods¶
- fit(X, y, sample_weight=None)
Fit the model based on the given training data.
- predict(X)
The prediction for the given dataset.
Overview¶
Members¶
- fit(X, y, sample_weight=None)¶
Fit the model based on the given training data.
Parameters¶
- X: {array-like} of shape (n_samples, n_features)
Training vector, where n_samples is the number of samples and n_features is the number of features.
- yarray-like of shape (n_samples,)
The target variable.
- sample_weightarray-like of shape (n_samples,), default=None
Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.
Returns¶
- selfobject
An instance of the estimator.
- class rehline.ReHLine(C=1.0, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100)¶
Bases:
rehline._base._BaseReHLine,sklearn.base.BaseEstimatorReHLine Minimization [1].
\[\begin{split}\min_{\mathbf{\beta} \in \mathbb{R}^d} \sum_{i=1}^n \sum_{l=1}^L \text{ReLU}( u_{li} \mathbf{x}_i^\intercal \mathbf{\beta} + v_{li}) + \sum_{i=1}^n \sum_{h=1}^H {\text{ReHU}}_{\tau_{hi}}( s_{hi} \mathbf{x}_i^\intercal \mathbf{\beta} + t_{hi}) + \frac{1}{2} \| \mathbf{\beta} \|_2^2, \\ \text{ s.t. } \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0},\end{split}\]where \(\mathbf{U} = (u_{li}),\mathbf{V} = (v_{li}) \in \mathbb{R}^{L \times n}\) and \(\mathbf{S} = (s_{hi}),\mathbf{T} = (t_{hi}),\mathbf{\tau} = (\tau_{hi}) \in \mathbb{R}^{H \times n}\) are the ReLU-ReHU loss parameters, and \((\mathbf{A},\mathbf{b})\) are the constraint parameters.
Parameters¶
- Cfloat, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive.
- _U, _V: array of shape (L, n_samples), default=np.empty(shape=(0, 0))
The parameters pertaining to the ReLU part in the loss function.
- _Tau, _S, _T: array of shape (H, n_samples), default=np.empty(shape=(0, 0))
The parameters pertaining to the ReHU part in the loss function.
- _A: array of shape (K, n_features), default=np.empty(shape=(0, 0))
The coefficient matrix in the linear constraint.
- _b: array of shape (K, ), default=np.empty(shape=0)
The intercept vector in the linear constraint.
- verboseint, default=0
Enable verbose output.
- max_iterint, default=1000
The maximum number of iterations to be run.
- tolfloat, default=1e-4
The tolerance for the stopping criterion.
- shrinkfloat, default=1
The shrinkage of dual variables for the ReHLine algorithm.
- warm_startbool, default=False
Whether to use the given dual params as an initial guess for the optimization algorithm.
- trace_freqint, default=100
The frequency at which to print the optimization trace.
Attributes¶
- coef_array-like
The optimized model coefficients.
- n_iter_int
The number of iterations performed by the ReHLine solver.
- opt_result_object
The optimization result object.
- dual_obj_array-like
The dual objective function values.
- primal_obj_array-like
The primal objective function values.
- _Lambda: array-like
The optimized dual variables for ReLU parts.
- _Gamma: array-like
The optimized dual variables for ReHU parts.
- _xi: array-like
The optimized dual variables for linear constraints.
Examples¶
>>> ## test SVM on simulated dataset >>> import numpy as np >>> from rehline import ReHLine
>>> # simulate classification dataset >>> n, d, C = 1000, 3, 0.5 >>> np.random.seed(1024) >>> X = np.random.randn(1000, 3) >>> beta0 = np.random.randn(3) >>> y = np.sign(X.dot(beta0) + np.random.randn(n))
>>> # Usage of ReHLine >>> n, d = X.shape >>> U = -(C*y).reshape(1,-1) >>> L = U.shape[0] >>> V = (C*np.array(np.ones(n))).reshape(1,-1) >>> clf = ReHLine(C=C) >>> clf._U, clf._V = U, V >>> clf.fit(X=X) >>> print('sol provided by rehline: %s' %clf.coef_) >>> sol provided by rehline: [ 0.7410154 -0.00615574 2.66990408] >>> print(clf.decision_function([[.1,.2,.3]])) >>> [0.87384162]
References¶
Overview¶
Members¶
- fit(X, sample_weight=None)¶
Fit the model based on the given training data.
Parameters¶
- X: {array-like} of shape (n_samples, n_features)
Training vector, where n_samples is the number of samples and n_features is the number of features.
- sample_weightarray-like of shape (n_samples,), default=None
Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.
Returns¶
- selfobject
An instance of the estimator.
- class rehline.plqERM_ElasticNet(loss, constraint=None, C=1.0, l1_ratio=0.5, omega=None, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100)¶
Bases:
rehline._base._BaseReHLine,sklearn.base.BaseEstimatorEmpirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a elastic net penalty.
\[\min_{\mathbf{\beta} \in \mathbb{R}^d} C \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta}) + \text{l1_ratio} \sum_{j=1}^d \omega_j | \beta_j | + \frac{1}{2} (1 - \text{l1_ratio}) \| \mathbf{\beta} \|_2^2, \ \text{ s.t. } \ \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0},\]- The function supports various loss functions, including:
‘hinge’, ‘svm’ or ‘SVM’
‘check’ or ‘quantile’ or ‘quantile regression’ or ‘QR’
‘sSVM’ or ‘smooth SVM’ or ‘smooth hinge’
‘TV’
‘huber’ or ‘Huber’
‘SVR’ or ‘svr’
- The following constraint types are supported:
‘nonnegative’ or ‘>=0’: A non-negativity constraint.
‘fair’ or ‘fairness’: A fairness constraint.
‘custom’: A custom constraint, where the user must provide the constraint matrix ‘A’ and vector ‘b’.
Parameters¶
- lossdict
A dictionary specifying the loss function parameters.
- constraintlist of dict
A list of dictionaries, where each dictionary represents a constraint. Each dictionary must contain a ‘name’ key, which specifies the type of constraint.
- Cfloat, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. C will be absorbed by the ReHLine parameters when self.make_ReLHLoss is conducted.
- l1_ratiofloat, default=0.5
The ElasticNet mixing parameter, with 0 <= l1_ratio < 1. For l1_ratio = 0 the penalty is an L2 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
- omegaarray of shape (n_features, ), default=np.empty(shape=0)
Non-negative weight coefficients for adaptive lasso. If not provided, all coefficients receive the same L1 penalty controlled by
l1_ratio.- verboseint, default=0
Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in liblinear that, if enabled, may not work properly in a multithreaded context.
- max_iterint, default=1000
The maximum number of iterations to be run.
- _U, _V: array of shape (L, n_samples), default=np.empty(shape=(0, 0))
The parameters pertaining to the ReLU part in the loss function.
- _Tau, _S, _T: array of shape (H, n_samples), default=np.empty(shape=(0, 0))
The parameters pertaining to the ReHU part in the loss function.
- _A: array of shape (K, n_features), default=np.empty(shape=(0, 0))
The coefficient matrix in the linear constraint.
- _b: array of shape (K, ), default=np.empty(shape=0)
The intercept vector in the linear constraint.
Attributes¶
- coef_array-like
The optimized model coefficients.
- n_iter_int
The number of iterations performed by the ReHLine solver.
- opt_result_object
The optimization result object.
- dual_obj_array-like
The dual objective function values.
- primal_obj_array-like
The primal objective function values.
Methods¶
- fit(X, y, sample_weight=None)
Fit the model based on the given training data.
- decision_function(X)
The decision function evaluated on the given dataset.
Notes¶
The plqERM_ElasticNet class is a subclass of _BaseReHLine and BaseEstimator, which suggests that it is part of a larger framework for implementing ReHLine algorithms.
Overview¶
Members¶
- fit(X, y, sample_weight=None)¶
Fit the model based on the given training data.
Parameters¶
- X: {array-like} of shape (n_samples, n_features)
Training vector, where n_samples is the number of samples and n_features is the number of features.
- yarray-like of shape (n_samples,)
The target variable.
- sample_weightarray-like of shape (n_samples,), default=None
Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.
Returns¶
- selfobject
An instance of the estimator.
- class rehline.plqERM_Ridge(loss, constraint=None, C=1.0, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100)¶
Bases:
rehline._base._BaseReHLine,sklearn.base.BaseEstimatorEmpirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty.
\[\min_{\mathbf{\beta} \in \mathbb{R}^d} \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta}) + \frac{1}{2} \| \mathbf{\beta} \|_2^2, \ \text{ s.t. } \ \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0},\]- The function supports various loss functions, including:
‘hinge’, ‘svm’ or ‘SVM’
‘check’ or ‘quantile’ or ‘quantile regression’ or ‘QR’
‘sSVM’ or ‘smooth SVM’ or ‘smooth hinge’
‘TV’
‘huber’ or ‘Huber’
‘SVR’ or ‘svr’
- The following constraint types are supported:
‘nonnegative’ or ‘>=0’: A non-negativity constraint.
‘fair’ or ‘fairness’: A fairness constraint.
‘custom’: A custom constraint, where the user must provide the constraint matrix ‘A’ and vector ‘b’.
Parameters¶
- lossdict
A dictionary specifying the loss function parameters.
- constraintlist of dict
A list of dictionaries, where each dictionary represents a constraint. Each dictionary must contain a ‘name’ key, which specifies the type of constraint.
- Cfloat, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. C will be absorbed by the ReHLine parameters when self.make_ReLHLoss is conducted.
- verboseint, default=0
Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in liblinear that, if enabled, may not work properly in a multithreaded context.
- max_iterint, default=1000
The maximum number of iterations to be run.
- _U, _V: array of shape (L, n_samples), default=np.empty(shape=(0, 0))
The parameters pertaining to the ReLU part in the loss function.
- _Tau, _S, _T: array of shape (H, n_samples), default=np.empty(shape=(0, 0))
The parameters pertaining to the ReHU part in the loss function.
- _A: array of shape (K, n_features), default=np.empty(shape=(0, 0))
The coefficient matrix in the linear constraint.
- _b: array of shape (K, ), default=np.empty(shape=0)
The intercept vector in the linear constraint.
Attributes¶
- coef_array-like
The optimized model coefficients.
- n_iter_int
The number of iterations performed by the ReHLine solver.
- opt_result_object
The optimization result object.
- dual_obj_array-like
The dual objective function values.
- primal_obj_array-like
The primal objective function values.
Methods¶
- fit(X, y, sample_weight=None)
Fit the model based on the given training data.
- decision_function(X)
The decision function evaluated on the given dataset.
Notes¶
The plqERM_Ridge class is a subclass of _BaseReHLine and BaseEstimator, which suggests that it is part of a larger framework for implementing ReHLine algorithms.
Overview¶
Members¶
- fit(X, y, sample_weight=None)¶
Fit the model based on the given training data.
Parameters¶
- X: {array-like} of shape (n_samples, n_features)
Training vector, where n_samples is the number of samples and n_features is the number of features.
- yarray-like of shape (n_samples,)
The target variable.
- sample_weightarray-like of shape (n_samples,), default=None
Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.
Returns¶
- selfobject
An instance of the estimator.
- class rehline.plqMF_Ridge(n_users, n_items, loss, biased=True, constraint_user=None, constraint_item=None, rank=10, C=1.0, rho=0.5, init_mean=0.0, init_sd=0.1, random_state=None, max_iter=10000, tol=0.0001, shrink=1, trace_freq=100, max_iter_CD=10, tol_CD=0.0001, verbose=0)¶
Bases:
rehline._base._BaseReHLine,sklearn.base.BaseEstimatorMatrix Factorization (MF) with a piecewise linear-quadratic objective and ridge penalty.
\[\begin{split}\min_{\substack{ \mathbf{P} \in \mathbb{R}^{n \times k}\ \pmb{\alpha} \in \mathbb{R}^n \\ \mathbf{Q} \in \mathbb{R}^{m \times k}\ \pmb{\beta} \in \mathbb{R}^m }} \left[ \sum_{(u,i)\in \Omega} C \cdot \text{PLQ}(r_{ui}, \ \mathbf{p}_u^T \mathbf{q}_i + \alpha_u + \beta_i) \right] + \left[ \frac{\rho}{n}\sum_{u=1}^n(\|\mathbf{p}_u\|_2^2 + \alpha_u^2) + \frac{1-\rho}{m}\sum_{i=1}^m(\|\mathbf{q}_i\|_2^2 + \beta_i^2) \right]\end{split}\]\[\begin{split}\ \text{ s.t. } \ \mathbf{A}_{\text{user}} \begin{pmatrix} \alpha_u \\ \mathbf{p}_u \end{pmatrix} + \mathbf{b}_{\text{user}} \geq \mathbf{0},\ u = 1,\dots,n \quad \text{and} \quad \mathbf{A}_{\text{item}} \begin{pmatrix} \beta_i \\ \mathbf{q}_i \end{pmatrix} + \mathbf{b}_{\text{item}} \geq \mathbf{0},\ i = 1,\dots,m\end{split}\]- The function supports various loss functions, including:
‘hinge’, ‘svm’ or ‘SVM’
‘MAE’ or ‘mae’ or ‘mean absolute error’
‘squared hinge’ or ‘squared svm’ or ‘squared SVM’
‘MSE’ or ‘mse’ or ‘mean squared error’
- The following constraint types are supported:
‘nonnegative’ or ‘>=0’: A non-negativity constraint.
‘fair’ or ‘fairness’: A fairness constraint.
‘custom’: A custom constraint, where the user must provide the constraint matrix ‘A’ and vector ‘b’.
Parameters¶
- n_usersint
Number of unique users in the dataset (or number of rows in target sparse matrix).
- n_itemsint
Number of unique items in the dataset (or number of columns in target sparse matrix).
- lossdict
A dictionary specifying the loss function parameters.
- constraint_userlist of dict
A list of dictionaries, where each dictionary represents a constraint to user side parameters. Each dictionary must contain a ‘name’ key, which specifies the type of constraint.
- constraint_itemlist of dict
A list of dictionaries, where each dictionary represents a constraint to item side parameters. Each dictionary must contain a ‘name’ key, which specifies the type of constraint.
- biasedbool, default=True
Whether to include user and item bias terms in the model.
- rankint, default=10
Dimensionality of the latent factor vectors (number of factors).
- Cfloat, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. C will be absorbed by the ReHLine parameters when _cast_sample_weight() is conducted.
- rhofloat, default=0.5
Regularization strength ratio between user and item factors. Must be within the range of (0,1).
- init_meanfloat, default=0.0
Mean of the Gaussian distribution for initializing latent factors.
- init_sdfloat, default=0.1
Standard deviation of the Gaussian distribution for initializing latent factors.
- random_stateint or RandomState, default=None
Random seed for reproducible initialization of latent factors.
- max_iterint, default=10000
The maximum number of iterations to be run for the ReHLine solver.
- tolfloat, default=1e-4
The tolerance for the stopping criterion for the ReHLine solver.
- shrinkfloat, default=1
The shrinkage of dual variables for the ReHLine solver.
- trace_freqint, default=100
The frequency at which to print the optimization trace for the ReHLine solver.
- max_iter_CDint, default=10
Maximum number of iterations for coordinate descent steps.
- tol_CDfloat, default=1e-4
The tolerance for the stopping criterion for coordinate descent steps.
- verboseint, default=0
- Verbosity level.
0: No output 1: CD iteration progress information 2: ReHLine solver optimization information 3: All information of CD and ReHLine
Attributes¶
- n_usersint
Number of unique users in the dataset (or number of rows in target sparse matrix).
- n_itemsint
Number of unique items in the dataset (or number of columns in target sparse matrix).
- n_ratingsint
Number of ratings in the training data. Available after fitting.
- Pndarray of shape (n_users, rank)
User latent factor matrix. Learned during fitting.
- Qndarray of shape (n_items, rank)
Item latent factor matrix. Learned during fitting.
- bundarray of shape (n_users,) or None
User bias terms. Learned during fitting. Only available if biased=True.
- bindarray of shape (n_items,) or None
Item bias terms. Learned during fitting. Only available if biased=True.
- Iulist of ndarray
List where each element contains indices of items rated by the corresponding user. Available after fitting.
- Uilist of ndarray
List where each element contains indices of users who rated the corresponding item. Available after fitting.
- historyndarray of shape (max_iter_CD + 1, 2)
Optimization history containing loss and objective values at each coordinate descent iteration. First column: cumulative loss term values. Second column: objective function values (with penalty).
- sample_weightndarray of shape (n_ratings,)
Sample weights used during fitting. Available after fitting.
Methods¶
- fit(X, y, sample_weight=None)
Fit the model based on the given training data.
- decision_function(X)
The decision function evaluated on the given dataset.
- obj(X, y)
Compute the values of loss term and objective function.
Notes¶
The plqMF_Ridge class is a subclass of _BaseReHLine and BaseEstimator, which suggests that it is part of a larger framework for implementing ReHLine algorithms.
Overview¶
Members¶
- fit(X, y, sample_weight=None)¶
Fit the model based on the given training data.
Parameters¶
- Xarray-like of shape (n_ratings, 2)
Input data where first column contains user ID and second column contains item ID.
- yarray-like of shape (n_ratings,)
Target rating values.
- sample_weightarray-like of shape (n_samples,), default=None
Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.
Returns¶
- selfobject
An instance of the estimator.
- decision_function(X)¶
The decision function evaluated on the given dataset
Parameters¶
- Xarray-like of shape (n_samples, 2)
Training data where first column contains user ID and second column contains item ID.
Returns¶
- predictionndarray of shape (n_samples,)
Predicted ratings for the input pairs.
- obj(X, y)¶
Compute the values of loss term and objective function.
Parameters¶
- Xarray-like of shape (n_ratings, 2)
User-item rating pairs.
- yarray-like of shape (n_ratings,)
Actual rating values.
Returns¶
- loss_termfloat
The data fitting term (sum of loss values).
- objective_valuefloat
The total objective value including regularization.
- class rehline.plq_ElasticNet_Classifier(loss, constraint=None, C=1.0, l1_ratio=0.5, omega=None, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, class_weight=None, multi_class=None, n_jobs=None)¶
Bases:
rehline._class.plqERM_ElasticNet,sklearn.base.ClassifierMixinEmpirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss and elastic net penalty, compatible with the scikit-learn API.
- This wrapper makes
plqERM_ElasticNetbehave as a classifier: Accepts arbitrary binary labels in the original label space.
Computes class weights on original labels (if
class_weightis set).Encodes labels with
LabelEncoderinto {0,1}, then maps to {-1,+1} for training.Supports optional intercept fitting (via an augmented constant feature).
Provides standard methods
fit,predict, anddecision_function.Integrates with scikit-learn ecosystem (e.g., GridSearchCV, Pipeline).
Supports multiclass classification via OvR or OvO method.
Parameters¶
- lossdict
Dictionary specifying the loss function parameters. Examples include: - {‘name’: ‘svm’} - {‘name’: ‘sSVM’} - {‘name’: ‘huber’} and other PLQ losses supported by
plqERM_ElasticNet.- constraintlist of dict, default=[]
Optional constraints. Each dictionary must include a
'name'key.- Cfloat, default=1.0
Inverse regularization strength (scales the loss term).
- l1_ratiofloat, default=0.5
The ElasticNet mixing parameter, 0 <= l1_ratio < 1. - l1_ratio = 0 → pure Ridge (equivalent to plq_Ridge_Classifier) - 0 < l1_ratio < 1 → combined L1 + L2 penalty Must be strictly less than 1.0 to avoid division by zero in rho/C_eff.
- omegaarray of shape (n_features, ), default=np.empty(shape=(0, 0))
Non-negative weight coefficients for adaptive lasso. If not provided, all non-intercept coefficients receive the same L1 penalty controlled by
l1_ratio. The penalty for the intercept can be scaled viaintercept_scaling.- fit_interceptbool, default=True
Whether to fit an intercept term via an augmented constant feature column.
- intercept_scalingfloat, default=1.0
Value of the constant feature column when
fit_intercept=True.- class_weightdict, ‘balanced’, or None, default=None
Class weights applied like in LinearSVC.
- multi_classstr or list, default=[]
Method for multiclass classification: - ‘ovr’: One-vs-Rest - ‘ovo’: One-vs-One - [] or ignored when only 2 classes are present.
- n_jobsint or None, default=None
Number of parallel jobs for multiclass fitting.
max_iter : int, default=1000 tol : float, default=1e-4 shrink : int, default=1 warm_start : int, default=0 verbose : int, default=0 trace_freq : int, default=100
Attributes¶
coef_: ndarray of shape (n_features,) for binary, (n_estimators, n_features) for multiclassintercept_: float for binary, ndarray of shape (n_estimators,) for multiclassclasses_: ndarray of shape (n_classes,)estimators_: list, only present for multiclass _label_encoder : LabelEncoderOverview¶
Members¶
- fit(X, y, sample_weight=None)¶
Fit the classifier to training data.
Parameters¶
X : array-like of shape (n_samples, n_features) y : array-like of shape (n_samples,) sample_weight : array-like of shape (n_samples,), default=None
Returns¶
self
- decision_function(X)¶
Compute the decision function for samples in X.
For binary: 1D array of shape (n_samples,). For OvR/OvO multiclass: 2D array of shape (n_samples, n_estimators).
- predict(X)¶
Predict class labels for samples in X.
Binary: threshold at 0. OvR: argmax across K classifiers. OvO: majority vote + normalized confidence tie-breaking.
- This wrapper makes
- class rehline.plq_ElasticNet_Regressor(loss=None, constraint=None, C=1.0, l1_ratio=0.5, omega=None, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0)¶
Bases:
rehline._class.plqERM_ElasticNet,sklearn.base.RegressorMixinEmpirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss and an elastic net penalty, implemented as a scikit-learn compatible estimator.
- This wrapper makes
plqERM_ElasticNetbehave as a regressor: Supports optional intercept fitting via an augmented constant feature column.
Provides standard methods
fit,predict, anddecision_function.Integrates with the scikit-learn ecosystem (e.g., GridSearchCV, Pipeline).
Notes¶
Intercept handling: if
fit_intercept=True, a constant column (value =intercept_scaling) is appended to the right of the design matrix before calling the base solver. The last learned coefficient is then split out asintercept_. Original feature indices are therefore unaffected;sen_idxin a'fair'constraint continues to reference the original columns.Sparse input: not supported. Convert to dense before fitting.
Parameters¶
- lossdict, default={‘name’: ‘QR’, ‘qt’: 0.5}
PLQ loss configuration. Examples:
{'name': 'QR', 'qt': 0.5},{'name': 'huber', 'tau': 1.0},{'name': 'SVR', 'epsilon': 0.1}.- constraintlist of dict, default=[]
- Constraint specifications:
{'name': 'nonnegative'}or{'name': '>=0'}{'name': 'fair', 'sen_idx': list[int], 'tol_sen': list[float]}{'name': 'custom', 'A': ndarray[K, d], 'b': ndarray[K]}
- Cfloat, default=1.0
Regularization parameter (scales the loss term).
- l1_ratiofloat, default=0.5
The ElasticNet mixing parameter, 0 <= l1_ratio < 1. - l1_ratio = 0 → pure Ridge (equivalent to plq_Ridge_Regressor) - 0 < l1_ratio < 1 → combined L1 + L2 penalty Must be strictly less than 1.0 to avoid division by zero in rho/C_eff.
- omegaarray of shape (n_features, ), default=np.empty(shape=(0, 0))
Non-negative weight coefficients for adaptive lasso. If not provided, all non-intercept coefficients receive the same L1 penalty controlled by
l1_ratio. The penalty for the intercept can be scaled viaintercept_scaling.- fit_interceptbool, default=True
If True, append a constant column (value =
intercept_scaling) to the design matrix before solving. The last learned coefficient is then extracted asintercept_.- intercept_scalingfloat, default=1.0
Scaling applied to the appended constant column when
fit_intercept=True.
max_iter : int, default=1000 tol : float, default=1e-4 shrink : int, default=1 warm_start : int, default=0 verbose : int, default=0 trace_freq : int, default=100
Attributes¶
coef_ndarray of shape (n_features,)Learned linear coefficients (excluding the intercept term).
intercept_floatIntercept term. 0.0 if
fit_intercept=False.n_features_in_intNumber of input features seen during
fit()(before intercept augmentation).
Overview¶
Members¶
- fit(X, y, sample_weight=None)¶
Fit the regressor to training data.
If
fit_intercept=True, a constant column (value =intercept_scaling) is appended to the right ofXbefore calling the base solver (plqERM_ElasticNet.fit). After solving, the last coefficient is split asintercept_and removed fromcoef_.Parameters¶
- Xndarray of shape (n_samples, n_features)
Training design matrix (dense). Sparse inputs are not supported.
- yndarray of shape (n_samples,)
Target values.
- sample_weightndarray of shape (n_samples,), default=None
Optional per-sample weights; forwarded to the underlying solver.
Returns¶
- selfobject
Fitted estimator.
- This wrapper makes
- class rehline.plq_Ridge_Classifier(loss, constraint=None, C=1.0, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, class_weight=None, multi_class=None, n_jobs=None)¶
Bases:
rehline._class.plqERM_Ridge,sklearn.base.ClassifierMixinEmpirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss and ridge penalty, compatible with the scikit-learn API.
- This wrapper makes
plqERM_Ridgebehave as a classifier: Accepts arbitrary binary labels in the original label space.
Computes class weights on original labels (if
class_weightis set).Encodes labels with
LabelEncoderinto {0,1}, then maps to {-1,+1} for training.Supports optional intercept fitting (via an augmented constant feature).
Provides standard methods
fit,predict, anddecision_function.Integrates with scikit-learn ecosystem (e.g., GridSearchCV, Pipeline).
Supports multiclass classification via OvR or OvO method.
Parameters¶
- lossdict
Dictionary specifying the loss function parameters. Examples include: - {‘name’: ‘svm’} - {‘name’: ‘sSVM’} - {‘name’: ‘huber’} and other PLQ losses supported by
plqERM_Ridge.- constraintlist of dict, default=[]
Optional constraints. Each dictionary must include a
'name'key. Examples: {‘name’: ‘nonnegative’}, {‘name’: ‘fair’}, {‘name’: ‘custom’}.- Cfloat, default=1.0
Inverse regularization strength.
- _U, _V, _Tau, _S, _Tndarray, default empty
Parameters for the PLQ representation of the loss function. Typically built internally by helper functions.
- _Andarray of shape (K, n_features), default empty
Linear-constraint coefficient matrix.
- _bndarray of shape (K,), default empty
Linear-constraint intercept vector.
- max_iterint, default=1000
Maximum number of iterations for the ReHLine solver.
- tolfloat, default=1e-4
Convergence tolerance.
- shrinkint, default=1
Shrinkage parameter for the solver.
- warm_startint, default=0
Whether to reuse the previous solution for initialization.
- verboseint, default=0
Verbosity level for the solver.
- trace_freqint, default=100
Frequency (in iterations) at which solver progress is traced when
verbose > 0.- fit_interceptbool, default=True
Whether to fit an intercept term. If True, a constant feature column is added to
Xduring training. The last learned coefficient is extracted asintercept_.- intercept_scalingfloat, default=1.0
Value used for the constant feature column when
fit_intercept=True. Matches the convention used in scikit-learn’sLinearSVC.- class_weightdict, ‘balanced’, or None, default=None
Class weights applied like in LinearSVC: - ‘balanced’ uses n_samples / (n_classes * n_j). - dict maps label -> weight in the ORIGINAL label space.
- multi_classstr or list, default=[]
Method for multiclass classification. Options: - ‘ovo’: One-vs-One, trains K*(K-1)/2 binary classifiers. - ‘ovr’: One-vs-Rest, trains K binary classifiers. - [ ] or ignored when only 2 classes are present.
- n_jobsint or None, default=None
Number of parallel jobs for multiclass fitting. None means 1 (serial). -1 means use all available CPUs. Passed directly to joblib.Parallel.
Attributes¶
coef_: ndarray of shape (n_features,) for binary, (n_estimators, n_features) for multiclassCoefficients of all fitted classifiers, excluding the intercept.
intercept_: float for binary, ndarray of shape (n_estimators,) for multiclassIntercept term(s). 0.0 if
fit_intercept=False.classes_ndarray of shape (n_classes,)Unique class labels in the original label space.
estimators_list, only present for multiclassFor OvR: list of (coef, intercept) tuples, length K. For OvO: list of (coef, intercept, cls_i, cls_j) tuples, length K*(K-1)/2.
- _label_encoderLabelEncoder
Encodes original labels into {0,1} for internal training.
Overview¶
Members¶
- fit(X, y, sample_weight=None)¶
Fit the classifier to training data.
Parameters¶
- Xarray-like of shape (n_samples, n_features)
Training features.
- yarray-like of shape (n_samples,)
Target labels.
- sample_weightarray-like of shape (n_samples,), default=None
Per-sample weights. If None, uniform weights are used.
Returns¶
- selfobject
Fitted estimator.
- decision_function(X)¶
Compute the decision function for samples in X.
For binary classification, returns a 1D array of scores. For OvR multiclass, returns a 2D array of shape (n_samples, K). For OvO multiclass, returns a 2D array of shape (n_samples, K*(K-1)/2).
Using
coef_.Tworks uniformly for both binary (n_features,) and multiclass (n_estimators, n_features) shapes.Parameters¶
- Xarray-like of shape (n_samples, n_features)
Input samples.
Returns¶
- ndarray of shape (n_samples,) or (n_samples, n_estimators)
Continuous scores for each sample.
- predict(X)¶
Predict class labels for samples in X. For binary classification, thresholds the decision score at 0. For OvR, takes the argmax across K classifiers. For OvO, uses majority voting across K*(K-1)/2 classifiers.
Parameters¶
- Xarray-like of shape (n_samples, n_features)
Input samples.
Returns¶
- y_predndarray of shape (n_samples,)
Predicted class labels in the original label space.
- This wrapper makes
- class rehline.plq_Ridge_Regressor(loss=None, constraint=None, C=1.0, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0)¶
Bases:
rehline._class.plqERM_Ridge,sklearn.base.RegressorMixinEmpirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss and a ridge penalty, implemented as a scikit-learn compatible estimator.
This wrapper adds standard sklearn conveniences while delegating loss/constraint construction to
plqERM_Ridge(via _make_loss_rehline_param / _make_constraint_rehline_param).Notes¶
Intercept handling: if
fit_intercept=True, a constant column (value =intercept_scaling) is appended to the right of the design matrix before calling the base solver. The last learned coefficient is then split out asintercept_. → The column indices of the original features remain; therefore,sen_idxin the constraintfairfollow the original index.Constraint handling: constraints are passed through unchanged; the base class will call
_make_constraint_rehline_param(constraint, X, y)on the matrix given to fit. With your updated implementation,fairmust be specified as{'name': 'fair', 'sen_idx': list[int], 'tol_sen': list[float]}.
Parameters¶
- lossdict, default={‘name’: ‘QR’, ‘qt’: 0.5}
PLQ loss configuration (e.g., median Quantile Regression). Examples:
{'name': 'QR', 'qt': 0.5},{'name': 'huber', 'tau': 1.0},{'name': 'SVR', 'epsilon': 0.1}. Required keys depend on the chosen loss and are consumed by the underlying solver.- constraintlist of dict, default=[]
- Constraint specifications. Supported by your updated _make_constraint_rehline_param:
{'name': 'nonnegative'}or{'name': '>=0'}{'name': 'fair', 'sen_idx': list[int], 'tol_sen': list[float]}{'name': 'custom', 'A': ndarray[K, d], 'b': ndarray[K]}
Note: when
fit_intercept=True, a constant column is appended as the last column; since you index sensitive columns bysen_idxon the original features, indices stay valid.- Cfloat, default=1.0
Regularization parameter (absorbed by ReHLine parameters inside the solver).
- _U, _V, _Tau, _S, _Tndarray, default empty
Advanced PLQ parameters for the underlying ReHLine formulation (usually left as defaults).
- _A, _bndarray, default empty
Optional linear constraint matrices (used only if
constraintcontains{'name': 'custom'}). (Your _make_constraint_rehline_param is responsible for validating their shapes.)- max_iterint, default=1000
Maximum iterations for the ReHLine solver.
- tolfloat, default=1e-4
Convergence tolerance for the ReHLine solver.
- shrinkint, default=1
Shrink parameter passed to the solver (see solver docs).
- warm_startint, default=0
Warm start flag passed to the solver (see solver docs).
- verboseint, default=0
Verbosity for the solver (0: silent).
- trace_freqint, default=100
Iteration frequency to trace solver internals (if
verboseis enabled).- fit_interceptbool, default=True
If
True, append a constant column (value =intercept_scaling) to the design matrix before calling the solver. The learned last coefficient is then split asintercept_.- intercept_scalingfloat, default=1.0
Scaling applied to the appended constant column when
fit_intercept=True.
Attributes¶
coef_ndarray of shape (n_features,)Learned linear coefficients (excluding the intercept term).
intercept_floatIntercept term extracted from the last coefficient when
fit_intercept=True, otherwise 0.0.n_features_in_intNumber of input features seen during
fit()(before intercept augmentation).
Notes¶
This estimator does not support sparse input. If you need sparse support, convert inputs to dense or wrap this estimator in a scikit-learn
Pipelinewith a transformer that densifies data (at the cost of memory).Overview¶
Members¶
- fit(X, y, sample_weight=None)¶
If
fit_intercept=True, a constant column (value =intercept_scaling) is appended to the right ofXbefore calling the base solver. The base class (plqERM_Ridge) will construct the loss and constraints via its internal helpers on the matrix passed here. After solving, the last coefficient is split asintercept_and removed fromcoef_.Parameters¶
- Xndarray of shape (n_samples, n_features)
Training design matrix (dense). Sparse inputs are not supported.
- yndarray of shape (n_samples,)
Target values.
- sample_weightndarray of shape (n_samples,), default=None
Optional per-sample weights; forwarded to the underlying solver.
Returns¶
self : object Fitted estimator.
Functions¶
- rehline.ReHLine_solver(X, U, V, Tau=None, S=None, T=None, A=None, b=None, rho=None, Lambda=None, Gamma=None, xi=None, mu=None, max_iter=1000, tol=0.0001, shrink=1, verbose=1, trace_freq=100)¶
- rehline.make_mf_dataset(n_users, n_items, n_factors=20, n_interactions=None, density=0.01, noise_std=0.1, seed=None, rating_min=1.0, rating_max=5.0, return_params=True)¶
Generate synthetic rating data using matrix factorization model.
Creates synthetic user-item rating data based on the matrix factorization approach commonly used in recommender systems. The ratings are generated as: rating = mu + user_bias + item_bias + user_factor * item_factor + noise
Parameters¶
- n_usersint
Number of users in the synthetic dataset
- n_itemsint
Number of items in the synthetic dataset
- n_factorsint, default=20
Number of latent factors for user and item embeddings
- n_interactionsint, optional
Exact number of user-item interactions. If None, calculated as density * total_pairs
- densityfloat, default=0.01
Density of the rating matrix (ignored if n_interactions is specified)
- noise_stdfloat, default=0.1
Standard deviation of Gaussian noise added to ratings
- seedint, optional
Random seed for reproducible results
- rating_minfloat, default=1.0
Minimum possible rating value
- rating_maxfloat, default=5.0
Maximum possible rating value
- return_paramsbool, default=True
If True, returns the underlying model parameters (P, Q, bu, bi, mu)
Returns¶
- dict
Dictionary containing:
- Xndarray of shape (n_interactions, 2)
User-item pairs where X[:, 0] are user indices and X[:, 1] are item indices
- yndarray of shape (n_interactions,)
Synthetic ratings for each user-item pair
- paramsdict, optional
Only returned if return_params=True. Contains:
- Pndarray of shape (n_users, n_factors)
User factor matrix
- Qndarray of shape (n_items, n_factors)
Item factor matrix
- bundarray of shape (n_users,)
User biases
- bindarray of shape (n_items,)
Item biases
- mufloat
Global mean rating
Notes¶
The rating generation follows the standard matrix factorization model:
r_ui = μ + b_u + b_i + p_u · q_i^T + ε
where ε ~ N(0, noise_std²)
The generated ratings are clipped to stay within [rating_min, rating_max] range.
- rehline.CQR_Ridge_path_sol(X, y, *, quantiles, eps=1e-05, n_Cs=50, Cs=None, max_iter=5000, tol=0.0001, verbose=0, shrink=1, warm_start=False, return_time=True)¶
Compute the regularization path for Composite Quantile Regression (CQR) with ridge penalty.
This function fits a series of CQR models using different values of the regularization parameter C. It reuses a single estimator and modifies C in-place before refitting.
Parameters¶
- Xndarray of shape (n_samples, n_features)
Feature matrix.
- yndarray of shape (n_samples,)
Response vector.
- quantileslist of float
Quantile levels (e.g. [0.1, 0.5, 0.9]).
- epsfloat, default=1e-5
Log-scaled lower bound for generated C values (used if Cs is None).
- n_Csint, default=50
Number of C values to generate.
- Csarray-like or None, default=None
Explicit values of regularization strength. If None, use eps and n_Cs to generate them.
- max_iterint, default=5000
Maximum number of solver iterations.
- tolfloat, default=1e-4
Solver convergence tolerance.
- verboseint, default=0
Verbosity level.
- shrinkfloat, default=1
Shrinkage parameter passed to solver.
- warm_startbool, default=False
Use previous dual solution to initialize the next fit.
- return_timebool, default=True
Whether to return a list of fit durations.
Returns¶
- Csndarray
List of regularization strengths.
- modelslist
List of fitted model objects.
- coefsndarray of shape (n_Cs, n_quantiles, n_features)
Coefficient matrices per quantile and C.
- interceptsndarray of shape (n_Cs, n_quantiles)
Intercepts per quantile and C.
- fit_timeslist of float, optional
Elapsed fit times (if return_time=True).
Example¶
>>> from sklearn.datasets import make_friedman1 >>> from sklearn.preprocessing import StandardScaler >>> import numpy as np >>> from rehline import CQR_Ridge_path_sol
>>> # Generate the data >>> X, y = make_friedman1(n_samples=500, n_features=6, noise=1.0, random_state=42) >>> X = StandardScaler().fit_transform(X) >>> y = y / y.std()
>>> # Set quantiles and Cs >>> quantiles = [0.1, 0.5, 0.9] >>> Cs = np.logspace(-5, 0, 30)
>>> # Fit CQR path >>> Cs, models, coefs, intercepts, fit_times = CQR_Ridge_path_sol( ... X, y, ... quantiles=quantiles, ... Cs=Cs, ... max_iter=100000, ... tol=1e-4, ... verbose=1, ... warm_start=True, ... return_time=True ... )
- rehline.plqERM_Ridge_path_sol(X, y, *, loss, constraint=None, eps=0.001, n_Cs=100, Cs=None, max_iter=5000, tol=0.0001, verbose=0, shrink=1, warm_start=False, return_time=True)¶
Compute the PLQ Empirical Risk Minimization (ERM) path over a range of regularization parameters. This function evaluates the model’s performance for different values of the regularization parameter and provides structured benchmarking output.
Parameters¶
- Xndarray of shape (n_samples, n_features)
Training input samples.
- yndarray of shape (n_samples,)
Target values corresponding to each input sample.
- lossdict
Dictionary describing the PLQ loss function parameters. Used to construct the loss object internally.
- constraintlist of dict, optional (default=[])
List of constraints applied to the optimization problem. Each constraint should be represented as a dictionary compatible with the solver.
- epsfloat, default=1e-3
Defines the length of the regularization path when Cs is not provided. The values of C will range from 10^log10(eps) to 10^-log10(eps).
- n_Csint, default=100
Number of regularization values to evaluate if Cs is not provided.
- Csarray-like of shape (n_Cs,), optional
Explicit values of regularization strength C to use. If None, the values are generated logarithmically between 1e-2 and 1e3.
- max_iterint, default=5000
Maximum number of iterations allowed for the optimization solver at each C.
- tolfloat, default=1e-4
Tolerance for solver convergence.
- verboseint, default=0
Controls verbosity level of output. Set to higher values (e.g., 1 or 2) for detailed progress logs. When verbose = 1, only print path results table; when verbose = 2, print path results table and path solution plot.
- shrinkfloat, default=1
Shrinkage factor for the solver, potentially influencing convergence behavior.
- warm_startbool, default=False
If True, reuse the previous solution to warm-start the next solver step, speeding up convergence.
- return_timebool, default=True
If True, return timing information for each value of C.
Returns¶
- Csndarray of shape (n_Cs,)
Array of regularization parameters used in the path.
- timeslist of float
Time in seconds taken to fit the model at each C. Returned only if return_time=True.
- n_iterslist of int
Number of iterations used by the solver at each regularization value.
- obj_valueslist of float
Final objective values (including loss and regularization terms) at each C.
- L2_normslist of float
L2 norm of the coefficients (excluding bias) at each C.
- coefsndarray of shape (n_features, n_Cs)
Learned model coefficients at each regularization strength.
Example¶
>>> # generate data >>> np.random.seed(42) >>> n, d, C = 1000, 5, 0.5 >>> X = np.random.randn(n, d) >>> beta0 = np.random.randn(d) >>> y = np.sign(X.dot(beta0) + np.random.randn(n)) >>> # define loss function >>> loss = {'name': 'svm'} >>> Cs = np.logspace(-1,3,15) >>> constraint = [{'name': 'nonnegative'}]
>>> # calculate >>> Cs, times, n_iters, losses, norms, coefs = plqERM_Ridge_path_sol( ... X, y, loss=loss, Cs=Cs, max_iter=100000,tol=1e-4,verbose=2, ... warm_start=False, constraint=constraint, return_time=True ... )