rehline

Overview

Classes

CQR_Ridge	Composite Quantile Regressor (CQR) with a ridge penalty.
ReHLine	ReHLine Minimization [1].
plqERM_ElasticNet	Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a elastic net penalty.
plqERM_Ridge	Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty.
plqMF_Ridge	Matrix Factorization (MF) with a piecewise linear-quadratic objective and ridge penalty.
plq_ElasticNet_Classifier	Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss
plq_ElasticNet_Regressor	Empirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss
plq_Ridge_Classifier	Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss
plq_Ridge_Regressor	Empirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss

Function

ReHLine_solver(X, U, V, Tau, S, T, A, b, rho, Lambda, Gamma, xi, mu, max_iter, tol, shrink, verbose, trace_freq, *None, coordinate_order, coordinate_seed, _quantile_count)	Solve a ReLU/ReHU problem with independently selectable update order.
make_mf_dataset(n_users, n_items, n_factors, n_interactions, density, noise_std, seed, rating_min, rating_max, return_params)	Generate synthetic rating data using matrix factorization model.
CQR_Ridge_path_sol(X, y, *None, quantiles, eps, n_Cs, Cs, max_iter, tol, verbose, shrink, coordinate_order, coordinate_seed, warm_start, return_time, compact)	Compute the regularization path for Composite Quantile Regression (CQR) with ridge penalty.
plqERM_Ridge_path_sol(X, y, *None, loss, constraint, eps, n_Cs, Cs, max_iter, tol, verbose, shrink, coordinate_order, coordinate_seed, warm_start, return_time)	Compute the PLQ Empirical Risk Minimization (ERM) path over a range of regularization parameters.

Classes

class rehline.CQR_Ridge(quantiles, C=1.0, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, *, coordinate_order='auto', coordinate_seed=None)

Bases: rehline._base._SparseInputMixin, rehline._base._BaseReHLine, sklearn.base.BaseEstimator

Composite Quantile Regressor (CQR) with a ridge penalty.

It allows for the fitting of a linear regression model that minimizes a composite quantile loss function.

Slopes are shared across quantiles and fitted jointly with one intercept per quantile. Both slopes and intercepts are regularized. The solver uses the original feature matrix implicitly, without expanding it to (n_samples * n_quantiles, n_features + n_quantiles). Loss and dual storage still grows with n_samples * n_quantiles.

\[\min_{\mathbf{\beta} \in \mathbb{R}^d, \mathbf{\beta_0} \in \mathbb{R}^K} C \sum_{k=1}^K \sum_{i=1}^n w_i \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta} + \mathbf{\beta_0k}) + \frac{1}{2} (\| \mathbf{\beta} \|_2^2 + \| \mathbf{\beta_0} \|_2^2).\]

Parameters

quantileslist of float (n_quantiles,)

The quantiles to be estimated.

Cfloat, default=1.0

Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. C will be absorbed by the ReHLine parameters when self.make_ReLHLoss is conducted.

verboseint, default=0

Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in liblinear that, if enabled, may not work properly in a multithreaded context.

max_iterint, default=1000

The maximum number of iterations to be run.

tolfloat, default=1e-4

Relative primal-dual gap and normalized feasibility tolerance. The absolute KKT residual is reported separately.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

warm_startbool, default=False

Whether to use the given dual params as an initial guess for the optimization algorithm.

trace_freqint, default=100

The frequency at which to print the optimization trace.

Attributes

coef_array-like

The optimized model coefficients.

intercept_array-like

The optimized model intercepts.

quantiles_: array-like

The quantiles to be estimated.

n_iter_int

The number of iterations performed by the ReHLine solver.

opt_result_object

The optimization result object.

dual_obj_array-like

The dual objective function values.

primal_obj_array-like

The primal objective function values.

Methods

fit(X, y, sample_weight=None)

Fit the model based on the given training data.

predict(X)

The prediction for the given dataset.

Overview

Methods

fit(X, y, sample_weight)	Fit the model based on the given training data.
to_inference()	Return an independent prediction snapshot without loss/dual state.
predict(X)	The prediction for the given dataset.

Members

fit(X, y, sample_weight=None)

Fit the model based on the given training data.

Parameters

X: {array-like} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features.

yarray-like of shape (n_samples,)

The target variable.

sample_weightarray-like of shape (n_samples,), default=None

Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.

Returns

selfobject

An instance of the estimator.

to_inference()

Return an independent prediction snapshot without loss/dual state.

The snapshot preserves fitted quantiles and final diagnostics, supports pickle/joblib serialization, and cannot be fitted or warm-started. The original estimator is unchanged and remains trainable.

predict(X)

The prediction for the given dataset.

Parameters

Xarray-like of shape (n_samples, n_features)

The data matrix.

Returns

ndarray of shape (n_samples, n_quantiles)

Returns the predicted quantile values for the samples.

class rehline.ReHLine(C=1.0, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, *, coordinate_order='auto', coordinate_seed=None)

Bases: rehline._base._SparseInputMixin, rehline._base._BaseReHLine, sklearn.base.BaseEstimator

ReHLine Minimization [1].

\[\begin{split}\min_{\mathbf{\beta} \in \mathbb{R}^d} \sum_{i=1}^n \sum_{l=1}^L \text{ReLU}( u_{li} \mathbf{x}_i^\intercal \mathbf{\beta} + v_{li}) + \sum_{i=1}^n \sum_{h=1}^H {\text{ReHU}}_{\tau_{hi}}( s_{hi} \mathbf{x}_i^\intercal \mathbf{\beta} + t_{hi}) + \frac{1}{2} \| \mathbf{\beta} \|_2^2, \\ \text{ s.t. } \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0},\end{split}\]

where \(\mathbf{U} = (u_{li}),\mathbf{V} = (v_{li}) \in \mathbb{R}^{L \times n}\) and \(\mathbf{S} = (s_{hi}),\mathbf{T} = (t_{hi}),\mathbf{\tau} = (\tau_{hi}) \in \mathbb{R}^{H \times n}\) are the ReLU-ReHU loss parameters, and \((\mathbf{A},\mathbf{b})\) are the constraint parameters.

Parameters

Cfloat, default=1.0

Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive.

_U, _V: array of shape (L, n_samples), default=np.empty(shape=(0, 0))

The parameters pertaining to the ReLU part in the loss function.

_Tau, _S, _T: array of shape (H, n_samples), default=np.empty(shape=(0, 0))

The parameters pertaining to the ReHU part in the loss function.

_A: array of shape (K, n_features), default=np.empty(shape=(0, 0))

The coefficient matrix in the linear constraint.

_b: array of shape (K, ), default=np.empty(shape=0)

The intercept vector in the linear constraint.

verboseint, default=0

Enable verbose output.

max_iterint, default=1000

The maximum number of iterations to be run.

tolfloat, default=1e-4

Relative primal-dual gap and normalized feasibility tolerance. The absolute KKT residual is reported separately.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

warm_startbool, default=False

Whether to use the given dual params as an initial guess for the optimization algorithm.

trace_freqint, default=100

The frequency at which to print the optimization trace.

Attributes

coef_array-like

The optimized model coefficients.

n_iter_int

The number of iterations performed by the ReHLine solver.

opt_result_object

The optimization result object.

dual_obj_array-like

The dual objective function values.

primal_obj_array-like

The primal objective function values.

_Lambda: array-like

The optimized dual variables for ReLU parts.

_Gamma: array-like

The optimized dual variables for ReHU parts.

_xi: array-like

The optimized dual variables for linear constraints.

Examples

>>> ## test SVM on simulated dataset
>>> import numpy as np
>>> from rehline import ReHLine

>>> # simulate classification dataset
>>> n, d, C = 1000, 3, 0.5
>>> np.random.seed(1024)
>>> X = np.random.randn(1000, 3)
>>> beta0 = np.random.randn(3)
>>> y = np.sign(X.dot(beta0) + np.random.randn(n))

>>> # Usage of ReHLine
>>> n, d = X.shape
>>> U = -(C*y).reshape(1,-1)
>>> L = U.shape[0]
>>> V = (C*np.array(np.ones(n))).reshape(1,-1)
>>> clf = ReHLine(C=C)
>>> clf._U, clf._V = U, V
>>> clf.fit(X=X)
>>> print('sol provided by rehline: %s' %clf.coef_)
>>> sol provided by rehline: [ 0.7410154  -0.00615574  2.66990408]
>>> print(clf.decision_function([[.1,.2,.3]]))
>>> [0.87384162]

References

[1] (1,2)

Dai, B., Qiu, Y,. (2023). ReHLine: Regularized Composite ReLU-ReHU Loss Minimization with Linear Computation and Linear Convergence

Overview

Methods

fit(X, sample_weight)	Fit the model based on the given training data.
decision_function(X)	The decision function evaluated on the given dataset

Members

fit(X, sample_weight=None)

Fit the model based on the given training data.

Parameters

X: {array-like} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features.

sample_weightarray-like of shape (n_samples,), default=None

Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.

Returns

selfobject

An instance of the estimator.

decision_function(X)

The decision function evaluated on the given dataset

Parameters

Xarray-like of shape (n_samples, n_features)

The data matrix.

Returns

ndarray of shape (n_samples, )

Returns the decision function of the samples.

class rehline.plqERM_ElasticNet(loss, constraint=None, C=1.0, l1_ratio=0.5, omega=None, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, *, coordinate_order='auto', coordinate_seed=None)

Bases: rehline._base._SparseInputMixin, rehline._base._BaseReHLine, sklearn.base.BaseEstimator

Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a elastic net penalty.

\[\min_{\mathbf{\beta} \in \mathbb{R}^d} C \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta}) + \text{l1_ratio} \sum_{j=1}^d \omega_j | \beta_j | + \frac{1}{2} (1 - \text{l1_ratio}) \| \mathbf{\beta} \|_2^2, \ \text{ s.t. } \ \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0},\]

The function supports various loss functions, including:

• ‘hinge’, ‘svm’ or ‘SVM’

• ‘check’ or ‘quantile’ or ‘quantile regression’ or ‘QR’

• ‘sSVM’ or ‘smooth SVM’ or ‘smooth hinge’

• ‘TV’

• ‘huber’ or ‘Huber’

• ‘SVR’ or ‘svr’

The following constraint types are supported:

• ‘nonnegative’ or ‘>=0’: A non-negativity constraint.

• ‘fair’ or ‘fairness’: A fairness constraint.

• ‘custom’: A custom constraint, where the user must provide the constraint matrix ‘A’ and vector ‘b’.

Parameters

lossdict

A dictionary specifying the loss function parameters.

constraintlist of dict

A list of dictionaries, where each dictionary represents a constraint. Each dictionary must contain a ‘name’ key, which specifies the type of constraint.

Cfloat, default=1.0

Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. C will be absorbed by the ReHLine parameters when self.make_ReLHLoss is conducted.

l1_ratiofloat, default=0.5

The ElasticNet mixing parameter, with 0 <= l1_ratio < 1. For l1_ratio = 0 the penalty is an L2 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.

omegaarray of shape (n_features, ), default=np.empty(shape=0)

Non-negative weight coefficients for adaptive lasso. If not provided, all coefficients receive the same L1 penalty controlled by l1_ratio.

verboseint, default=0

Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in liblinear that, if enabled, may not work properly in a multithreaded context.

max_iterint, default=1000

The maximum number of iterations to be run.

U, V, S, T, TauNone or empty array, default=None

Legacy parameters retained for constructor compatibility. Named-loss estimators generate these matrices from loss; nonempty values raise ValueError at fit time. Use ReHLine or ReHLine_solver for manual matrices.

A: array of shape (K, n_features), default=None

Additional linear-constraint coefficients. When constraint is also supplied, both constraint lists are enforced, with a warning per fit.

b: array of shape (K,), default=None

Offsets in A @ coef_ + b >= 0; must be supplied together with A.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

Attributes

coef_array-like

The optimized model coefficients.

n_iter_int

The number of iterations performed by the ReHLine solver.

opt_result_object

The optimization result object.

dual_obj_array-like

The dual objective function values.

primal_obj_array-like

The primal objective function values.

Methods

fit(X, y, sample_weight=None)

Fit the model based on the given training data.

decision_function(X)

The decision function evaluated on the given dataset.

Notes

The plqERM_ElasticNet class is a subclass of _BaseReHLine and BaseEstimator, which suggests that it is part of a larger framework for implementing ReHLine algorithms.

Overview

Methods

fit(X, y, sample_weight)	Fit the model based on the given training data.
decision_function(X)	The decision function evaluated on the given dataset

Members

fit(X, y, sample_weight=None)

Fit the model based on the given training data.

Parameters

X: {array-like} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features.

yarray-like of shape (n_samples,)

The target variable.

sample_weightarray-like of shape (n_samples,), default=None

Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.

Returns

selfobject

An instance of the estimator.

decision_function(X)

The decision function evaluated on the given dataset

Parameters

Xarray-like of shape (n_samples, n_features)

The data matrix.

Returns

ndarray of shape (n_samples, )

Returns the decision function of the samples.

class rehline.plqERM_Ridge(loss, constraint=None, C=1.0, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, *, coordinate_order='auto', coordinate_seed=None)

Bases: rehline._base._SparseInputMixin, rehline._base._BaseReHLine, sklearn.base.BaseEstimator

Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty.

\[\min_{\mathbf{\beta} \in \mathbb{R}^d} \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta}) + \frac{1}{2} \| \mathbf{\beta} \|_2^2, \ \text{ s.t. } \ \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0},\]

The function supports various loss functions, including:

• ‘hinge’, ‘svm’ or ‘SVM’

• ‘check’ or ‘quantile’ or ‘quantile regression’ or ‘QR’

• ‘sSVM’ or ‘smooth SVM’ or ‘smooth hinge’

• ‘TV’

• ‘huber’ or ‘Huber’

• ‘SVR’ or ‘svr’

The following constraint types are supported:

• ‘nonnegative’ or ‘>=0’: A non-negativity constraint.

• ‘fair’ or ‘fairness’: A fairness constraint.

• ‘custom’: A custom constraint, where the user must provide the constraint matrix ‘A’ and vector ‘b’.

Parameters

lossdict

A dictionary specifying the loss function parameters.

constraintlist of dict

A list of dictionaries, where each dictionary represents a constraint. Each dictionary must contain a ‘name’ key, which specifies the type of constraint.

Cfloat, default=1.0

Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. C will be absorbed by the ReHLine parameters when self.make_ReLHLoss is conducted.

verboseint, default=0

Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in liblinear that, if enabled, may not work properly in a multithreaded context.

max_iterint, default=1000

The maximum number of iterations to be run.

U, V, S, T, TauNone or empty array, default=None

Legacy parameters retained for constructor compatibility. Named-loss estimators generate these matrices from loss; nonempty values raise ValueError at fit time. Use ReHLine or ReHLine_solver for manual matrices.

A: array of shape (K, n_features), default=None

Additional linear-constraint coefficients. When constraint is also supplied, both constraint lists are enforced, with a warning per fit.

b: array of shape (K,), default=None

Offsets in A @ coef_ + b >= 0; must be supplied together with A.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

Attributes

coef_array-like

The optimized model coefficients.

n_iter_int

The number of iterations performed by the ReHLine solver.

opt_result_object

The optimization result object.

dual_obj_array-like

The dual objective function values.

primal_obj_array-like

The primal objective function values.

Methods

fit(X, y, sample_weight=None)

Fit the model based on the given training data.

decision_function(X)

The decision function evaluated on the given dataset.

Notes

The plqERM_Ridge class is a subclass of _BaseReHLine and BaseEstimator, which suggests that it is part of a larger framework for implementing ReHLine algorithms.

Overview

Methods

fit(X, y, sample_weight)	Fit the model based on the given training data.
decision_function(X)	The decision function evaluated on the given dataset

Members

fit(X, y, sample_weight=None)

Fit the model based on the given training data.

Parameters

X: {array-like} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features.

yarray-like of shape (n_samples,)

The target variable.

sample_weightarray-like of shape (n_samples,), default=None

Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.

Returns

selfobject

An instance of the estimator.

decision_function(X)

The decision function evaluated on the given dataset

Parameters

Xarray-like of shape (n_samples, n_features)

The data matrix.

Returns

ndarray of shape (n_samples, )

Returns the decision function of the samples.

class rehline.plqMF_Ridge(n_users, n_items, loss, biased=True, constraint_user=None, constraint_item=None, rank=10, C=1.0, rho=0.5, init_mean=0.0, init_sd=0.1, random_state=None, max_iter=10000, tol=0.0001, shrink=1, trace_freq=100, max_iter_CD=10, tol_CD=0.0001, verbose=0, *, coordinate_order='auto', coordinate_seed=None)

Bases: rehline._base._BaseReHLine, sklearn.base.BaseEstimator

Matrix Factorization (MF) with a piecewise linear-quadratic objective and ridge penalty.

\[\begin{split}\min_{\substack{ \mathbf{P} \in \mathbb{R}^{n \times k}\ \pmb{\alpha} \in \mathbb{R}^n \\ \mathbf{Q} \in \mathbb{R}^{m \times k}\ \pmb{\beta} \in \mathbb{R}^m }} \left[ \sum_{(u,i)\in \Omega} C \cdot \text{PLQ}(r_{ui}, \ \mathbf{p}_u^T \mathbf{q}_i + \alpha_u + \beta_i) \right] + \left[ \frac{\rho}{n}\sum_{u=1}^n(\|\mathbf{p}_u\|_2^2 + \alpha_u^2) + \frac{1-\rho}{m}\sum_{i=1}^m(\|\mathbf{q}_i\|_2^2 + \beta_i^2) \right]\end{split}\]

\[\begin{split}\ \text{ s.t. } \ \mathbf{A}_{\text{user}} \begin{pmatrix} \alpha_u \\ \mathbf{p}_u \end{pmatrix} + \mathbf{b}_{\text{user}} \geq \mathbf{0},\ u = 1,\dots,n \quad \text{and} \quad \mathbf{A}_{\text{item}} \begin{pmatrix} \beta_i \\ \mathbf{q}_i \end{pmatrix} + \mathbf{b}_{\text{item}} \geq \mathbf{0},\ i = 1,\dots,m\end{split}\]

The function supports various loss functions, including:

• ‘hinge’, ‘svm’ or ‘SVM’

• ‘MAE’ or ‘mae’ or ‘mean absolute error’

• ‘squared hinge’ or ‘squared svm’ or ‘squared SVM’

• ‘MSE’ or ‘mse’ or ‘mean squared error’

The following constraint types are supported:

• ‘nonnegative’ or ‘>=0’: A non-negativity constraint.

• ‘fair’ or ‘fairness’: A fairness constraint.

• ‘custom’: A custom constraint, where the user must provide the constraint matrix ‘A’ and vector ‘b’.

Parameters

n_usersint

Number of unique users in the dataset (or number of rows in target sparse matrix).

n_itemsint

Number of unique items in the dataset (or number of columns in target sparse matrix).

lossdict

A dictionary specifying the loss function parameters.

constraint_userlist of dict

A list of dictionaries, where each dictionary represents a constraint to user side parameters. Each dictionary must contain a ‘name’ key, which specifies the type of constraint.

constraint_itemlist of dict

A list of dictionaries, where each dictionary represents a constraint to item side parameters. Each dictionary must contain a ‘name’ key, which specifies the type of constraint.

biasedbool, default=True

Whether to include user and item bias terms in the model.

rankint, default=10

Dimensionality of the latent factor vectors (number of factors).

Cfloat, default=1.0

Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. C will be absorbed by the ReHLine parameters when _cast_sample_weight() is conducted.

rhofloat, default=0.5

Regularization strength ratio between user and item factors. Must be within the range of (0,1).

init_meanfloat, default=0.0

Mean of the Gaussian distribution for initializing latent factors.

init_sdfloat, default=0.1

Standard deviation of the Gaussian distribution for initializing latent factors.

random_stateint, RandomState or Generator, default=None

Random seed for reproducible initialization of latent factors.

max_iterint, default=10000

The maximum number of iterations to be run for the ReHLine solver.

tolfloat, default=1e-4

Convergence tolerance for each ReHLine block solve and the final row-normalized factor constraint violation.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

trace_freqint, default=100

The frequency at which to print the optimization trace for the ReHLine solver.

max_iter_CDint, default=10

Maximum number of iterations for coordinate descent steps.

tol_CDfloat, default=1e-4

The tolerance for the stopping criterion for coordinate descent steps.

verboseint, default=0

Verbosity level.

0: No output 1: CD iteration progress information 2: ReHLine solver optimization information 3: All information of CD and ReHLine

Attributes

n_usersint

Number of unique users in the dataset (or number of rows in target sparse matrix).

n_itemsint

Number of unique items in the dataset (or number of columns in target sparse matrix).

n_ratingsint

Number of ratings in the training data. Available after fitting.

Pndarray of shape (n_users, rank)

User latent factor matrix. Learned during fitting.

Qndarray of shape (n_items, rank)

Item latent factor matrix. Learned during fitting.

bundarray of shape (n_users,) or None

User bias terms. Learned during fitting. Only available if biased=True.

bindarray of shape (n_items,) or None

Item bias terms. Learned during fitting. Only available if biased=True.

Iulist of ndarray

List where each element contains indices of items rated by the corresponding user. Available after fitting.

Uilist of ndarray

List where each element contains indices of users who rated the corresponding item. Available after fitting.

historyndarray of shape (max_iter_CD + 1, 2)

Optimization history containing loss and objective values at each coordinate descent iteration. First column: weighted cumulative loss. Second column: weighted objective including the penalty. Unused rows after early stopping are NaN.

objective_float

Final training objective, including sample weights and both penalties.

n_iter_int

Number of completed outer coordinate-descent sweeps.

converged_bool

Whether the weighted objective stopping criterion, inner convergence and final feasibility checks passed. This does not certify a global MF optimum. False when the outer iteration budget is exhausted without convergence. An exhausted outer budget emits ConvergenceWarning mentioning max_iter_CD.

inner_converged_bool

Whether all block solves in the last sweep converged.

constraint_violation_float

Maximum violation of the final user/item constraints in original units.

scaled_constraint_violation_float

Maximum violation after each constraint row (A, b) is divided by max(abs(A)). A zero row uses scale 1. This diagnostic is compared with tol for outer convergence and constraint warnings; it is invariant to positive row rescaling within floating-point accuracy.

sample_weightndarray of shape (n_ratings,)

Sample weights used during fitting. Available after fitting.

Methods

fit(X, y, sample_weight=None)

Fit the model based on the given training data.

decision_function(X)

The decision function evaluated on the given dataset.

obj(X, y)

Compute the values of loss term and objective function.

Notes

The plqMF_Ridge class is a subclass of _BaseReHLine and BaseEstimator, which suggests that it is part of a larger framework for implementing ReHLine algorithms.

Overview

Methods

fit(X, y, sample_weight)	Fit the model based on the given training data.
decision_function(X)	The decision function evaluated on the given dataset
obj(X, y, sample_weight)	Compute the values of loss term and objective function.

Members

fit(X, y, sample_weight=None)

Fit the model based on the given training data.

Parameters

Xarray-like of shape (n_ratings, 2)

Input data where first column contains user ID and second column contains item ID.

yarray-like of shape (n_ratings,)

Target rating values.

sample_weightarray-like of shape (n_samples,), default=None

Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.

Returns

selfobject

An instance of the estimator.

decision_function(X)

The decision function evaluated on the given dataset

Parameters

Xarray-like of shape (n_samples, 2)

Training data where first column contains user ID and second column contains item ID.

Returns

predictionndarray of shape (n_samples,)

Predicted ratings for the input pairs.

obj(X, y, sample_weight=None)

Compute the values of loss term and objective function.

Parameters

Xarray-like of shape (n_ratings, 2)

User-item rating pairs.

yarray-like of shape (n_ratings,)

Actual rating values.

sample_weightarray-like or float, default=None

Evaluation weights; None means equal weights. Training weights are not implicitly reused for evaluation on another dataset.

Returns

loss_termfloat

The data fitting term (sum of loss values).

objective_valuefloat

The total objective value including regularization.

class rehline.plq_ElasticNet_Classifier(loss, constraint=None, C=1.0, l1_ratio=0.5, omega=None, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, class_weight=None, multi_class=None, n_jobs=None, decision_function_shape='ovr', *, coordinate_order='auto', coordinate_seed=None)

Bases: _ReHLineClassifier

Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss and elastic net penalty, compatible with the scikit-learn API.

This wrapper makes plqERM_ElasticNet behave as a classifier:

• Accepts arbitrary binary labels in the original label space.

• Computes class weights on original labels (if class_weight is set).

• Encodes labels with LabelEncoder into {0,1}, then maps to {-1,+1} for training.

• Supports optional intercept fitting (via an augmented constant feature).

• Provides standard methods fit, predict, and decision_function.

• Integrates with scikit-learn ecosystem (e.g., GridSearchCV, Pipeline).

• Supports multiclass classification via OvR or OvO method.

Parameters

lossdict

Dictionary specifying the loss function parameters. Examples include: - {‘name’: ‘svm’} - {‘name’: ‘sSVM’} - {‘name’: ‘huber’} and other PLQ losses supported by plqERM_ElasticNet.

constraintlist of dict or None, default=None

Optional constraints. Each dictionary must include a 'name' key.

Cfloat, default=1.0

Inverse regularization strength (scales the loss term).

l1_ratiofloat, default=0.5

The ElasticNet mixing parameter, 0 <= l1_ratio < 1. - l1_ratio = 0 → pure Ridge (equivalent to plq_Ridge_Classifier) - 0 < l1_ratio < 1 → combined L1 + L2 penalty Must be strictly less than 1.0 to avoid division by zero in rho/C_eff.

omegaarray of shape (n_features, ), default=np.empty(shape=(0, 0))

Non-negative weight coefficients for adaptive lasso. If not provided, all non-intercept coefficients receive the same L1 penalty controlled by l1_ratio. The penalty for the intercept can be scaled via intercept_scaling.

fit_interceptbool, default=True

Whether to fit an intercept term via an augmented constant feature column.

intercept_scalingfloat, default=1.0

Value of the constant feature column when fit_intercept=True.

class_weightdict, ‘balanced’, or None, default=None

‘balanced’ gives sample i effective weight w_i * sum(w) / (n_classes * sum(w[y == y_i])), computed on retained original classes before binary/OvR/OvO decomposition. Zero-weight rows are excluded. The definition matches LinearSVC >= 1.7 on every supported sklearn version. A dict maps original labels to multipliers of sample_weight.

multi_classstr or None, default=None

Method for multiclass classification: - ‘ovr’: One-vs-Rest - ‘ovo’: One-vs-One - None selects OvR; binary problems use one estimator.

decision_function_shape{‘ovr’, ‘ovo’}, default=’ovr’

Score output format, independent of the training strategy. ‘ovr’ returns one score per class; ‘ovo’ requires OvO training and returns one margin per sorted class pair, positive for the first class (like SVC). Binary output is always 1D, positive for classes_[1]. Changing this parameter after fitting does not change predictions or the fitted optimization.

n_jobsint or None, default=None

Number of parallel jobs for multiclass fitting.

max_iter : int, default=1000 tol : float, default=1e-4 shrink : int, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

warm_start : int, default=0 verbose : int, default=0 trace_freq : int, default=100

Attributes

coef_ndarray of shape (n_features,) for binary, (n_estimators, n_features) for multiclass

OvO training coefficients favor the second class of each sorted pair; raw multiclass decision scores use the opposite sign, like SVC.

intercept_ : float for binary, ndarray of shape (n_estimators,) for multiclass classes_ : ndarray of shape (n_classes,) estimators_ : list, only present for multiclass _label_encoder : LabelEncoder

class rehline.plq_ElasticNet_Regressor(loss=None, constraint=None, C=1.0, l1_ratio=0.5, omega=None, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, *, coordinate_order='auto', coordinate_seed=None)

Bases: sklearn.base.RegressorMixin, _SklearnReHLine

Empirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss and an elastic net penalty, implemented as a scikit-learn compatible estimator.

This wrapper makes plqERM_ElasticNet behave as a regressor:

• Supports optional intercept fitting via an augmented constant feature column.

• Provides standard methods fit, predict, and score.

• Integrates with the scikit-learn ecosystem (e.g., GridSearchCV, Pipeline).

Notes

• Intercept handling: if fit_intercept=True, a constant column (value = intercept_scaling) is appended to the right of the design matrix before calling the base solver. The last learned coefficient is then multiplied by intercept_scaling to obtain intercept_. Original feature indices are therefore unaffected; sen_idx in a 'fair' constraint continues to reference the original columns.

• Sparse input: not supported. Convert to dense before fitting.

Parameters

lossdict or None, default=None

None selects {‘name’: ‘QR’, ‘qt’: 0.5} at fit time. PLQ loss configuration. Examples: {'name': 'QR', 'qt': 0.5}, {'name': 'huber', 'tau': 1.0}, {'name': 'SVR', 'epsilon': 0.1}.

constraintlist of dict or None, default=None

Constraint specifications:

• {'name': 'nonnegative'} or {'name': '>=0'}

• {'name': 'fair', 'sen_idx': list[int], 'tol_sen': list[float]}

• {'name': 'custom', 'A': ndarray[K, d], 'b': ndarray[K]}

Cfloat, default=1.0

Regularization parameter (scales the loss term).

l1_ratiofloat, default=0.5

The ElasticNet mixing parameter, 0 <= l1_ratio < 1. - l1_ratio = 0 → pure Ridge (equivalent to plq_Ridge_Regressor) - 0 < l1_ratio < 1 → combined L1 + L2 penalty Must be strictly less than 1.0 to avoid division by zero in rho/C_eff.

omegaarray of shape (n_features, ), default=np.empty(shape=(0, 0))

Non-negative weight coefficients for adaptive lasso. If not provided, all non-intercept coefficients receive the same L1 penalty controlled by l1_ratio. The penalty for the intercept can be scaled via intercept_scaling.

fit_interceptbool, default=True

If True, append a constant column (value = intercept_scaling) to the design matrix before solving. The last learned coefficient is then extracted as intercept_.

intercept_scalingfloat, default=1.0

Scaling applied to the appended constant column when fit_intercept=True.

max_iter : int, default=1000 tol : float, default=1e-4 shrink : int, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

warm_start : int, default=0 verbose : int, default=0 trace_freq : int, default=100

Attributes

coef_ndarray of shape (n_features,)

Learned linear coefficients (excluding the intercept term).

intercept_float

Intercept term. 0.0 if fit_intercept=False.

n_features_in_int

Number of input features seen during fit() (before intercept augmentation).

class rehline.plq_Ridge_Classifier(loss, constraint=None, C=1.0, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, class_weight=None, multi_class=None, n_jobs=None, decision_function_shape='ovr', *, coordinate_order='auto', coordinate_seed=None)

Bases: _ReHLineClassifier

Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss and ridge penalty, compatible with the scikit-learn API.

This wrapper makes plqERM_Ridge behave as a classifier:

• Accepts arbitrary binary labels in the original label space.

• Computes class weights on original labels (if class_weight is set).

• Encodes labels with LabelEncoder into {0,1}, then maps to {-1,+1} for training.

• Supports optional intercept fitting (via an augmented constant feature).

• Provides standard methods fit, predict, and decision_function.

• Integrates with scikit-learn ecosystem (e.g., GridSearchCV, Pipeline).

• Supports multiclass classification via OvR or OvO method.

Parameters

lossdict

Dictionary specifying the loss function parameters. Examples include: - {‘name’: ‘svm’} - {‘name’: ‘sSVM’} - {‘name’: ‘huber’} and other PLQ losses supported by plqERM_Ridge.

constraintlist of dict or None, default=None

Optional constraints. Each dictionary must include a 'name' key. Examples: {‘name’: ‘nonnegative’}, {‘name’: ‘fair’}, {‘name’: ‘custom’}.

Cfloat, default=1.0

Inverse regularization strength.

U, V, S, T, TauNone or empty array, default=None

Legacy constructor parameters. Nonempty values raise ValueError at fit time because these matrices are generated from loss. Use ReHLine or ReHLine_solver for manually specified loss matrices.

_Andarray of shape (K, n_features), default empty

Linear-constraint coefficient matrix.

_bndarray of shape (K,), default empty

Linear-constraint intercept vector.

max_iterint, default=1000

Maximum number of iterations for the ReHLine solver.

tolfloat, default=1e-4

Relative primal-dual gap and normalized feasibility tolerance. The absolute KKT residual is reported separately.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

warm_startint, default=0

Whether to reuse the previous solution for initialization.

verboseint, default=0

Verbosity level for the solver.

trace_freqint, default=100

Frequency (in iterations) at which solver progress is traced when verbose > 0.

fit_interceptbool, default=True

Whether to fit an intercept term. If True, a constant feature column is added to X during training. The last learned coefficient is extracted as intercept_.

intercept_scalingfloat, default=1.0

Value used for the constant feature column when fit_intercept=True. Matches the convention used in scikit-learn’s LinearSVC.

class_weightdict, ‘balanced’, or None, default=None

‘balanced’ gives sample i effective weight w_i * sum(w) / (n_classes * sum(w[y == y_i])). Balancing uses retained original classes before binary/OvR/OvO decomposition. Zero-weight rows are excluded. This matches LinearSVC >= 1.7 and has the same definition with every supported sklearn version. A dict maps original labels to multipliers of sample_weight.

multi_classstr or None, default=None

Method for multiclass classification. Options: - ‘ovo’: One-vs-One, trains K*(K-1)/2 binary classifiers. - ‘ovr’: One-vs-Rest, trains K binary classifiers. - None selects OvR; binary problems use one estimator.

decision_function_shape{‘ovr’, ‘ovo’}, default=’ovr’

Score output format, independent of the training strategy. ‘ovr’ returns one score per class; ‘ovo’ requires OvO training and returns one margin per sorted class pair, positive for the first class (like SVC). Binary output is always 1D, positive for classes_[1]. Changing this parameter after fitting does not change predictions or the fitted optimization.

n_jobsint or None, default=None

Number of parallel jobs for multiclass fitting. None means 1 (serial). -1 means use all available CPUs. Passed directly to joblib.Parallel.

Attributes

coef_: ndarray of shape (n_features,) for binary, (n_estimators, n_features) for multiclass

Coefficients of all fitted classifiers, excluding the intercept. OvO training coefficients favor the second class of each sorted pair; raw multiclass decision scores use the opposite sign, like SVC.

intercept_: float for binary, ndarray of shape (n_estimators,) for multiclass

Intercept term(s). 0.0 if fit_intercept=False.

classes_ndarray of shape (n_classes,)

Unique class labels in the original label space.

estimators_list, only present for multiclass

For OvR: list of (coef, intercept) tuples, length K. For OvO: list of (coef, intercept, cls_i, cls_j) tuples, length K*(K-1)/2.

_label_encoderLabelEncoder

Encodes original labels into {0,1} for internal training.

class rehline.plq_Ridge_Regressor(loss=None, constraint=None, C=1.0, U=None, V=None, Tau=None, S=None, T=None, A=None, b=None, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, *, coordinate_order='auto', coordinate_seed=None)

Bases: sklearn.base.RegressorMixin, _SklearnReHLine

Empirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss and a ridge penalty, implemented as a scikit-learn compatible estimator.

This wrapper adds standard sklearn conveniences while delegating loss/constraint construction to plqERM_Ridge (via _make_loss_rehline_param / _make_constraint_rehline_param).

Notes

• Intercept handling: if fit_intercept=True, a constant column (value = intercept_scaling) is appended to the right of the design matrix before calling the base solver. The last learned coefficient is multiplied by intercept_scaling to obtain intercept_. → The column indices of the original features remain; therefore, sen_idx in the constraint fair follow the original index.

• Constraint handling: built-in constraints act on original feature coefficients. Custom A accepts n_features columns, or n_features + 1 columns to explicitly constrain the actual intercept.

Parameters

lossdict or None, default=None

None selects {‘name’: ‘QR’, ‘qt’: 0.5} at fit time. PLQ loss configuration (e.g., median Quantile Regression). Examples: {'name': 'QR', 'qt': 0.5}, {'name': 'huber', 'tau': 1.0}, {'name': 'SVR', 'epsilon': 0.1}. Required keys depend on the chosen loss and are consumed by the underlying solver.

constraintlist of dict or None, default=None

Constraint specifications. Supported by your updated _make_constraint_rehline_param:

• {'name': 'nonnegative'} or {'name': '>=0'}

• {'name': 'fair', 'sen_idx': list[int], 'tol_sen': list[float]}

• {'name': 'custom', 'A': ndarray[K, d], 'b': ndarray[K]}

Note: when fit_intercept=True, a constant column is appended as the last column; since you index sensitive columns by sen_idx on the original features, indices stay valid.

Cfloat, default=1.0

Regularization parameter (absorbed by ReHLine parameters inside the solver).

U, V, S, T, TauNone or empty array, default=None

Legacy constructor parameters. Nonempty values raise ValueError at fit time because these matrices are generated from loss. Use ReHLine or ReHLine_solver for manually specified loss matrices.

A, bndarray or None, default=None

Additional linear constraints, combined with every entry in constraint.

max_iterint, default=1000

Maximum iterations for the ReHLine solver.

tolfloat, default=1e-4

Convergence tolerance for the ReHLine solver.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

warm_startint, default=0

Warm start flag passed to the solver (see solver docs).

verboseint, default=0

Verbosity for the solver (0: silent).

trace_freqint, default=100

Iteration frequency to trace solver internals (if verbose is enabled).

fit_interceptbool, default=True

If True, append a constant column (value = intercept_scaling) to the design matrix before calling the solver. The learned last coefficient is then split as intercept_.

intercept_scalingfloat, default=1.0

Scaling applied to the appended constant column when fit_intercept=True.

Attributes

coef_ndarray of shape (n_features,)

Learned linear coefficients (excluding the intercept term).

intercept_float

Intercept term extracted from the last coefficient when fit_intercept=True, otherwise 0.0.

n_features_in_int

Number of input features seen during fit() (before intercept augmentation).

Notes

This estimator does not support sparse input. If you need sparse support, convert inputs to dense or wrap this estimator in a scikit-learn Pipeline with a transformer that densifies data (at the cost of memory).

Functions

rehline.ReHLine_solver(X, U, V, Tau=None, S=None, T=None, A=None, b=None, rho=None, Lambda=None, Gamma=None, xi=None, mu=None, max_iter=1000, tol=0.0001, shrink=1, verbose=1, trace_freq=100, *, coordinate_order='auto', coordinate_seed=None, _quantile_count=0)

Solve a ReLU/ReHU problem with independently selectable update order.

X and A independently accept dense arrays or SciPy sparse matrices/2-D sparse arrays. Sparse inputs use float64 CSR internally; dimensions and stored-entry counts must fit in int32. Loss parameters, b, rho and dual variables remain dense. Sparse constraint updates visit only stored entries. Optional primal refinement uses a dense active constraint matrix only when it contains at most 1,000,000 entries; larger refinements are skipped without changing the stopping certificate.

coordinate_order is 'cyclic' or 'random' (a new permutation within each coordinate group per sweep). 'auto' preserves the legacy order: cyclic for shrink=0, random for shrink>0. Shrinking still depends only on shrink. coordinate_seed=None uses the positive shrink value, or 1 when shrinking is disabled. An explicit seed is a non-negative int32, is reset on every call, and is ignored in cyclic order.

With shrinking enabled, an internal eps_shrink requests full sweeps before each restricted problem reaches the final tol. It starts at max(tol, 0.1 * first_full_sweep_PG_violation) and halves after each restoration, down to tol. Before restoring, the solver may check the full objective gap and feasibility using beta reconstructed from the duals. Such probes require two full scans’ worth of coordinate visits since the previous certificate check. Deferred or failed probes restore as usual; failed probes preserve the incremental state. Full-working-set stopping checks remain independent of this internal work budget. Final convergence requires scaled constraint violation at most tol and abs(P-D) / max(1, abs(P), abs(D)) <= tol, where P and D are the primal and dual objectives. The absolute KKT residual is reported separately; it is not an additional stopping requirement. For iterates feasible only within tolerance, the gap is an approximate diagnostic rather than an exact primal bound.

rehline.make_mf_dataset(n_users, n_items, n_factors=20, n_interactions=None, density=0.01, noise_std=0.1, seed=None, rating_min=1.0, rating_max=5.0, return_params=True)

Generate synthetic rating data using matrix factorization model.

Creates synthetic user-item rating data based on the matrix factorization approach commonly used in recommender systems. The ratings are generated as: rating = mu + user_bias + item_bias + user_factor * item_factor + noise

Parameters

n_usersint

Non-negative number of users in the synthetic dataset.

n_itemsint

Non-negative number of items. n_users * n_items must fit in int64.

n_factorsint, default=20

Positive number of latent factors for user and item embeddings.

n_interactionsint, optional

Non-negative number of unique user-item pairs, capped at total_pairs. Zero produces empty data. If None, uses int(density * total_pairs).

densityfloat, default=0.01

Finite density in [0, 1], ignored if n_interactions is specified.

noise_stdfloat, default=0.1

Standard deviation of Gaussian noise added to ratings

seedint, optional

Random seed for reproducible results

rating_minfloat, default=1.0

Minimum possible rating value

rating_maxfloat, default=5.0

Maximum possible rating value

return_paramsbool, default=True

If True, returns the underlying model parameters (P, Q, bu, bi, mu)

Returns

dict

Dictionary containing:

• Xndarray of shape (n_interactions, 2)

  User-item pairs where X[:, 0] are user indices and X[:, 1] are item indices

• yndarray of shape (n_interactions,)

  Synthetic ratings for each user-item pair

• paramsdict, optional

  Only returned if return_params=True. Contains:

  • Pndarray of shape (n_users, n_factors)

    User factor matrix

  • Qndarray of shape (n_items, n_factors)

    Item factor matrix

  • bundarray of shape (n_users,)

    User biases

  • bindarray of shape (n_items,)

    Item biases

  • mufloat

    Global mean rating

Notes

The rating generation follows the standard matrix factorization model:

r_ui = μ + b_u + b_i + p_u · q_i^T + ε

where ε ~ N(0, noise_std²)

The generated ratings are clipped to stay within [rating_min, rating_max] range.

Pairs are sampled uniformly without replacement. Pair-index storage is O(n_interactions); sparse sampling does not allocate all n_users * n_items pairs. Factors still require O((n_users + n_items) * n_factors) storage, even when return_params=False.

The same seed and arguments reproduce results within this implementation. Sparse sampling (fewer than 10% of pairs) uses a different random draw sequence from the previous full-population sampler, so those datasets and their ratings can differ from earlier releases for the same seed.

rehline.CQR_Ridge_path_sol(X, y, *, quantiles, eps=1e-05, n_Cs=50, Cs=None, max_iter=5000, tol=0.0001, verbose=0, shrink=1, coordinate_order='auto', coordinate_seed=None, warm_start=False, return_time=True, compact=False)

Compute the regularization path for Composite Quantile Regression (CQR) with ridge penalty.

This function fits a series of CQR models using different values of the regularization parameter C. It reuses a single estimator and modifies C in-place before refitting.

Parameters

Xndarray of shape (n_samples, n_features)

Feature matrix.

yndarray of shape (n_samples,)

Response vector.

quantileslist of float

Quantile levels (e.g. [0.1, 0.5, 0.9]).

epsfloat, default=1e-5

Log-scaled lower bound for generated C values (used if Cs is None).

n_Csint, default=50

Number of C values to generate.

Csarray-like or None, default=None

Explicit values of regularization strength. If None, use eps and n_Cs to generate them.

max_iterint, default=5000

Maximum number of solver iterations.

tolfloat, default=1e-4

Solver convergence tolerance.

verboseint, default=0

Verbosity level.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

warm_startbool, default=False

Use previous dual solution to initialize the next fit.

return_timebool, default=True

Whether to return a list of fit durations.

compactbool, default=False

Return independent inference snapshots in models instead of full fitted estimators. Snapshots retain prediction metadata and final objective/convergence diagnostics but omit training loss/dual arrays. They cannot fit or warm-start. The working estimator still uses warm_start across the path, and returned array shapes are unchanged.

Returns

Csndarray

List of regularization strengths.

modelslist

Full fitted models, or inference snapshots when compact=True.

coefsndarray of shape (n_Cs, n_quantiles, n_features)

Coefficient matrices per quantile and C.

interceptsndarray of shape (n_Cs, n_quantiles)

Intercepts per quantile and C.

fit_timeslist of float, optional

Elapsed fit times (if return_time=True).

Example

>>> from sklearn.datasets import make_friedman1
>>> from sklearn.preprocessing import StandardScaler
>>> import numpy as np
>>> from rehline import CQR_Ridge_path_sol

>>> # Generate the data
>>> X, y = make_friedman1(n_samples=500, n_features=6, noise=1.0, random_state=42)
>>> X = StandardScaler().fit_transform(X)
>>> y = y / y.std()

>>> # Set quantiles and Cs
>>> quantiles = [0.1, 0.5, 0.9]
>>> Cs = np.logspace(-5, 0, 30)

>>> # Fit CQR path
>>> Cs, models, coefs, intercepts, fit_times = CQR_Ridge_path_sol(
...     X, y,
...     quantiles=quantiles,
...     Cs=Cs,
...     max_iter=100000,
...     tol=1e-4,
...     verbose=1,
...     warm_start=True,
...     return_time=True
... )

rehline.plqERM_Ridge_path_sol(X, y, *, loss, constraint=None, eps=0.001, n_Cs=100, Cs=None, max_iter=5000, tol=0.0001, verbose=0, shrink=1, coordinate_order='auto', coordinate_seed=None, warm_start=False, return_time=True)

Compute the PLQ Empirical Risk Minimization (ERM) path over a range of regularization parameters. This function evaluates the model’s performance for different values of the regularization parameter and provides structured benchmarking output.

Parameters

Xndarray of shape (n_samples, n_features)

Training input samples.

yndarray of shape (n_samples,)

Target values corresponding to each input sample.

lossdict

Dictionary describing the PLQ loss function parameters. Used to construct the loss object internally.

constraintlist of dict, optional (default=[])

List of constraints applied to the optimization problem. Each constraint should be represented as a dictionary compatible with the solver.

epsfloat, default=1e-3

Defines the length of the regularization path when Cs is not provided. The values of C will range from 10^log10(eps) to 10^-log10(eps).

n_Csint, default=100

Number of regularization values to evaluate if Cs is not provided.

Csarray-like of shape (n_Cs,), optional

Explicit values of regularization strength C to use. If None, the values are generated logarithmically between 1e-2 and 1e3.

max_iterint, default=5000

Maximum number of iterations allowed for the optimization solver at each C.

tolfloat, default=1e-4

Tolerance for solver convergence.

verboseint, default=0

Controls verbosity level of output. Set to higher values (e.g., 1 or 2) for detailed progress logs. When verbose = 1, only print path results table; when verbose = 2, print path results table and path solution plot.

shrinkint, default=1

Zero disables shrinking; a positive integer enables it. With coordinate_seed=None, the positive value also supplies the legacy seed.

coordinate_order{“auto”, “cyclic”, “random”}, default=”auto”

Coordinate order within each group. Auto uses cyclic when shrink=0 and random permutations when shrink>0. Explicit orders work with either shrinking setting. The group order remains xi, Lambda, Gamma, mu.

coordinate_seedint or None, default=None

Non-negative int32 seed for random coordinate permutations. None uses the positive shrink value, or 1 when shrink=0. Reset for each solver call, including warm refits. Ignored for cyclic order.

warm_startbool, default=False

If True, reuse the previous solution to warm-start the next solver step, speeding up convergence.

return_timebool, default=True

If True, return timing information for each value of C.

Returns

Csndarray of shape (n_Cs,)

Array of regularization parameters used in the path.

timeslist of float

Time in seconds taken to fit the model at each C. Returned only if return_time=True.

n_iterslist of int

Number of iterations used by the solver at each regularization value.

obj_valueslist of float

Final objective values (including loss and regularization terms) at each C.

L2_normslist of float

L2 norm of the coefficients (excluding bias) at each C.

coefsndarray of shape (n_features, n_Cs)

Learned model coefficients at each regularization strength.

Example

>>> # generate data
>>> np.random.seed(42)
>>> n, d, C = 1000, 5, 0.5
>>> X = np.random.randn(n, d)
>>> beta0 = np.random.randn(d)
>>> y = np.sign(X.dot(beta0) + np.random.randn(n))
>>> # define loss function
>>> loss = {'name': 'svm'}
>>> Cs = np.logspace(-1,3,15)
>>> constraint = [{'name': 'nonnegative'}]

>>> # calculate
>>> Cs, times, n_iters, losses, norms, coefs = plqERM_Ridge_path_sol(
...     X, y, loss=loss, Cs=Cs, max_iter=100000,tol=1e-4,verbose=2,
...     warm_start=False, constraint=constraint, return_time=True
... )