{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "k8JuVt-Lddja" }, "source": [ "# MSE\n", "[](https://rehline-python.readthedocs.io/en/latest/)\n", "\n", "The regularized MSE regression solves the following optimization problem:\n", "\n", "$$\n", "\\min_{\\mathbf{\\beta} \\in \\mathbb{R}^d}\n", "C \\sum_{i=1}^n (y_i - \\mathbf{x}_i^\\top \\mathbf{\\beta})^2\n", "+ \\frac{1}{2}\\|\\mathbf{\\beta}\\|_2^2,\n", "$$\n", "\n", "where $\\mathbf{x}_i \\in \\mathbb{R}^d$ is a feature vector, and $y_i \\in \\mathbb{R}$ is the response variable.\n", "\n", "> **Note.** Since the squared loss is a plq function, we can optimize it using `rehline.plq_Ridge_Regressor`. \n", "> Moreover, this wrapper adapts the `plqERM_Ridge` into a regressor, compatible with the scikit-learn API." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "cY_04uxlIp6C" }, "outputs": [], "source": [ "## install rehline\n", "%pip install rehline -q" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "id": "6A65T6tzF3Lj" }, "outputs": [], "source": [ "import warnings\n", "\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "import seaborn as sns\n", "from sklearn.datasets import make_regression\n", "from sklearn.preprocessing import StandardScaler" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "iH3AockWGKbl" }, "outputs": [], "source": [ "# Simulate data\n", "np.random.seed(42)\n", "scaler_mse = StandardScaler()\n", "\n", "n, d = 10000, 5\n", "X, y = make_regression(n_samples=n, n_features=d, noise=5.0)\n", "X = scaler_mse.fit_transform(X)\n", "y = y / y.std()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 80 }, "id": "1OGXUMlKGL3U", "outputId": "be99ca6b-ec7a-41ac-d252-fa24ea46f411" }, "outputs": [ { "data": { "text/html": [ "
plq_Ridge_Regressor(loss={'name': 'mse'})In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. plq_Ridge_Regressor(loss={'name': 'mse'})Pipeline(steps=[('scaler', StandardScaler()),\n",
" ('reg', plq_Ridge_Regressor(loss={'name': 'mse'}))])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. Pipeline(steps=[('scaler', StandardScaler()),\n",
" ('reg', plq_Ridge_Regressor(loss={'name': 'mse'}))])StandardScaler()
plq_Ridge_Regressor(loss={'name': 'mse'})