{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "W9_lmFre4Rse" }, "source": [ "# Smooth SVM\n", "Smooth SVMs solve the following optimization problem:\n", "$$\n", " \\min_{\\mathbf{\\beta} \\in \\mathbb{R}^d} \\\n", " C \\sum_{i=1}^n V(y_i \\mathbf{\\beta}^\\intercal \\mathbf{x}_i) + \\frac{1}{2} \\| \\mathbf{\\beta} \\|_2^2\n", "$$\n", "where $\\mathbf{x}_i \\in \\mathbb{R}^d$ is a feature vector, $y_i \\in \\{-1, 1\\}$ is a binary label, and $V(·)$ is the smoothed hinge loss (shown here with a default smoothing parameter τ=1):\n", "$$\n", " V(z) = \\begin{cases}\n", " 0 & \\text{if } z \\ge 1 \\\\\n", " \\frac{(1-z)^2}{2} & \\text{if } 0 < z < 1 \\\\\n", " \\frac{1}{2}- z & \\text{if } z \\le 0\n", " \\end{cases}\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": { "id": "lO5s9H4tbAP4" }, "source": [ "> **Note.** Since the smooth hinge loss is a plq function, we can optimize it using `rehline.plq_Ridge_Classifier`. \n", "> Moreover, this wrapper adapts the `plqERM_Ridge` into a classifier, compatible with the scikit-learn API." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sRNXaXpO4Pt2" }, "outputs": [], "source": [ "## install rehline\n", "%pip install rehline -q" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "id": "lRz5G7RAaFtj" }, "outputs": [], "source": [ "## set up plotting style\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "custom_palette = [\"#FFE4E1\", \"#3D325C\"]\n", "sns.set_palette(custom_palette)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "ZawpNRuQR9Uu" }, "outputs": [], "source": [ "## simulate data\n", "import numpy as np\n", "from sklearn.datasets import make_classification\n", "from sklearn.preprocessing import StandardScaler\n", "\n", "scaler = StandardScaler()\n", "\n", "n, d = 10000, 5\n", "X, y = make_classification(n_samples=n, n_features=d, random_state=42)\n", "y = 2 * y - 1\n", "X = scaler.fit_transform(X)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 80 }, "id": "lskTdleDZwpa", "outputId": "3408050f-4e9d-49a3-f702-8158500bf421" }, "outputs": [ { "data": { "text/html": [ "
plq_Ridge_Classifier(loss={'name': 'sSVM'})
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "plq_Ridge_Classifier(loss={'name': 'sSVM'})" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## solve Smooth SVM via `plq_Ridge_Classifier`\n", "from rehline import plq_Ridge_Classifier\n", "\n", "clf = plq_Ridge_Classifier(loss={\"name\": \"sSVM\"}, C=1.0)\n", "clf.fit(X=X, y=y)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 472 }, "id": "g5DCBXT5Z5uR", "outputId": "78025cf1-710b-4965-ac8c-e3ba6792716a" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAANvZJREFUeJzt3X10FNX9x/HPbswjkATIE2ASUNAQngWEFFQCkRSjBUEtQikgomKgCr8ijVKIaEXQCiVGsK2CWlM8tCoUKBiCiJKIGgvl+ZQWfqEJSRYRwmMSNvP7w7K/xmQTSDa7m+H9OmfPcefenfnOHJSPM3futRiGYQgAAMCkrJ4uAAAAoCkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgBcMywWi6ZPn+7pMgC4GWEHQK327Nmj++67T7GxsQoICFCHDh105513KiMjw9Ol1Sk3N1fp6ek6deqUy/dts9n0xBNPKC4uToGBgYqIiNCtt96qOXPm6OzZs6qsrFRYWJgGDx7sdB+GYSg6Olq33HKLJGnbtm2yWCyyWCz6wx/+UOtvBg0aJIvFou7du7v8nIBrAWEHQA25ubnq16+fdu/eralTp+rVV1/Vww8/LKvVqt/85jeeLq9Oubm5evbZZ10edk6ePKl+/frp7bffVkpKipYtW6ZZs2apc+fOWr58uU6cOCFfX1/df//9ys3N1f/+7//Wup/t27fr3//+t37yk59U2x4QEKCsrKwa/Y8eParc3FwFBAS49HyAa8l1ni4AgPf51a9+pZCQEH355ZcKDQ2t1lZaWuqZojzsjTfeUEFBgXbs2KEf/OAH1drKysrk5+cnSRo/frxWrFihP/7xj/rFL35RYz9ZWVmyWq0aO3Zste133XWX1q1bpxMnTigsLKxa/8jISHXp0kXffvttE5wZYH7c2QFQwz//+U9169atRtCRpIiIiGrfL4+DWbNmjeLj4xUYGKiEhATt2bNHkvT666+rc+fOCggI0JAhQ3T06NEa+1yzZo369u2rwMBAhYWF6Sc/+YkKCwtr9Nu6datuu+02tWjRQqGhoRo5cqQOHDjgaE9PT9fs2bMlSZ06dXI8Hvr+MT/88EN1795d/v7+6tatmzZt2nRF18THx0cDBw6s0RYcHOy48zJo0CB17Nix1rs0lZWV+tOf/qTExES1b9++WtvIkSPl7++vNWvWVNuelZWlBx54QD4+PvXWCKB2hB0ANcTGxio/P1979+69ov6ffvqp/ud//kcTJ05Uenq6Dhw4oLvvvluZmZlatmyZHn/8cc2ePVt5eXl66KGHqv121apVjr/MFy5cqKlTp+r999/X4MGDqz2K2rJli5KTk1VaWqr09HTNmjVLubm5GjRokCPMjB49Wg8++KAkacmSJXrnnXf0zjvvKDw83LGfzz77TI8//rjGjh2rxYsX6+LFixozZoy++eabeq+J3W7XO++8U2c/i8WicePGac+ePdq3b1+1tk2bNunkyZMaP358jd8FBQVp5MiR+uMf/+jYtnv3bu3bt0/jxo2r85gA6mEAwPd89NFHho+Pj+Hj42MkJCQYTz31lLF582ajoqKiRl9Jhr+/v3HkyBHHttdff92QZERFRRllZWWO7WlpaYYkR9+KigojIiLC6N69u3HhwgVHv/Xr1xuSjHnz5jm29e7d24iIiDC++eYbx7bdu3cbVqvV+OlPf+rY9tJLL1U7xvdr9fPzMw4fPlxtH5KMjIyMOq9JcXGxER4ebkgy4uLijMcee8zIysoyTp06VaPvvn37DElGWlpate1jx441AgICjNOnTzu2ffzxx4YkY82aNcb69esNi8ViFBQUGIZhGLNnzzZuuOEGwzAM44477jC6detWZ40AasedHQA13HnnncrLy9OPfvQj7d69W4sXL1ZycrI6dOigdevW1eg/bNgwdezY0fF9wIABkqQxY8aoVatWNbb/61//kiR99dVXKi0t1eOPP15tAG5KSori4uK0YcMGSdLx48e1a9cuTZo0SW3atHH069mzp+68805t3Ljxis8tKSlJN954Y7V9BAcHO2pyJjIyUrt379Zjjz2mb7/9VitWrNC4ceMUERGh5557ToZhOPrGx8erT58+Wr16tWPbuXPntG7dOt19990KDg6u9RjDhw9XmzZttHr1ahmGodWrVzvuVAFoOMIOgFr1799f77//vr799lt98cUXSktL05kzZ3Tfffdp//791frGxMRU+x4SEiJJio6OrnX75YG2l99Yuvnmm2scPy4uztFeV7+uXbvqxIkTOnfu3BWd1/drlaTWrVtf0eDfdu3aafny5Tp+/LgOHTqkZcuWKTw8XPPmzdMbb7xRre/48eN15MgR5ebmSvpunND58+drfYR12eW3ubKysrR9+3YdO3aMR1iACxB2ANTJz89P/fv31wsvvKDly5ersrKyxiBaZ4NnnW3/77sg7uaKmiwWi2666SbNmDFD27dvl9Vq1bvvvlutz4MPPiir1eoYqJyVlaXWrVvrrrvuqnPf48aN065du5Senq5evXopPj7+iusCUDvCDoAr1q9fP0nfPVZyhdjYWEnSoUOHarQdOnTI0V5Xv4MHDyosLEwtWrSQ9F0QcacbbrhBrVu3rnFN2rdvr8TERK1Zs0YlJSXKzs7Wfffd53hF3ZnBgwcrJiZG27Zt464O4CKEHQA1fPzxx7Xe6bg8Nqa2x0kN0a9fP0VERGjFihUqLy93bP/rX/+qAwcOKCUlRdJ3j4969+6tt956q9obWnv37tVHH31U7W7J5dDj6kkFd+7cWeujsi+++ELffPNNrddk/PjxKi0t1aOPPqrKyso6H2FdZrFYtGzZMs2fP18TJkxwSe3AtY5JBQHUMGPGDJ0/f1733nuv4uLiVFFRodzcXL333nvq2LGjJk+e7JLj+Pr6atGiRZo8ebLuuOMOPfjggyopKdFvfvMbdezYUTNnznT0femllzRixAglJCRoypQpunDhgjIyMhQSEqL09HRHv759+0qSnnnmGY0dO1a+vr665557HCGood555x29++67uvfee9W3b1/5+fnpwIEDevPNNxUQEKCnn366xm/GjBmjxx9/XGvXrlV0dLRuv/32KzrWyJEjNXLkyEbVC+D/EXYA1PDyyy9rzZo12rhxo37729+qoqJCMTExevzxxzV37txaJxtsqEmTJikoKEgvvvii5syZoxYtWujee+/VokWLqh0nKSlJmzZt0vz58zVv3jz5+vrqjjvu0KJFi9SpUydHv/79++u5557TihUrtGnTJlVVVenIkSONDjuPPvqogoKClJOTo7Vr16qsrEzh4eEaPny40tLS1KdPnxq/CQ4O1j333KM1a9bowQcfdPsjNgDfsRieHCkIAADQxBizAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI15diRVVVWpqKhIrVq1Yh4MAACaCcMwdObMGbVv315Wq/P7N4QdSUVFRTVWZwYAAM3DsWPHdP311zttJ+xIatWqlaTvLlZwcLCHqwEAAFeirKxM0dHRjr/HnSHs6P9XSQ4ODibsAADQzNQ3BIUBygAAwNQIOwAAwNQIOwAAwNQYswMAQDNht9tVWVnp6TLcxtfXVz4+Po3eD2EHAAAvZxiGiouLderUKU+X4nahoaGKiopq1Dx4hB0AALzc5aATERGhoKCga2ICXMMwdP78eZWWlkqS2rVr1+B9EXYAAPBidrvdEXTatm3r6XLcKjAwUJJUWlqqiIiIBj/SYoAyAABe7PIYnaCgIA9X4hmXz7sxY5UIOwAANAPXwqOr2rjivAk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AACgmrfffltt27ZVeXl5te2jRo3ShAkTPFRVwxF2AABANffff7/sdrvWrVvn2FZaWqoNGzbooYce8mBlDcOkggDcamhiomw2m9P28PBwbf34YzdWBOD7AgMDNW7cOK1cuVL333+/JOkPf/iDYmJiNGTIEM8W1wCEHQBuZbPZtHPLZqftA5KS3VgNAGemTp2q/v37q7CwUB06dNCqVas0adKkZjnfD2EHAADU0KdPH/Xq1Utvv/22hg8frn379mnDhg2eLqtBCDsAvEphYaF6dO/utJ3HXID7PPzww1q6dKkKCwuVlJSk6OhoT5fUIIQdAF7FMKp4zAV4iXHjxunnP/+5fve73+ntt9/2dDkNxttYAACgViEhIRozZoxatmypUaNGebqcBiPsAAAApwoLCzV+/Hj5+/t7upQG4zEWAACo4dtvv9W2bdu0bds2vfbaa54up1EIOwBgQoMH3aaSkmKn7ZGRUfpsx6durAjNTZ8+ffTtt99q0aJFuvnmmz1dTqMQdgDAhEpKijUz9Wmn7UsyX3BjNWiOjh496ukSXIawAwDNUH13bgoLi9xYDeDdCDsA0AzVd+dm5pxpbqwG8G68jQUAAEyNsAMAAEyNsAMAAEyNsAMAAEyNAcoAADRTBQUFOnHihFuOFRYWppiYGLccy9UIOwBcamhiomw2m9P240W8Eg24QkFBgbp27arz58+75XhBQUE6cOBAowLP+++/rxUrVig/P18nT57U3/72N/Xu3dt1RTpB2AHgUjabrc5VyzvExbuxGsC8Tpw4ofPnz+uNV5fp5i5dmvRYh/7xD02Z/jOdOHGiUWHn3LlzGjx4sB544AFNnTrVhRXWjbADAEAzdnOXLurTs4eny7giEyZMkOT+2ZkZoAwAAEyNsAMAAEyNsAMAAFzu3XffVcuWLR2fTz/91GO1MGYHAAC43I9+9CMNGDDA8b1Dhw4eq4WwAwAAXK5Vq1Zq1aqVp8uQRNgBAABucvLkSRUUFKjoP/NtHTp0SJIUFRWlqKioJjsuYQcAgGbs0D/+0WyOsW7dOk2ePNnxfezYsZKk+fPnKz093SXHqA1hBwCAZigsLExBQUGaMv1nbjleUFCQwsLCGrWPSZMmadKkSa4p6CoQdgAAaIZiYmJ04MAB1sa6AoQdAACaqZiYmGYbQNyJeXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpMc8OgKsyNDFRNpvNafvx/6x5A6DpFRQUMKngFSDsALgqNptNO7dsdtreIS7ejdUA166CggLFxcXpwoULbjleYGCgDh48eFWBZ/v27XrppZeUn5+v48eP64MPPtCoUaOarkgnCDsA4IUGD7pNJSXFTtsLC7mDdq07ceKELly4oInjpyoqsn2THqu4pEhvvfs7nThx4qrCzrlz59SrVy899NBDGj16dBNWWDfCDgB4oZKSYs1Mfdpp+8w509xYDbxZVGR7xVwf6+kyajVixAiNGDHC02UwQBkAAJgbYQcAAJgaYQcAAJgaYQcAAJia14SdF198URaLRU8++aRj28WLF5Wamqq2bduqZcuWGjNmjEpKSqr9rqCgQCkpKQoKClJERIRmz56tS5cuubl6AADgrbzibawvv/xSr7/+unr27Flt+8yZM7VhwwatWbNGISEhmj59ukaPHq0dO3ZIkux2u1JSUhQVFaXc3FwdP35cP/3pT+Xr66sXXnjBE6cCAAD+4+zZszp8+LDj+5EjR7Rr1y61adPGrRMUejzsnD17VuPHj9fvfvc7Pf/8847tp0+f1htvvKGsrCwNHTpUkrRy5Up17dpVn3/+uQYOHKiPPvpI+/fv15YtWxQZGanevXvrueee05w5c5Seni4/Pz9PnRYA1Il5dOAqxSVN/2elocf46quvlJiY6Pg+a9YsSdLEiRO1atUqV5R2RTwedlJTU5WSkqKkpKRqYSc/P1+VlZVKSkpybIuLi1NMTIzy8vI0cOBA5eXlqUePHoqMjHT0SU5O1rRp07Rv3z716dOn1mOWl5ervLzc8b2srKwJzgwAnGMeHTRWWFiYAgMD9da7v3PL8QIDAxUWFnZVvxkyZIgMw2iiiq6cR8PO6tWr9fXXX+vLL7+s0VZcXCw/Pz+FhoZW2x4ZGani4mJHn/8OOpfbL7c5s3DhQj377LONrB4AAM+JiYnRwYMHWRvrCngs7Bw7dkxPPPGEsrOzFRAQ4NZjp6WlOW6lSd/d2YmOjnZrDYA3qm+RT4mFPgFvEhMT02wDiDt5LOzk5+ertLRUt9xyi2Ob3W7X9u3b9eqrr2rz5s2qqKjQqVOnqt3dKSkpUVRUlCQpKipKX3zxRbX9Xn5b63Kf2vj7+8vf39+FZwOYQ32LfEos9Amg+fHYq+fDhg3Tnj17tGvXLsenX79+Gj9+vOOffX19lZOT4/jNoUOHVFBQoISEBElSQkKC9uzZo9LSUkef7OxsBQcHKz6e/yADAAAP3tlp1aqVunfvXm1bixYt1LZtW8f2KVOmaNasWWrTpo2Cg4M1Y8YMJSQkaODAgZKk4cOHKz4+XhMmTNDixYtVXFysuXPnKjU1lTs3AABT8YaBvp7givP2+NtYdVmyZImsVqvGjBmj8vJyJScn67XXXnO0+/j4aP369Zo2bZoSEhLUokULTZw4UQsWLPBg1QAAuI6vr68k6fz58woMDPRwNe53/vx5Sf9/HRrCq8LOtm3bqn0PCAhQZmamMjMznf4mNjZWGzdubOLKAADwDB8fH4WGhjqGbAQFBclisXi4qqZnGIbOnz+v0tJShYaGysfHp8H78qqwAwAAarr80s1/j1G9VoSGhtb50tGVIOwAAODlLBaL2rVrp4iICFVWVnq6HLfx9fVt1B2dywg7AAA0Ez4+Pi75y/9a4zWrngMAADQF7uwAwDWosLBIXTp3qbNPZGSUPtvxqZsqApoOYQcArkF2u73OhUglaUnmC26qBmhaPMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmxttYAJqVwsJC9ejevc4+4eHh2vrxx26qCIC3I+wAaFYMo0o7t2yus8+ApGQ3VQOgOeAxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDUmFQQAFxs86DaVlBTX2aewsMhN1QAg7ACAi5WUFGtm6tN19pk5Z5qbqgHAYywAAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqvHoOAKhVYWGRunTu4rQ9MjJKn+341I0VAQ1D2AEA1Mput9c5X9CSzBfcWA3QcDzGAgAApsadHQCmU1hYqB7duzttDw8P19aPP3ZjRQA8ibADwHQMo0o7t2x22j4gKdmN1QDwNB5jAQAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU7vO0wUAQHMzeNBtKikpdtpeWFjkxmoA1IewAwBXqaSkWDNTn3baPnPONDdWA6A+PMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACm5tGws3z5cvXs2VPBwcEKDg5WQkKC/vrXvzraL168qNTUVLVt21YtW7bUmDFjVFJSUm0fBQUFSklJUVBQkCIiIjR79mxdunTJ3acCAAC8lEfDzvXXX68XX3xR+fn5+uqrrzR06FCNHDlS+/btkyTNnDlTf/nLX7RmzRp98sknKioq0ujRox2/t9vtSklJUUVFhXJzc/XWW29p1apVmjdvnqdOCQAAeJnrPHnwe+65p9r3X/3qV1q+fLk+//xzXX/99XrjjTeUlZWloUOHSpJWrlyprl276vPPP9fAgQP10Ucfaf/+/dqyZYsiIyPVu3dvPffcc5ozZ47S09Pl5+fnidMCAABexGvG7Njtdq1evVrnzp1TQkKC8vPzVVlZqaSkJEefuLg4xcTEKC8vT5KUl5enHj16KDIy0tEnOTlZZWVljrtDAADg2ubROzuStGfPHiUkJOjixYtq2bKlPvjgA8XHx2vXrl3y8/NTaGhotf6RkZEqLi6WJBUXF1cLOpfbL7c5U15ervLycsf3srIyF50NAADwNh6/s3PzzTdr165d2rlzp6ZNm6aJEydq//79TXrMhQsXKiQkxPGJjo5u0uMBAADP8XjY8fPzU+fOndW3b18tXLhQvXr10m9+8xtFRUWpoqJCp06dqta/pKREUVFRkqSoqKgab2dd/n65T23S0tJ0+vRpx+fYsWOuPSkAAOA1PB52vq+qqkrl5eXq27evfH19lZOT42g7dOiQCgoKlJCQIElKSEjQnj17VFpa6uiTnZ2t4OBgxcfHOz2Gv7+/43X3yx8AAGBOHh2zk5aWphEjRigmJkZnzpxRVlaWtm3bps2bNyskJERTpkzRrFmz1KZNGwUHB2vGjBlKSEjQwIEDJUnDhw9XfHy8JkyYoMWLF6u4uFhz585Vamqq/P39PXlqAADAS3g07JSWluqnP/2pjh8/rpCQEPXs2VObN2/WnXfeKUlasmSJrFarxowZo/LyciUnJ+u1115z/N7Hx0fr16/XtGnTlJCQoBYtWmjixIlasGCBp04JAAB4GY+GnTfeeKPO9oCAAGVmZiozM9Npn9jYWG3cuNHVpQEAAJPwujE7AAAAruTxeXYAAM1TYWGRunTu4rQ9MjJKn+341I0VAbUj7AAAGsRut2tm6tNO25dkvuDGagDneIwFAABMjTs7APA9gwfdppIS50vOFBYWubEaAI1F2AGA7ykpKa7z8czMOdPcWA2AxuIxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDWWiwCuIUMTE2Wz2Zy2Hy9izScA5kPYAa4hNptNO7dsdtreIS7ejdUAgHs06DHWDTfcoG+++abG9lOnTumGG25odFEAAACu0qCwc/ToUdnt9hrby8vLVVhY2OiiAAAAXOWqHmOtW7fO8c+bN29WSEiI47vdbldOTo46duzosuIAXB3G5ABATVcVdkaNGiVJslgsmjhxYrU2X19fdezYUb/+9a9dVhyAq8OYHACo6arCTlVVlSSpU6dO+vLLLxUWFtYkRQEAALhKg97GOnLkiKvrAAAAaBINfvU8JydHOTk5Ki0tddzxuezNN99sdGEAAACu0KCw8+yzz2rBggXq16+f2rVrJ4vF4uq6AAAAXKJBYWfFihVatWqVJkyY4Op6AAAAXKpB8+xUVFToBz/4gatrAQAAcLkGhZ2HH35YWVlZrq4FAADA5Rr0GOvixYv67W9/qy1btqhnz57y9fWt1v7KK6+4pDgAAIDGalDY+fvf/67evXtLkvbu3VutjcHKAADAmzQo7Hz88ceurgMAAKBJNGjMDgAAQHPRoDs7iYmJdT6u2rp1a4MLAgAAcKUGhZ3L43Uuq6ys1K5du7R3794aC4QCAAB4UoPCzpIlS2rdnp6errNnzzaqIAAAAFdy6Zidn/zkJ6yLBQAAvIpLw05eXp4CAgJcuUsAAIBGadBjrNGjR1f7bhiGjh8/rq+++kq//OUvXVIYAACAKzQo7ISEhFT7brVadfPNN2vBggUaPny4SwoDAABwhQaFnZUrV7q6DgAAgCbRoLBzWX5+vg4cOCBJ6tatm/r06eOSogAAAFylQWGntLRUY8eO1bZt2xQaGipJOnXqlBITE7V69WqFh4e7skYAAIAGa9DbWDNmzNCZM2e0b98+nTx5UidPntTevXtVVlamn/3sZ66uEQAAoMEadGdn06ZN2rJli7p27erYFh8fr8zMTAYoAwAAr9KgOztVVVXy9fWtsd3X11dVVVWNLgoAAMBVGhR2hg4dqieeeEJFRUWObYWFhZo5c6aGDRvmsuIAAAAaq0Fh59VXX1VZWZk6duyoG2+8UTfeeKM6deqksrIyZWRkuLpGAACABmvQmJ3o6Gh9/fXX2rJliw4ePChJ6tq1q5KSklxaHAAAQGNd1Z2drVu3Kj4+XmVlZbJYLLrzzjs1Y8YMzZgxQ/3791e3bt306aefNlWtAAAAV+2qws7SpUs1depUBQcH12gLCQnRo48+qldeecVlxQEAADTWVT3G2r17txYtWuS0ffjw4Xr55ZcbXRQAoPkrLCxSl85dnLZHRkbpsx08DUDTu6qwU1JSUusr546dXXedbDZbo4sCADR/drtdM1Ofdtq+JPMFN1aDa9lVPcbq0KGD9u7d67T973//u9q1a9foogAAAFzlqsLOXXfdpV/+8pe6ePFijbYLFy5o/vz5uvvuu11WHAAAQGNd1WOsuXPn6v3339dNN92k6dOn6+abb5YkHTx4UJmZmbLb7XrmmWeapFAAAICGuKqwExkZqdzcXE2bNk1paWkyDEOSZLFYlJycrMzMTEVGRjZJoQDgKoWFherRvbvTdput1I3VAGhqVz2pYGxsrDZu3Khvv/1Whw8flmEY6tKli1q3bt0U9QGAyxlGlXZu2ey0vf1NXZ22AWh+GjSDsiS1bt1a/fv3d2UtAAAALtegtbEAAACaC8IOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNY+GnYULF6p///5q1aqVIiIiNGrUKB06dKhan4sXLyo1NVVt27ZVy5YtNWbMGJWUlFTrU1BQoJSUFAUFBSkiIkKzZ8/WpUuX3HkqAADAS3k07HzyySdKTU3V559/ruzsbFVWVmr48OE6d+6co8/MmTP1l7/8RWvWrNEnn3yioqIijR492tFut9uVkpKiiooK5ebm6q233tKqVas0b948T5wSAADwMg2eVNAVNm3aVO37qlWrFBERofz8fN1+++06ffq03njjDWVlZWno0KGSpJUrV6pr1676/PPPNXDgQH300Ufav3+/tmzZosjISPXu3VvPPfec5syZo/T0dPn5+Xni1AAAgJfwqjE7p0+fliS1adNGkpSfn6/KykolJSU5+sTFxSkmJkZ5eXmSpLy8PPXo0aPamlzJyckqKyvTvn37aj1OeXm5ysrKqn0AAIA5eU3Yqaqq0pNPPqlBgwap+38W6CsuLpafn59CQ0Or9Y2MjFRxcbGjz/cXH738/XKf71u4cKFCQkIcn+joaBefDQAA8BZeE3ZSU1O1d+9erV69usmPlZaWptOnTzs+x44da/JjAgAAz/DomJ3Lpk+frvXr12v79u26/vrrHdujoqJUUVGhU6dOVbu7U1JSoqioKEefL774otr+Lr+tdbnP9/n7+8vf39/FZwEAALyRR+/sGIah6dOn64MPPtDWrVvVqVOnau19+/aVr6+vcnJyHNsOHTqkgoICJSQkSJISEhK0Z88elZaWOvpkZ2crODhY8fHx7jkRAADgtTx6Zyc1NVVZWVlau3atWrVq5RhjExISosDAQIWEhGjKlCmaNWuW2rRpo+DgYM2YMUMJCQkaOHCgJGn48OGKj4/XhAkTtHjxYhUXF2vu3LlKTU3l7g0AAPBs2Fm+fLkkaciQIdW2r1y5UpMmTZIkLVmyRFarVWPGjFF5ebmSk5P12muvOfr6+Pho/fr1mjZtmhISEtSiRQtNnDhRCxYscNdpAAAAL+bRsGMYRr19AgIClJmZqczMTKd9YmNjtXHjRleWBgAATMJr3sYCAABoCoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgatd5ugAAcLdLl+zKyMhw3m63u7EaAE2NsAPgGmQoof9tdbQ7D0JwncLCInXp3MVpe2RklD7b8akbK4JZEXYAAB5ht9s1M/Vpp+1LMl9wYzUwM8bsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU2MGZaAZGZqYKJvN5rT9eFGRG6sBgOaBsAM0IzabTTu3bHba3iEu3o3VAEDzwGMsAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgasyzA3gRJg30DkZVlRa9ku603Wq1uK8YAI1G2AG8CJMGeger1ap3Ml5z2n7X+AfcWA2AxuIxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMLXrPF0AALjapUt2ZWRkOO9gNG7/RlWVFr2S7rTdarU07gAAXIqwA8CEDCX0v62O9jqC0BWwWq16J+M1p+13jX+gUfsH4Fo8xgIAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKbm0bCzfft23XPPPWrfvr0sFos+/PDDau2GYWjevHlq166dAgMDlZSUpH/84x/V+pw8eVLjx49XcHCwQkNDNWXKFJ09e9aNZwEAALyZR8POuXPn1KtXL2VmZtbavnjxYi1btkwrVqzQzp071aJFCyUnJ+vixYuOPuPHj9e+ffuUnZ2t9evXa/v27XrkkUfcdQoAAMDLeXSenREjRmjEiBG1thmGoaVLl2ru3LkaOXKkJOntt99WZGSkPvzwQ40dO1YHDhzQpk2b9OWXX6pfv36SpIyMDN111116+eWX1b59e7edCwAA8E5eO6ngkSNHVFxcrKSkJMe2kJAQDRgwQHl5eRo7dqzy8vIUGhrqCDqSlJSUJKvVqp07d+ree++tdd/l5eUqLy93fC8rK2u6EwH+Y2hiomw2W519jhcVuakawPsVFhapS+cudfaJjIzSZzs+dVNFaK68NuwUFxdLkiIjI6ttj4yMdLQVFxcrIiKiWvt1112nNm3aOPrUZuHChXr22WddXDFQN5vNpp1bNtfZp0NcvJuqab7qXQpCavRyEPAOdrtdM1OfrrPPkswX3FQNmjOvDTtNKS0tTbNmzXJ8LysrU3R0tAcrAnDl6lsKQmrschAAzMVrXz2PioqSJJWUlFTbXlJS4miLiopSaWlptfZLly7p5MmTjj618ff3V3BwcLUPAAAwJ6+9s9OpUydFRUUpJydHvXv3lvTdHZidO3dq2rRpkqSEhASdOnVK+fn56tu3ryRp69atqqqq0oABAzxVOgDATeob18OYHkgeDjtnz57V4cOHHd+PHDmiXbt2qU2bNoqJidGTTz6p559/Xl26dFGnTp30y1/+Uu3bt9eoUaMkSV27dtUPf/hDTZ06VStWrFBlZaWmT5+usWPH8iYWAFwD6hvXw5geSB4OO1999ZUSExMd3y+Po5k4caJWrVqlp556SufOndMjjzyiU6dOafDgwdq0aZMCAgIcv3n33Xc1ffp0DRs2TFarVWPGjNGyZcvcfi4AAMA7eTTsDBkyRIbh/LUJi8WiBQsWaMGCBU77tGnTRllZWU1RHgAAMAGvHaAMAADgCl47QBlobuqbNJAJAwHAMwg7gIvUN2kgEwYCgGfwGAsAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgay0UAgIsZVVVa9Ep6nX2sVot7igFA2AEAV7NarXon47U6+9w1/gE3VQOAx1gAAMDUuLMDXKGhiYmy2WxO248XFbmxGgDAlSLs4JpRX1gJDw/X1o8/dtpus9m0c8tmp+0d4uIbVR8AoGkQdnDNqC+sDEhKdmM1AAB3YcwOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNd7GAv6jsLBQPbp3d9rOPDoA0DwRdoD/MIwq5tGB29S3fhZrZwGuQ9gBAA+ob/0s1s4CXIewA8CrXLpkV0ZGhvMOhvtqAWAOhB0AXsZQQv/b6mivIwgB31NYWKQunbs4bY+MjNJnOz51Y0XwBMIOAMC07Ha7ZqY+7bR9SeYLbqwGnsKr5wAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNR4GwuAWzGPDgB3I+wAcDPm0QHgXjzGAgAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApsbbWACAaxarol8bCDsAgGsWq6JfGwg7MIWhiYmy2Wx19jleVOSmagAA3oSwg2ahvjBzvKhIBfv31LmPDnHxri4L8Bir1aJFr6TX2Q7gO4QdNAs2m007t2x22k6QgdkYVVV1hhkfH6veyXjNaftd4x9ogqqA5omwAwBeyGqtO8ykEGaAK0bYAQATqu/OEI+5cC0h7ACACdV3Z4jHXLiWEHYAuNSlS3ZlZNSxcrnhvloAQCLsALhK9YcZQwn9b6tjD3X8FgCaAGEHblHfq+Ph4eHa+vHHbqwIDUeYAdC8EHbgFvW9Oj4gKdmN1QAAriWEHXiFwsJC9eje3Wk7sx8DABqKsAOvYBhVTBoIAGgShB0AAJxgVXRzIOwAAOAEq6Kbg9XTBQAAADQl7uwAwDWovuUkJJaUgHkQdgA41DthoMQMyCZR33ISEktKXIn6xvRIUqnNpojwcKftjPtpeoQdAP+lvgkDJSYNvHawmGj96hvTI0kz50xj3I+HEXbgEvXNkMw8OUDzw2KiMAvCDlyivhmSmScHMJ/67vz4+Fi4MwSvYJqwk5mZqZdeeknFxcXq1auXMjIydOutt3q6LLdg3SkAnlDfnZ+U8Q9wZwhewRRh57333tOsWbO0YsUKDRgwQEuXLlVycrIOHTqkiIgIT5fX5Jp63an6wpTEYyoAgPcyRdh55ZVXNHXqVE2ePFmStGLFCm3YsEFvvvmmfvGLX3i4usZr6vEwV7L/gv176twHj6m8Q31vU12qvFT321a8aQU3YgD0d5iluek1+7BTUVGh/Px8paWlObZZrVYlJSUpLy/Pg5W5TlOPh2G8zZWpN0hcsruxGmfqe5sqo952wF1cMQDaam3+44KYpbnpNfuwc+LECdntdkVGRlbbHhkZqYMHD9b6m/LycpWXlzu+nz59WpJUVlbm8vruTknRiRMnnLaHhYVp/YYNde7Dbrer7MwZp+2GYdTZ/u9//1vxXbs6bT9WUKCXXn7ZaXtl5aU693+5T2P2Ud85NLb9imqsqKyz3aiqUs9ufets94breO78+SZrd8cxqNE72r2hhiq7XQtfnldnjVarRSsWOf93ZsxDE+rch8UiXbh4oc72un5vtdbdfiXHMAyjzvaqqqom+fvpagy/M1k2W6nTdtuJEwoPC3PaHh4eoY+ynf9PdUNdvi6GUc9taaOZKywsNCQZubm51bbPnj3buPXWW2v9zfz58w19d8OeDx8+fPjw4dPMP8eOHaszKzT7OzthYWHy8fFRSUlJte0lJSWKioqq9TdpaWmaNWuW43tVVZVOnjyptm3bymJp2C3PsrIyRUdH69ixYwoODm7QPq51XMPG4xo2Dtev8biGjcc1vHKGYejMmTNq3759nf2afdjx8/NT3759lZOTo1GjRkn6Lrzk5ORo+vTptf7G399f/v7+1baFhoa6pJ7g4GD+cDYS17DxuIaNw/VrPK5h43ENr0xISEi9fZp92JGkWbNmaeLEierXr59uvfVWLV26VOfOnXO8nQUAAK5dpgg7P/7xj2Wz2TRv3jwVFxerd+/e2rRpU41BywAA4NpjirAjSdOnT3f62Mod/P39NX/+/BqPx3DluIaNxzVsHK5f43ENG49r6HoWw6jvfS0AAIDmy+rpAgAAAJoSYQcAAJgaYQcAAJgaYQcAAJgaYaeJbNiwQQMGDFBgYKBat27tmPAQV6e8vFy9e/eWxWLRrl27PF1Os3H06FFNmTJFnTp1UmBgoG688UbNnz9fFRUVni7Nq2VmZqpjx44KCAjQgAED9MUXX3i6pGZj4cKF6t+/v1q1aqWIiAiNGjVKhw4d8nRZzdaLL74oi8WiJ5980tOlmAJhpwn8+c9/1oQJEzR58mTt3r1bO3bs0Lhx4zxdVrP01FNP1TsNOGo6ePCgqqqq9Prrr2vfvn1asmSJVqxYoaefdr6y8rXuvffe06xZszR//nx9/fXX6tWrl5KTk1Va6nzxQ/y/Tz75RKmpqfr888+VnZ2tyspKDR8+XOfOnfN0ac3Ol19+qddff109e/b0dCnm4ZrlOHFZZWWl0aFDB+P3v/+9p0tp9jZu3GjExcUZ+/btMyQZf/vb3zxdUrO2ePFio1OnTp4uw2vdeuutRmpqquO73W432rdvbyxcuNCDVTVfpaWlhiTjk08+8XQpzcqZM2eMLl26GNnZ2cYdd9xhPPHEE54uyRS4s+NiX3/9tQoLC2W1WtWnTx+1a9dOI0aM0N69ez1dWrNSUlKiqVOn6p133lFQUJCnyzGF06dPq02bNp4uwytVVFQoPz9fSUlJjm1Wq1VJSUnKy8vzYGXN1+nTpyWJP3NXKTU1VSkpKdX+LKLxCDsu9q9//UuSlJ6errlz52r9+vVq3bq1hgwZopMnT3q4uubBMAxNmjRJjz32mPr16+fpckzh8OHDysjI0KOPPurpUrzSiRMnZLfbaywxExkZqeLiYg9V1XxVVVXpySef1KBBg9S9e3dPl9NsrF69Wl9//bUWLlzo6VJMh7BzhX7xi1/IYrHU+bk8TkKSnnnmGY0ZM0Z9+/bVypUrZbFYtGbNGg+fhWdd6TXMyMjQmTNnlJaW5umSvc6VXsP/VlhYqB/+8Ie6//77NXXqVA9VjmtJamqq9u7dq9WrV3u6lGbj2LFjeuKJJ/Tuu+8qICDA0+WYDstFXCGbzaZvvvmmzj433HCDduzYoaFDh+rTTz/V4MGDHW0DBgxQUlKSfvWrXzV1qV7rSq/hAw88oL/85S+yWCyO7Xa7XT4+Pho/frzeeuutpi7Va13pNfTz85MkFRUVaciQIRo4cKBWrVolq5X/v6lNRUWFgoKC9Kc//anam5MTJ07UqVOntHbtWs8V18xMnz5da9eu1fbt29WpUydPl9NsfPjhh7r33nvl4+Pj2Ga322WxWGS1WlVeXl6tDVfHNAuBNrXw8HCFh4fX269v377y9/fXoUOHHGGnsrJSR48eVWxsbFOX6dWu9BouW7ZMzz//vON7UVGRkpOT9d5772nAgAFNWaLXu9JrKH13RycxMdFxd5Gg45yfn5/69u2rnJwcR9ipqqpSTk6ORxcYbk4Mw9CMGTP0wQcfaNu2bQSdqzRs2DDt2bOn2rbJkycrLi5Oc+bMIeg0EmHHxYKDg/XYY49p/vz5io6OVmxsrF566SVJ0v333+/h6pqHmJiYat9btmwpSbrxxht1/fXXe6KkZqewsFBDhgxRbGysXn75ZdlsNkdbVFSUByvzXrNmzdLEiRPVr18/3XrrrVq6dKnOnTunyZMne7q0ZiE1NVVZWVlau3atWrVq5RjrFBISosDAQA9X5/1atWpVY3xTixYt1LZtW8Y9uQBhpwm89NJLuu666zRhwgRduHBBAwYM0NatW9W6dWtPl4ZrRHZ2tg4fPqzDhw/XCIg8ua7dj3/8Y9lsNs2bN0/FxcXq3bu3Nm3aVGPQMmq3fPlySdKQIUOqbV+5cqUmTZrk/oKA/8KYHQAAYGo8xAcAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAfbfyOQBzIuwA8Gp/+tOf1KNHDwUGBqpt27ZKSkrSuXPnJElvvvmmunXrJn9/f7Vr167aCuUFBQUaOXKkWrZsqeDgYD3wwAMqKSlxtKenp6t37976/e9/r06dOikgIECSdOrUKT388MMKDw9XcHCwhg4dqt27d7v3pAG4FGEHgNc6fvy4HnzwQT300EM6cOCAtm3bptGjR8swDC1fvlypqal65JFHtGfPHq1bt06dO3eWJFVVVWnkyJE6efKkPvnkE2VnZ+tf//qXfvzjH1fb/+HDh/XnP/9Z77//vnbt2iVJuv/++1VaWqq//vWvys/P1y233KJhw4bp5MmT7j59AC7CQqAAvNbXX3+tvn376ujRo4qNja3W1qFDB02ePFnPP/98jd9lZ2drxIgROnLkiKKjoyVJ+/fvV7du3fTFF1+of//+Sk9P1wsvvKDCwkKFh4dLkj777DOlpKSotLRU/v7+jv117txZTz31lB555JEmPFsATeU6TxcAAM706tVLw4YNU48ePZScnKzhw4frvvvuU2VlpYqKijRs2LBaf3fgwAFFR0c7go4kxcfHKzQ0VAcOHFD//v0lSbGxsY6gI0m7d+/W2bNn1bZt22r7u3Dhgv75z382wRkCcAfCDgCv5ePjo+zsbOXm5uqjjz5SRkaGnnnmGeXk5Lhk/y1atKj2/ezZs2rXrp22bdtWo29oaKhLjgnA/Qg7ALyaxWLRoEGDNGjQIM2bN0+xsbHKzs5Wx44dlZOTo8TExBq/6dq1q44dO6Zjx45Ve4x16tQpxcfHOz3WLbfcouLiYl133XXq2LFjU50SADcj7ADwWjt37lROTo6GDx+uiIgI7dy5UzabTV27dlV6eroee+wxRUREaMSIETpz5ox27NihGTNmKCkpST169ND48eO1dOlSXbp0SY8//rjuuOMO9evXz+nxkpKSlJCQoFGjRmnx4sW66aabVFRUpA0bNujee++t87cAvBdhB4DXCg4O1vbt27V06VKVlZUpNjZWv/71rzVixAhJ0sWLF7VkyRL9/Oc/V1hYmO677z5J390NWrt2rWbMmKHbb79dVqtVP/zhD5WRkVHn8SwWizZu3KhnnnlGkydPls1mU1RUlG6//XZFRkY2+fkCaBq8jQUAAEyNeXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICp/R9m5X3KpffJoQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import warnings\n", "\n", "import pandas as pd\n", "import seaborn as sns\n", "\n", "warnings.filterwarnings(\"ignore\", \"is_categorical_dtype\")\n", "warnings.filterwarnings(\"ignore\", \"use_inf_as_na\")\n", "\n", "score = clf.decision_function(X)\n", "df = pd.DataFrame({\"score\": score, \"y\": y})\n", "sns.histplot(df, x=\"score\", hue=\"y\").set_title(\"Smooth SVM\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "Mfu2LWh7MuGj" }, "source": [ "## With Pipeline\n", "`plq_Ridge_Classifier` can be integrated into a scikit-learn Pipeline to streamline preprocessing including scaling." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "id": "MCEXWUchaaP_" }, "outputs": [], "source": [ "## simulate data\n", "from sklearn.datasets import make_classification\n", "from sklearn.pipeline import Pipeline\n", "from sklearn.preprocessing import StandardScaler\n", "\n", "n, d = 10000, 5\n", "X, y = make_classification(n_samples=n, n_features=d, random_state=42)\n", "y = 2 * y - 1" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 155 }, "id": "PwMysLRWaiv_", "outputId": "428843f0-59b4-4a72-a2db-664b4b269b47" }, "outputs": [ { "data": { "text/html": [ "
Pipeline(steps=[('scaler', StandardScaler()),\n",
       "                ('clf', plq_Ridge_Classifier(loss={'name': 'sSVM'}))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "Pipeline(steps=[('scaler', StandardScaler()),\n", " ('clf', plq_Ridge_Classifier(loss={'name': 'sSVM'}))])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## solve SVM via `plq_Ridge_Classifier`\n", "from rehline import plq_Ridge_Classifier\n", "\n", "pipe = Pipeline([(\"scaler\", StandardScaler()), (\"clf\", plq_Ridge_Classifier(loss={\"name\": \"sSVM\"}, C=1.0))])\n", "pipe.fit(X=X, y=y)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 472 }, "id": "ZMofqZ_zakNc", "outputId": "f051e47f-aeea-41f4-f283-4c3e4ab4cf93" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAANvZJREFUeJzt3X10FNX9x/HPbswjkATIE2ASUNAQngWEFFQCkRSjBUEtQikgomKgCr8ijVKIaEXQCiVGsK2CWlM8tCoUKBiCiJKIGgvl+ZQWfqEJSRYRwmMSNvP7w7K/xmQTSDa7m+H9OmfPcefenfnOHJSPM3futRiGYQgAAMCkrJ4uAAAAoCkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgBcMywWi6ZPn+7pMgC4GWEHQK327Nmj++67T7GxsQoICFCHDh105513KiMjw9Ol1Sk3N1fp6ek6deqUy/dts9n0xBNPKC4uToGBgYqIiNCtt96qOXPm6OzZs6qsrFRYWJgGDx7sdB+GYSg6Olq33HKLJGnbtm2yWCyyWCz6wx/+UOtvBg0aJIvFou7du7v8nIBrAWEHQA25ubnq16+fdu/eralTp+rVV1/Vww8/LKvVqt/85jeeLq9Oubm5evbZZ10edk6ePKl+/frp7bffVkpKipYtW6ZZs2apc+fOWr58uU6cOCFfX1/df//9ys3N1f/+7//Wup/t27fr3//+t37yk59U2x4QEKCsrKwa/Y8eParc3FwFBAS49HyAa8l1ni4AgPf51a9+pZCQEH355ZcKDQ2t1lZaWuqZojzsjTfeUEFBgXbs2KEf/OAH1drKysrk5+cnSRo/frxWrFihP/7xj/rFL35RYz9ZWVmyWq0aO3Zste133XWX1q1bpxMnTigsLKxa/8jISHXp0kXffvttE5wZYH7c2QFQwz//+U9169atRtCRpIiIiGrfL4+DWbNmjeLj4xUYGKiEhATt2bNHkvT666+rc+fOCggI0JAhQ3T06NEa+1yzZo369u2rwMBAhYWF6Sc/+YkKCwtr9Nu6datuu+02tWjRQqGhoRo5cqQOHDjgaE9PT9fs2bMlSZ06dXI8Hvr+MT/88EN1795d/v7+6tatmzZt2nRF18THx0cDBw6s0RYcHOy48zJo0CB17Nix1rs0lZWV+tOf/qTExES1b9++WtvIkSPl7++vNWvWVNuelZWlBx54QD4+PvXWCKB2hB0ANcTGxio/P1979+69ov6ffvqp/ud//kcTJ05Uenq6Dhw4oLvvvluZmZlatmyZHn/8cc2ePVt5eXl66KGHqv121apVjr/MFy5cqKlTp+r999/X4MGDqz2K2rJli5KTk1VaWqr09HTNmjVLubm5GjRokCPMjB49Wg8++KAkacmSJXrnnXf0zjvvKDw83LGfzz77TI8//rjGjh2rxYsX6+LFixozZoy++eabeq+J3W7XO++8U2c/i8WicePGac+ePdq3b1+1tk2bNunkyZMaP358jd8FBQVp5MiR+uMf/+jYtnv3bu3bt0/jxo2r85gA6mEAwPd89NFHho+Pj+Hj42MkJCQYTz31lLF582ajoqKiRl9Jhr+/v3HkyBHHttdff92QZERFRRllZWWO7WlpaYYkR9+KigojIiLC6N69u3HhwgVHv/Xr1xuSjHnz5jm29e7d24iIiDC++eYbx7bdu3cbVqvV+OlPf+rY9tJLL1U7xvdr9fPzMw4fPlxtH5KMjIyMOq9JcXGxER4ebkgy4uLijMcee8zIysoyTp06VaPvvn37DElGWlpate1jx441AgICjNOnTzu2ffzxx4YkY82aNcb69esNi8ViFBQUGIZhGLNnzzZuuOEGwzAM44477jC6detWZ40AasedHQA13HnnncrLy9OPfvQj7d69W4sXL1ZycrI6dOigdevW1eg/bNgwdezY0fF9wIABkqQxY8aoVatWNbb/61//kiR99dVXKi0t1eOPP15tAG5KSori4uK0YcMGSdLx48e1a9cuTZo0SW3atHH069mzp+68805t3Ljxis8tKSlJN954Y7V9BAcHO2pyJjIyUrt379Zjjz2mb7/9VitWrNC4ceMUERGh5557ToZhOPrGx8erT58+Wr16tWPbuXPntG7dOt19990KDg6u9RjDhw9XmzZttHr1ahmGodWrVzvuVAFoOMIOgFr1799f77//vr799lt98cUXSktL05kzZ3Tfffdp//791frGxMRU+x4SEiJJio6OrnX75YG2l99Yuvnmm2scPy4uztFeV7+uXbvqxIkTOnfu3BWd1/drlaTWrVtf0eDfdu3aafny5Tp+/LgOHTqkZcuWKTw8XPPmzdMbb7xRre/48eN15MgR5ebmSvpunND58+drfYR12eW3ubKysrR9+3YdO3aMR1iACxB2ANTJz89P/fv31wsvvKDly5ersrKyxiBaZ4NnnW3/77sg7uaKmiwWi2666SbNmDFD27dvl9Vq1bvvvlutz4MPPiir1eoYqJyVlaXWrVvrrrvuqnPf48aN065du5Senq5evXopPj7+iusCUDvCDoAr1q9fP0nfPVZyhdjYWEnSoUOHarQdOnTI0V5Xv4MHDyosLEwtWrSQ9F0QcacbbrhBrVu3rnFN2rdvr8TERK1Zs0YlJSXKzs7Wfffd53hF3ZnBgwcrJiZG27Zt464O4CKEHQA1fPzxx7Xe6bg8Nqa2x0kN0a9fP0VERGjFihUqLy93bP/rX/+qAwcOKCUlRdJ3j4969+6tt956q9obWnv37tVHH31U7W7J5dDj6kkFd+7cWeujsi+++ELffPNNrddk/PjxKi0t1aOPPqrKyso6H2FdZrFYtGzZMs2fP18TJkxwSe3AtY5JBQHUMGPGDJ0/f1733nuv4uLiVFFRodzcXL333nvq2LGjJk+e7JLj+Pr6atGiRZo8ebLuuOMOPfjggyopKdFvfvMbdezYUTNnznT0femllzRixAglJCRoypQpunDhgjIyMhQSEqL09HRHv759+0qSnnnmGY0dO1a+vr665557HCGood555x29++67uvfee9W3b1/5+fnpwIEDevPNNxUQEKCnn366xm/GjBmjxx9/XGvXrlV0dLRuv/32KzrWyJEjNXLkyEbVC+D/EXYA1PDyyy9rzZo12rhxo37729+qoqJCMTExevzxxzV37txaJxtsqEmTJikoKEgvvvii5syZoxYtWujee+/VokWLqh0nKSlJmzZt0vz58zVv3jz5+vrqjjvu0KJFi9SpUydHv/79++u5557TihUrtGnTJlVVVenIkSONDjuPPvqogoKClJOTo7Vr16qsrEzh4eEaPny40tLS1KdPnxq/CQ4O1j333KM1a9bowQcfdPsjNgDfsRieHCkIAADQxBizAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI15diRVVVWpqKhIrVq1Yh4MAACaCcMwdObMGbVv315Wq/P7N4QdSUVFRTVWZwYAAM3DsWPHdP311zttJ+xIatWqlaTvLlZwcLCHqwEAAFeirKxM0dHRjr/HnSHs6P9XSQ4ODibsAADQzNQ3BIUBygAAwNQIOwAAwNQIOwAAwNQYswMAQDNht9tVWVnp6TLcxtfXVz4+Po3eD2EHAAAvZxiGiouLderUKU+X4nahoaGKiopq1Dx4hB0AALzc5aATERGhoKCga2ICXMMwdP78eZWWlkqS2rVr1+B9EXYAAPBidrvdEXTatm3r6XLcKjAwUJJUWlqqiIiIBj/SYoAyAABe7PIYnaCgIA9X4hmXz7sxY5UIOwAANAPXwqOr2rjivAk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AACgmrfffltt27ZVeXl5te2jRo3ShAkTPFRVwxF2AABANffff7/sdrvWrVvn2FZaWqoNGzbooYce8mBlDcOkggDcamhiomw2m9P28PBwbf34YzdWBOD7AgMDNW7cOK1cuVL333+/JOkPf/iDYmJiNGTIEM8W1wCEHQBuZbPZtHPLZqftA5KS3VgNAGemTp2q/v37q7CwUB06dNCqVas0adKkZjnfD2EHAADU0KdPH/Xq1Utvv/22hg8frn379mnDhg2eLqtBCDsAvEphYaF6dO/utJ3HXID7PPzww1q6dKkKCwuVlJSk6OhoT5fUIIQdAF7FMKp4zAV4iXHjxunnP/+5fve73+ntt9/2dDkNxttYAACgViEhIRozZoxatmypUaNGebqcBiPsAAAApwoLCzV+/Hj5+/t7upQG4zEWAACo4dtvv9W2bdu0bds2vfbaa54up1EIOwBgQoMH3aaSkmKn7ZGRUfpsx6durAjNTZ8+ffTtt99q0aJFuvnmmz1dTqMQdgDAhEpKijUz9Wmn7UsyX3BjNWiOjh496ukSXIawAwDNUH13bgoLi9xYDeDdCDsA0AzVd+dm5pxpbqwG8G68jQUAAEyNsAMAAEyNsAMAAEyNsAMAAEyNAcoAADRTBQUFOnHihFuOFRYWppiYGLccy9UIOwBcamhiomw2m9P240W8Eg24QkFBgbp27arz58+75XhBQUE6cOBAowLP+++/rxUrVig/P18nT57U3/72N/Xu3dt1RTpB2AHgUjabrc5VyzvExbuxGsC8Tpw4ofPnz+uNV5fp5i5dmvRYh/7xD02Z/jOdOHGiUWHn3LlzGjx4sB544AFNnTrVhRXWjbADAEAzdnOXLurTs4eny7giEyZMkOT+2ZkZoAwAAEyNsAMAAEyNsAMAAFzu3XffVcuWLR2fTz/91GO1MGYHAAC43I9+9CMNGDDA8b1Dhw4eq4WwAwAAXK5Vq1Zq1aqVp8uQRNgBAABucvLkSRUUFKjoP/NtHTp0SJIUFRWlqKioJjsuYQcAgGbs0D/+0WyOsW7dOk2ePNnxfezYsZKk+fPnKz093SXHqA1hBwCAZigsLExBQUGaMv1nbjleUFCQwsLCGrWPSZMmadKkSa4p6CoQdgAAaIZiYmJ04MAB1sa6AoQdAACaqZiYmGYbQNyJeXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpMc8OgKsyNDFRNpvNafvx/6x5A6DpFRQUMKngFSDsALgqNptNO7dsdtreIS7ejdUA166CggLFxcXpwoULbjleYGCgDh48eFWBZ/v27XrppZeUn5+v48eP64MPPtCoUaOarkgnCDsA4IUGD7pNJSXFTtsLC7mDdq07ceKELly4oInjpyoqsn2THqu4pEhvvfs7nThx4qrCzrlz59SrVy899NBDGj16dBNWWDfCDgB4oZKSYs1Mfdpp+8w509xYDbxZVGR7xVwf6+kyajVixAiNGDHC02UwQBkAAJgbYQcAAJgaYQcAAJgaYQcAAJia14SdF198URaLRU8++aRj28WLF5Wamqq2bduqZcuWGjNmjEpKSqr9rqCgQCkpKQoKClJERIRmz56tS5cuubl6AADgrbzibawvv/xSr7/+unr27Flt+8yZM7VhwwatWbNGISEhmj59ukaPHq0dO3ZIkux2u1JSUhQVFaXc3FwdP35cP/3pT+Xr66sXXnjBE6cCAAD+4+zZszp8+LDj+5EjR7Rr1y61adPGrRMUejzsnD17VuPHj9fvfvc7Pf/8847tp0+f1htvvKGsrCwNHTpUkrRy5Up17dpVn3/+uQYOHKiPPvpI+/fv15YtWxQZGanevXvrueee05w5c5Seni4/Pz9PnRYA1Il5dOAqxSVN/2elocf46quvlJiY6Pg+a9YsSdLEiRO1atUqV5R2RTwedlJTU5WSkqKkpKRqYSc/P1+VlZVKSkpybIuLi1NMTIzy8vI0cOBA5eXlqUePHoqMjHT0SU5O1rRp07Rv3z716dOn1mOWl5ervLzc8b2srKwJzgwAnGMeHTRWWFiYAgMD9da7v3PL8QIDAxUWFnZVvxkyZIgMw2iiiq6cR8PO6tWr9fXXX+vLL7+s0VZcXCw/Pz+FhoZW2x4ZGani4mJHn/8OOpfbL7c5s3DhQj377LONrB4AAM+JiYnRwYMHWRvrCngs7Bw7dkxPPPGEsrOzFRAQ4NZjp6WlOW6lSd/d2YmOjnZrDYA3qm+RT4mFPgFvEhMT02wDiDt5LOzk5+ertLRUt9xyi2Ob3W7X9u3b9eqrr2rz5s2qqKjQqVOnqt3dKSkpUVRUlCQpKipKX3zxRbX9Xn5b63Kf2vj7+8vf39+FZwOYQ32LfEos9Amg+fHYq+fDhg3Tnj17tGvXLsenX79+Gj9+vOOffX19lZOT4/jNoUOHVFBQoISEBElSQkKC9uzZo9LSUkef7OxsBQcHKz6e/yADAAAP3tlp1aqVunfvXm1bixYt1LZtW8f2KVOmaNasWWrTpo2Cg4M1Y8YMJSQkaODAgZKk4cOHKz4+XhMmTNDixYtVXFysuXPnKjU1lTs3AABT8YaBvp7givP2+NtYdVmyZImsVqvGjBmj8vJyJScn67XXXnO0+/j4aP369Zo2bZoSEhLUokULTZw4UQsWLPBg1QAAuI6vr68k6fz58woMDPRwNe53/vx5Sf9/HRrCq8LOtm3bqn0PCAhQZmamMjMznf4mNjZWGzdubOLKAADwDB8fH4WGhjqGbAQFBclisXi4qqZnGIbOnz+v0tJShYaGysfHp8H78qqwAwAAarr80s1/j1G9VoSGhtb50tGVIOwAAODlLBaL2rVrp4iICFVWVnq6HLfx9fVt1B2dywg7AAA0Ez4+Pi75y/9a4zWrngMAADQF7uwAwDWosLBIXTp3qbNPZGSUPtvxqZsqApoOYQcArkF2u73OhUglaUnmC26qBmhaPMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmxttYAJqVwsJC9ejevc4+4eHh2vrxx26qCIC3I+wAaFYMo0o7t2yus8+ApGQ3VQOgOeAxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDUmFQQAFxs86DaVlBTX2aewsMhN1QAg7ACAi5WUFGtm6tN19pk5Z5qbqgHAYywAAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqvHoOAKhVYWGRunTu4rQ9MjJKn+341I0VAQ1D2AEA1Mput9c5X9CSzBfcWA3QcDzGAgAApsadHQCmU1hYqB7duzttDw8P19aPP3ZjRQA8ibADwHQMo0o7t2x22j4gKdmN1QDwNB5jAQAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU7vO0wUAQHMzeNBtKikpdtpeWFjkxmoA1IewAwBXqaSkWDNTn3baPnPONDdWA6A+PMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACm5tGws3z5cvXs2VPBwcEKDg5WQkKC/vrXvzraL168qNTUVLVt21YtW7bUmDFjVFJSUm0fBQUFSklJUVBQkCIiIjR79mxdunTJ3acCAAC8lEfDzvXXX68XX3xR+fn5+uqrrzR06FCNHDlS+/btkyTNnDlTf/nLX7RmzRp98sknKioq0ujRox2/t9vtSklJUUVFhXJzc/XWW29p1apVmjdvnqdOCQAAeJnrPHnwe+65p9r3X/3qV1q+fLk+//xzXX/99XrjjTeUlZWloUOHSpJWrlyprl276vPPP9fAgQP10Ucfaf/+/dqyZYsiIyPVu3dvPffcc5ozZ47S09Pl5+fnidMCAABexGvG7Njtdq1evVrnzp1TQkKC8vPzVVlZqaSkJEefuLg4xcTEKC8vT5KUl5enHj16KDIy0tEnOTlZZWVljrtDAADg2ubROzuStGfPHiUkJOjixYtq2bKlPvjgA8XHx2vXrl3y8/NTaGhotf6RkZEqLi6WJBUXF1cLOpfbL7c5U15ervLycsf3srIyF50NAADwNh6/s3PzzTdr165d2rlzp6ZNm6aJEydq//79TXrMhQsXKiQkxPGJjo5u0uMBAADP8XjY8fPzU+fOndW3b18tXLhQvXr10m9+8xtFRUWpoqJCp06dqta/pKREUVFRkqSoqKgab2dd/n65T23S0tJ0+vRpx+fYsWOuPSkAAOA1PB52vq+qqkrl5eXq27evfH19lZOT42g7dOiQCgoKlJCQIElKSEjQnj17VFpa6uiTnZ2t4OBgxcfHOz2Gv7+/43X3yx8AAGBOHh2zk5aWphEjRigmJkZnzpxRVlaWtm3bps2bNyskJERTpkzRrFmz1KZNGwUHB2vGjBlKSEjQwIEDJUnDhw9XfHy8JkyYoMWLF6u4uFhz585Vamqq/P39PXlqAADAS3g07JSWluqnP/2pjh8/rpCQEPXs2VObN2/WnXfeKUlasmSJrFarxowZo/LyciUnJ+u1115z/N7Hx0fr16/XtGnTlJCQoBYtWmjixIlasGCBp04JAAB4GY+GnTfeeKPO9oCAAGVmZiozM9Npn9jYWG3cuNHVpQEAAJPwujE7AAAAruTxeXYAAM1TYWGRunTu4rQ9MjJKn+341I0VAbUj7AAAGsRut2tm6tNO25dkvuDGagDneIwFAABMjTs7APA9gwfdppIS50vOFBYWubEaAI1F2AGA7ykpKa7z8czMOdPcWA2AxuIxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDWWiwCuIUMTE2Wz2Zy2Hy9izScA5kPYAa4hNptNO7dsdtreIS7ejdUAgHs06DHWDTfcoG+++abG9lOnTumGG25odFEAAACu0qCwc/ToUdnt9hrby8vLVVhY2OiiAAAAXOWqHmOtW7fO8c+bN29WSEiI47vdbldOTo46duzosuIAXB3G5ABATVcVdkaNGiVJslgsmjhxYrU2X19fdezYUb/+9a9dVhyAq8OYHACo6arCTlVVlSSpU6dO+vLLLxUWFtYkRQEAALhKg97GOnLkiKvrAAAAaBINfvU8JydHOTk5Ki0tddzxuezNN99sdGEAAACu0KCw8+yzz2rBggXq16+f2rVrJ4vF4uq6AAAAXKJBYWfFihVatWqVJkyY4Op6AAAAXKpB8+xUVFToBz/4gatrAQAAcLkGhZ2HH35YWVlZrq4FAADA5Rr0GOvixYv67W9/qy1btqhnz57y9fWt1v7KK6+4pDgAAIDGalDY+fvf/67evXtLkvbu3VutjcHKAADAmzQo7Hz88ceurgMAAKBJNGjMDgAAQHPRoDs7iYmJdT6u2rp1a4MLAgAAcKUGhZ3L43Uuq6ys1K5du7R3794aC4QCAAB4UoPCzpIlS2rdnp6errNnzzaqIAAAAFdy6Zidn/zkJ6yLBQAAvIpLw05eXp4CAgJcuUsAAIBGadBjrNGjR1f7bhiGjh8/rq+++kq//OUvXVIYAACAKzQo7ISEhFT7brVadfPNN2vBggUaPny4SwoDAABwhQaFnZUrV7q6DgAAgCbRoLBzWX5+vg4cOCBJ6tatm/r06eOSogAAAFylQWGntLRUY8eO1bZt2xQaGipJOnXqlBITE7V69WqFh4e7skYAAIAGa9DbWDNmzNCZM2e0b98+nTx5UidPntTevXtVVlamn/3sZ66uEQAAoMEadGdn06ZN2rJli7p27erYFh8fr8zMTAYoAwAAr9KgOztVVVXy9fWtsd3X11dVVVWNLgoAAMBVGhR2hg4dqieeeEJFRUWObYWFhZo5c6aGDRvmsuIAAAAaq0Fh59VXX1VZWZk6duyoG2+8UTfeeKM6deqksrIyZWRkuLpGAACABmvQmJ3o6Gh9/fXX2rJliw4ePChJ6tq1q5KSklxaHAAAQGNd1Z2drVu3Kj4+XmVlZbJYLLrzzjs1Y8YMzZgxQ/3791e3bt306aefNlWtAAAAV+2qws7SpUs1depUBQcH12gLCQnRo48+qldeecVlxQEAADTWVT3G2r17txYtWuS0ffjw4Xr55ZcbXRQAoPkrLCxSl85dnLZHRkbpsx08DUDTu6qwU1JSUusr546dXXedbDZbo4sCADR/drtdM1Ofdtq+JPMFN1aDa9lVPcbq0KGD9u7d67T973//u9q1a9foogAAAFzlqsLOXXfdpV/+8pe6ePFijbYLFy5o/vz5uvvuu11WHAAAQGNd1WOsuXPn6v3339dNN92k6dOn6+abb5YkHTx4UJmZmbLb7XrmmWeapFAAAICGuKqwExkZqdzcXE2bNk1paWkyDEOSZLFYlJycrMzMTEVGRjZJoQDgKoWFherRvbvTdput1I3VAGhqVz2pYGxsrDZu3Khvv/1Whw8flmEY6tKli1q3bt0U9QGAyxlGlXZu2ey0vf1NXZ22AWh+GjSDsiS1bt1a/fv3d2UtAAAALtegtbEAAACaC8IOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNY+GnYULF6p///5q1aqVIiIiNGrUKB06dKhan4sXLyo1NVVt27ZVy5YtNWbMGJWUlFTrU1BQoJSUFAUFBSkiIkKzZ8/WpUuX3HkqAADAS3k07HzyySdKTU3V559/ruzsbFVWVmr48OE6d+6co8/MmTP1l7/8RWvWrNEnn3yioqIijR492tFut9uVkpKiiooK5ebm6q233tKqVas0b948T5wSAADwMg2eVNAVNm3aVO37qlWrFBERofz8fN1+++06ffq03njjDWVlZWno0KGSpJUrV6pr1676/PPPNXDgQH300Ufav3+/tmzZosjISPXu3VvPPfec5syZo/T0dPn5+Xni1AAAgJfwqjE7p0+fliS1adNGkpSfn6/KykolJSU5+sTFxSkmJkZ5eXmSpLy8PPXo0aPamlzJyckqKyvTvn37aj1OeXm5ysrKqn0AAIA5eU3Yqaqq0pNPPqlBgwap+38W6CsuLpafn59CQ0Or9Y2MjFRxcbGjz/cXH738/XKf71u4cKFCQkIcn+joaBefDQAA8BZeE3ZSU1O1d+9erV69usmPlZaWptOnTzs+x44da/JjAgAAz/DomJ3Lpk+frvXr12v79u26/vrrHdujoqJUUVGhU6dOVbu7U1JSoqioKEefL774otr+Lr+tdbnP9/n7+8vf39/FZwEAALyRR+/sGIah6dOn64MPPtDWrVvVqVOnau19+/aVr6+vcnJyHNsOHTqkgoICJSQkSJISEhK0Z88elZaWOvpkZ2crODhY8fHx7jkRAADgtTx6Zyc1NVVZWVlau3atWrVq5RhjExISosDAQIWEhGjKlCmaNWuW2rRpo+DgYM2YMUMJCQkaOHCgJGn48OGKj4/XhAkTtHjxYhUXF2vu3LlKTU3l7g0AAPBs2Fm+fLkkaciQIdW2r1y5UpMmTZIkLVmyRFarVWPGjFF5ebmSk5P12muvOfr6+Pho/fr1mjZtmhISEtSiRQtNnDhRCxYscNdpAAAAL+bRsGMYRr19AgIClJmZqczMTKd9YmNjtXHjRleWBgAATMJr3sYCAABoCoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgatd5ugAAcLdLl+zKyMhw3m63u7EaAE2NsAPgGmQoof9tdbQ7D0JwncLCInXp3MVpe2RklD7b8akbK4JZEXYAAB5ht9s1M/Vpp+1LMl9wYzUwM8bsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU2MGZaAZGZqYKJvN5rT9eFGRG6sBgOaBsAM0IzabTTu3bHba3iEu3o3VAEDzwGMsAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgasyzA3gRJg30DkZVlRa9ku603Wq1uK8YAI1G2AG8CJMGeger1ap3Ml5z2n7X+AfcWA2AxuIxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMLXrPF0AALjapUt2ZWRkOO9gNG7/RlWVFr2S7rTdarU07gAAXIqwA8CEDCX0v62O9jqC0BWwWq16J+M1p+13jX+gUfsH4Fo8xgIAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKbm0bCzfft23XPPPWrfvr0sFos+/PDDau2GYWjevHlq166dAgMDlZSUpH/84x/V+pw8eVLjx49XcHCwQkNDNWXKFJ09e9aNZwEAALyZR8POuXPn1KtXL2VmZtbavnjxYi1btkwrVqzQzp071aJFCyUnJ+vixYuOPuPHj9e+ffuUnZ2t9evXa/v27XrkkUfcdQoAAMDLeXSenREjRmjEiBG1thmGoaVLl2ru3LkaOXKkJOntt99WZGSkPvzwQ40dO1YHDhzQpk2b9OWXX6pfv36SpIyMDN111116+eWX1b59e7edCwAA8E5eO6ngkSNHVFxcrKSkJMe2kJAQDRgwQHl5eRo7dqzy8vIUGhrqCDqSlJSUJKvVqp07d+ree++tdd/l5eUqLy93fC8rK2u6EwH+Y2hiomw2W519jhcVuakawPsVFhapS+cudfaJjIzSZzs+dVNFaK68NuwUFxdLkiIjI6ttj4yMdLQVFxcrIiKiWvt1112nNm3aOPrUZuHChXr22WddXDFQN5vNpp1bNtfZp0NcvJuqab7qXQpCavRyEPAOdrtdM1OfrrPPkswX3FQNmjOvDTtNKS0tTbNmzXJ8LysrU3R0tAcrAnDl6lsKQmrschAAzMVrXz2PioqSJJWUlFTbXlJS4miLiopSaWlptfZLly7p5MmTjj618ff3V3BwcLUPAAAwJ6+9s9OpUydFRUUpJydHvXv3lvTdHZidO3dq2rRpkqSEhASdOnVK+fn56tu3ryRp69atqqqq0oABAzxVOgDATeob18OYHkgeDjtnz57V4cOHHd+PHDmiXbt2qU2bNoqJidGTTz6p559/Xl26dFGnTp30y1/+Uu3bt9eoUaMkSV27dtUPf/hDTZ06VStWrFBlZaWmT5+usWPH8iYWAFwD6hvXw5geSB4OO1999ZUSExMd3y+Po5k4caJWrVqlp556SufOndMjjzyiU6dOafDgwdq0aZMCAgIcv3n33Xc1ffp0DRs2TFarVWPGjNGyZcvcfi4AAMA7eTTsDBkyRIbh/LUJi8WiBQsWaMGCBU77tGnTRllZWU1RHgAAMAGvHaAMAADgCl47QBlobuqbNJAJAwHAMwg7gIvUN2kgEwYCgGfwGAsAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgay0UAgIsZVVVa9Ep6nX2sVot7igFA2AEAV7NarXon47U6+9w1/gE3VQOAx1gAAMDUuLMDXKGhiYmy2WxO248XFbmxGgDAlSLs4JpRX1gJDw/X1o8/dtpus9m0c8tmp+0d4uIbVR8AoGkQdnDNqC+sDEhKdmM1AAB3YcwOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNd7GAv6jsLBQPbp3d9rOPDoA0DwRdoD/MIwq5tGB29S3fhZrZwGuQ9gBAA+ob/0s1s4CXIewA8CrXLpkV0ZGhvMOhvtqAWAOhB0AXsZQQv/b6mivIwgB31NYWKQunbs4bY+MjNJnOz51Y0XwBMIOAMC07Ha7ZqY+7bR9SeYLbqwGnsKr5wAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNR4GwuAWzGPDgB3I+wAcDPm0QHgXjzGAgAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApsbbWACAaxarol8bCDsAgGsWq6JfGwg7MIWhiYmy2Wx19jleVOSmagAA3oSwg2ahvjBzvKhIBfv31LmPDnHxri4L8Bir1aJFr6TX2Q7gO4QdNAs2m007t2x22k6QgdkYVVV1hhkfH6veyXjNaftd4x9ogqqA5omwAwBeyGqtO8ykEGaAK0bYAQATqu/OEI+5cC0h7ACACdV3Z4jHXLiWEHYAuNSlS3ZlZNSxcrnhvloAQCLsALhK9YcZQwn9b6tjD3X8FgCaAGEHblHfq+Ph4eHa+vHHbqwIDUeYAdC8EHbgFvW9Oj4gKdmN1QAAriWEHXiFwsJC9eje3Wk7sx8DABqKsAOvYBhVTBoIAGgShB0AAJxgVXRzIOwAAOAEq6Kbg9XTBQAAADQl7uwAwDWovuUkJJaUgHkQdgA41DthoMQMyCZR33ISEktKXIn6xvRIUqnNpojwcKftjPtpeoQdAP+lvgkDJSYNvHawmGj96hvTI0kz50xj3I+HEXbgEvXNkMw8OUDzw2KiMAvCDlyivhmSmScHMJ/67vz4+Fi4MwSvYJqwk5mZqZdeeknFxcXq1auXMjIydOutt3q6LLdg3SkAnlDfnZ+U8Q9wZwhewRRh57333tOsWbO0YsUKDRgwQEuXLlVycrIOHTqkiIgIT5fX5Jp63an6wpTEYyoAgPcyRdh55ZVXNHXqVE2ePFmStGLFCm3YsEFvvvmmfvGLX3i4usZr6vEwV7L/gv176twHj6m8Q31vU12qvFT321a8aQU3YgD0d5iluek1+7BTUVGh/Px8paWlObZZrVYlJSUpLy/Pg5W5TlOPh2G8zZWpN0hcsruxGmfqe5sqo952wF1cMQDaam3+44KYpbnpNfuwc+LECdntdkVGRlbbHhkZqYMHD9b6m/LycpWXlzu+nz59WpJUVlbm8vruTknRiRMnnLaHhYVp/YYNde7Dbrer7MwZp+2GYdTZ/u9//1vxXbs6bT9WUKCXXn7ZaXtl5aU693+5T2P2Ud85NLb9imqsqKyz3aiqUs9ufets94breO78+SZrd8cxqNE72r2hhiq7XQtfnldnjVarRSsWOf93ZsxDE+rch8UiXbh4oc72un5vtdbdfiXHMAyjzvaqqqom+fvpagy/M1k2W6nTdtuJEwoPC3PaHh4eoY+ynf9PdUNdvi6GUc9taaOZKywsNCQZubm51bbPnj3buPXWW2v9zfz58w19d8OeDx8+fPjw4dPMP8eOHaszKzT7OzthYWHy8fFRSUlJte0lJSWKioqq9TdpaWmaNWuW43tVVZVOnjyptm3bymJp2C3PsrIyRUdH69ixYwoODm7QPq51XMPG4xo2Dtev8biGjcc1vHKGYejMmTNq3759nf2afdjx8/NT3759lZOTo1GjRkn6Lrzk5ORo+vTptf7G399f/v7+1baFhoa6pJ7g4GD+cDYS17DxuIaNw/VrPK5h43ENr0xISEi9fZp92JGkWbNmaeLEierXr59uvfVWLV26VOfOnXO8nQUAAK5dpgg7P/7xj2Wz2TRv3jwVFxerd+/e2rRpU41BywAA4NpjirAjSdOnT3f62Mod/P39NX/+/BqPx3DluIaNxzVsHK5f43ENG49r6HoWw6jvfS0AAIDmy+rpAgAAAJoSYQcAAJgaYQcAAJgaYQcAAJgaYaeJbNiwQQMGDFBgYKBat27tmPAQV6e8vFy9e/eWxWLRrl27PF1Os3H06FFNmTJFnTp1UmBgoG688UbNnz9fFRUVni7Nq2VmZqpjx44KCAjQgAED9MUXX3i6pGZj4cKF6t+/v1q1aqWIiAiNGjVKhw4d8nRZzdaLL74oi8WiJ5980tOlmAJhpwn8+c9/1oQJEzR58mTt3r1bO3bs0Lhx4zxdVrP01FNP1TsNOGo6ePCgqqqq9Prrr2vfvn1asmSJVqxYoaefdr6y8rXuvffe06xZszR//nx9/fXX6tWrl5KTk1Va6nzxQ/y/Tz75RKmpqfr888+VnZ2tyspKDR8+XOfOnfN0ac3Ol19+qddff109e/b0dCnm4ZrlOHFZZWWl0aFDB+P3v/+9p0tp9jZu3GjExcUZ+/btMyQZf/vb3zxdUrO2ePFio1OnTp4uw2vdeuutRmpqquO73W432rdvbyxcuNCDVTVfpaWlhiTjk08+8XQpzcqZM2eMLl26GNnZ2cYdd9xhPPHEE54uyRS4s+NiX3/9tQoLC2W1WtWnTx+1a9dOI0aM0N69ez1dWrNSUlKiqVOn6p133lFQUJCnyzGF06dPq02bNp4uwytVVFQoPz9fSUlJjm1Wq1VJSUnKy8vzYGXN1+nTpyWJP3NXKTU1VSkpKdX+LKLxCDsu9q9//UuSlJ6errlz52r9+vVq3bq1hgwZopMnT3q4uubBMAxNmjRJjz32mPr16+fpckzh8OHDysjI0KOPPurpUrzSiRMnZLfbaywxExkZqeLiYg9V1XxVVVXpySef1KBBg9S9e3dPl9NsrF69Wl9//bUWLlzo6VJMh7BzhX7xi1/IYrHU+bk8TkKSnnnmGY0ZM0Z9+/bVypUrZbFYtGbNGg+fhWdd6TXMyMjQmTNnlJaW5umSvc6VXsP/VlhYqB/+8Ie6//77NXXqVA9VjmtJamqq9u7dq9WrV3u6lGbj2LFjeuKJJ/Tuu+8qICDA0+WYDstFXCGbzaZvvvmmzj433HCDduzYoaFDh+rTTz/V4MGDHW0DBgxQUlKSfvWrXzV1qV7rSq/hAw88oL/85S+yWCyO7Xa7XT4+Pho/frzeeuutpi7Va13pNfTz85MkFRUVaciQIRo4cKBWrVolq5X/v6lNRUWFgoKC9Kc//anam5MTJ07UqVOntHbtWs8V18xMnz5da9eu1fbt29WpUydPl9NsfPjhh7r33nvl4+Pj2Ga322WxWGS1WlVeXl6tDVfHNAuBNrXw8HCFh4fX269v377y9/fXoUOHHGGnsrJSR48eVWxsbFOX6dWu9BouW7ZMzz//vON7UVGRkpOT9d5772nAgAFNWaLXu9JrKH13RycxMdFxd5Gg45yfn5/69u2rnJwcR9ipqqpSTk6ORxcYbk4Mw9CMGTP0wQcfaNu2bQSdqzRs2DDt2bOn2rbJkycrLi5Oc+bMIeg0EmHHxYKDg/XYY49p/vz5io6OVmxsrF566SVJ0v333+/h6pqHmJiYat9btmwpSbrxxht1/fXXe6KkZqewsFBDhgxRbGysXn75ZdlsNkdbVFSUByvzXrNmzdLEiRPVr18/3XrrrVq6dKnOnTunyZMne7q0ZiE1NVVZWVlau3atWrVq5RjrFBISosDAQA9X5/1atWpVY3xTixYt1LZtW8Y9uQBhpwm89NJLuu666zRhwgRduHBBAwYM0NatW9W6dWtPl4ZrRHZ2tg4fPqzDhw/XCIg8ua7dj3/8Y9lsNs2bN0/FxcXq3bu3Nm3aVGPQMmq3fPlySdKQIUOqbV+5cqUmTZrk/oKA/8KYHQAAYGo8xAcAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAfbfyOQBzIuwA8Gp/+tOf1KNHDwUGBqpt27ZKSkrSuXPnJElvvvmmunXrJn9/f7Vr167aCuUFBQUaOXKkWrZsqeDgYD3wwAMqKSlxtKenp6t37976/e9/r06dOikgIECSdOrUKT388MMKDw9XcHCwhg4dqt27d7v3pAG4FGEHgNc6fvy4HnzwQT300EM6cOCAtm3bptGjR8swDC1fvlypqal65JFHtGfPHq1bt06dO3eWJFVVVWnkyJE6efKkPvnkE2VnZ+tf//qXfvzjH1fb/+HDh/XnP/9Z77//vnbt2iVJuv/++1VaWqq//vWvys/P1y233KJhw4bp5MmT7j59AC7CQqAAvNbXX3+tvn376ujRo4qNja3W1qFDB02ePFnPP/98jd9lZ2drxIgROnLkiKKjoyVJ+/fvV7du3fTFF1+of//+Sk9P1wsvvKDCwkKFh4dLkj777DOlpKSotLRU/v7+jv117txZTz31lB555JEmPFsATeU6TxcAAM706tVLw4YNU48ePZScnKzhw4frvvvuU2VlpYqKijRs2LBaf3fgwAFFR0c7go4kxcfHKzQ0VAcOHFD//v0lSbGxsY6gI0m7d+/W2bNn1bZt22r7u3Dhgv75z382wRkCcAfCDgCv5ePjo+zsbOXm5uqjjz5SRkaGnnnmGeXk5Lhk/y1atKj2/ezZs2rXrp22bdtWo29oaKhLjgnA/Qg7ALyaxWLRoEGDNGjQIM2bN0+xsbHKzs5Wx44dlZOTo8TExBq/6dq1q44dO6Zjx45Ve4x16tQpxcfHOz3WLbfcouLiYl133XXq2LFjU50SADcj7ADwWjt37lROTo6GDx+uiIgI7dy5UzabTV27dlV6eroee+wxRUREaMSIETpz5ox27NihGTNmKCkpST169ND48eO1dOlSXbp0SY8//rjuuOMO9evXz+nxkpKSlJCQoFGjRmnx4sW66aabVFRUpA0bNujee++t87cAvBdhB4DXCg4O1vbt27V06VKVlZUpNjZWv/71rzVixAhJ0sWLF7VkyRL9/Oc/V1hYmO677z5J390NWrt2rWbMmKHbb79dVqtVP/zhD5WRkVHn8SwWizZu3KhnnnlGkydPls1mU1RUlG6//XZFRkY2+fkCaBq8jQUAAEyNeXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICp/R9m5X3KpffJoQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import warnings\n", "\n", "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import seaborn as sns\n", "\n", "warnings.filterwarnings(\"ignore\", \"is_categorical_dtype\")\n", "warnings.filterwarnings(\"ignore\", \"use_inf_as_na\")\n", "\n", "score = pipe.decision_function(X)\n", "df = pd.DataFrame({\"score\": score, \"y\": y})\n", "sns.histplot(df, x=\"score\", hue=\"y\").set_title(\"Smooth SVM\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "k_h8PGzS7AC2" }, "source": [ "## Hyperparameter Tuning with GridSearchCV\n", "\n", "Due to its compatibility with the scikit-learn API, `GridSearchCV` can be applied to determine the optimal hyperparameters for the ReHLine model." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "dZ-8KTfP7B2M", "outputId": "56ed5d61-a189-4935-d16b-3fc6ec2d720d" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Best Parameters: {'clf__C': 0.1}\n", "Best CV Accuracy: 0.8920\n" ] } ], "source": [ "from sklearn.model_selection import GridSearchCV\n", "\n", "# Define the parameter grid to search\n", "param_grid = {\"clf__C\": [0.1, 1.0, 10.0]}\n", "\n", "# Create the GridSearchCV object\n", "grid_search = GridSearchCV(pipe, param_grid, cv=5)\n", "grid_search.fit(X, y)\n", "\n", "# Print the best parameters and score\n", "print(f\"Best Parameters: {grid_search.best_params_}\")\n", "print(f\"Best CV Accuracy: {grid_search.best_score_:.4f}\")" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 472 }, "id": "UgMTfMwk9jdf", "outputId": "7df98ae9-b1ce-47b3-c3c4-4e25bb42c2f6" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAO5lJREFUeJzt3XtcVVX+//H3AeWmAl64aYCWFuINw0uklReClCkt0yxz1MqmQqf0O2ZWo3S1tNSJIW2a0rIc+zrTxbRU1LwkpoZpaupMk34xkIuZ4hUQ9u+PhvObEx5AOJzL9vV8PM7jMWettff+7JON7/Zeey+LYRiGAAAATMrL1QUAAAA0JMIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOgAZlsVg0YcIEV5fhFIMHD9b48eNdXUaDeeKJJ9S7d29XlwFcMsIO4Eb27NmjO++8U9HR0fLz81ObNm108803Kz093dWlVSsrK0tpaWk6ceKEw/ddVFSkRx99VDExMfL391doaKh69eqlqVOn6vTp0yorK1OrVq3Ut29fu/swDEORkZG69tprJUkbNmyQxWKRxWLRe++9d9Ft+vTpI4vFos6dO9eqzi1btmjNmjWaOnVqlb6CggL94Q9/UExMjAICAtSkSRPFx8fr+eefd9hvlpWVpb59+yogIEDh4eH6/e9/r9OnT9dq2/nz52v48OGKioqSxWLR2LFjLzruscce0+7du7V8+XKH1Aw4C2EHcBNZWVnq0aOHdu/erfHjx+vPf/6zHnjgAXl5eelPf/qTq8urVlZWlp555hmHh53jx4+rR48eevfdd5WSkqLXXntNkydPVvv27TV//nwdO3ZMjRs31vDhw5WVlaX/+7//u+h+Nm3apB9//FH33nuvTbufn5+WLFlSZfzhw4eVlZUlPz+/Wtc6e/ZsDRw4UO3bt7dp37Fjhzp37qyMjAzdcMMNmjNnjl599VV1795dL730kkaMGFHrY9iza9cuDRw4UGfPntWcOXP0wAMP6C9/+YuGDx9eq+1ffvllrV+/Xp06dVKjRo3sjgsPD9eQIUP0yiuv1LtmwJns/6kG4FQvvPCCgoKCtGPHDgUHB9v0FRYWuqYoF3vrrbeUk5OjLVu26Prrr7fpKy4ulo+PjyRp1KhRWrBggf72t7/piSeeqLKfJUuWyMvLSyNHjrRpHzx4sJYvX65jx46pVatWNuPDwsLUoUMH/fzzzzXWWVhYqJUrV2rBggU27SdOnNDtt98ub29vffPNN4qJibHpf+GFF/Tmm2/WuP+aPPnkk2revLk2bNigwMBASVLbtm01fvx4rVmzRklJSdVuv3HjRutVnaZNm1Y7dsSIERo+fLh++OEHXXnllfWuHXAGruwAbuLf//63OnXqVCXoSFJoaKjN98p5MMuWLVNsbKz8/f2VkJCgPXv2SJLeeOMNtW/fXn5+furXr58OHz5cZZ/Lli1TfHy8/P391apVK917773Kzc2tMm79+vW64YYb1KRJEwUHB2vIkCHav3+/tT8tLU1TpkyRJLVr1856e+jXx/z444/VuXNn+fr6qlOnTlq1alWtfhNvb29dd911VfoCAwOtV1769Omjtm3bXvQqTVlZmf7+97+rf//+at26tU3fkCFD5Ovrq2XLltm0L1myRCNGjJC3t3eNNUrSypUrdeHCBSUmJtq0v/HGG8rNzdWcOXOqBB1JCgsL09NPP12rY9hTXFyszMxM3XvvvdagI0m//e1v1bRpU/3v//5vjfuIjo6WxWKp1fEqz/GTTz6pW8GACxB2ADcRHR2t7Oxs7d27t1bjN2/erP/5n//RmDFjlJaWpv379+s3v/mNMjIy9Nprr+mRRx7RlClTtHXrVt1333022y5atMj6l/nMmTM1fvx4ffjhh+rbt6/Nrai1a9cqOTlZhYWFSktL0+TJk5WVlaU+ffpYw8wdd9yhu+++W5I0d+5cLV68WIsXL1ZISIh1P19++aUeeeQRjRw5UrNmzdL58+c1bNgw/fTTTzX+JuXl5Vq8eHG14ywWi+655x7t2bNH+/bts+lbtWqVjh8/rlGjRlXZLiAgQEOGDNHf/vY3a9vu3bu1b98+3XPPPdUe879lZWWpZcuWio6Otmlfvny5/P39deedd9ZqP6dPn9axY8dq/Jw8edK6zZ49e3ThwgX16NHDZl8+Pj6Ki4vTN998U+vzqI2goCBdddVV2rJli0P3CzQoA4BbWLNmjeHt7W14e3sbCQkJxuOPP26sXr3aKC0trTJWkuHr62scOnTI2vbGG28Ykozw8HCjuLjY2j5t2jRDknVsaWmpERoaanTu3Nk4d+6cddyKFSsMScb06dOtbXFxcUZoaKjx008/Wdt2795teHl5Gb/97W+tbbNnz7Y5xq9r9fHxMb7//nubfUgy0tPTq/1N8vPzjZCQEEOSERMTYzz00EPGkiVLjBMnTlQZu2/fPkOSMW3aNJv2kSNHGn5+fsbJkyetbV988YUhyVi2bJmxYsUKw2KxGDk5OYZhGMaUKVOMK6+80jAMw7jpppuMTp06VVujYRhG3759jfj4+CrtzZs3N7p161bj9pXGjBljSKrxc9NNN1m3WbZsmSHJ2LRpU5X9DR8+3AgPD6/18Q3DMJo0aWKMGTOm2jFJSUlGx44dL2m/gCtxZQdwEzfffLO2bt2q2267Tbt379asWbOUnJysNm3aXPTpl4EDB6pt27bW75WPBA8bNkzNmjWr0v7DDz9Ikr7++msVFhbqkUcesZmAm5KSopiYGK1cuVKSdPToUe3atUtjx45VixYtrOO6du2qm2++WZ999lmtzy0xMVFXXXWVzT4CAwOtNdkTFham3bt366GHHtLPP/+sBQsW6J577lFoaKiee+45GYZhHRsbG6vu3btr6dKl1rYzZ85o+fLl+s1vfmNzi+e/JSUlqUWLFlq6dKkMw9DSpUutV6pq66efflLz5s2rtBcXF9v8s6jJ448/rszMzBo/r776qnWbc+fOSZJ8fX2r7M/Pz8/a70jNmzfXsWPHHL5foKEwQRlwIz179tSHH36o0tJS7d69Wx999JHmzp2rO++8U7t27VJsbKx1bFRUlM22QUFBkqTIyMiLtldOtK18Yumaa66pcvyYmBh9+eWXNY7r2LGjVq9erTNnzqhJkyY1nteva5V++QuzNpN/IyIiNH/+fL3++uv617/+pdWrV+vll1/W9OnTFRERoQceeMA6dtSoUfrDH/6grKwsXX/99fr444919uzZi97CqlT5NNeSJUvUq1cvHTly5JJuYVX67+BVKTAwUKdOnar1PmJjY23+GdeGv7+/JKmkpKRK3/nz5639jmQYRq3n+ADugCs7gBvy8fFRz5499eKLL2r+/PkqKyurMonW3uRZe+0X+8vYWRxRk8Vi0dVXX62JEydq06ZN8vLy0vvvv28z5u6775aXl5d1ovKSJUvUvHlzDR48uNp933PPPdq1a5fS0tLUrVu3Sw4cLVu2vGhwi4mJ0T//+U+VlpbWaj8nT55Ufn5+jZ/jx49bt4mIiJD0y5W4Xzt69GiVSdmO8PPPP9s8vQa4O8IO4OYqJ55e7C+zuqicRHvw4MEqfQcPHrT2VzfuwIEDatWqlfWqjrP/K//KK69U8+bNq/wmrVu3Vv/+/bVs2TIVFBQoMzNTd955p/URdXv69u2rqKgobdiwoU5XdWJiYnTo0KEq7bfeeqvOnTunf/zjH7Xaz6OPPqqIiIgaP3fccYd1m86dO6tRo0b6+uuvbfZVWlqqXbt2KS4u7pLPpyaHDh1Sx44dHb5foKEQdgA38cUXX1z0Skfl3JiL3U6qix49eig0NFQLFiywufXx+eefa//+/UpJSZH0yxWDuLg4vfPOOzZPaO3du1dr1qyxuVpSGXoc/VLBbdu26cyZM1Xat2/frp9++umiv8moUaNUWFio3/3udyorK6v2FlYli8Wi1157TTNmzNDo0aMvuc6EhAT9/PPPVeYgPfTQQ4qIiND//M//6J///GeV7QoLC/X8889bv9dlzk5QUJASExP13nvv2dwyW7x4sU6fPm3zYsGzZ8/qwIED9Zpvc/LkSf373/+u8t4jwJ0xZwdwExMnTtTZs2d1++23KyYmRqWlpcrKytIHH3ygtm3baty4cQ45TuPGjfXyyy9r3Lhxuummm3T33XeroKBAf/rTn9S2bVtNmjTJOnb27NkaNGiQEhISdP/99+vcuXNKT09XUFCQ0tLSrOPi4+MlSU899ZRGjhypxo0b69Zbb63VfJ7qLF68WO+//75uv/12xcfHy8fHR/v379fbb78tPz8/Pfnkk1W2GTZsmB555BF98sknioyM1I033lirYw0ZMkRDhgypU50pKSlq1KiR1q5dqwcffNDa3rx5c3300UcaPHiw4uLidO+991p/q507d+pvf/ubEhISrOPrMmdH+uXlhNdff71uuukmPfjgg/rxxx/16quvKikpSbfccot13Pbt29W/f3/NmDHD5p/fp59+qt27d0v65b1E3377rTWE3Xbbberatat17Nq1a2UYRp1/K8AlXPcgGID/9vnnnxv33XefERMTYzRt2tTw8fEx2rdvb0ycONEoKCiwGSvJSE1NtWk7dOiQIcmYPXu2Tft/P2b93z744AOje/fuhq+vr9GiRQtj1KhRxo8//lilrrVr1xp9+vQx/P39jcDAQOPWW281vvvuuyrjnnvuOaNNmzaGl5eXzWPoF6vVMAwjOjq6xkecv/32W2PKlCnGtddea7Ro0cJo1KiRERERYQwfPtzYuXOn3e2GDx9uSDIef/zxi/bb+01+rbaPnhuGYdx2223GwIEDL9qXl5dnTJo0ybj66qsNPz8/IyAgwIiPjzdeeOEFm0fi62Pz5s3G9ddfb/j5+RkhISFGamqqzSsIDOP/n/eMGTNs2qt75H3hwoU2Y++66y6jb9++DqkZcBaLYbhw1iIAmMTmzZvVr18/HThwQB06dHB1OQ0iPz9f7dq109KlS7myA49C2AEABxk0aJCuuOIKh6x35Y6eeOIJrV+/Xtu3b3d1KcAlIewAAABT42ksAABgaoQdAABgaoQdAABgaoQdAABgarxUUFJFRYXy8vLUrFkzFrcDAMBDGIahU6dOqXXr1vLysn/9hrAjKS8vr8pK0QAAwDMcOXJEV1xxhd1+wo6kZs2aSfrlxwoMDHRxNQAAoDaKi4sVGRlp/XvcHsKO/v+KzYGBgYQdAAA8TE1TUJigDAAATI2wAwAATI2wAwAATI05OwAAeIjy8nKVlZW5ugynady4sby9veu9H8IOAABuzjAM5efn68SJE64uxemCg4MVHh5er/fgEXYAAHBzlUEnNDRUAQEBl8ULcA3D0NmzZ1VYWChJioiIqPO+CDsAALix8vJya9Bp2bKlq8txKn9/f0lSYWGhQkND63xLiwnKAAC4sco5OgEBAS6uxDUqz7s+c5UIOwAAeIDL4dbVxTjivAk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AADAxrvvvquWLVuqpKTEpn3o0KEaPXq0i6qqO8IOAACwMXz4cJWXl2v58uXWtsLCQq1cuVL33XefCyurG14qCMCpBvTvr6KiIrv9lS8PsyckJETrv/iiIUoD8B/+/v665557tHDhQg0fPlyS9N577ykqKkr9+vVzbXF1QNgB4FRFRUXatna13f42MbHV9vdOTG6IsgD8yvjx49WzZ0/l5uaqTZs2WrRokcaOHeuR7/sh7AAAgCq6d++ubt266d1331VSUpL27dunlStXurqsOiHsAACAi3rggQc0b9485ebmKjExUZGRka4uqU6YoAwAAC7qnnvu0Y8//qg333zTIycmVyLsAACAiwoKCtKwYcPUtGlTDR061NXl1Bm3sQDAhPr2uUEFBfl2+8PCwvXlls1OrAieKjc3V6NGjZKvr6+rS6kzwg4AmFBBQb4mpT5pt39uxotOrAae6Oeff9aGDRu0YcMGvf76664up14IOwAAoIru3bvr559/1ssvv6xrrrnG1eXUC2EHADxQTbepcnPznFgNzOjw4cOuLsFhCDsA4IFquk01aerDTqwGcG88jQUAAEyNsAMAAEyN21gAHKqmhT6P5jGXBIBzEXYAOFRtFvoEAGfiNhYAADA1ruwAAOChcnJydOzYMaccq1WrVoqKinLKsRyNsAMAgAfKyclRx44ddfbsWaccLyAgQPv3769X4Pnwww+1YMECZWdn6/jx4/rmm28UFxfnuCLtIOwAAOCBjh07prNnz+qtP7+mazp0aNBjHfzXv3T/hN/r2LFj9Qo7Z86cUd++fTVixAiNHz/egRVWj7ADAIAHu6ZDB3Xv2sXVZdTK6NGjJTn/7cxMUAYAAKZG2AEAAKZG2AEAAA73/vvvq2nTptbP5s2bXVYLc3YAAIDD3Xbbberdu7f1e5s2bVxWC2EHAAA4XLNmzdSsWTNXlyGJsAMAAJzk+PHjysnJUd5/1sg7ePCgJCk8PFzh4eENdlzCDgAAHuzgv/7lMcdYvny5xo0bZ/0+cuRISdKMGTOUlpbmkGNcDGEHAAAP1KpVKwUEBOj+Cb93yvECAgLUqlWreu1j7NixGjt2rGMKugSEHQAAPFBUVJT279/P2li1QNgBAMBDRUVFeWwAcSbeswMAAEyNsAMAAEyNsAMAAEyNOTsALsmA/v1VVFRkt//of96fAQDugrAD4JIUFRVp29rVdvvbxMQ6sRoAqBm3sQAAgKkRdgAAgKlxGwsAAA+Vk5PDSwVrgbADAG6ob58bVFCQb7c/N5eJ4Je7nJwcxcTE6Ny5c045nr+/vw4cOHBJgWfTpk2aPXu2srOzdfToUX300UcaOnRowxVpB2EHANxQQUG+JqU+abd/0tSHnVgN3NGxY8d07tw5jRk1XuFhrRv0WPkFeXrn/Td17NixSwo7Z86cUbdu3XTffffpjjvuaMAKq0fYAQDAg4WHtVbUFdGuLuOiBg0apEGDBrm6DCYoAwAAcyPsAAAAU3ObsPPSSy/JYrHoscces7adP39eqampatmypZo2baphw4apoKDAZrucnBylpKQoICBAoaGhmjJlii5cuODk6gEAgLtyi7CzY8cOvfHGG+ratatN+6RJk/Tpp59q2bJl2rhxo/Ly8mwmOJWXlyslJUWlpaXKysrSO++8o0WLFmn69OnOPgUAAOCmXB52Tp8+rVGjRunNN99U8+bNre0nT57UW2+9pTlz5mjAgAGKj4/XwoULlZWVpa+++kqStGbNGn333Xd67733FBcXp0GDBum5555TRkaGSktLXXVKAADAjbg87KSmpiolJUWJiYk27dnZ2SorK7Npj4mJUVRUlLZu3SpJ2rp1q7p06aKwsDDrmOTkZBUXF2vfvn12j1lSUqLi4mKbDwA4U98+N6hD+w52P7xHB2Zw+vRp7dq1S7t27ZIkHTp0SLt27VJOTo5T63Dpo+dLly7Vzp07tWPHjip9+fn58vHxUXBwsE17WFiY8vPzrWP+O+hU9lf22TNz5kw988wz9aweAOqO9+jAUfILGj4Y1/UYX3/9tfr372/9PnnyZEnSmDFjtGjRIkeUVisuCztHjhzRo48+qszMTPn5+Tn12NOmTbP+4JJUXFysyMhIp9YAuKMB/furqKio2jFH87jiALiDVq1ayd/fX++8/6ZTjufv769WrVpd0jb9+vWTYRgNVFHtuSzsZGdnq7CwUNdee621rby8XJs2bdKf//xnrV69WqWlpTpx4oTN1Z2CggKFh4dLksLDw7V9+3ab/VY+rVU55mJ8fX3l6+vrwLMBzKGoqEjb1q6udkybmFgnVQOgOlFRUTpw4ABrY9WCy8LOwIEDtWfPHpu2cePGKSYmRlOnTlVkZKQaN26sdevWadiwYZKkgwcPKicnRwkJCZKkhIQEvfDCCyosLFRoaKgkKTMzU4GBgYqN5f+QAQDmFhUV5bEBxJlcFnaaNWumzp0727Q1adJELVu2tLbff//9mjx5slq0aKHAwEBNnDhRCQkJuu666yRJSUlJio2N1ejRozVr1izl5+fr6aefVmpqKlduAACAJDdfG2vu3Lny8vLSsGHDVFJSouTkZL3++uvWfm9vb61YsUIPP/ywEhIS1KRJE40ZM0bPPvusC6sGAADuxK3CzoYNG2y++/n5KSMjQxkZGXa3iY6O1meffdbAlQEA4FruMNHXFRxx3i5/zw4AALCvcePGkqSzZ8+6uBLXqDzvyt+hLtzqyg4AwDlyc/PUoX2HaseEhYXryy2bnVQR7PH29lZwcLAKCwslSQEBAbJYLC6uquEZhqGzZ8+qsLBQwcHB8vb2rvO+CDsAcBkqLy+v9qWGkjQ340UnVYOaVL5OpTLwXE6Cg4OrfZ1MbRB2AABwcxaLRREREQoNDVVZWZmry3Gaxo0b1+uKTiXCDgAAHsLb29shf/lfbpigDAAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI03KAPwKLm5uerSuXO1Y0JCQrT+iy+cVBEAd0fYAeBRDKNC29aurnZM78RkJ1UDwBNwGwsAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaT2MBgIP17XODCgryqx2Tm5vnpGoAEHYAwMEKCvI1KfXJasdMmvqwk6oBwG0sAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgajx6DsB0cnNz1aVzZ7v9ISEhWv/FF06sCIArEXYAmI5hVGjb2tV2+3snJjuxGs+Vm5unDu072O0PCwvXl1s2O7EioG4IOwCAiyovL6/25YhzM150YjVA3TFnBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmFojVxcAAJ6mb58bVFCQb7c/NzfPidUAqAlhBwAuUUFBvialPmm3f9LUh51YDYCacBsLAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYmkvDzvz589W1a1cFBgYqMDBQCQkJ+vzzz63958+fV2pqqlq2bKmmTZtq2LBhKigosNlHTk6OUlJSFBAQoNDQUE2ZMkUXLlxw9qkAAAA35dKwc8UVV+ill15Sdna2vv76aw0YMEBDhgzRvn37JEmTJk3Sp59+qmXLlmnjxo3Ky8vTHXfcYd2+vLxcKSkpKi0tVVZWlt555x0tWrRI06dPd9UpAQAAN9PIlQe/9dZbbb6/8MILmj9/vr766itdccUVeuutt7RkyRINGDBAkrRw4UJ17NhRX331la677jqtWbNG3333ndauXauwsDDFxcXpueee09SpU5WWliYfHx9XnBYAAHAjbjNnp7y8XEuXLtWZM2eUkJCg7OxslZWVKTEx0TomJiZGUVFR2rp1qyRp69at6tKli8LCwqxjkpOTVVxcbL06dDElJSUqLi62+QAAAHNyedjZs2ePmjZtKl9fXz300EP66KOPFBsbq/z8fPn4+Cg4ONhmfFhYmPLz8yVJ+fn5NkGnsr+yz56ZM2cqKCjI+omMjHTsSQEAALfh8rBzzTXXaNeuXdq2bZsefvhhjRkzRt99912DHnPatGk6efKk9XPkyJEGPR4AAHAdl87ZkSQfHx+1b99ekhQfH68dO3boT3/6k+666y6VlpbqxIkTNld3CgoKFB4eLkkKDw/X9u3bbfZX+bRW5ZiL8fX1la+vr4PPBAAAuCOXX9n5tYqKCpWUlCg+Pl6NGzfWunXrrH0HDx5UTk6OEhISJEkJCQnas2ePCgsLrWMyMzMVGBio2NhYp9cOAADcj0uv7EybNk2DBg1SVFSUTp06pSVLlmjDhg1avXq1goKCdP/992vy5Mlq0aKFAgMDNXHiRCUkJOi6666TJCUlJSk2NlajR4/WrFmzlJ+fr6efflqpqalcuQEAAJJcHHYKCwv129/+VkePHlVQUJC6du2q1atX6+abb5YkzZ07V15eXho2bJhKSkqUnJys119/3bq9t7e3VqxYoYcfflgJCQlq0qSJxowZo2effdZVpwQAANyMS8POW2+9VW2/n5+fMjIylJGRYXdMdHS0PvvsM0eXBgAATMLlE5QBAJ4pNzdPHdp3sNsfFhauL7dsdmJFwMURdgAAdVJeXq5JqU/a7Z+b8aITqwHsc7unsQAAAByJsAMAAEyNsAMAAEyNOTsA8Ct9+9ygggL76+vl5uY5sRoA9UXYAYBfKSjIr3bi7aSpDzuxGgD1xW0sAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgajx6DlxGBvTvr6KiIrv9R/N4fwwA8yHsAJeRoqIibVu72m5/m5hYJ1YDAM7BbSwAAGBqhB0AAGBqhB0AAGBqhB0AAGBqdQo7V155pX766acq7SdOnNCVV15Z76IAAAAcpU5h5/DhwyovL6/SXlJSotzc3HoXBQAA4CiX9Oj58uXLrf979erVCgoKsn4vLy/XunXr1LZtW4cVBwAAUF+XFHaGDh0qSbJYLBozZoxNX+PGjdW2bVu9+uqrDisOAACgvi4p7FRUVEiS2rVrpx07dqhVq1YNUhSAuuENyQBQVZ3eoHzo0CFH1wHAAXhDMgBUVeflItatW6d169apsLDQesWn0ttvv13vwgAAAByhTmHnmWee0bPPPqsePXooIiJCFovF0XUBAAA4RJ3CzoIFC7Ro0SKNHj3a0fUAAAA4VJ3es1NaWqrrr7/e0bUAAAA4XJ3CzgMPPKAlS5Y4uhYAAACHq9NtrPPnz+svf/mL1q5dq65du6px48Y2/XPmzHFIcQAAAPVVp7Dz7bffKi4uTpK0d+9emz4mKwMAAHdSp7DzxRdfOLoOAACABlGnOTsAAACeok5Xdvr371/t7ar169fXuSAAAABHqlPYqZyvU6msrEy7du3S3r17qywQCgAA4Ep1Cjtz5869aHtaWppOnz5dr4IAAAAcyaFzdu69917WxQIAAG7FoWFn69at8vPzc+QuAQAA6qVOt7HuuOMOm++GYejo0aP6+uuv9cc//tEhhQEAADhCncJOUFCQzXcvLy9dc801evbZZ5WUlOSQwgAAAByhTmFn4cKFjq4DAACgQdQp7FTKzs7W/v37JUmdOnVS9+7dHVIUAACAo9Qp7BQWFmrkyJHasGGDgoODJUknTpxQ//79tXTpUoWEhDiyRgAAgDqr09NYEydO1KlTp7Rv3z4dP35cx48f1969e1VcXKzf//73jq4RAACgzup0ZWfVqlVau3atOnbsaG2LjY1VRkYGE5QBAIBbqdOVnYqKCjVu3LhKe+PGjVVRUVHvogAAABylTmFnwIABevTRR5WXl2dty83N1aRJkzRw4ECHFQcAAFBfdQo7f/7zn1VcXKy2bdvqqquu0lVXXaV27dqpuLhY6enpjq4RAACgzuo0ZycyMlI7d+7U2rVrdeDAAUlSx44dlZiY6NDiAAAA6uuSruysX79esbGxKi4ulsVi0c0336yJEydq4sSJ6tmzpzp16qTNmzc3VK0AAACX7JLCzrx58zR+/HgFBgZW6QsKCtLvfvc7zZkzx2HFAQAA1NclhZ3du3frlltusduflJSk7OzsehcFAADgKJcUdgoKCi76yHmlRo0aqaioqN5FAQAAOMolTVBu06aN9u7dq/bt21+0/9tvv1VERIRDCgMAeLbc3Dx1aN/Bbn9YWLi+3MI8TzS8Swo7gwcP1h//+Efdcsst8vPzs+k7d+6cZsyYod/85jcOLRAA4JnKy8s1KfVJu/1zM150YjW4nF1S2Hn66af14Ycf6uqrr9aECRN0zTXXSJIOHDigjIwMlZeX66mnnmqQQgHAUXJzc9Wlc2e7/UVFhU6sBkBDu6SwExYWpqysLD388MOaNm2aDMOQJFksFiUnJysjI0NhYWENUigAOIphVGjb2tV2+1tf3dFuHwDPc8kvFYyOjtZnn32mn3/+Wd9//70Mw1CHDh3UvHnzhqgPAACgXur0BmVJat68uXr27OnIWgAAAByuTmtjAQAAeArCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDWXhp2ZM2eqZ8+eatasmUJDQzV06FAdPHjQZsz58+eVmpqqli1bqmnTpho2bJgKCgpsxuTk5CglJUUBAQEKDQ3VlClTdOHCBWeeCgAAcFMuDTsbN25UamqqvvrqK2VmZqqsrExJSUk6c+aMdcykSZP06aefatmyZdq4caPy8vJ0xx13WPvLy8uVkpKi0tJSZWVl6Z133tGiRYs0ffp0V5wSAABwM3V+z44jrFq1yub7okWLFBoaquzsbN144406efKk3nrrLS1ZskQDBgyQJC1cuFAdO3bUV199peuuu05r1qzRd999p7Vr1yosLExxcXF67rnnNHXqVKWlpcnHx8cVpwYAANyEW83ZOXnypCSpRYsWkqTs7GyVlZUpMTHROiYmJkZRUVHaunWrJGnr1q3q0qWLzTIVycnJKi4u1r59+y56nJKSEhUXF9t8AACAOblN2KmoqNBjjz2mPn36qPN/FujLz8+Xj4+PgoODbcaGhYUpPz/fOubX63FVfq8c82szZ85UUFCQ9RMZGengswEAAO7CbcJOamqq9u7dq6VLlzb4saZNm6aTJ09aP0eOHGnwYwIAANdw6ZydShMmTNCKFSu0adMmXXHFFdb28PBwlZaW6sSJEzZXdwoKChQeHm4ds337dpv9VT6tVTnm13x9feXr6+vgswAAAO7IpVd2DMPQhAkT9NFHH2n9+vVq166dTX98fLwaN26sdevWWdsOHjyonJwcJSQkSJISEhK0Z88eFRYWWsdkZmYqMDBQsbGxzjkRAADgtlx6ZSc1NVVLlizRJ598ombNmlnn2AQFBcnf319BQUG6//77NXnyZLVo0UKBgYGaOHGiEhISdN1110mSkpKSFBsbq9GjR2vWrFnKz8/X008/rdTUVK7eAAAA14ad+fPnS5L69etn075w4UKNHTtWkjR37lx5eXlp2LBhKikpUXJysl5//XXrWG9vb61YsUIPP/ywEhIS1KRJE40ZM0bPPvuss04DAAC4MZeGHcMwahzj5+enjIwMZWRk2B0THR2tzz77zJGlAQAAk3Cbp7EAAAAaAmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYWiNXFwAAuDzl5uapQ/sOdvvDwsL15ZbNTqwIZkXYAQC4RHl5uSalPmm3f27Gi06sBmbGbSwAAGBqXNkBcNm5cKFc6enp9vvLy51YDYCGRtgBcBkylNDzhmr67QchAJ6H21gAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUWBsL8CAD+vdXUVGR3f6jeXlOrAYAPANhB/AgRUVF2rZ2td3+NjGxTqwGADwDt7EAAICpEXYAAICpEXYAAICpEXYAAICpMUEZcCM8bQUAjkfYAdwIT1u5B6OiQi/PSbPb7+VlcV4xAOqNsAMAv+Ll5aXF6a/b7R88aoQTqwFQX8zZAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAAptbI1QUAgKNduFCu9PR0+wMM59UCwPUIOwBMyFBCzxuq6a8mCNVm7xUVenlOmt1+Ly9LvfYPwLEIOwBwiby8vLQ4/XW7/YNHjXBiNQBqwpwdAABgaoQdAABgaoQdAABgaoQdAABgai4NO5s2bdKtt96q1q1by2Kx6OOPP7bpNwxD06dPV0REhPz9/ZWYmKh//etfNmOOHz+uUaNGKTAwUMHBwbr//vt1+vRpJ54FAABwZy4NO2fOnFG3bt2UkZFx0f5Zs2bptdde04IFC7Rt2zY1adJEycnJOn/+vHXMqFGjtG/fPmVmZmrFihXatGmTHnzwQWedAgAAcHMuffR80KBBGjRo0EX7DMPQvHnz9PTTT2vIkCGSpHfffVdhYWH6+OOPNXLkSO3fv1+rVq3Sjh071KNHD0lSenq6Bg8erFdeeUWtW7d22rkANRnQv7+KioqqHXM0L89J1QDA5cNt37Nz6NAh5efnKzEx0doWFBSk3r17a+vWrRo5cqS2bt2q4OBga9CRpMTERHl5eWnbtm26/fbbXVE6cFFFRUXatnZ1tWPaxMQ6qRrA/eXm5qlD+w7VjgkLC9eXWzY7qSJ4KrcNO/n5+ZKksLAwm/awsDBrX35+vkJDQ236GzVqpBYtWljHXExJSYlKSkqs34uLix1VNgDAQcrLyzUp9clqx8zNeNFJ1cCTuW3YaUgzZ87UM8884+oyANRBjeteSax9BcCG24ad8PBwSVJBQYEiIiKs7QUFBYqLi7OOKSwstNnuwoULOn78uHX7i5k2bZomT55s/V5cXKzIyEgHVg+g4dS07pVU37WvAJiL275np127dgoPD9e6deusbcXFxdq2bZsSEhIkSQkJCTpx4oSys7OtY9avX6+Kigr17t3b7r59fX0VGBho8wEAAObk0is7p0+f1vfff2/9fujQIe3atUstWrRQVFSUHnvsMT3//PPq0KGD2rVrpz/+8Y9q3bq1hg4dKknq2LGjbrnlFo0fP14LFixQWVmZJkyYoJEjR/IkFgBcBmqaxMwEZkguDjtff/21+vfvb/1eeWtpzJgxWrRokR5//HGdOXNGDz74oE6cOKG+fftq1apV8vPzs27z/vvva8KECRo4cKC8vLw0bNgwvfbaa04/FwCA89U0iZkJzJBcHHb69esnw7A/k9BisejZZ5/Vs88+a3dMixYttGTJkoYoDwAAmIDbztkBAABwBMIOAAAwNcIOAAAwNbd9zw7gaWpa+4p1rwDANQg7gIPUtPYV614BgGtwGwsAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaLxUEAAczKir08py0asd4eVmcUwwAwg4AOJqXl5cWp79e7ZjBo0Y4qRoA3MYCAACmRtgBAACmxm0soJZY1RwAPBNhB5eNmsJKSEiI1n/xhd1+VjUHAM9E2MFlo6aw0jsx2YnVAACchTk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AADA1HgaC/iP3Nxcdenc2W4/79EBAM9E2AH+wzAqeI8OnKamxUJZKBRwHMIOALdy4UK50tPT7Q8wnFdLQ6ppsVAWCgUch7ADwM0YSuh5QzX91QQhALgIwg4AwLRyc/PUoX0Hu/1hYeH6cstmJ1YEVyDsAABMq7y8XJNSn7TbPzfjRSdWA1fh0XMAAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqvGcHgFNdLstBAHAfhB0ATsZyEACci9tYAADA1Ag7AADA1Ag7AADA1JizAwC4bLEq+uWBsAMAuGyxKvrlgbADUxjQv7+KioqqHXM0L89J1QAA3AlhBx6hpjBzNC9POd/tqXYfbWJiHV0W4DJeXha9PCet2n4AvyDswCMUFRVp29rVdvsJMrjceHt7aXH663b7B48a4cRqAPdG2AEAN2RUVFR75aa+23PlB5cTwg4AuCEvr+qv3KTUcOWmpu258oPLCe/ZAQAApsaVHQAOxUKfANwNYQfAJak5zLDQJwD3QtiBU9T06HhISIjWf/GFEytC3RFmAHgWwg6coqZHx6/o2FldOne2288LAQEAdUXYgVswjAreowMAaBCEHQAA7GChUHMg7AAAYAcLhZoD79kBAACmxpUdALgM1WY5CpaUgFkQdgDgMlTTchISS0rAPAg7AKxqfGGgxBuQgf9S0wRmSSosKlJoSIjdfiY5NzzCDoD/UtMLAyVeGnj5YOX0mtU0gVmSJk19mEnOLkbYgUPU9IZkXgoIeB5WTodZEHbgEDW9IZmXAgLmU9OVH29vC1eG4BZME3YyMjI0e/Zs5efnq1u3bkpPT1evXr1cXZZTsO4UAFeo6cpPyqgRXBmCWzBF2Pnggw80efJkLViwQL1799a8efOUnJysgwcPKjQ01NXlNbiarqr0Tkyu1/5rClMSt6kAAO7LFGFnzpw5Gj9+vMaNGydJWrBggVauXKm3335bTzzxhIurq7+Gng9Tm/3nfLen2n1wm8o91PQ01YWyC9U/bcWTVnAiJkD/giUpGp7Hh53S0lJlZ2dr2rRp1jYvLy8lJiZq69atLqzMcRp6PgzzbWqnxiBxodyJ1dhT09NU6TX2A87iiAnQXl6ePy+IJSkanseHnWPHjqm8vFxhYWE27WFhYTpw4MBFtykpKVFJSYn1+8mTJyVJxcXFDq/vNykpOnbsmN3+Vq1aacXKldXuo7y8XMWnTtntNwyj2v4ff/xRsR072u0/kpOj2a+8Yre/rOxCtfuvHFOffdR0DvXtr1WNpWXV9hsVFeraKb7afnf4Hc+cPdtg/c44BjW6R7871FBRXq6Zr0yvtkYvL4sWvGz/35lh942udh8Wi3Tu/Llq+6vb3sur+v7aHMMwjGr7KyoqGuTvp0uRdHOyiooK7fYXHTumkFat7PaHhIRqTab9/6iuq8rfxTBquCxteLjc3FxDkpGVlWXTPmXKFKNXr14X3WbGjBmGfrlgz4cPHz58+PDx8M+RI0eqzQoef2WnVatW8vb2VkFBgU17QUGBwsPDL7rNtGnTNHnyZOv3iooKHT9+XC1btpTFcmmXPIuLixUZGakjR44oMDDw0k8A/IYOwu9Yf/yGjsHvWH/8hrVjGIZOnTql1q1bVzvO48OOj4+P4uPjtW7dOg0dOlTSL+Fl3bp1mjBhwkW38fX1la+vr01bcHBwveoIDAzkD2Q98Rs6Br9j/fEbOga/Y/3xG9YsKCioxjEeH3YkafLkyRozZox69OihXr16ad68eTpz5oz16SwAAHD5MkXYueuuu1RUVKTp06crPz9fcXFxWrVqVZVJywAA4PJjirAjSRMmTLB726oh+fr6asaMGVVui6H2+A0dg9+x/vgNHYPfsf74DR3LYhg1Pa8FAADgubxcXQAAAEBDIuwAAABTI+wAAABTI+wAAABTI+w42MqVK9W7d2/5+/urefPm1hcd4tKVlJQoLi5OFotFu3btcnU5HuPw4cO6//771a5dO/n7++uqq67SjBkzVFpa6urS3F5GRobatm0rPz8/9e7dW9u3b3d1SR5l5syZ6tmzp5o1a6bQ0FANHTpUBw8edHVZHu2ll16SxWLRY4895upSPBphx4H+8Y9/aPTo0Ro3bpx2796tLVu26J577nF1WR7r8ccfr/EV4KjqwIEDqqio0BtvvKF9+/Zp7ty5WrBggZ580v6qypA++OADTZ48WTNmzNDOnTvVrVs3JScnq7DQ/uKHsLVx40alpqbqq6++UmZmpsrKypSUlKQzZ864ujSPtGPHDr3xxhvq2rWrq0vxfI5ZjhNlZWVGmzZtjL/+9a+uLsUUPvvsMyMmJsbYt2+fIcn45ptvXF2SR5s1a5bRrl07V5fh1nr16mWkpqZav5eXlxutW7c2Zs6c6cKqPFthYaEhydi4caOrS/E4p06dMjp06GBkZmYaN910k/Hoo4+6uiSPxpUdB9m5c6dyc3Pl5eWl7t27KyIiQoMGDdLevXtdXZrHKSgo0Pjx47V48WIFBAS4uhxTOHnypFq0aOHqMtxWaWmpsrOzlZiYaG3z8vJSYmKitm7d6sLKPNvJkycliT97dZCamqqUlBSbP5OoO8KOg/zwww+SpLS0ND399NNasWKFmjdvrn79+un48eMurs5zGIahsWPH6qGHHlKPHj1cXY4pfP/990pPT9fvfvc7V5fito4dO6by8vIqS8yEhYUpPz/fRVV5toqKCj322GPq06ePOnfu7OpyPMrSpUu1c+dOzZw509WlmAZhpwZPPPGELBZLtZ/KORKS9NRTT2nYsGGKj4/XwoULZbFYtGzZMhefhevV9ndMT0/XqVOnNG3aNFeX7HZq+xv+t9zcXN1yyy0aPny4xo8f76LKcTlKTU3V3r17tXTpUleX4lGOHDmiRx99VO+//778/PxcXY5psFxEDYqKivTTTz9VO+bKK6/Uli1bNGDAAG3evFl9+/a19vXu3VuJiYl64YUXGrpUt1bb33HEiBH69NNPZbFYrO3l5eXy9vbWqFGj9M477zR0qW6rtr+hj4+PJCkvL0/9+vXTddddp0WLFsnLi/+2sae0tFQBAQH6+9//bvME5ZgxY3TixAl98sknrivOA02YMEGffPKJNm3apHbt2rm6HI/y8ccf6/bbb5e3t7e1rby8XBaLRV5eXiopKbHpQ+2YZiHQhhISEqKQkJAax8XHx8vX11cHDx60hp2ysjIdPnxY0dHRDV2m26vt7/jaa6/p+eeft37Py8tTcnKyPvjgA/Xu3bshS3R7tf0NpV+u6PTv3996hZGgUz0fHx/Fx8dr3bp11rBTUVGhdevWuWSBYU9lGIYmTpyojz76SBs2bCDo1MHAgQO1Z88em7Zx48YpJiZGU6dOJejUEWHHQQIDA/XQQw9pxowZioyMVHR0tGbPni1JGj58uIur8xxRUVE235s2bSpJuuqqq3TFFVe4oiSPk5ubq379+ik6OlqvvPKKioqKrH3h4eEurMy9TZ48WWPGjFGPHj3Uq1cvzZs3T2fOnNG4ceNcXZrHSE1N1ZIlS/TJJ5+oWbNm1vlOQUFB8vf3d3F1nqFZs2ZV5jg1adJELVu2ZO5TPRB2HGj27Nlq1KiRRo8erXPnzql3795av369mjdv7urScBnJzMzU999/r++//75KQOSutX133XWXioqKNH36dOXn5ysuLk6rVq2qMmkZ9s2fP1+S1K9fP5v2hQsXauzYsc4vCPgP5uwAAABT40Y+AAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAOiXlc8BmBNhB4Bb+/vf/64uXbrI399fLVu2VGJios6cOSNJevvtt9WpUyf5+voqIiLCZoXynJwcDRkyRE2bNlVgYKBGjBihgoICa39aWpri4uL017/+Ve3atZOfn58k6cSJE3rggQcUEhKiwMBADRgwQLt373buSQNwKMIOALd19OhR3X333brvvvu0f/9+bdiwQXfccYcMw9D8+fOVmpqqBx98UHv27NHy5cvVvn17SVJFRYWGDBmi48ePa+PGjcrMzNQPP/ygu+66y2b/33//vf7xj3/oww8/1K5duyRJw4cPV2FhoT7//HNlZ2fr2muv1cCBA3X8+HFnnz4AB2EhUABua+fOnYqPj9fhw4cVHR1t09emTRuNGzdOzz//fJXtMjMzNWjQIB06dEiRkZGSpO+++06dOnXS9u3b1bNnT6WlpenFF19Ubm6uQkJCJElffvmlUlJSVFhYKF9fX+v+2rdvr8cff1wPPvhgA54tgIbSyNUFAIA93bp108CBA9WlSxclJycrKSlJd955p8rKypSXl6eBAwdedLv9+/crMjLSGnQkKTY2VsHBwdq/f7969uwpSYqOjrYGHUnavXu3Tp8+rZYtW9rs79y5c/r3v//dAGcIwBkIOwDclre3tzIzM5WVlaU1a9YoPT1dTz31lNatW+eQ/Tdp0sTm++nTpxUREaENGzZUGRscHOyQYwJwPsIOALdmsVjUp08f9enTR9OnT1d0dLQyMzPVtm1brVu3Tv3796+yTceOHXXkyBEdOXLE5jbWiRMnFBsba/dY1157rfLz89WoUSO1bdu2oU4JgJMRdgC4rW3btmndunVKSkpSaGiotm3bpqKiInXs2FFpaWl66KGHFBoaqkGDBunUqVPasmWLJk6cqMTERHXp0kWjRo3SvHnzdOHCBT3yyCO66aab1KNHD7vHS0xMVEJCgoYOHapZs2bp6quvVl5enlauXKnbb7+92m0BuC/CDgC3FRgYqE2bNmnevHkqLi5WdHS0Xn31VQ0aNEiSdP78ec2dO1d/+MMf1KpVK915552Sfrka9Mknn2jixIm68cYb5eXlpVtuuUXp6enVHs9iseizzz7TU089pXHjxqmoqEjh4eG68cYbFRYW1uDnC6Bh8DQWAAAwNd6zAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATO3/AX3mCfMKVdoTAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import warnings\n", "\n", "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import seaborn as sns\n", "\n", "warnings.filterwarnings(\"ignore\", \"is_categorical_dtype\")\n", "warnings.filterwarnings(\"ignore\", \"use_inf_as_na\")\n", "\n", "score = grid_search.decision_function(X)\n", "df = pd.DataFrame({\"score\": score, \"y\": y})\n", "sns.histplot(data=df, x=\"score\", hue=\"y\").set_title(\"Smooth SVM (C=0.1)\")\n", "plt.show()" ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 0 }