![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
],
"metadata": {
"id": "RGsdu3aXY5O_"
}
},
{
"cell_type": "markdown",
"source": [
"# Regresion multivariada ecuacion normal, repaso breve.\n",
"\n",
"Supongamos que tenemos un conjunto de caracteristicas $X = X_1,X_2...X_j...X_n$ para realizar una predicción $y$ con valores esperados $\\hat{y}$. \n",
"\n",
"Cada X, puede ser escrito como:\n",
" $X_1 = x_1^{(1)},x_1^{(2)}, x_1^{(3)}...x_1^{(m)}$, \n",
"\n",
" $X_2 = x_2^{(1)},x_2^{(2)}, x_2^{(3)}...x_2^{(m)}$, \n",
" \n",
" .\n",
" \n",
" .\n",
" \n",
" .\n",
" \n",
" $X_n = x_n^{(1)},x_n^{(2)}, x_n^{(3)}...x_n^{(m)}$. \n",
" \n",
"\n",
"Siendo n el número de caracteristicas y m el número de datos de datos, \n",
"$\\hat{y} = \\hat{y}_1^{(1)}, \\hat{y}_1^{(2)}...\\hat{y}_1^{(m)} $, el conjunto de datos etiquetados y $y = y_1^{(1)}, y_1^{(2)}...y_1^{(m)} $ los valores predichos por un modelo\n",
"\n",
"\n",
"\n",
"\n",
"Lo anterior puede ser resumido como:\n",
"\n",
"\n",
"\n",
"|Training|$\\hat{y}$ | X_1 | X_2 | . | .|. |. | X_n|\n",
"|--------|-------|------|------|-----|--|--|--|----|\n",
"|1|$\\hat{y}_1^{1}$ | $x_1^{1}$|$x_2^{1}$| . | .|. |. | $x_n^{1}$|\n",
"|2|$\\hat{y}_1^{2}$ | $x_1^{2}$|$x_2^{2}$| . | .|. |. | $x_n^{2}$|\n",
"|.|. | . |.| . | .|. |. | |\n",
"|.|. | . |.| . | .|. |. | |\n",
"|.|. | . |.| . | .|. |. | |\n",
"|m|$\\hat{y}_1^{m}$ | $x_1^{m}$ |$x_2^{m}$| . | .|. |. | $x_n^{m}$|\n",
"\n",
"\n",
"y el el modelo puede ser ajustado como sigue: \n",
"\n",
"Para un solo conjunto de datos de entrenamiento tentemos que:\n",
"\n",
"$y = h(\\theta_0,\\theta_1,\\theta_2,...,\\theta_n ) = \\theta_0 + \\theta_1 x_1+\\theta_2 x_2 + \\theta_3 x_3 +...+ \\theta_n x_n $.\n",
"\n",
"\\begin{equation}\n",
"h_{\\Theta}(x) = [\\theta_0,\\theta_1,...,\\theta_n ]\\begin{bmatrix}\n",
"1^{(1)}\\\\\n",
"x_1^{(1)}\\\\\n",
"x_2^{(1)}\\\\\n",
".\\\\\n",
".\\\\\n",
".\\\\\n",
"x_n^{(1)}\\\\\n",
"\\end{bmatrix} = \\Theta^T X^{(1)}\n",
"\\end{equation}\n",
"\n",
"\n",
"\n",
"Para todo el conjunto de datos, tenemos que:\n",
"\n",
"Sea $\\Theta^T = [\\theta_0,\\theta_1,\\theta_2,...,\\theta_n]$ una matrix $1 \\times (n+1)$ y \n",
"\n",
"\n",
"\\begin{equation}\n",
"X =\n",
"\\begin{bmatrix}\n",
"1& 1 & 1 & .&.&.&1\\\\\n",
"x_1^{(1)}&x_1^{(2)} & x_1^{(3)} & .&.&.&x_1^{(m)}\\\\\n",
".&. & . &.&.&.& .\\\\\n",
".&. & . & .&.&.&.\\\\\n",
".&. & . & .&.&.&.\\\\\n",
"x_n^{(1)}&x_n^{(2)} & x^{(3)} & .&.&.&x_n^{(m)}\\\\\n",
"\\end{bmatrix}_{(n+1) \\times m}\n",
"\\end{equation}\n",
"\n",
"\n",
"\n",
"\n",
"luego $h = \\Theta^{T} X $ con dimension $1\\times m$\n",
"\n",
"\n",
"\n",
"\n",
"La anterior ecuación, es un hiperplano en $\\mathbb{R}^n$. Notese que en caso de tener una sola característica, la ecuación puede ser análizada según lo visto en la sesión de regresion lineal.\n",
"\n",
"\n",
"Para la optimización, vamos a definir la función de coste **$J(\\theta_1,\\theta_2,\\theta_3, ...,\\theta_n )$** , como la función asociada a la minima distancia entre dos puntos, según la metrica euclidiana. \n",
"\n",
"- Metrica Eculidiana\n",
"\n",
"\\begin{equation}\n",
"J(\\theta_1,\\theta_2,\\theta_3, ...,\\theta_n )=\\frac{1}{2m} \\sum_{i=1}^m ( h_{\\Theta} (X)-\\hat{y}^{(i)})^2 =\\frac{1}{2m} \\sum_{i = 1}^m (\\Theta^{T} X - \\hat{y}^{(i)})^2\n",
"\\end{equation}\n",
"\n",
"Otras métricas pueden ser definidas como sigue en la siguiente referencia. [Metricas](https://jmlb.github.io/flashcards/2018/04/21/list_cost_functions_fo_neuralnets/).\n",
"\n",
"Nuestro objetivo será encontrar los valores mínimos \n",
"$\\Theta = \\theta_0,\\theta_1,\\theta_2,...,\\theta_n$ que minimizan el error, respecto a los valores etiquetados y esperados $\\hat{y}$ \n",
"\n",
"\n",
"Para encontrar $\\Theta$ optimo, se necesita minimizar la función de coste. Ecnontremos los valores exactos.\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n"
],
"metadata": {
"id": "GnE9vnCkY2FK"
}
},
{
"cell_type": "markdown",
"source": [
"# Normal equation\n",
"Se puede encontrar una solucion exacta para theta sin necesidad de emplear el gradiente descente de la sesiones pasadas, para ellos se puede encontrar el valor minimo de theta y a partir de alli determinar el valor de theta que minimiza J. \n",
"\n",
"Los pasos para esta minimizacion se dejan como tarea, y pueden ser calculados según lo siguiente:\n",
"\n",
"Si J es la funcion de coste dada por:\n",
"\n",
"\\begin{equation}\n",
"J(\\theta_1,\\theta_2,\\theta_3, ...,\\theta_n )=\\frac{1}{2m} \\sum_{i = 1}^m (\\Theta^{T} X - \\hat{y}^{(i)})^2\n",
"\\end{equation}\n",
"\n",
"\n",
"Demostrar que:\n",
"\n",
"- $J(\\theta_1,\\theta_2,\\theta_3, ...,\\theta_n ) = \\frac{1}{2m} (\\Theta ^ T X - y)^T (\\Theta ^ T X - y)$\n",
"\n",
"- $ \\nabla _{\\theta} J = \\frac{1}{m}( (X^T X) \\Theta - X^T y)$\n",
"\n",
"\n",
"Para encontrar el valor minimo de \\theta, $\\nabla _{\\theta} J = 0$, \n",
"\n",
"- $\\Theta = (X^T X)^{-1} X^T y$\n",
"\n",
"\n",
"\n",
"\n",
"Para la demostracion anterior emplee las siguientes propiedades:\n",
"\n",
"- $z^T z= \\sum_i z_i^2$\n",
"- $a^T b = b^Ta$\n",
"- $\\nabla _x b^T x = b$\n",
"- $\\nabla _x x^T A x = 2Ax$\n",
"\n",
"donde a, b, x son matrices, $\\nabla_x$ es la derivada respecto al vector x, y A es una matriz simétrica"
],
"metadata": {
"id": "mT4-yW7jZTmz"
}
},
{
"cell_type": "code",
"source": [
"from sklearn.datasets import load_boston\n",
"import pandas as pd\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib import cm\n",
"#from matplotlib.ticker import LinearLocator\n",
"import numpy as np"
],
"metadata": {
"id": "9W1oojJTv89P"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Regresion lineal simple\n",
"N = 10\n",
"x1 = np.linspace(-1, 1, N)\n",
"y = 2*x1 #- 3*x2 + 0.0\n",
"df = pd.DataFrame({\"Y\":y, \"X1\":x1})\n",
"df[\"ones\"] = np.ones(N)\n"
],
"metadata": {
"id": "1oyJaDpCv5aq"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"plt.plot(df.X1,df.Y,\"ro\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 282
},
"id": "RVUm-SbqFb9N",
"outputId": "9649c174-8d50-4b31-f2ec-e51865e21e32"
},
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[| \n", " | mean_ | \n", "rm | \n", "ones | \n", "
|---|---|---|---|
| 0 | \n", "24.0 | \n", "6.575 | \n", "1.0 | \n", "
| 1 | \n", "21.6 | \n", "6.421 | \n", "1.0 | \n", "
| 2 | \n", "34.7 | \n", "7.185 | \n", "1.0 | \n", "
| 3 | \n", "33.4 | \n", "6.998 | \n", "1.0 | \n", "
| 4 | \n", "36.2 | \n", "7.147 | \n", "1.0 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "
| 501 | \n", "22.4 | \n", "6.593 | \n", "1.0 | \n", "
| 502 | \n", "20.6 | \n", "6.120 | \n", "1.0 | \n", "
| 503 | \n", "23.9 | \n", "6.976 | \n", "1.0 | \n", "
| 504 | \n", "22.0 | \n", "6.794 | \n", "1.0 | \n", "
| 505 | \n", "11.9 | \n", "6.030 | \n", "1.0 | \n", "
506 rows × 3 columns
\n", "