logistic regression
This commit is contained in:
Ritchie committed
1 parent
f351e89421
commit
5b68b7ec43
3 files changed
+428
No files matched your search
Binary file not shown.
|
After Width: | Height: | Size: 107 KiB |
@@ -0,0 +1,382 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Logistic Regression\n",
|
||||
"\n",
|
||||
"* labels: y\n",
|
||||
"* inputs: x\n",
|
||||
"\n",
|
||||
"**model**\n",
|
||||
"\n",
|
||||
"$$ w^T x = w_0 + w_1x_1 + w_2x_2 + w_3x_3 $$\n",
|
||||
"\n",
|
||||
"$$ P(y|x) = σ(w^Tx) $$\n",
|
||||
"\n",
|
||||
"## Given data:\n",
|
||||
"\n",
|
||||
"D = ((x1, y1), ..., (xn, yn))\n",
|
||||
"x ∈ R\n",
|
||||
"y ∈ {0, 1}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## Pro's:\n",
|
||||
"\n",
|
||||
"- interpretable\n",
|
||||
"- small # params (d + 1)\n",
|
||||
"- computationally efficient\n",
|
||||
"- extensible to multi-class\n",
|
||||
"\n",
|
||||
"## Cons\n",
|
||||
"\n",
|
||||
"- performance as good as other models\n",
|
||||
"\n",
|
||||
"## Maximum Likelihood Estimation MLE\n",
|
||||
"\n",
|
||||
"likelihood \n",
|
||||
"\n",
|
||||
"$$l(w)= P(y_i | x_i, w) \\cdot P(y_j | x_j, w) ... P(y_n | x_n,w)$$\n",
|
||||
"\n",
|
||||
"in product notation\n",
|
||||
"\n",
|
||||
"$$l(w) = \\prod_{i=1}^{N}P(y_i |x_i, w) $$\n",
|
||||
"\n",
|
||||
"So we need to find w that makes the function $ l(w) $ as large as possible. This is done with gradient ascent. (If we define a loss function, we use gradient descent)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"*$ll$ is the log likelihood, because this is a nice convex function*\n",
|
||||
"\n",
|
||||
"$\\triangledown f$ = vector of partial derivatives of f\n",
|
||||
"\n",
|
||||
"$$ \\triangledown ll(w) = \n",
|
||||
"\\left(\n",
|
||||
"\\begin{array}{}\n",
|
||||
"\\frac{\\partial ll}{\\partial w_0} \\\\\n",
|
||||
"\\frac{\\partial ll}{\\partial w_1} \\\\\n",
|
||||
"\\vdots \\\\\n",
|
||||
"\\frac{\\partial ll}{\\partial w_n}\n",
|
||||
"\\end{array}\n",
|
||||
"\\right) $$\n",
|
||||
"\n",
|
||||
"## Derivative of log likelihood\n",
|
||||
"\n",
|
||||
"$$\\frac{\\partial ll(w)}{\\partial w_j} = \\sum_{i=1}^{N} h_i(x) (y_i - P(y_i|x_i, w)$$\n",
|
||||
"\n",
|
||||
"or\n",
|
||||
"\n",
|
||||
"$$\\frac{\\partial ll(w)}{\\partial w_j} = \\sum_{i=1}^{N} h(x) (y_i - \\hat{y_i})$$\n",
|
||||
"\n",
|
||||
"$h_j(x_i)$ is the individual input value taken from the $x$ vector (containing all input values)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Deriving log-likelihood\n",
|
||||
"\n",
|
||||
"log of the likelihood function\n",
|
||||
"\n",
|
||||
"$$\\ln l(w) = ln \\prod_{i = 1}^{N} P(y_i|x_i, w)$$\n",
|
||||
"\n",
|
||||
"By taking the log, the products becom the sum.\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\ln a \\cdot b = \\ln a + \\ln b \\\\\n",
|
||||
"\\ln \\frac{a}{b} = \\ln a - \\ln b\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"The $\\hat{w}$ that maximizes $l(w)$ is equal to the $\\hat{w}$ that maximizes $\\ln l(w)$.\n",
|
||||
"\n",
|
||||
"In other words the peaks of the parabola are on the same places."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 55,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8FPX9x/HXZ3MTEs4Qwk3kDCJBIqKAJ4rYereKtYqK\nRVu14k9ttdbqr5dXay+tFSpq6614UIuitVrFChqQKwHkEjlyAUISjpDj+/sjA82PJiRkdzObzfv5\neOwjs9+ZyXyYLO+d/e7MfM05h4iIRK+A3wWIiEh4KehFRKKcgl5EJMop6EVEopyCXkQkyinoRUSi\nnIJeRCTKKehFRKKcgl5EJMrF+l0AQNeuXV2/fv38LkNEpFVZtGjRNudcWmPLRUTQ9+vXj9zcXL/L\nEBFpVcxsY1OWU9eNiEiUU9CLiEQ5Bb2ISJRT0IuIRDkFvYhIlGs06M2st5m9Z2b5ZpZnZjd57Z3N\n7B0zW+P97FRnnTvMbK2ZrTazieH8B4iIyOE15Yi+CrjFOZcFjAGuN7Ms4HbgXefcQOBd7znevMnA\nMOAs4I9mFhOO4kVEpHGNnkfvnCsACrzpMjNbCfQEzgNO8RZ7Cngf+KHX/rxzrgLYYGZrgdHAx6Eu\nXhpXU+Mo319F6d5Kdu2tpHRvFaX7KindW0npvirK91WRFB8gNTGO1KQ472fswecpibHExaiHT6Q1\nO6ILpsysHzASWAike28CAIVAujfdE1hQZ7XNXtuhv2saMA2gT58+R1JGm+eco2DXPvK3lrK6qIwd\nu/d7IV7phfh/wrysoopghwVuFx9Dh3reBFITY+mQFEdmWnuG9Uilf9dkYvWmIBJxmhz0ZtYemA1M\nd86VmtnBec45Z2ZHFCfOuRnADICcnByNUN6Aquoa1pXsJr9gF/lbS8nbWkp+QSk791QeXKZ9Qiyp\nibEHj8h7dExkSGJK7XMvkA89Wj8Q3MkJMVRU1VC6r84R/8E3jNqj/kPfQIrK9rGmuPzgMjXeXy8h\nNsCQ7ilk9ehAVo9UsjJSGZqRQrv4iLgAW6TNatL/QDOLozbkn3HOveI1F5lZhnOuwMwygGKvfQvQ\nu87qvbw2acTuiipWFZb+v0BfVVjG/qoaAOK9IJ10dHeyMlLJ6tGBId1TSE4ILkhjYwIkJ8SS0SHp\niNetrK5hvfdGlLeltua5ywt47pMvATCD/l2TycpIZVidN4C0lISgahaRpjPXyOd6qz10fwrY4Zyb\nXqf9QWC7c+4+M7sd6Oyc+4GZDQOepbZfvge1X9QOdM5VN7SNnJwc19budeOcY21xOe+uKmb5ll2s\n3FrKhu27D3azdGwX54VjKlk9akMys5V0jTjn2Op1LeVt/c8nkS079x5cpltKwsHQHzewK6P7dW4V\n/zaRSGJmi5xzOY0u14SgHwd8CCwHarzmH1HbT/8i0AfYCFzsnNvhrXMncDW1Z+xMd869ebhttJWg\nr6lxfLZpJ2/nF/J2XhEbtu0GoFenpP93xDusRyoZHRKp2z0WDXbtqSS/wAv/gtpPLmuKy6mucXRq\nF8fpQ9M5MyudkwalkRinE7VEGhOyoG8J0Rz0+6tq+Hj9dt7OK+Sd/CKKyyqIDRgnHNWFM4d154yh\n6XTvkOh3mb7ZXVHFB5+X8HZ+Ee+uLKJ0XxVJcTGcPCiNM4elc/qQdDq0i/O7TJGI1NSg17dkYVBe\nUcW/VpcwL6+Q91YVU1ZRRbv4GE4ZnMaZWd05dUg3OiQpvACSE2KZNDyDScMzqKyuYeH6HczLK+Tt\n/ELeyiskJmCMyezMxGHdOSMrvVnfI4i0dTqiD5Ft5RW8u7KIeXlFzF+7jf1VNXROjmfC0G5MHNad\nsQO6qjviCNTUOJZt2VUb+nmFrCup7eYa0asDZw7rzsRh6QzoluJzlSL+UtdNC9i5Zz+zF29h3opC\ncjfuoMZBz45JTPSCaFTfTvqCMUTWFpfzdn4h8/KKWLppJwCZacmcmdWdb4zqxYBu7X2uUKTlKejD\naF1JObPmb2D24s3sq6xhSPeUg0eZWRmpUfclaqQp2LWXf+TXfnpasH47VTWOUwanMXVcf8YN6Kr9\nL22Ggj7EnHN8tHY7j89fz3urS4iPCXBedg+uHtefoRmpfpfXZm0rr+CZBV/y1wVfsK18P4PTU7h6\nXD/Oy+6prjKJegr6ENlXWc2cJVuZ9dEGVhWW0bV9PN8e05fLju+ri34iSEVV7d/p8fm1f6cuyfFc\nNqYvl4/R30mil4I+SCVlFTy9YCPPLNzItvL9DOmewtXj+nPuiB46Uoxgzjk+XredWR9t4N1VxcQF\nApyb3YOrx/Ynq4c+eUl00emVzbSyoJTH529gzpKt7K+u4fQh3Zg6rj8nHNVFfb+tgJlx4oCunDig\nKxu27eaJjzbwUu5mXl60mROP6sLVY/tz2pBuBAL6W0rboSN6ak/le291MY/P38C/120nKS6Gb4zq\nxVVj+5GZprM5Wrtdeyp57tMveerfX1Cwax/9uyZz1dh+XHRsr6DvEyTiJ3XdNEFFVTUv5m7mifkb\nWL9tNxkdEplyYj8mH9ebju3iW7weCa/K6hreXFHI4/M3sHTTTlITY7n0+D5MHdefbilt9+pkab0U\n9IfhnGNeXiH3vrmKjdv3MKJ3R6aO68+ko7trkI02YtHGr5g1fwNvriggKS6G7506gKnj+uv7F2lV\n1EffgBVbdvGzN/JZuGEHA7u156mrR3PSQJ173daM6tuJUX07sWHbbn45dyUPzlvNswu/5IeThnDO\nMRl6PUhUaTNH9MWl+3hw3mpeXryZTu3iufmMQVx6XG9duSoA/HvtNn7295WsLChlVN9O3PX1LLJ7\nd/S7LJHDUteNZ19lNTM/WM+j/1pHZXUNV43tz/WnDtBNxeS/VNc4Xl60iQfnfc628grOz+7BD84a\nQo+OupGaRKY2H/TOOeYs3cr9b65i6659nDWsO3ecPYS+XZJDuh2JPuUVVTz6/lpmfriBgMG08Zlc\ne/JROkNHIk6bDvpFG7/iZ2/ks2TTTob1SOWur2cxJrNLyH6/tA2bduzh/rdW8cayArqlJHDbxMFc\ndGwvnYMvESOUI0zNAr4OFDvnjvbaXgAGe4t0BHY657LNrB+wEljtzVvgnLuusSJCFfSbv9rD/W+t\n5m9Lt+o/poTMoo07+OkbK1m6aSdH90zlrq9lcbwOHCQChDLoTwLKgb8cCPpD5v8a2OWc+6kX9G/U\nt9zhBBv0Bz5q//nDDQBce5I+akto1dR4XYFvraJAXYESIUJ2eqVz7gMvwOvbiAEXA6cdaYGhUF3j\nmL1oMw++vZqSsgrO874866kvzyTEAgHj/JE9mTisOzM/XM+j76/jn6uKuXJsP248bQApifpyXyJX\nsOcWjgeKnHNr6rT1N7MlZvYvMxvf0IpmNs3Mcs0st6SkpFkbX7B+Oz+YvYxenZJ49Xsn8rvJIxXy\nElZJ8TF8//SBvH/bKZyb3YOZH67nrN9+yML12/0uTaRBTfoytqEuGTN7FFjrnPu19zwBaO+c225m\no4DXgGHOudLD/f7mdt0cuEf82AG64Zj4Y9HGr7jlxSVs3LGH74zP5JYzB5EQq6trpWU0teum2Uf0\nZhYLXAi8cKDNOVfhnNvuTS8C1gGDmruNJtTAOF3VKj4a1bcTf//+eC4d3YcZH6znvIc/In/rYY9r\nRFpcMF03E4BVzrnNBxrMLM3MYrzpTGAgsD64EkUiW3JCLL+8YDhPXHkc23fv57xH5vPo++uorvH/\n1GURaELQm9lzwMfAYDPbbGZTvVmTgecOWfwkYJmZLQFeBq5zzu0IZcEikerUId2YN/0kJgxN5/63\nVjF5xsds2rHH77JEovOCKRE/Oed49bMt3P16HjXO8ZNzsrg4p7e6GCXkwt5HLyL1MzMuPLYXb918\nEsf06sgPZy/nO3/JpaSswu/SpI1S0IuESc+OSTxzzfH8+GtD+WDNNs767Qe8nVfod1nSBinoRcIo\nEDCuGZ/JGzeOo3uHRKb9dRG3vbSUsn2VfpcmbYiCXqQFDEpP4dXvjeX6U49i9uLNTPqdLrKSlqOg\nF2kh8bEBbps4hJeuO4GYgDF55gLunbuSiqpqv0uTKKegF2lho/p2Zq53kdVjH6znkscWUFy6z++y\nJIop6EV8cOAiqz99+1g+LyrjnIfns3TTTr/LkiiloBfx0VlHZzD7uycSFxPgm499zGufbfG7JIlC\nCnoRnw3NSGXODeMY2bsj019Ywr1vrtTtEySkFPQiEaBzcjxPX3M83x7Th8f+tZ5rnvqUUp2CKSGi\noBeJEHExAX5+/nB+fv7RfLhmGxc88hHrS8r9LkuigIJeJMJ8e0xfnr7meHbs3s/5j3zEB583b2Ae\nkQMU9CIRaExmF+bcMI4eHZO48olP+POH64mEGxBK66SgF4lQvTu3Y/Z3T+TMrO78/O8rufWlZeyr\n1MVVcuQU9CIRLDkhlj9edizTJwxk9uLNXDpTF1fJkVPQi0S4QMCYPmEQj152LKsKyjj34Y90cZUc\nkaaMMDXLzIrNbEWdtnvMbIuZLfEeZ9eZd4eZrTWz1WY2MVyFi7Q1k4bXXlwVEzAufuxjXl+ii6uk\naZpyRP8kcFY97b9xzmV7j7kAZpZF7RCDw7x1/nhgDFkRCV5Wj1Tm3DCWEb07ctPzurhKmqbRoHfO\nfQA0ddzX84DnnXMVzrkNwFpgdBD1icghurRP4Ompx3PZ8bUXV93w7GLdAVMOK5g++hvNbJnXtdPJ\na+sJbKqzzGavTURCKD42wC8uGM6PvzaUN1cU8p2/LGLvfoW91K+5Qf8okAlkAwXAr4/0F5jZNDPL\nNbPckhJdECLSHNeMz+S+C4fz4ZoSpsz6RLdNkHo1K+idc0XOuWrnXA0wk/90z2wBeLine truncated
|
||||
"text/plain": [
|
||||
"<matplotlib.figure.Figure at 0x7fd9c1058780>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD9CAYAAABHnDf0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt0XOV57/HvM7paM7ZlSyPfZNlINriAsQFhY25xSUNt\nQ6CcpDk0CYQkreskTaFNm0N7ziFpVrty0lxOA26hhFyAkltPgNAEk5CEJBCCwXZ8IzbGMr5iW7Js\nS5Zk3Z/zx2w5sixZI2lGezT+fdaaNTN7v5r9sBn/5p1373m3uTsiIpJdImEXICIiqadwFxHJQgp3\nEZEspHAXEclCCncRkSykcBcRyUK5yTQys93ACaAL6HT36j7rlwLfB94MFj3h7p9JXZkiIjIUSYV7\n4Pfd/chZ1r/g7jeNtCARERk5DcuIiGShZMPdgZ+Y2XozWzlAm6vMbLOZrTGzi1JUn4iIDEOywzLX\nuPsBMysDnjOz7e7+y17rNwAV7t5kZiuAp4C5fV8k+GBYCRCNRi+fN2/eCMsXETm3rF+//oi7xwdr\nZ0OdW8bMPg00ufsXztJmN1B9tjH66upqX7du3ZC2LSJyrjOz9X1PaunPoMMyZhY1s/E9j4EbgK19\n2kw1MwseLwpet344hYuIyMglMywzBXgyyO5c4Jvu/qyZrQJw9weBdwMfMbNO4CRwm2u6SRGR0Awa\n7u6+C1jQz/IHez1eDaxObWkiIjJcOhVSRCQLKdxFRLKQwl1EJAsp3EVEstBQ5paREHR0ddPS1kVz\neyct7Z00tXXR0t5JZ5fT1e10djtd3d3BvQ+8/LT13UQiRl5OhJyIkRuxXveR3z3PGXh5UX4u0fwc\nigpyieXnUlSQQ16O+goimULhnkZd3c6xlnaONrdT35S4P9rcxrGWDprbOhOB3dZFU1snLe2JAG9u\n66Q5CPDm9i7aO7vD/s9IWn5OhGhBTiL4g/tYQS5F+TlECxLLovm5RAtymVSUx+RoAZOj+aduk4ry\nyNUHhEhKKNyHwN053tLBwYZW6pvbTgvt+iC4EwGeuB0/2cFAZ/sX5EYSwRcEXlF+DrGCXKaMLzy1\nLFrwu95xtFdAFublkJ8TITcn0qfXbeRGIuTk9O2NB8uD55GI0d27N9/dfXrvvtvp6hpgeXc37Z3O\nyY7ffQg1tXXR0pb4MEo87+z1baOLI01tpz7Imts7ae3o/wPLDCaOy2NyNJ+SU6FfQEk0n0m9lpXE\n8pk2cRyTivIIfn8hIn0o3Htp7+zmUEMrB46f5K2eW8NJDhxv5cCxFt463srJjq4z/i5inNYDnTd1\nApOieaeC6VRYxXp6qPmhD2FEIkZ+pCcYc0Z12x1d3ae+0RxtaufoGd9u2qlvbuPNI82s33OMYy0d\ndHWf+Sk5Li+H6cWFTC8ex4zicUw/dSukvLiIqRMLyc/VNwE5N51T4d7R1c2e+hbePNKcCOs+QV57\nou2MnnZprIAZxYWcP2U8Sy8oS4THxEJKYgWnQnviuDwiEfUgk5WXE6FsfCFl4wuTat/d7TS2dgTf\njto5cqKNgw2tp334bt9eS92JttP+zgzisYJe4V946kOgMh5lVkk09A9ZkXTJynA/0drBrrpmdtY2\nUVPXdOp+T30Lnb16gAW5kVP/2N92fvxUz29GcJs6sZDCvNHt1cqZIhGjuCif4qJ8qs4yF15rRxeH\ngtBPfGj/7gNg28FGfrLtMG29jmHkRoyKkiLmxGNUlcVO3VfFo4wvzBuF/zKR9Bmz4e7u1J5oo6a2\niZ11Tb3umznU2HqqXW7EmF0aZU5ZjGUXT6UqHqMyHqN80jhKovkas80ihXk5zC6NMrs02u96d+do\nczv7j51k15HEe6Xng//512vp6PrdB/+UCQXMKYtRFY+duq+Kx5gyoUDvGRkTxly4v/jGET7/49fZ\nVdvEibbOU8tjBblUlcW4ak7Jaf8oKyYX6au3AGBmlMQKKIkVsGBm8WnrOrq62Xu05bROws66Jp7c\ncKDf99lF0ydwyYyJXFJezNwpMb3HJOOMuXAvzIsQK8jh1stmnBbiZePVo5Lhy8uJnOqd39BreX/f\nEHccbuK/Nr3FN9fuBRLDexdNn8Al5cXMnzGRBTMncl5pjBwdh5EQDfliHamii3XIWNbd7ew52sLm\n/cfZvL+BLfsb2PpWAy3tibOpovk5XDxjIpeUT2R+eTELyidSMblIHRAZsWQv1jHmeu4imSASMc4r\njXJeaZRbFs4AEj9aq6lrYtO+42w50MDm/Q088us9tHe+CSTO4Z8fBP6i8yaz6LzJFOXrn6Ckh3ru\nImnU3tnNjsMnEr37A8fZtK+BHYdP0Nnt5OUYC2cWc1VVKVfPKWXhzGKdly+DSrbnnlS4B9dEPQF0\nAZ19Xzi4xN6XgRVAC3Cnu28422sq3OVc1dLeybrdx/hVzRFe2lnP1rcacE/8KOuK8yZzdVUJV88p\n5femTdC4vZwhHcMyv3+WC14vB+YGt8XAA8G9iPRRlJ/LdefHue78xEn7DS0d/HpXPS/VHOGlmno+\nu2Y7kBjGWVJZwtVzSlhSVUpVPKoxe0laqgb8bgEeDa6b+rKZFZvZNHc/mKLXF8laE4vyWHbxVJZd\nPBWAw42tiaDfWc9LNfU8+9ohIHHu/dVVpSypKuH6eWWUxArCLFsyXLLh7sBPzKwL+Hd3f6jP+hnA\nvl7P9wfLFO4iQzRlQiG3XlrOrZeW4+7sqW/hpZp6flVzhJ/vqOOJ3xwgYnBlZQnL509j2UVTiY9X\n0Mvpkg33a9z9gJmVAc+Z2XZ3/+VQN2ZmK4GVABUVFUP9c5Fzjpmd+tXtexdX0N3t/PZgI89uPcQz\nWw7yv5/ayqe+v5VF501mRRD0ZROSm7NHstuQz5Yxs08DTe7+hV7L/h34ubt/K3j+OrD0bMMyOqAq\nMjLuzuuHT/DMlkTQ76xtwgyumDWZFfOnsuziaUydqKDPNik7W8bMokDE3U8Ej58DPuPuz/ZqcyPw\nFyTOllkM3Ofui872ugp3kdR64/AJfrjlIGu2HOL1wycAqJ41ieXzp7H84qlMLx4XcoWSCqkM90rg\nyeBpLvBNd/8nM1sF4O4PBqdCrgaWkTgV8oPuftbkVriLpM/O2ibWbDnID7ccZPuhRNBfWlHMjfOn\nseziqZRPKgq5QhmulJ7nng4Kd5HRsauuiTXBGP1rbzUCcPWcEu5YMpu3zyvTpQ3HGIW7iJxh95Fm\n/mvTW3zrlb281dDKjOJxvHdxBbddMVOnVo4RCncRGVBnVzc/2VbLo7/ezUs19eTnRLhpwTQ+sGT2\nGdMhS2ZRuItIUt44fIJHf72HJzbsp7m9iwUzi/nAklmsmD9NVyLLQAp3ERmSE60dPLHhAI/8eje7\n6popiebz36+YyfuunMUMnWmTMRTuIjIs7s6vdtbzyK9389NthwF4x4VT+MCS2SypKtH8NiHTfO4i\nMixmxjVzS7lmbin7jrbw+Nq9fOfVvfzotcPMKYtxx5JZ/PHlMxmXryGbTKaeu4gMqrWjix9sPsgj\nL+1my4EGpkwo4BPvuIB3XV6uaYlHmYZlRCTl3J1X3jzKZ9dsZ+O+41wwZTz3LJ/H0gviGq4ZJcmG\nu369ICJJMzMWV5bw5Eev4l/fexmtnV188Buv8r6H17L1QEPY5UkvCncRGTIz48ZLpvHcX72NT73z\nQrYdbOSm+1/k7m//hn1HW8IuT9CwjIikQGNrBw/8vIavvfgm7nDn1bP52NI5TCzKC7u0rKMxdxEZ\ndW8dP8mXntvB9zbsZ0JhHh+/fg63L5lFQa7OrEkVjbmLyKibXjyOL/zxAn748WtZMLOYf/zhNt7+\nxV/w/Y0H6O4OpyN5rlK4i0jKXTh9Ao9+aBGPfXgR4wvzuOvbG7nlX3/FSzVHwi7tnKFwF5G0uXZu\nnB9+/Bq++McLqG9q471fWcuHvvEqe+t10DXdFO4iklaRiPGuy8v52d8s5Z7l83j1zaPceN8L/HDz\ngFfhlBRQuIvIqCjMy2HV26pYc/e1zJkS42Pf3MD/emoLrR1dYZeWlZIOdzPLMbPfmNkP+lm31Mwa\nzGxjcLs3tWWKSLYon1TEd/98CX9+XSX/8fJebv23l6ipawq7rKwzlJ77XcC2s6x/wd0XBrfPjLAu\nEclieTkR/m7F7/H1O6/gUMNJ3nn/izz5m/1hl5VVkgp3MysHbgQeTm85InIu+f15ZTxz17VcPH0i\nf/WdTXzy/23iZLuGaVIh2Z77vwCfBLrP0uYqM9tsZmvM7KKRlyYi54JpE8fxzT9bzMevn8N/rt/P\nzatfZMfhE2GXNeYNGu5mdhNQ6+7rz9JsA1Dh7pcA9wNPDfBaK81snZmtq6urG1bBIpJ9cnMifOKG\nC3jsQ4s51tLBzatf5Luv7iOsX9Bng0GnHzCzzwK3A51AITABeMLd33+Wv9kNVLv7gL9Y0PQDItKf\n2hOt3P3tjbxUU88fLZzOP946n1iBrivUI2XTD7j737l7ubvPBm4DftY32M1sqgWTOZvZouB164dV\nuYic08rGF/LYhxfz1+84n6c3vcXN97/Ia29pOuGhGvZ57ma2ysxWBU/fDWw1s03AfcBtru9TIjJM\nORHjL98+l2/+2ZU0t3dy67+9xGMv79EwzRBoVkgRyWj1TW389Xc38Ysdddw4fxqffdd8JhSeu1MJ\na1ZIEckKJbECvn7nFdyzfB7PvnaIm+57ke2HGsMuK+Mp3EUk40Uixqq3VfHdP7+Sts4u3v/wK+yp\nbw67rIymcBeRMePyWZN5/E8X09Xdzfu/upbaxtawS8pYCncRGVPmlI3n6x9cRH1TO3d87RUaWjrC\nLikjKdxFZMxZOLOYh26vpqauiQ8/8qqmLOiHwl1ExqRr5pby5dsuZf3eY3z08fV0dLine truncated
|
||||
"text/plain": [
|
||||
"<matplotlib.figure.Figure at 0x7fd9c1395898>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"# function\n",
|
||||
"def fn(x):\n",
|
||||
" return -(x-5)**2 +200\n",
|
||||
"\n",
|
||||
"x = np.arange(0, 20)\n",
|
||||
"plt.plot(x, fn(x))\n",
|
||||
"plt.show()\n",
|
||||
"\n",
|
||||
"plt.plot(x, np.log(fn(x)))\n",
|
||||
"plt.show()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\\ln l(w) = ln \\prod_{i = 1}^{N} P(y_i|x_i, w) = \\sum_{i=1}^{N} \\ln P(y_i|x_i, w) \\tag{1}$$\n",
|
||||
"\n",
|
||||
"We also add the label for simpler math:\n",
|
||||
"\n",
|
||||
"$$\\sum_{i=1}^{N} y_i \\ln P(y_i|x_i, w) \\tag{1}$$\n",
|
||||
"\n",
|
||||
"$$P(y=0|x, w) = 1 - P(y= 1 | x, w) = 1 - \\frac{1}{e^{-w^Th(x)}} \\tag{2}$$\n",
|
||||
"\n",
|
||||
"note:\n",
|
||||
"\n",
|
||||
"$$1 - \\frac{1}{x+1} = \\frac{x+1}{x+1} - \\frac{1}{x+1} = \\frac{x}{x+1}$$\n",
|
||||
"\n",
|
||||
"When applied to the formula (2):\n",
|
||||
"\n",
|
||||
"$$P(y=0|x, w) = \\frac{e^{-w^Th(x)}}{1 + e^{-w^Th(x)}} \\tag{3}$$\n",
|
||||
"\n",
|
||||
"Both probabilities are:\n",
|
||||
"\n",
|
||||
"$$P(y=1|x, w) = \\frac{1}{1 + e^{-w^Th(x)}} \\tag{1.1}$$\n",
|
||||
"\n",
|
||||
"$$P(y=0|x, w) = \\frac{e^{-w^Th(x)}}{1 + e^{-w^Th(x)}} \\tag{2.1}$$\n",
|
||||
"\n",
|
||||
"Total formula:\n",
|
||||
"\n",
|
||||
"$$ \\ln l(w) = y_i \\ln \\frac{1}{1 + e^{-w^Th(x)}} + y_i \\ln \\frac{e^{-w^Th(x)}}{1 + e^{-w^Th(x)}} \\tag{4}$$\n",
|
||||
"\n",
|
||||
"Simplifying:\n",
|
||||
"\n",
|
||||
"$$ \\ln \\frac{1}{1 + e^{-w^Th(x)}} = -\\ln (1 + e^{-w^Th(x)}) \\tag{1.2}$$\n",
|
||||
"\n",
|
||||
"$$ \\ln \\frac{e^{-w^Th(x)}}{1 + e^{-w^Th(x)}} = \\ln (e^{-w^Th(x)}) -\\ln(1 + e^{-w^Th(x)}) \\tag{2.2}$$\n",
|
||||
"\n",
|
||||
"Further simplifying:\n",
|
||||
"\n",
|
||||
"$$ \\ln e^a = a$$\n",
|
||||
"\n",
|
||||
"$$ -w^Th(x) -\\ln(1 + e^{-w^Th(x)}) \\tag{2.3}$$\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"After all simplifications the log likelihood function becomes (Cannot replicate it) :(: \n",
|
||||
"\n",
|
||||
"$$ \\ln l = \\sum_{i=1}^{N}y_i w^T x_i - \\ln(1+ e^{w^T x_i})$$\n",
|
||||
"\n",
|
||||
"$$\\triangledown \\ln l= \\sum_{i=1}^{N} h(x) (y_i - \\hat{y_i})$$\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 143,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAsUAAAHVCAYAAAAZ9YYeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXmQXNd15vk79y2ZWQv2wk4CBLgIBEVK3EBRFElJpixZ\npi1Lsluy2rvaM9ETHTMxExMxMRET7onomZgYR0yE/+ietqdn2tFuuy1b+y5rpURxB/cVXAFiXwu1\nZeZb7pk/zs2sKlRhKQAkQOL+IiCisl6+vO+9VOB7533nO6KqRCKRSCQSiUQilzLuQi8gEolEIpFI\nJBK50ERRHIlEIpFIJBK55ImiOBKJRCKRSCRyyRNFcSQSiUQikUjkkieK4kgkEolEIpHIJU8UxZFI\nJBKJRCKRS54oiiORSCQSiUQilzxRFEcikUgkEolELnmiKI5EIpFIJBKJXPKkF+JDV6xYoRs3brwQ\nHx2JRCKRSCQSuYTYvn37YVUdOd12F0QUb9y4kccee+xCfHQkEolEIpFI5BJCRHaeyXbRPhGJRCKR\nSCQSueSJojgSiUQikUgkcskTRXEkEolEIpFI5JIniuJIJBKJRCKRyCVPFMWRSCQSiUQikUueKIoj\nkUgkEolEIpc8URRHIpFIJBKJRC55oiiORCKRSCQSiVzyRFEciUQikUgkErnkiaI4EolEIpFIJHLJ\nE0VxJBKJRCKRSOSSJ4riSCQSiUQikcglTxTFkUgkEolEIpFLniiKI5FIJBKJRCKXPOmFXkAkEolE\nIpHI2aLqAY9IlDQnovURtN4NOCTdiLjhC72ki5r4DYpEIpFIJPKOQbULOoXSgPIxKJ4EajTdjDQ+\nhLilF3qJFwW++zAU9wMOVFFxaPPXcdmVF3ppFy1RFEcikUgkErnoUa3Q7i+hfAJQKF8DNwTZtUAC\n1U603g+Dv4dI60Iv94Ki9UETxG5Vv4Ku2oHOd9H0TxFpXuAVXpxET3EkEolEIpGLHi0ehvJRcCMg\nw6DjUO8HHUfEIckI6ARavnqhl3peUfWon0S1OvP3VDsBN8tSYkK4gnrf+V/ku4RYKY5EIpFIJHJR\no1pC8XiofCZmoZAUJIdqF+TLwpYZ+KMXdK3nE1/ugO59oBNAhubbkPwmRE5T0zzZ71UBh/qjoCW4\n5dGLPYN4JiKRSCQSiVzcaAFUiGT2s7QABU2A9owNC3Ar3/71vQVotQs63zTh6tagWkD3p6g4JL/p\nlO+V9Aq0ex/qx8AfB2qgAQjavR/8Acws0ECbv4rLNr31B/QOINonIpFIJBKJXNzIALilqJ+wH2UA\nkvWgB8ENo9pF633gRpB3icDT4lGQob4/WiSHZBV0HwmJGydH3DJItkL3p1CEZsTiUbNO1EeQZC2S\nrAbXhM43rHIciaI4EolEIpHIxY2IQP4R0HG0PmTiWBZBsgXcetAO5Dchrc+YeHw34I+Givg0Ig2s\nMl6e8q2qBdSvQH43ZO+1G4jkMnvNZTP21wIELXec9+W/E4n2iUgkEolEIhctqh4Rh8suR90X0PIZ\n8Ecguw7JrkPc0IVe4ltDsgGqFyGZtoOonwC3DDiN8K8PAl07T/VOIAWdgvoIFC9A8/YZG4ffRaIo\njkQikUgkcnGh6tHyKSgeAZ1Ekw1I4w4kWYUkH73Qy3tbkMbNaP2yxau5YdC2VcQbn7bK+Snf7MBP\nmiCWxfazDoDsg3o3qt1QdQa0g6Qb3voDegcQ7RORSCQSiUQuKrR4BDo/AmmAWw3+ANr+EuqPXeil\nvW2IW4YM/C5kWwEHyeXIwOdx2cbTv9mtAq0tYWJmEoVbBqJQv4H6o2j1JmRXQXIG+7wEiJXiSCQS\niUQiFw2qXSgehmTNdFyYLEPrA2jxNNK868Iu8G1E3FKkufDKuEiCNj8Ck6+AH53+RX6rWSXS94AM\nI00TxCLJeVz1O5coiiORSCQSiVw86CTg5+bnyiD4gxdmSVpZM1r1PJAh+XWQbDq9jeECIvmNaHET\niAdycIvMfsEw0vqt6Xi7txDVLqDvmAl6URRHIpFIJBK5eJAhIEW1nC3cdAKS97zty1H1aOe7UL4E\nbjHg0fYOyG9DGh9629dzpogbQlufhs53gU5Is1iEtO59ywWx+gm0+1OoXrafk41I8yOIW/KWfu65\nEkVxJBKJRCKRiwaRHG18ALo/QWW5+Yr9MZAMyd779i+o3g3VDkjW9yvDqsPQfRTNrkfc4rd/TadA\n6/1o9TJojaSb0ME/QfQQkIBbefppeOf6+Vqj7a/ZNXOr7cV6Lzr1ZRj8/Ys6Mi+K4kgkEolEIhcV\nkt2EMgjlIyau0k1I4wMLFqCWYvE8lNstjSG7GslvRdyiM3u/P452fgHVAcgWWTYywbMrgD8UqscL\nQ7VAy9dAj4BbgaRXnBex6IvHbWAHKeBsAEh+M9L88Dnv+4yp94A/iCTrpl9LVqD1Hqh3QXrl27eW\nBRJFcSQSiUQi54j6CbR6CepDkKxGsmv6k8giC0dEkHwL5FvOaT9aPAjdX4JbbrFm5XNo9RoMfAFx\ng6d8ry9fhs53LPO3fhP0KJpcgWQzRN1ZXGNf7YWpvwfaIMNAZRXxgc+cU+ay+gno3heqwWaPUPVQ\nbkezLTbB7u1Ap0Dnq0YL6ie4eF3YURRHIpFIJHJOqD+KTn0JfAdcA6rn0eIxGPidM65IRs4/6qds\ntHGydrppL1mFVvvR6kUkv+nk79UudL4PbonFmOkxUIXqddStAEpwI+DWnPl6tEI7P4XON2yohgxC\nssaSIPwhtHj4rJIm+vj9WFPbzIl1DiVBqz1nJYptrHYBsvjMEyrcchDtD12ZsTfELV/wGt5OoiiO\nRCKRSOQc0O4vgn9zWiBpvR8tHj03kRM5N3QMYG6KhWtCvf/U7633g1b91ATNboTyeYs3q16GxgeR\n5j0L8udq+SSUT4LvBjEtUO0BGbARzOXzcBbfF1++DMUDUL1ugzmkhbilMz5YYYHWDPVTaPdHUL1q\nL8gg2vjYGWUkSzKCptdC+QzqlgJiNxXpZphpqbgIicM7IpFIJBI5S1Q9VK+F0bszcMstreAiQ7VC\nq91otRPVzlnuQ9HqTXznPnz3lzZx7WJEhgCPaj37de2AW3G6N9uQi95PbhjybZC/D1q/iRv4tL22\nEIrHIRkBSQAPIhaTVu0C6gULVwBfvgTtr9uQjuRKQKH7S9QfB0B1CiRF0isWtF/t/iB8r1chyRog\nhc7XUX/0jN4vzXug+WvgBq1RsvErSPPX3/Imv3MlVoojkUgkEjlrBMiBKvy3Rwlu4MIs6SRofRBt\nfwN0HFRAHNr8OC675sz3oYp2f2ICTxqAosVDaOMeXH79W7f4k63Hj4M/Dm7RHKuKuCE0e595at0q\nILWKpWRIdu1J9jeKVjtBK8wDOz5D/BZAbhnFZ7XYrlWF08tDBXYJVpsszbfcuHNhu1OF7i8gWY6I\nfdc0/wAUD0HxBJpdDbSQ1qcW5FVWfywI4rX9tA1xA2g9jpYvII0PnnYfIgmSb4V864KO6UITRXEk\nEolEImeJiKD5TVDcj7q15uHUGurD0Pz4hV5eH9UKbX8dAEnWhte60PkumowgJ1S6VX0YlFHPatzC\n74fiCUjW9at+qqXFp6WbT9u8dl6Pp/NTKJ8KlVe1eLTG3bPsEtK4E5VBKLYDHRu40bhj3iqvL56H\n7vdBsXsdPwE6iiaLwwsOmh+fc67OmGwLlM9CssEyl+uD4KcgWQrZViS/cYE7LEHHELd2+njdEJp/\nCPwhZOBz4EYWnkmsXcDNHUwiGfixBa7xnUUUxZFIJBKJnAOS32wVy+pZFAEUGrch2VlWFN8K6v2g\nE31BDCDSQAEtX0Ma00JP60No+1ugo5gYbKDNX8NlG9Fqlz2On/EYXCRDUfAHwG06p2WqnwoNYynq\n1ljF8YRH7qoFOvk30P0hMAjJCkg2Q/Ek6pYg+c0z1pYijW1ofivWhDb/43tLbvgni0cLNgZ1pZ23\n/MN2g+APQ/Ui6nJINi94mp3k26wK7Q+AW2trT1No/Q4uW5i9wchAFqN+Cpn1VGIS0itnXesF4ZZh\nw1OK2TFxOgXJxrPb5zuEKIojkUgkEjkHRFKkdQ/qt1kFUBadU7TWW0MNKlbF1qNmD5Bh+o/vA1ZR\n/hqgMyrKU9D5Jpr8kVkmVOfZv3KuksIXz0Dne1Dvg+oNoItm16L5nUjzrn5GsXZ+ZFm8siKsZwyq\nJyB7v1WEZ4jiHiZgTyFi671APUsEmth3UNwPdECWgE7Y+clvRRp3Lej4xA3D4D9Hy1eCMF6BZFef\n9QhkEUEbd0D7WyjerBk6Br6DtLad1T5tvznauBu6P0BpWoVYx214Sbb5rPf7TiCK4kgkEolEzgPm\nab1II9iSVdZg1v05UANqOtYNQuuz09vVe+epKA+gOopWr9mENPkZqu1+DrP64yawz7YyCWh9GNpf\nMzFcvQy0bX31UaiOoPVuGPoT0DZUL4I0rTFNBBg0X3F9FNxZZkOLY17RrGPgu0j+vumXdAi629Hs\nhgWPLRZpIPlWVK/lVJXrM8Vl1+BJzEfsD0GyxsY4J2ceFTfvfvP3oskytHgWdBLSDyDZey7qaXTn\ngyiKI5FIJBJ519MAEnsELk3AgZSgPnhIe5TMKw5FQAvELUKbvwGd76GMAmoZtq3fmPMW1cJizc6g\n4VDLV0wQ6yiWiztsy9BJa46rXjCbR7LI1pesNk9umDCHpGa7yD8ePtvTb4ybITy1PmLiUVoWD+aP\no8UvLSmkfA5N25BeEbzhXTtfbuSEU5GgImanWKAoVu2g3YfMC42i6Rak8cFzerLgsishO/9T4iRZ\nh7Qu7gi1800UxZFIJBKJXARovRftPmDV2mQFZLfhsnPz6E7v/Bi4DBp3m5jTCpLloLine truncated
|
||||
"text/plain": [
|
||||
"<matplotlib.figure.Figure at 0x7fd9c165d470>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"np.random.seed(0)\n",
|
||||
"num_observations = 5000\n",
|
||||
"\n",
|
||||
"x1 = np.random.multivariate_normal([0, 0], [[1, .75], [.75, 1]], num_observations)\n",
|
||||
"x2 = np.random.multivariate_normal([1, 4], [[1, .75], [.75, 1]], num_observations)\n",
|
||||
"\n",
|
||||
"features = np.vstack((x1, x2))\n",
|
||||
"labels = np.hstack((np.zeros(num_observations), np.ones(num_observations)))\n",
|
||||
"plt.figure(figsize=(12, 8))\n",
|
||||
"plt.scatter(features[:,0], features[:,1], alpha=.4, c=labels)\n",
|
||||
"plt.show()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 158,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Score: -5954.892449294746\n",
|
||||
"Score: -498.20255757965316\n",
|
||||
"Score: -350.5808749406438\n",
|
||||
"Score: -292.93254194794474\n",
|
||||
"Score: -261.1174211867518\n",
|
||||
"Score: -240.59940601572433\n",
|
||||
"Score: -226.12429319454873\n",
|
||||
"Score: -215.29620999117014\n",
|
||||
"Score: -206.8547580368404\n",
|
||||
"Score: -200.068756580291\n",
|
||||
"Score: -194.48262542221624\n",
|
||||
"Score: -189.79657736766856\n",
|
||||
"Score: -185.80464746775425\n",
|
||||
"Score: -182.36029469165794\n",
|
||||
"Score: -179.35619655046423\n",
|
||||
"Score: -176.71181606704357\n",
|
||||
"Score: -174.36544807294422\n",
|
||||
"Score: -172.26896020650537\n",
|
||||
"Score: -170.3842154300599\n",
|
||||
"Score: -168.680577514434\n",
|
||||
"Score: -167.1331334953521\n",
|
||||
"Score: -165.72140245990641\n",
|
||||
"Score: -164.4283813996932\n",
|
||||
"Score: -163.23982921970457\n",
|
||||
"Score: -162.14372195511714\n",
|
||||
"Score: -161.12983300913268\n",
|
||||
"Score: -160.18940599227997\n",
|
||||
"Score: -159.3148970463256\n",
|
||||
"Score: -158.49976993054196\n",
|
||||
"Score: -157.7383316131317\n",
|
||||
"Score: -157.02559927358772\n",
|
||||
"Score: -156.3571918923439\n",
|
||||
"Score: -155.72924125417734\n",
|
||||
"Score: -155.13831840477053\n",
|
||||
"Score: -154.58137250094836\n",
|
||||
"Score: -154.05567967120572\n",
|
||||
"Score: -153.5588000151348\n",
|
||||
"Score: -153.08854126145337\n",
|
||||
"Score: -152.64292790552287\n",
|
||||
"Score: -152.22017488097782\n",
|
||||
"Score: -151.81866500278323\n",
|
||||
"Score: -151.43692956282422\n",
|
||||
"Score: -151.07363157302348\n",
|
||||
"Score: -150.72755124174546\n",
|
||||
"Score: -150.39757334200755\n",
|
||||
"Score: -150.08267618865483\n",
|
||||
"Score: -149.78192198917554\n",
|
||||
"Score: -149.4944483715164\n",
|
||||
"Score: -149.2194609239191\n",
|
||||
"Score: -148.95622660781115\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def sigmoid(z):\n",
|
||||
" return 1 / (1 + np.exp(-z))\n",
|
||||
"\n",
|
||||
"def log_likelihood(z, y):\n",
|
||||
" return np.sum(y * z - np.log(1 + np.exp(z)))\n",
|
||||
"\n",
|
||||
"def gradient(x, y, a):\n",
|
||||
" return np.dot(x.T, (y - a))\n",
|
||||
"\n",
|
||||
"def log_regr(x, y, n, eta):\n",
|
||||
" intercept = np.ones((x.shape[0], 1)) # bias as 1 * wbias = wbias\n",
|
||||
" x = np.hstack((intercept, x)) # add to the inputs \n",
|
||||
" w = np.random.standard_normal(x.shape[1])\n",
|
||||
"\n",
|
||||
" for i in range(n):\n",
|
||||
" z = np.dot(x, w)\n",
|
||||
" \n",
|
||||
" ll = log_likelihood(z, y)\n",
|
||||
" prediction = sigmoid(z)\n",
|
||||
"\n",
|
||||
" dw = gradient(x, y, prediction)\n",
|
||||
" \n",
|
||||
" w += eta * dw\n",
|
||||
" \n",
|
||||
" if i % 100 == 0:\n",
|
||||
" print(f\"Score: {ll}\") # , w: {w}, dw, {dw}\")\n",
|
||||
" \n",
|
||||
" return w\n",
|
||||
" \n",
|
||||
"w = log_regr(features, labels, 5000, 1e-4)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 173,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
|
||||
" intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
|
||||
" penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
|
||||
" verbose=0, warm_start=False)"
|
||||
]
|
||||
},
|
||||
"execution_count": 173,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.linear_model import LogisticRegression\n",
|
||||
"\n",
|
||||
"lrc = LogisticRegression(fit_intercept=True)\n",
|
||||
"lrc.fit(features, labels)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 174,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"0.995"
|
||||
]
|
||||
},
|
||||
"execution_count": 174,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"lrc.score(features, labels)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.1"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
Reference in new issue
Block a user