multiple-linear-regression/multiple-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multiple Regression (Interpretation) on House sales data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fire up graphlab create"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import graphlab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load house sales data\n",
    "\n",
    "Dataset is from house sales in King County, the region where the city of Seattle, WA is located."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] graphlab.cython.cy_server: GraphLab Create v2.1 started. Logging: C:\\Users\\agraw\\AppData\\Local\\Temp\\graphlab_server_1504899806.log.0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This non-commercial license of GraphLab Create for academic use is assigned to agrawal.pr@husky.neu.edu and will expire on March 12, 2018.\n"
     ]
    }
   ],
   "source": [
    "sales = graphlab.SFrame('kc_house_data.gl/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore house sales data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">id</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">date</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">price</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">bedrooms</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">bathrooms</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_living</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_lot</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">floors</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">waterfront</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7129300520</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2014-10-13 00:00:00+00:00</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">221900.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1180.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5650</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "    </tr>\n",
       "</table>\n",
       "<table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">view</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">condition</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">grade</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_above</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_basement</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">yr_built</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">yr_renovated</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">zipcode</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">lat</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1180</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1955</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">98178</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">47.51123398</td>\n",
       "    </tr>\n",
       "</table>\n",
       "<table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">long</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_living15</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_lot15</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-122.25677536</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1340.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5650.0</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[1 rows x 21 columns]<br/>\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tid\tstr\n",
       "\tdate\tdatetime\n",
       "\tprice\tfloat\n",
       "\tbedrooms\tfloat\n",
       "\tbathrooms\tfloat\n",
       "\tsqft_living\tfloat\n",
       "\tsqft_lot\tint\n",
       "\tfloors\tstr\n",
       "\twaterfront\tint\n",
       "\tview\tint\n",
       "\tcondition\tint\n",
       "\tgrade\tint\n",
       "\tsqft_above\tint\n",
       "\tsqft_basement\tint\n",
       "\tyr_built\tint\n",
       "\tyr_renovated\tint\n",
       "\tzipcode\tstr\n",
       "\tlat\tfloat\n",
       "\tlong\tfloat\n",
       "\tsqft_living15\tfloat\n",
       "\tsqft_lot15\tfloat\n",
       "\n",
       "Rows: 1\n",
       "\n",
       "Data:\n",
       "+------------+---------------------------+----------+----------+-----------+\n",
       "|     id     |            date           |  price   | bedrooms | bathrooms |\n",
       "+------------+---------------------------+----------+----------+-----------+\n",
       "| 7129300520 | 2014-10-13 00:00:00+00:00 | 221900.0 |   3.0    |    1.0    |\n",
       "+------------+---------------------------+----------+----------+-----------+\n",
       "+-------------+----------+--------+------------+------+-----------+-------+------------+\n",
       "| sqft_living | sqft_lot | floors | waterfront | view | condition | grade | sqft_above |\n",
       "+-------------+----------+--------+------------+------+-----------+-------+------------+\n",
       "|    1180.0   |   5650   |   1    |     0      |  0   |     3     |   7   |    1180    |\n",
       "+-------------+----------+--------+------------+------+-----------+-------+------------+\n",
       "+---------------+----------+--------------+---------+-------------+\n",
       "| sqft_basement | yr_built | yr_renovated | zipcode |     lat     |\n",
       "+---------------+----------+--------------+---------+-------------+\n",
       "|       0       |   1955   |      0       |  98178  | 47.51123398 |\n",
       "+---------------+----------+--------------+---------+-------------+\n",
       "+---------------+---------------+-----+\n",
       "|      long     | sqft_living15 | ... |\n",
       "+---------------+---------------+-----+\n",
       "| -122.25677536 |     1340.0    | ... |\n",
       "+---------------+---------------+-----+\n",
       "[1 rows x 21 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sales[0:1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Split the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_data, test_data = sales.random_split(.8,seed=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning a multiple regression model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Linear regression:</pre>"
      ],
      "text/plain": [
       "Linear regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 17384</pre>"
      ],
      "text/plain": [
       "Number of examples          : 17384"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of features          : 3</pre>"
      ],
      "text/plain": [
       "Number of features          : 3"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 3</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 3"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 4</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 4"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting Newton Method</pre>"
      ],
      "text/plain": [
       "Starting Newton Method"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 2        | 1.032561     | 4146407.600631     | 258679.804477 |</pre>"
      ],
      "text/plain": [
       "| 1         | 2        | 1.032561     | 4146407.600631     | 258679.804477 |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>SUCCESS: Optimal solution found.</pre>"
      ],
      "text/plain": [
       "SUCCESS: Optimal solution found."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre></pre>"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "example_features = ['sqft_living', 'bedrooms', 'bathrooms']\n",
    "example_model = graphlab.linear_regression.create(train_data, target = 'price', features = example_features, \n",
    "                                                  validation_set = None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extracting the regression weights (coefficients) as an SFrame as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------------+-------+----------------+---------------+\n",
      "|     name    | index |     value      |     stderr    |\n",
      "+-------------+-------+----------------+---------------+\n",
      "| (intercept) |  None | 87910.0724924  |  7873.3381434 |\n",
      "| sqft_living |  None | 315.403440552  | 3.45570032585 |\n",
      "|   bedrooms  |  None | -65080.2155528 | 2717.45685442 |\n",
      "|  bathrooms  |  None | 6944.02019265  | 3923.11493144 |\n",
      "+-------------+-------+----------------+---------------+\n",
      "[4 rows x 4 columns]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "example_weight_summary = example_model.get(\"coefficients\")\n",
    "print example_weight_summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Making Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "271789.505878\n"
     ]
    }
   ],
   "source": [
    "example_predictions = example_model.predict(train_data)\n",
    "print example_predictions[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compute RSS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RSS function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_residual_sum_of_squares(model, data, outcome):\n",
    "    # First get the predictions\n",
    "    predictions = model.predict(data)\n",
    "\n",
    "    # Then compute the residuals/errors\n",
    "    residuals = outcome - predictions\n",
    "\n",
    "    # Then square and add them up\n",
    "    residuals_squared = residuals*residuals\n",
    "    RSS = residuals_squared.sum()\n",
    "\n",
    "    return(RSS)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run example model on test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RSS for example model: 2.7376153833e+14\n"
     ]
    }
   ],
   "source": [
    "rss_example_train = get_residual_sum_of_squares(example_model, test_data, test_data['price'])\n",
    "print \"The RSS for example model: \" +str(rss_example_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create new features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transformations of existing features e.g. the log of the squarefeet or even \"interaction\" features such as the product of bedrooms and bathrooms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from math import log"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* bedrooms_squared = bedrooms\\*bedrooms\n",
    "* bed_bath_rooms = bedrooms\\*bathrooms\n",
    "* log_sqft_living = log(sqft_living)\n",
    "* lat_plus_long = lat + long "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create the bedroom squared new feature in both TEST and TRAIN data\n",
    "train_data['bedrooms_squared'] = train_data['bedrooms'].apply(lambda x: x**2)\n",
    "test_data['bedrooms_squared'] = test_data['bedrooms'].apply(lambda x: x**2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# create the bathroom bedroom new feature in both TEST and TRAIN data\n",
    "train_data['bed_bath_rooms'] = train_data.apply(lambda x: x['bedrooms']*x['bathrooms'])\n",
    "test_data['bed_bath_rooms'] = test_data.apply(lambda x: x['bedrooms']*x['bathrooms'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# create the log of sqft_living new feature in both TEST and TRAIN data\n",
    "train_data['log_sqft_living'] = train_data['sqft_living'].apply(lambda x: log(x))\n",
    "test_data['log_sqft_living'] = test_data['sqft_living'].apply(lambda x: log(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create the lat long new feature in both TEST and TRAIN data\n",
    "train_data['lat_plus_long'] = train_data.apply(lambda x: x['lat'] + x['long'])\n",
    "test_data['lat_plus_long'] = test_data.apply(lambda x: x['lat'] + x['long'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Squaring bedrooms will increase the separation between not many bedrooms (e.g. 1) and lots of bedrooms (e.g. 4) since 1^2 = 1 but 4^2 = 16. Consequently this feature will mostly affect houses with many bedrooms.\n",
    "* bedrooms times bathrooms is an \"interaction\" feature. It is large when *both* of them are large.\n",
    "* Taking the log of squarefeet has the effect of bringing large values closer together and spreading out small values.\n",
    "* Adding latitude to longitude is totally non-sensical but we will see why later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">id</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">date</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">price</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">bedrooms</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">bathrooms</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_living</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_lot</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">floors</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">waterfront</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7129300520</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2014-10-13 00:00:00+00:00</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">221900.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1180.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5650</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "    </tr>\n",
       "</table>\n",
       "<table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">view</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">condition</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">grade</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_above</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_basement</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">yr_built</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">yr_renovated</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">zipcode</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">lat</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1180</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1955</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">98178</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">47.51123398</td>\n",
       "    </tr>\n",
       "</table>\n",
       "<table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">long</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_living15</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sqft_lot15</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">bedrooms_squared</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">bed_bath_rooms</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">log_sqft_living</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">lat_plus_long</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-122.25677536</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1340.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5650.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7.07326971746</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-74.74554138</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[1 rows x 25 columns]<br/>\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tid\tstr\n",
       "\tdate\tdatetime\n",
       "\tprice\tfloat\n",
       "\tbedrooms\tfloat\n",
       "\tbathrooms\tfloat\n",
       "\tsqft_living\tfloat\n",
       "\tsqft_lot\tint\n",
       "\tfloors\tstr\n",
       "\twaterfront\tint\n",
       "\tview\tint\n",
       "\tcondition\tint\n",
       "\tgrade\tint\n",
       "\tsqft_above\tint\n",
       "\tsqft_basement\tint\n",
       "\tyr_built\tint\n",
       "\tyr_renovated\tint\n",
       "\tzipcode\tstr\n",
       "\tlat\tfloat\n",
       "\tlong\tfloat\n",
       "\tsqft_living15\tfloat\n",
       "\tsqft_lot15\tfloat\n",
       "\tbedrooms_squared\tfloat\n",
       "\tbed_bath_rooms\tfloat\n",
       "\tlog_sqft_living\tfloat\n",
       "\tlat_plus_long\tfloat\n",
       "\n",
       "Rows: 1\n",
       "\n",
       "Data:\n",
       "+------------+---------------------------+----------+----------+-----------+\n",
       "|     id     |            date           |  price   | bedrooms | bathrooms |\n",
       "+------------+---------------------------+----------+----------+-----------+\n",
       "| 7129300520 | 2014-10-13 00:00:00+00:00 | 221900.0 |   3.0    |    1.0    |\n",
       "+------------+---------------------------+----------+----------+-----------+\n",
       "+-------------+----------+--------+------------+------+-----------+-------+------------+\n",
       "| sqft_living | sqft_lot | floors | waterfront | view | condition | grade | sqft_above |\n",
       "+-------------+----------+--------+------------+------+-----------+-------+------------+\n",
       "|    1180.0   |   5650   |   1    |     0      |  0   |     3     |   7   |    1180    |\n",
       "+-------------+----------+--------+------------+------+-----------+-------+------------+\n",
       "+---------------+----------+--------------+---------+-------------+\n",
       "| sqft_basement | yr_built | yr_renovated | zipcode |     lat     |\n",
       "+---------------+----------+--------------+---------+-------------+\n",
       "|       0       |   1955   |      0       |  98178  | 47.51123398 |\n",
       "+---------------+----------+--------------+---------+-------------+\n",
       "+---------------+---------------+-----+\n",
       "|      long     | sqft_living15 | ... |\n",
       "+---------------+---------------+-----+\n",
       "| -122.25677536 |     1340.0    | ... |\n",
       "+---------------+---------------+-----+\n",
       "[1 rows x 25 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# checking new features\n",
    "train_data[0:1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " **What is the Mean (arithmetic average) values of 4 new features on TEST data?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12.44667770158429"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data['bedrooms_squared'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7.503901631591395"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data['bed_bath_rooms'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7.55027467964594"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data['log_sqft_living'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-74.65333497217306"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data['lat_plus_long'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning Multiple Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Model 1: squarefeet, # bedrooms, # bathrooms, latitude & longitude\n",
    "* Model 2: add bedrooms\\*bathrooms\n",
    "* Model 3: add log squarefeet, bedrooms squared, and the (nonsensical) latitude + longitude"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model_1_features = ['sqft_living', 'bedrooms', 'bathrooms', 'lat', 'long']\n",
    "model_2_features = model_1_features + ['bed_bath_rooms']\n",
    "model_3_features = model_2_features + ['bedrooms_squared', 'log_sqft_living', 'lat_plus_long']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Linear regression:</pre>"
      ],
      "text/plain": [
       "Linear regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 17384</pre>"
      ],
      "text/plain": [
       "Number of examples          : 17384"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of features          : 5</pre>"
      ],
      "text/plain": [
       "Number of features          : 5"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 5</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 5"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 6</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 6"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting Newton Method</pre>"
      ],
      "text/plain": [
       "Starting Newton Method"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 2        | 0.019049     | 4074878.213096     | 236378.596455 |</pre>"
      ],
      "text/plain": [
       "| 1         | 2        | 0.019049     | 4074878.213096     | 236378.596455 |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>SUCCESS: Optimal solution found.</pre>"
      ],
      "text/plain": [
       "SUCCESS: Optimal solution found."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre></pre>"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_1 = graphlab.linear_regression.create(train_data, target = 'price', features = model_1_features, validation_set = None)\n",
    "model_1_summary = model_1.get('coefficients')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Model 2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Linear regression:</pre>"
      ],
      "text/plain": [
       "Linear regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 17384</pre>"
      ],
      "text/plain": [
       "Number of examples          : 17384"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of features          : 6</pre>"
      ],
      "text/plain": [
       "Number of features          : 6"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 6</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 6"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 7</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 7"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting Newton Method</pre>"
      ],
      "text/plain": [
       "Starting Newton Method"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 2        | 0.019052     | 4014170.932927     | 235190.935428 |</pre>"
      ],
      "text/plain": [
       "| 1         | 2        | 0.019052     | 4014170.932927     | 235190.935428 |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>SUCCESS: Optimal solution found.</pre>"
      ],
      "text/plain": [
       "SUCCESS: Optimal solution found."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre></pre>"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_2 = graphlab.linear_regression.create(train_data, target = 'price', features = model_2_features, validation_set = None)\n",
    "model_2_summary = model_2.get('coefficients')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Linear regression:</pre>"
      ],
      "text/plain": [
       "Linear regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 17384</pre>"
      ],
      "text/plain": [
       "Number of examples          : 17384"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of features          : 9</pre>"
      ],
      "text/plain": [
       "Number of features          : 9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 9</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 10</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 10"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting Newton Method</pre>"
      ],
      "text/plain": [
       "Starting Newton Method"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 2        | 0.015042     | 3193229.177894     | 228200.043155 |</pre>"
      ],
      "text/plain": [
       "| 1         | 2        | 0.015042     | 3193229.177894     | 228200.043155 |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+--------------------+---------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+--------------------+---------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>SUCCESS: Optimal solution found.</pre>"
      ],
      "text/plain": [
       "SUCCESS: Optimal solution found."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre></pre>"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_3 = graphlab.linear_regression.create(train_data, target = 'price', features = model_3_features, validation_set = None)\n",
    "model_3_summary = model_3.get('coefficients')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Weights of each of the models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">name</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">index</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">value</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">stderr</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">(intercept)</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-56140675.7444</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1649985.42028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">sqft_living</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">310.263325778</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3.18882960408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bedrooms</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-59577.1160682</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2487.27977322</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bathrooms</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">13811.8405418</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3593.54213297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">lat</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">629865.789485</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">13120.7100323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">long</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-214790.285186</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">13284.2851607</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[6 rows x 4 columns]<br/>\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tname\tstr\n",
       "\tindex\tstr\n",
       "\tvalue\tfloat\n",
       "\tstderr\tfloat\n",
       "\n",
       "Rows: 6\n",
       "\n",
       "Data:\n",
       "+-------------+-------+----------------+---------------+\n",
       "|     name    | index |     value      |     stderr    |\n",
       "+-------------+-------+----------------+---------------+\n",
       "| (intercept) |  None | -56140675.7444 | 1649985.42028 |\n",
       "| sqft_living |  None | 310.263325778  | 3.18882960408 |\n",
       "|   bedrooms  |  None | -59577.1160682 | 2487.27977322 |\n",
       "|  bathrooms  |  None | 13811.8405418  | 3593.54213297 |\n",
       "|     lat     |  None | 629865.789485  | 13120.7100323 |\n",
       "|     long    |  None | -214790.285186 | 13284.2851607 |\n",
       "+-------------+-------+----------------+---------------+\n",
       "[6 rows x 4 columns]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_1_summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">name</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">index</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">value</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">stderr</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">(intercept)</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-54410676.1152</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1650405.16541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">sqft_living</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">304.449298057</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3.20217535637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bedrooms</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-116366.043231</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4805.54966546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bathrooms</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-77972.3305135</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7565.05991091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">lat</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">625433.834953</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">13058.3530972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">long</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-203958.60296</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">13268.1283711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bed_bath_rooms</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">26961.6249092</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1956.36561555</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[7 rows x 4 columns]<br/>\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tname\tstr\n",
       "\tindex\tstr\n",
       "\tvalue\tfloat\n",
       "\tstderr\tfloat\n",
       "\n",
       "Rows: 7\n",
       "\n",
       "Data:\n",
       "+----------------+-------+----------------+---------------+\n",
       "|      name      | index |     value      |     stderr    |\n",
       "+----------------+-------+----------------+---------------+\n",
       "|  (intercept)   |  None | -54410676.1152 | 1650405.16541 |\n",
       "|  sqft_living   |  None | 304.449298057  | 3.20217535637 |\n",
       "|    bedrooms    |  None | -116366.043231 | 4805.54966546 |\n",
       "|   bathrooms    |  None | -77972.3305135 | 7565.05991091 |\n",
       "|      lat       |  None | 625433.834953  | 13058.3530972 |\n",
       "|      long      |  None | -203958.60296  | 13268.1283711 |\n",
       "| bed_bath_rooms |  None | 26961.6249092  | 1956.36561555 |\n",
       "+----------------+-------+----------------+---------------+\n",
       "[7 rows x 4 columns]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_2_summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">name</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">index</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">value</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">stderr</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">(intercept)</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-52974974.0602</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1615194.9439</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">sqft_living</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">529.196420564</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7.69913498511</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bedrooms</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">28948.5277313</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9395.72889106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bathrooms</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">65661.207231</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">10795.3380703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">lat</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">704762.148408</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1292011141.66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">long</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-137780.01994</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1292011141.57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bed_bath_rooms</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-8478.36410518</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2858.95391257</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">bedrooms_squared</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-6072.38466067</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1494.97042777</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">log_sqft_living</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-563467.784269</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">17567.8230814</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">lat_plus_long</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-83217.1979248</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1292011141.58</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[10 rows x 4 columns]<br/>\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tname\tstr\n",
       "\tindex\tstr\n",
       "\tvalue\tfloat\n",
       "\tstderr\tfloat\n",
       "\n",
       "Rows: 10\n",
       "\n",
       "Data:\n",
       "+------------------+-------+----------------+---------------+\n",
       "|       name       | index |     value      |     stderr    |\n",
       "+------------------+-------+----------------+---------------+\n",
       "|   (intercept)    |  None | -52974974.0602 |  1615194.9439 |\n",
       "|   sqft_living    |  None | 529.196420564  | 7.69913498511 |\n",
       "|     bedrooms     |  None | 28948.5277313  | 9395.72889106 |\n",
       "|    bathrooms     |  None |  65661.207231  | 10795.3380703 |\n",
       "|       lat        |  None | 704762.148408  | 1292011141.66 |\n",
       "|       long       |  None | -137780.01994  | 1292011141.57 |\n",
       "|  bed_bath_rooms  |  None | -8478.36410518 | 2858.95391257 |\n",
       "| bedrooms_squared |  None | -6072.38466067 | 1494.97042777 |\n",
       "| log_sqft_living  |  None | -563467.784269 | 17567.8230814 |\n",
       "|  lat_plus_long   |  None | -83217.1979248 | 1292011141.58 |\n",
       "+------------------+-------+----------------+---------------+\n",
       "[10 rows x 4 columns]"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_3_summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparing multiple models\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RSS of train data using all three models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Compute the RSS on TRAINING data for each of the three models and record the values:\n",
    "rss_train_model_1 = get_residual_sum_of_squares(model_1, train_data, train_data['price'])\n",
    "rss_train_model_2 = get_residual_sum_of_squares(model_2, train_data, train_data['price'])\n",
    "rss_train_model_3 = get_residual_sum_of_squares(model_3, train_data, train_data['price'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Which model (1, 2 or 3) has lowest RSS on TRAINING Data?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RSS of train data using model 1: 9.71328233544e+14\n"
     ]
    }
   ],
   "source": [
    "print \"RSS of train data using model 1: \" + str(rss_train_model_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RSS of train data using model 2: 9.61592067856e+14\n"
     ]
    }
   ],
   "source": [
    "print \"RSS of train data using model 2: \" + str(rss_train_model_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RSS of train data using model 3: 9.05276314555e+14\n"
     ]
    }
   ],
   "source": [
    "print \"RSS of train data using model 3: \" + str(rss_train_model_3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### So RSS of the model 3 is lowest on the train data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RSS of test data using all three models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Compute the RSS on TEST data for each of the three models and record the values:\n",
    "rss_test_model_1 = get_residual_sum_of_squares(model_1, test_data, test_data['price'])\n",
    "rss_test_model_2 = get_residual_sum_of_squares(model_2, test_data, test_data['price'])\n",
    "rss_test_model_3 = get_residual_sum_of_squares(model_3, test_data, test_data['price'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Which model (1, 2 or 3) has lowest RSS on TEST Data?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RSS of test data using model 1: 2.26568089093e+14\n"
     ]
    }
   ],
   "source": [
    "print \"RSS of test data using model 1: \" + str(rss_test_model_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RSS of test data using model 2: 2.24368799994e+14\n"
     ]
    }
   ],
   "source": [
    "print \"RSS of test data using model 2: \" + str(rss_test_model_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RSS of test data using model 3: 2.51829318952e+14\n"
     ]
    }
   ],
   "source": [
    "print \"RSS of test data using model 3: \" + str(rss_test_model_3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question: Which model (1, 2 or 3) has lowest RSS on TESTING Data?** Is this what you expected? Think about the features that were added to each model from the previous."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### So RSS of the model 2 is lowest on the test data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hence model 2 is the optimal model"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}