notebooks/clinc_oos.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "410e30fd-d2cf-477e-9dc2-ee99a38e6b12",
   "metadata": {},
   "source": [
    "# Perform a data-driven analysis of CLINC-150\n",
    "\n",
    "CLINC-150 is a dataset proposed by [Larson et al.](https://paperswithcode.com/paper/an-evaluation-dataset-for-intent) in the goal of doing intent analysis with an `out-of-scope` intent. This last intent should be output by the model when no prediction can be made.\n",
    "This is particularly important in intent classification as users often make unsupported requests. For example, you could go on any chatbot and enter \"I would like some pizza\". This is a valid sentence, but this is not part of the intents that we support. Hence, we should output `oos`.\n",
    "\n",
    "Today, I would like to analyze this dataset and perform a data-driven analysis. To do so, I will use `sentence-transformers` to extract features and `spectral_metric` to perform a spectral analysis of the dataset.\n",
    "\n",
    "I will keep this tutorial short, do not hesitate to reach out if you have questions.\n",
    "\n",
    "**Note** This notebook should run as-is if you have `spectral_metric`, `seaborn` and `sentence-transformers` installed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "37f78f81-2ef1-4b0e-9622-342825928837",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Reusing dataset clinc_oos (/home/fred/.cache/huggingface/datasets/clinc_oos/small/1.0.0/abcc41d382f8137f039adc747af44714941e8196e845dfbdd8ae7a7e020e6ba1)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "09b5102bfadb42aba30290e9fbbc1d30",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "from sentence_transformers import SentenceTransformer\n",
    "\n",
    "# We perform the analysis on the train set.\n",
    "ds = load_dataset('clinc_oos', \"small\")[\"train\"]\n",
    "class_names = ds.features['intent'].names\n",
    "# I use all-MiniLM-L12-v2 as it is a good compromise between speed and performance.\n",
    "embedder = SentenceTransformer(\"all-MiniLM-L12-v2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "bf2df6a9-b059-4b60-af0b-5f048a58c107",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['text', 'intent'],\n",
       "    num_rows: 7600\n",
       "})"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Show dataset info\n",
    "display(ds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fc8049b7-edf1-48a5-9b1c-e7cf936dc05a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will get **normalized** features for the dataset using our embedder.\n",
    "import torch\n",
    "features = embedder.encode(ds['text'], device=0 if torch.cuda.is_available() else -1, normalize_embeddings=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7cfe042-d178-4782-bde1-03501b0073a9",
   "metadata": {},
   "source": [
    "### Running the spectral analysis\n",
    "\n",
    "Now that we have our embeddings extracted by our sentence embedder, we can make an in-depth analysis of these features.\n",
    "\n",
    "To do so, we will use CSG (Branchaud-Charron et al, 2019), a technique that combines Probability Product Kernels (Jebara et al, 2004) and spectral clustering to analyze a dataset without training a model. \n",
    "\n",
    "In this notebook, we won't use the actual CSG metrics, but we will use the $W$ matrix. \n",
    "This matrix is computed as:\n",
    "* Run a Probabilistic K-NN on the dataset (optionally done via Monte-Carlo)\n",
    "* Compute the average prediction per class (results in the $S$ matrix)\n",
    "* Symetrize this matrix using Bray-Curtis distance metric, a metric that was made to compare samplings from a distribution.\n",
    "\n",
    "These steps are all done by `spectral_metric.estimator.CumulativeGradientEstimator`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "09ae06d4-ac15-42d0-84e6-723c010c40c0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/data/workspace/spectral_metric/spectral_metric/lib.py:117: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
      "  return np.array(class_samples), class_indices\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<spectral_metric.estimator.CumulativeGradientEstimator at 0x7facb8a22550>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's compute the W matrix from Branchaud et al. and try to find overlapping classes.\n",
    "from spectral_metric.estimator import CumulativeGradientEstimator\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from spectral_metric.visualize import make_graph\n",
    "\n",
    "X, y = features, np.array(ds['intent'])  # Your dataset with shape [N, ?], [N]\n",
    "estimator = CumulativeGradientEstimator(M_sample=250, k_nearest=5, distance=\"cosine\")\n",
    "estimator.fit(data=X, target=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6898788c-0695-479b-8bea-bca3e22d4e2c",
   "metadata": {},
   "source": [
    "### Visualize the $W$ matrix\n",
    "\n",
    "Now that we have a simitrized similarity matrix, we can run some analysis on it.\n",
    "Here's what we will do:\n",
    "\n",
    "1. Plot the matrix\n",
    "2. Find the intent with the highest confusion (entropy)\n",
    "3. Find the most similar pairs of classes.\n",
    "4. Show the points in the dataset that contribute to this confusion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b5c41828-91a4-44b0-8197-eff9c3f4b512",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAFICAYAAACLGl7PAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAACN7klEQVR4nO29d5wcR5n//3l2pZVW0mqVs4ScjTG2AdkmnS1jDgzHYYIJNnC2MQg40h1whOMOfIQj/uDgCPc1YAyHydiHMQbbGAfAWLacc5aVc9pVWkn7/P7onp5nRv1MV011z/RIz1uveam2p7q6uqq6p+qpJxAzwzAMwzAMoyx0tbsChmEYhmEYEpucGIZhGIZRKmxyYhiGYRhGqbDJiWEYhmEYpcImJ4ZhGIZhlAqbnBiGYRiGUSpscmI0BRG9iYiuafLcvyGih8XfS4noxQF1GSSiQ5s9v66soLqUnTzbyjAMoyhscmKoENELiehmItpKRJuI6C9EdCIAMPOlzPySZspl5j8x81F51ZOZxzHzE3GdLyGiz+RVtg+dMLGRbZUFETERHZ7HdYloIRGtyKOsUIjoSCL6BRFtiMf2PUT0ASLqJqL58X2PSDnvQiL6kfibieheIuoSxz5DRJeIv3vi8x4lou3xGLmYiOYrdZtJRFcQ0aq4/Pl1319CREPxJLPy6Rbfn05EDxHRDiK6noieFtRYhtEmbHJipEJE4wFcCeC/AUwCMBvAfwDY3c56SdJ+QAyjEUR0GIDFAJYDeCYz9wN4HYAFAPqaKHIWgDc2+P6XAF4J4BwA/QCOB3A7gNOV/MMAfg/gtQ3K/GI8yax89gEAEU0BcBmAf0f0zC4B8DOPezGM8sDM9rHPfh9EL+stDb4/D8Cfxd8M4B8BPApgAMCnARwG4GYA2wD8HEBPnHchgBXi3KUAXhynTwLwVwBbAKwG8I3KeeI6746v86Q4djiARQD2ABgCMAjgNwD+BcCv6ur+dQBfU+5rKYCPAXgAwGYA3wcwWnz/CgB3xfW7GcBx8fH/RfTDsjO+9ocB/ADAB+PvZ1fqHv99GIBNALoalRt/NwvArwCsB/AkgPeJ7y6M2/aHcbvfD2BBg35jAIfH6UsAfBPAb+NzFwM4LP7upjjv9vh+3uBQz6UAPgTgHgBbEf0wjgYwNm6X4biswfieTkL0A7oNwFoAX2lQ77cDeCxusysAzKq7p3fGY2JLfE+klPMjAL9tcJ35cXkjUr67EMCP6q77kfi6I+JjnwFwSZx+cXzfc5t4/kbE5c+vO34JgM8o5ywCcLP4u9LuR7f7fWIf+/h+2l4B+5TzA2A8gI2IfmBfBmBi3ffnYf/Jya/j856BSMJyHYBDEa0YHwBwbpx3IfTJyXMAPDd+Oc8H8CCAf6q7zrWIVoa94pj8wf2MyD8T0Q/shPjvEQDWAXiOct9LAdwHYG58jb9UygPwrPjckwF0Azg3zj+q/j7iv98K4Ddx+hwAjwP4mfju11nlIpJu3g7gEwB64vZ8AsBL43MvBLALwMvjcz8H4JYG/VrfVhsRTRJGALgUwE/T8nrc/62IJh6T4r57Z1qfx8f+CuAtcXocgOcqdX4RgA0Anh23yX8DuKmunlcCmABgHqJJ3BlKWWsAnN+gfebDb3JyRNw/b4uPycnJ5wHc2OTz12hysin+3A7gteK7rwH4dl3++2Qe+9inUz62rWOkwszbALwQ0QvyOwDWx3vh0xuc9kVm3sbM9yN6KV7DzE8w81YAv0P045Z13duZ+RZm3svMSwH8PwCn1mX7HDNvYuadDuWtRiQFeF186AwAG5j59ganfYOZlzPzJgCfBXB2fHwRgP/HzIuZeR8z/wDRJOy5Sjk3AnhhrJNwCoAvAnhB/N2p8fdZ5Z4IYCozf4qZhzjSF/kOarcS/szMV3Ek3v9fRFsHrlzOzLcy815Ek5MTGuR1uf+vM/OquO1+k1HeHgCHE9EUZh5k5luUfG8CcDEz38HMuxFJtp5Xp4/xeWbewszLAFzf4LqTEUnk8oIRbaP8OxH1FHwtIJL6HQFgWnzdS4ioMqbGIZJYSbaiue0qw2grNjkxVJj5QWY+j5nnADgW0Yr4vxqcslakd6b8PS7rmrGy4pVEtIaItgH4TwBT6rItd6m/4AcA3hyn34zoB7wRsvynEN03ADwNwAeJaEvlg0jCMgspMPPjiKQ2JwD4G0Sr+1VEdBRqJyeNyn0agFl13/0rADlJXCPSOwCM9tDHqT+3UR+53L9PeRcAOBLAQ0R0GxG9Qsk3C1E/AACYeRCRxGd2E9fdiEialhvMfBWAFQDe4XOt2GqtotR6v+O17mDmjfHk/SpEE8rXxF8PIpJcSsYj2rIzjI7CJieGE8z8ECKR8rEFX+rbAB4CcAQzj0f0Q0z11Wlwftp3/wfgOCI6FpHOxKUZdZgr0vMArIrTywF8lpkniM8YZv5Jg2vfCOAsRHozK+O/zwUwEZHuRla5yxHp1sjv+pj55Rn3UARZ99+I/dqGmR9l5rMRSQG+AOCXRDQ25dxViCZGAIA4z2QAK5u4hz+gsbJps3wc0VgdU3etk4hoTtoJHFmtVZRan9HkdRnV5+N+CKlZ3E6HxccNo6OwyYmRChEdTUQfrLxYiWguou0NTfSeF32IFCQHiehoAO/yPH8tIr2MBGbehchq4scAbo1F/414NxHNIaJJiH50KhYP3wHwTiI6mSLGEtHfEVFFbL7ftRFNRt6DaGsJAG6I//5zvA2TVe6tAAaI6CNE1Bubux5bMekumPr7ybr/rLImE1F/5QARvZmIpjLzMCJFViBSmq3nJwDOJ6ITiGgUImna4njbz5dPAng+EX2JiGbE9TiciH5ERBNEvlFENFp8Gr4rmfkGRFuZ54pjf0CkH3U5ET2HiEYQUR8RvZOI3qqVRUSjEenWJPUQ351FROOIqIuIXoJIEnhF/PXlAI4lotfG53wCwD3xwsIwOgqbnBgaA4gUHxcT0XZEk5L7AHyw4Ot+CJHy6ACiH0NfU8jvATgm3nb4P3H8BwCeiewtHSCaxFyDSPH0cURKjmDmJYisRr6ByJLnMUSKwRU+B+Df4mt/KD52I6IJV2Vy8mdEq+vK3w3LjScwr0C0NfQkIsXQ7yJSMi6aCwH8IL6f1zvcv0r8A/kTAE/E5c1CpP9zPxENIlLmfGOaHlH8I//viCyWViOSBjQy321Uj8cBPA+R4uv9RLQ1LncJarc/BhFtRVY+L3Io/t8QKQJLzgJwFaJxvBXRM7QAkVRFo2LxBURSRNkm70ckMdoC4EsA3h5PjMDM6xFJhT6LqH9ORpPtZBjthpgbScgN48CAiOYhetHPiJV9DcMwjJJikhPjgCcWyX8AkZmsTUwMwzBKTmGTEyI6g4geJqLHiOijRV3HMBoRKwVuA/C3iPQNDMMwjJygKBzDOiK6T/meiOjr8VzgHiJ6tku5hUxOKIr18E1EzruOAXA2ER1TxLUMoxHMvL1iDcHMvibIhmEYRmMuQaQ/pvEyRL55jkDkK+nbLoUWJTk5CcBjsQOuIQA/BXBmQdcyDMMwDKMNMPNNiDwWa5wJ4IcccQuACUSU6WuoqMnJbNQ6slqBWodJhmEYhmEc+DQ1HyhFVNc9G55ITIYmP60acX7HnuwAuF1U9c81rFgeyTyH9lcnbI9vWZWkNZsl6f1LyzN+VNXv0rbdO1LPzSpDMm/8tGp5Q9uT9NC+vanpvcP7kEZ3V3XuuW84zX2EG2n3Aej3MqIrieCOYR4W6eiM6WMnJMfW79i63/f1zO6bnKRXDmxMvc4+0QYubSzHRBfJdkovx2UcyDy9I0cl6axxrPXTlDFVZ5+bdlatXLV2koztSVxjoKe7+phv3jmYlr2mPaQF35y+qnPe5QMbMq/rguz/wT27kvSOoSgdYj/oMuZd3hlaf8vjx0+uuoC5a+MTSVr22+BQ9f527R1K0nLsas9v2nM3rqc3SQ8M7WyYFwC6Ha6TF3LMbRf3LZH11OqW1T8TRlf99O0Ubbp7757MOrr0vQtzxXOxcrD6TtLKlGNiw46qTv7eoZVa1xWC/K11pWfqYe9AtB1T4SJmvii/WqVTlORkJWq9bM5BnTdHIlpEREuIaMl3f+jiYNIwDMMwjKYZ3uf9YeaLmHmB+PhOTDLnA2kU4uckjuvxCIDT40rcBuCcOCDcfowfe2hSiY1PVX0T9c76m+xribTLilYjpBVcVgw+uEgqXFYAeUlO8ka2l6yXXFlKtFWHS99ryPabOLoahmXjznzCkPis0LS8vhIrmb9L9P30MROS9KrBRlvDrUHW8zlTjkjSSzY82vrKBKA9X38zrar7/5f1DybpkJV6hVfOfE6SvmJ1o9iVrUeTIEtk31OTUoynja+GlXr6mKok/Pdr7so8N693Yp+QYA0KCZaLNFlKiVouOVn7sPcgHDn9qMw6UhSE80pm3i+8CRH9HSKv2C9H5Bjw68x8UmaZRTlhI6KXIwoS140oouhntbwjemanVmLnqj8laZeJSidw1MRqmI2HN69oY03ccRFBG+0hLzF1WQmZgB4olHWREYpv31byH0jjoOWTk9UP+k9OZj69YR2J6CcAFiIK0LoWkcuGkQDAzP9D0Sz0G4gsenYAOD/2Nt2QwnRO4oiZVxVVvmEYhmEY7jDnP7mNg3c2+p4BvNu33FIoxGpIaUmIFMVFbN6q2XinSEskmrSk2VX7qBEjk7SLEpskD7HwgYSvQmen0cl1B8L6oSIxKZu0JOT5lWjtoUmKKvnzur7GAS2NLNlYakSpJyeGYRiGYeREAZKToihschKHH/8ugGMRTXrfysx/bbY8TYoy57CXJ2lNmTFEWqLN4n3KCVk9+SpF+jLGweRVq7/LqiLt3PE9VcW59Xu3Igvt+iH6Ui6mor6mxJJmpXXfnnZakn7Xuuubvv4IYT68R5id+zK5ty9Jy+fLZXXpoquU9ypVa6fRI3qStKZ47VKOJEQXJG89Eu09McrzviVa30hpRchzIU345fjIao8hT2mJHIcsaqldx2Ucyv4bTpHu1DNj3MQkvWZwc2b5hdFBOoNFSk6+BuD3zHwWEfUgChNvGIZhGEY7ONglJ0TUD+AUAOcBQOzCXp22aw6gtFmolJaseLyqc6vpomh6ClJqoJkAa7NrF7NeeTw5z3N1KFfAe8UKeKRy3EfrHXBzdJeXtKfChJ6q6a50wqbtJZMyPkLQ2lVeS664pPRBG6+U0t8uyFXYBzffnJrHt18lmgRDW/XK+xtykLpo13Wx7PLpz4m91XGzRTiTcylhz3D2fWirYQ1S7lwz6ZbvEhdpSVr5qhRRKSPtHeSKiwRBky5qeWqPN6cvVfOcKXWslfBW21q+N7U+cJGeyr5xqo84LsdZy+kgnZOiWukQAOsBfJ+I7iSi78bRYQ3DMAzDaAPMw96fdlGUE7YFAG4B8AJmXkxEXwOwjZn/PS2/5ufEl7z8ojRrf++SvxM1wfOuc9FtcNzkQ5L0PRufzKXMdlu/aFKlEBf+vpi/G52Q8ZHVrp34ziiCitTDV4eqaL29EFrt52T347d43/aow57b0jpWKErnZAWAFcy8OP77lwA+KjMQ0SLE/vqpux9dXeGClbxMj317zyd/ES8XF8+MIUzurXpoldswvlRewjUmwAHKmhL5Ar83pwmJpIitJR+0F7IUWeelBKvRjgmJttWimZEXbWaqETIisrbaXLYNiqaIyblLmdoWqk/ZMoaPxGU8t3tRkjsH+7YOM68BsJyIjooPnQ7ggbo8ib/+PCYmhmEYhmE0gIf9P22iSGud9wK4NLbUeQLA+VpGF4VYX9NMFylKXrPiIsWuLmUPeEpLfMXzUloS0maVa8nrh7Sdi5m3hqYoqNWhWbNpF0LaQOZnIS2RbSOfKZfI3e12VChRFdKV+3CRlhR9TyHly/yV+C2DSvRhlxhLed1fiNuFkDKHa8Zu9P5weXZl2ZpkSiPvWGlAiUIQdNB2bJHu6+8CsKCo8g3DMAzD8OBgNyX25dD+amTJx7esSs2jreZcdAHy0kXRcHEhXsFltfD6mdWAjZevuyNJd4nrhCg/hugO5K0g7CIp0Fb1LmbeGnLcuJl1ZpfvIulLQxvbz/RU7JVXkfdUY7roKRmaIMx3Nwvz3RDOm/W8JP2/axYn6VatKEOkCbJ/XCIqy+i1A0IC4rKSlvnTrq/R7DgMxcX8WtKshMfF3YSGi16Wi7REU0TvLrvSeAfpnJRicmIYhmEYRsF0kOQkyJSYiC4G8AoA65j52PjYlwD8PSKna48jCo+8pVE5IxVTYhf34S77g9q5UooydvYpSVpzpKa1lZwtyxn17thdtOaIyWVVoznskU6AQmbovg7wfDXs0+5LWhdJfRmXa0pCrFMmjK4qYUsLpxorEMUttVYfea6PZKbWMVS1L311P3wtFORYlVY/2jh3ua7miM5X1yUNX2dyGi5OtqDkcamP9sxKd+1aP6fh4tTNRQ+jSxmfmsTD14JGq5uGixPAtLYJ0afqVRxvukiyXFzWu4xR+Wzu2rWstabE91ztb0p83EvbYkocaq1zCYAz6o5dC+BYZj4OwCMAPhZ4DcMwDMMwAmHe5/1pF0HbOsx8ExHNrzt2jfjzFgBnZZbjci3luMv+oHaulJZsX3lTktZ0UbRVllwFpc2WtZWJyx6wnMXLIGxbc/Jn4rsP7aLjk1XmThGALKTvQ3x5yP18Wd8ThR7BbesfSdKyvzWJTY2Lag+34S734dtOchxK3x/y+PcnVcf5Wzbc4HCF7OuO6q5eyyXQnM/4y2sPXxvDLitgl/qoAeUgJB0OekBp9dJX9dX6jhRpGZ5CSuhCfMBo0hhfnROX8lO/dwi0p5XXpcjHXCQ9b5pxcpL+4ar0GLbamNCezZZzsGzrAEA8Obmysq1T991vAPyMmX/UqIy8PMTmZUJXhNJshXabZhrNUVbvqKWJdpoDU8ZEzv427NiW+r09O0YoZXuOW+0hdtcdV3g/OqOf/coDykMsiOjjAPYCuFT5PncPsYZhGIZhKHSQ5KSoqMTnIVKUPZ0V0QwzXwTgIqBWITbE2Y/vlFBTItNMj8fNOTVJa1Eps0T0Lkq+GlIsKxUYXaJrThnTn6SlUzWtTE10GuIevFKffqGMumXXdq8yJCMDFGK1+9CUU11WWbLMnq5qOWkmoRKXftVWfGuFtERb3miKijIq954aBc306w6JdtLGq9x63LhzQMkl6iaeHU1iknVN3/hCLuNGMwH2RVOOnT5mQpJeNbipYRku7wkpQZPtvkeJZO7yvEx0MCOvSLsAYKPoP5f32egRPUnaZQswC62dZB/0jar2q69pvMu7b974aUl62bZ1qXVoR/iLhBJIi1zJfXJCRGcA+DCAU5k5/0AvhmEYhmH400GSk1BT4p8AWAhgCoC1AD6JyDpnFICNcbZbmPmdjcrJS+fEF5cViZzxDq64MUnnoYtStIMkWXcX0+M8zIRdaKVjqHa5je4E/Ygi6phHe5dNLyAvNClKHuNSSsGkEqwLIRLIThjnkqLH1uy+yUl65cDG1DzPnDQ/Sd+55i+t1Tm55Wf+OifPfUPn6Zww89kph78XUqZhGIZhGAXQQZKTjvQQ28rw3VLioOmijBHHferjKzXwlTj4mty56MPkEb5dOqMaLtiO3ndV6uKgqxNWiBJ9Lz59FRkUiDAHKYDvita3b1ykOyHSBI0iJXc7FZPhPUowSFmXkPtzcS2g0UoJagUOeHpdxpkmLZGOJ+/dtLTpOgTTQe7rg5ywEdHFRLSOiO4Tx04goluI6C4iWkJEJzUqwzAMwzAMQxIqObkEwDcA/FAc+yKA/2Dm3xHRy+O/FzYqRM4q9wmxk+ZgTXN1rDlU0mbo3YqWv0Rzxy2lJTtykKJoyLqPExYEQ9L5l7iS1CJ3WdVobSPPlXmkk60dw+l723LlJq9baeNDxs9IjnULKcpDm5enlqeFKJjbNyVJr9petXhwkRjJMbd7X7XNap3oSRf02at5OVJGeVgiaFYAcqU7cXTVckJatcg8sm9k//WI/pAWCvsUV/ayDWRf7q0Zc8jMI9HuUVps7BneG9er+dWdJh2oWfU6hAXQ7kNz1S+fHTm2pMNBX8uZNGtCOQ6kVU5tXar3Ldt9qEnrOsDtXfasKYcl6bs2PJ6k5XtrUFg+Te4Vlj47haWP8h6vtNl4Ud52ITHarTh3rHH/L943TNmu77XQAWNHVt9Jg4o1V02QUk4fiy2ngyQnuTthI6KrAVzMzD8jorMB/D0zn9OojHYpxOZlyiYH244CHbgdKHTyFolhGEZe77BWO2HbedMl3tXtPeW8zlOIVfgnAFcT0ZcRbRs9v4BrGIZhGIbhQwdJToqYnLwLwD8z86+I6PWIrHde3OiEkFmor1KVvJYUA4bUQeZPU5p9xbPenRy7bu09nqVXkZF0pRg+y3lVPSH36hsdNy1PSDThvNCkZtIkc6diktmjbFFo52a1cdFKuNpWhxalVuIS9dVlq0PL3yrJWV6RiyUu755D+2cm6Se2rna4cnNo9S1CsVcSotTabFRiX3yjTWtoDho1w4Ai76lpOshaJzQqcRrnArgsTv8CQKpCLBEtihVmlwwPN+8l1DAMwzAMB4aH/T9togjJySoApwK4AcCLADyalklzX+9CXhtgUsEpr4iakorE5Mo7v5kcc9FD0e5Prkr35TQD9nWaFWI6GHpensjIrVLteo+i+FozVjSJgGybsOrtX15Am0klwH3I7uOQZ6FW4VxRTlXM0YscFXm1pcTFJF6OM18JUx4MF7xSHnZQLtZo9q6DJL85mT7vUc7VFMvbKi2RdJDkJGhyIj3EEtEKRB5i3w7ga0Q0AtF7f1FoJQ3DMAzDCKSDdE6CrXXyQLPWOWrinCT98OYVmeXkta9X5J74zgBrHhfdBCh5Qsh737oM1jq+bSlpR51dgpr5Ot3Lqw5FUrR+VBnIem/lHSAvT1rZPxWzaN+AoxJvSbFIFzGGWm6tc/U3/K11XvqeA8ZaxzAMwzCMstFBkpNST05cpCWSvNzBayvpPPaJNRf4LlIU7YpFu/DPW8u/iP3/vHCxQGoHWr1CVn+t0hEIIeSa5RpZwDGT5iXpBzctS9JZ7xIXaUkRVjMutLKNQyQmFbTnRWuDIqRBac71WsbBMDkhormIPMNOR9QPFzHz18T3HwTwZQBTmXlDaEWLwOUhrhmoOf+QhkxUiqBVL5oyKIeV+SWcN0VPajuNdm33PCAmJHnj8ky5LMZaSTuuq22R5VUXl3LMlNiNEMnJXgAfZOY7iKgPwO1EdC0zPxBPXF4CoLin0TAMwzAMdw4GyQkzrwawOk4PENGDAGYDeADAVwF8GMCvXcqaN35akl6+bV31Gs1WrgGvn1l1u/Lz1bcWcIX9cVmpaVKU8XNPq56bEqemUZntIut+Qxzn5XWvRSvG5aGcfdzkQ5L0PRufbKoMwP9eJRN7q7FcZFyeEN4nxvo31/wlSRcZtVcSMobkOHjWlMOT9B0bHkvNI+PKDIgYLM32SYgzuaJX7K2SSIVcJy8XDNrzrZkSl0Y5+yCRnCTE8XWeBWAxEZ0JYCUz3615nTQMwzAMo8UcDJKTCkQ0DsCvEMXU2QvgXxFt6TizbcjPQ6yc8ri4Qpez3MvX3ZGkQ1aUWvl5rE6ktGTb8uuT9LT51WaVkTDbsapvhFwt7thbdeNeaWP5/bbdO5q+ToiiJykqrrkpximOxtLK19zI37/5Kb9rivT0cROT9NrBzV7lSI4eVzXn/+vOh5K0U/Rfpcxvrbm5msdj/Lk8r3k90y4s37E+M48c/xKXulX0I4KCkpZY+bwd7uu1aNOSEMnuXsWJo0+U8kLpIMlJkPt6IhqJaGJyKTNfBuAwAIcAuJuIlgKYA+AOIpqRcm7ivn7X0NaQahiGYRiGkcXB4L6eoin59wA8yMxfAQBmvhfANJFnKYAFadY60n39+LGH8lA8o/WdJ7vMhOXst0uZCU/u7UvSG3cOJGnNAZmcXWuSgEqgPrl62SL27VUrClFHKS1Zt/SaJP38485L0veJFbZ0V6055ZLlu6wS5MqgW+ypanovw+KvNFfoPaJNnUzvlNVwt7K/6zKGpo+dkKRXDWxM0q8TOklXbri7el3RruNGjk7ScqzIVbu0Ctg+VHWQn1Y3eR9yjI0SAQZdVtpSirhlV1UaOU3c69rtW5L07L7JSVqO251iZXff1nTpjazP2J5qe8ixtUMJnNgs2spfjiEXKY7LmHdxPLh9z67U47I/+0eNSdKbdw2m5tHKr6yw5TgYUsxpZX21+5PvPm2FL3F5NuV15bvHNxRGTZ0zzH2194FLfScp73mJpkPysmnHJ+mr1t2Vml+zBnJpb6OWEMnJCwC8BcCLiOiu+PPynOplGIZhGEaeHAySE2b+MzJ8UzHzfJeyhhSJhLaqkUd9XavL1b5cvWxVdB+0wFmybkPKdSuzbrnqdlnVyzpK3RIpLbn5nkuStOYXRdt/D3HXPuxwL1l7qr5SDm3FzAF671JSIEu5aaAap3L3Pj+nT+y5Mq3QLQLIySv6rrbk+N8nniNpJSLZsHNb6rWkPs4uhzbYKSQkLkrwzQajc3kfaMd99Qg0iaw8c6ciGZL3J4MDyvHh0wZSWqJJDWR5e/ZV05r1iAu+z6aLTkuNdEM5V9OTSSvd1+Rip6e+h2yzm7dW3w2s6JFp1kCyf+R7ueWUTO+oEaXwEBsi8tJeIuOFOHVA+SFyua6LeFL7kdywY9t+x1xEj9rkQW7faKbH4+acmqRdvCG6vKgr21NA7XaBdi/adSs/ENpkTkP9UVK2p7JeaIAefVgqj54w5bAkfc+mqimvizjYx5ulNpmT7aj9uMo+WDD1yCS9ZP0jSVr7EXWpo8vLv6Z/HMaTj5Kj9iLXtiyl6bPcQvVVrHT6YVaOy2ut35GuT5f1jADVcXzKtGckx25cd3/6NZXy8tpOcHpveTq1lNtVcixq5VQWonIS7hsjy0UZVZ77ihnPTtK/WXPH/pnr0BbItRPTNk4QDiZrHcMwDMMwOoAOmpwERSUmotEAbgIwCtFE55fM/EkiOgTATwFMBnA7gLcwszpl1aISH4iEOOPRtmnk8cEVNybpvNzg5xWbpUy4mMJqIvR24xKhuEz1zYsQB2SdMj7T+rDM/drKNq5cq5V9WXTbtzwq8Y8+7t18vW/+bFscloVufu0G8CJmPh7ACQDOIKLnAvgCgK8y8+EANgO4IPA6hmEYhmGEUJBCLBGdQUQPE9FjRPTRlO/nEdH1RHQnEd3jYjwTtK3DkdilsrE7Mv4wgBcBOCc+/gMAFwL4tlaOtq9chlWCtpKWe7w+0YpDZv3DirREtlMRwQTzlpZIc7vdDsG3QlYvmkKgps8hFRhD9usXTj82Sd+w9r6myvB1CKe1jba3X4REzFfx1Ce/b110B3z55NGQY65GgdxBFyqtD+WxItyjh4yDHjG2pL6Fi1sCiY+Oj9a+LuPNV0HYxdmfpv8kKdrxpTMFXJuIugF8E8DfAlgB4DYiuoKZHxDZ/g3Az5n520R0DICrAMxvVG6w2jARdRPRXQDWAbgWwOMAtjBzZaSuQBRzxzAMwzCMdlGM5OQkAI8x8xOx+sZPAZxZl4cBjI/T/QBWZRUarBDLzPsAnEBEEwBcDuBo3zLKICHRmDKmP0lLJ0qaU7MicbGIkfhKUVq1ty2tm9xMq5uvi7Y60i2W8rnvP62vLhqaXS3mJc3wsRwKvW7vyFFJWjqf08hjFek7bkMscVzQxpxcPf/P1IVJ+h3rrk/Nn6ZjoZUtnYvJ95RL+2o5jp00P0nfu2lpap4Jo6qWfNLBn8u1mjVz1vJq5cnx0Teq6jDTN5Cl5uxPIh0brhTOHTXT45bTxLuUiBYBWCQOXRQ7Ua0wG8By8fcKACfXFXMhgGuI6L0AxgJ4cdZ1c7PWYeYtRHQ9gOcBmEBEI2LpyRwAK+vzyxum7n50dY2tz2IYhmEYRl404WNIenMP4GwAlzDz/0dEzwPwv0R0LLNeoaDJCRFNBbAnnpj0Itpz+gKA6wGchUi8cy6AX9efK284xFrHZWUX4ohI81PgQtoMOWQPX9uzdmkDFylKiIRijFgxZ7kt14Lu5UWIlCHIek2k85Y8uehGaLi4+c/L6sJFWuKDS73aJXn11V2RUgxNWiKpBItz0cvSfO+EoElLJC7SEg3fd3HW+1QrT/q82uIgLdEce7o8F1JaIimL5R8PFyLlXwlgrvg7TSBxAYAzAICZ/xpb+k5BpA6SSqjkZCaAH8QKMV2IFF6uJKIHAPyUiD4D4E5EMXhyw+eHEHATs4bE2NDKqaRCtktqYmPI6LVKHheKUJr1iaMy5Ol5VSOr3RsxRsSD0X5QQ5TYfM7tUryqhojkaydK2VtbmldOl6jfEnkvcqshzSEhkP1s5KXc6YvTxMMh4q+mRO8SpbZy3G0CqnmiTd9OcGkblzE8wiHWVkj5kmb7c2B3uodkDd/3vET1yl0Wz6zFTIxuA3BE7EJkJYA3omoQU2EZgNMBXEJETwcwGkDDsN6h1jr3AHhWyvEnECnJGIZhGIZRBpoMHdGwSOa9RPQeAFcD6AZwMTPfT0SfArCEma8A8EEA3yGif0Y0zzyPM2ZsQU7Y8qJdTthcVhK+JmN5Xz8vXKQ3LlKUPOocss2WF53moKuI+nZaG2iMdZCClQEX6UYe5OWszuWd0QljSL5vpDJ+EdsrLm0mJTBDu1e0VD92xzff491NY979jbbo8Jr7esMwDMM4GCixZWw9TU9OGriuJwCfAfA6APsAfJuZv55HZfPGZQqpSUu0/XqfVVArVxp5mR6XdXXkS6fdR6fVt5UUIS0pQiLQKkVIX8dy7VI0bpUEVXNWVwQubWaB/9wIkZxUXNcPEtFIAH8mot8BeDoizd2jmXmYiKblUVHDMAzDMA4Omp6cNHBd/y4A51Tsl5lZNRWqoM3ofV01y728yb3jk3SIObA2x9Wum+Ue2MX6R0O77wmjqz5itu7anprHlyIseir1HBL3rfWri+mdzC/daEs0B2QuKyjfOmhjIo+VkmahJkMByP102a6a9K8Ii4oQ0pyOaa7KpbRS9o18vvZ6moFKfHU1fM/VkGav23bvAFDbx5plT17j0FtfRXn3uUiZSbFS06i0g2wDl/uW1xnZVR0fWltKXPSZXNr4QHZfXxRB7uvrXdcz82IAhwF4AxEtIaLfEdEROdTTMAzDMIwQCgr8VwShpsQ1ruuJ6FhEOii7mHkBEb0GwMUAmlpua/uQ2txPzkiltMQlsJWmNzJKCWwl84/qrubZMVxd1VJKXl9piaRbWUVuEdISScj+qmynvKQolXpO7B2XHNstfJ7IFbDL3q1sV18X7Zr/Cen/ZIeyUnIZTz64rFZ3Kn5ktNWfXKn1KlIXKWVQA9R5rrTyshSpoOoiKP5/XJ4vF12HvFzia04UNcd4FWmJRDph0+re5RA81Ve64tsGmqSxRlonyp87turq/altazPLr4x1rU0lNfch83vKtaS0RGt73zbTAt22hGKcsBVCLtY6wnX9GYj86l8Wf3U5gO+nnePrvj5ELOayNaMNcvVHT+SXE5K08vMy19acHBUhXtbIY6Iit57aJeLUtjrky8i3XV3Gk095vnk0x3yag7yQibJGEdsnFbQfV98xFBKNVuKkUK/UzaUOaeNPOy+kvlodtTInC+d6NZ5pPftBm5BoUbSrl0nfPtLwXfRIZB+EjJuaNm7rtk7nKMQ2PYUjoqmxxATCdf1DAP4PwGlxtlMBPJJ2PjNfxMwLmHmBxdUxDMMwjIIZZv9PmwiRnGiu6/8M4NLYE9wggLflUM/2KhGVBJcVuwu+7v81ilCa9SFkTOQlrSgTWn1bFW26aDqtP0Ip6/0WEcdH4rtFm4VUKHaJUyQpax80C3fQ8x9iraO5rt8C4O8C6mQYhmEYRt4cbDonoYQ44wlRwGvWeVo9mnlws1GJ5XnjenqT9LA4WypC+q6GQ6QlGpoUpW/OwiSd1re+kV01XRvf3stLqdVFETJrbPnWxaWdpBKsr/mpPO4SoC5vJdgQWmmyqUmkqCZP8++2SvkuSq2+dSyCVgZd9MHFZNiFkHdDaaSXHaRzUorJiWEYhmEYBXMwSU5inZMlAFYy8yuI6FIACwDsAXArgHcwc8NNxGHP2VxeJrJ5WdFo5TRbujxvx96qlGO4Q/YLpbRkYMUNSboiXclLYlUEZapNiOWLdHSntbGLDpO22g9Z3RbZxq0cT9rzKGuwL2e37P4SwtbFbMtLck0OpsLtQDO/drEekrsDbZWcdMhvCBDohC3m/QAeFH9fCuBoAM8E0IucFGINwzAMwwjgILHWARHNQaT8+lkAHwAAZr5KfH8rgDlZ5eTlt0SSV1AplxVi3quj2rIdnB8VLImQ7vE1h28S2d5puijj556WHNuH7P10lxW+7G+tLtq5GiHtmncASM33Q5fiAjyvQGouYSPyIs19fdr3gO7wq+g6Slz0n3zr8JIZxyfpa9bcDSDsXVZkQL1mcJFcy2cnTZ/P99nVdAJdnlHNstH3fZC3BVLTHEQ6J/8F4MMA+uq/iIMBvgWRZKUh08dOSNKDe6pOsKRDLPlyHt9TjT8xoafqbfTxravk9ZO09sMlB95OoTQlB3C/+GGW3hu7qPpyPGT8jCT92JaV1evGD4V8COVERhNrygdLKsT2yLghohwpwneJuyJFvUP75A9duiBNli+9u251mKjIe6xMSrYtvz45NmHei5K0/JGR99cn2mBgaGe1PBGHRN53rfJZ+st5ypj+JL1ld/U+Joyq9remSCcVk7WXvyxHml6mTTJkfeVYlS806YVYjk+p+Dpa5JF1nNVb9cT54OZlSVqOLdl+8/qqsTq7xZiQz5esjzTP1LzRSuT4lmM0zQuubJsxI9KfV9lmfaOq97Rl52A1j3h25o6bmqSXbluTpOUzOG98tQ2WbauGB5P1kX28eVf1WuNGVr0M7xHjQz6/Evle+cPae5J05f0kx0yXeHbluOpTFOclcjzLLSkt3o2MQyPfE1LJt188g/I9t1Xck3xG5Dv3qP7quvWuDY+nli/HeqWfpynP7pDiVHDy6OrP0+zeKUn69g2PJmltAiPLlHUfK/pY1kFbLBw2fmaSfmjz8tQ8LaGDdE5CnLC9AsA6Zr5dyfItADcx85/SviSiRXH8nSXbd29uthqGYRiGYTjAw8Pen3ZBzSocEdHnEElG9gIYDWA8gMuY+c1E9ElEPlBeU4lO3IieUXOSSvSIFVleJmCaeD4vEfDRE+cm6SJnxWUw2XQxicu6rlwJbln2xyStOW/TypMSt7Xbt6SeqyGjjcpnoGZ1qUizXES6PvldxPa+0VGlFKxXSCek5EnSLlPHvLZffSh668fFtUCz181y7Q7kt83rW993znphkv6fVX/OzN9sPaX0dmB3dTy7jJ9uJaaW7/tUErJFvHdoZes0lgEMfuQ13gNi3Bcua2kdKzQtOWHmjzHzHGaeD+CNAP4YT0zeBuClAM52mZgYhmEYhtECOkghtmnJSU0hRAsBfCg2Jd4L4CkAlQ3Ry5j5U43OH9Ezu+lKuKwkjDDydq6kOaYKcYEfsvLXznWRVrSKVip6tovK6lJbRUu9il1CB6KIAIa+FKmU3krHcgcKndJmLZec/Mur/SUnX7q8LZKTvKIS3wDghjhtjt0MwzAMo2x00GZGx08kTFpSPJrZpi+VKbu2NxwSSDCkXtq57ZaWSFzMa11MW8tM1gpX05cpA0Wuzsu88i8r1mYKB4O1TgUi6iaiO4noyvjv04noDiK6i4j+TESHh1fTMAzDMIwQeJi9P+0iD8lJxUPs+PjvbwM4k5kfJKJ/BPBvAM5rVMDsvqofht1iL3nDjm2p+UOC9/m6HXYhb90EbYNPSjBk3bX7zssCw/fcLK12qTswKFbD8i5cpCh5Bf4LCc6mkUfbu1gWaNIS+VxMGl21bvANd1900MU+4SdD+vvoBEYrQRGPm3xIkn5A+JVpNqipbN8jJsxO0o8Kn0oarbTC0hyWSboUnyo+dZsxbmKSlu9bF8laXhLFv5/x7CR95Zo7MsucKnyzrN+xNeDKgXSQ5CR3D7GI+qcyUekHsCrl1BpWDmzMvpZIu3gZ1NBMiUPIW/yve6L1e7m0K4ZDVo/4iue1icoYT6VZiaxjESasebS9bxnynuSPgO+ERCvT5bhvmWWakPgqUWquDu7Z+GSSzisOWAWXCYmkiHeA9i52iXZe065KG2um2BXWDDbvFyuvn+bfiAmJC22dkEg6KLZOER5i3wbgKiLaCWAbgOcGXsMwDMMwjFAOBsmJ9BAbmxJX+GcAL2fmxUT0LwC+gozgf7URG6ur2BoxckCkSk20OSJjhg5kz+IBYG5f1SXy8oENXnXLQtt+kO7oXbZ4NFxWi9p2gcRFIlXJId3OSwdhmiM1WZ6UluwIMD2eNW5Skl4nVjVy/Mn2lmNIU8J2GcdpnD/r+Un6+6tuTs0z3mH7Q9ZRuteWK1pNSvS08dOT9FPb1ibpZu+pnlYp7vo6CZS4PDsujrjkMyXd+UsJq89z57KdG7LV7Yt8LmRE+cP6ZyVpFwmP5oBvtBKqoYJ8J8vr+8Yhk7i8+2qOK79HXcpWbF5GBcEcDJMTAC8A8EoiejliD7FE9FsARzPz4jjPzwD8Pu1kIloEYBEAjBgxEd3d49KyGYZhGIaRA3n4NWsVuTphA/AqAGsAPJ+ZHyGiCxBJUV7b6PyRwglbXk3XSic87XL9XaHoe83L0V1aAMa9NZKK5ttR6qKMm3OqVzna6r1Mjpy0VWYRCt7touKWfLMI2KdJBKQyqpQMtcuEOq+xkvYMtMPFfytoNhyHS94QN/VaOUW821vthG3b21/iPTjHf+eaznXCVoGZ9xLR2wH8ioiGAWwG8NY8r2EYhmEYRhMcJNs6CXUeYi8HcLnX+XlUArUmfPcqGvNFdM1wgdISl7r7Sr/a7aBLrkbkSjBkZSKlJYMrbkzSLroorZSWNFu+tnoO0cVyQatvEe0kJSYVakoW19mpWIa0S1qSV9uTeDorqX0O0pJ2P9PNUGQ9Q97JNdK6nGw6y9I/7fRb4kvHe4g1DMMwDMMBm5y0B+lfoAi0/cciu9ulbC2Pttr2rW9eIQIqdXBZCfoi+yPEDX5eUgBtpdRs+ZpPjaJ1nLT6tkMHp2yv1SLaoFmdkpCalGVVn4VP3fLyw5OXjk9p2rVz3JyET06IaCmiCMT7AOxl5gXiuw8C+DKAqcycr41tG2irCVgTHEjKc80SMlFxQXUOmEvphlE8nag4bDTHwbitc1r95IOI5gJ4CYBl6acYhmEYhtEyDsLJSRpfReQ99tdZGUMc47g4UtPiOcgYL9KlupSQuJjRag6yKuaOI8X2yqDiul3eqdw+kgpZ08dOSL3OHiEh2TNcbQN5H5pa1xgRF2iHcBKlOcrSnAxp+SWVPFNEnIkde6uKjVoYAE06IR2prRrcpFy1iiZFkcq0o4QDKDlWZN/L7TK53SJjLMnjWRI3GVtqtbiPLhLOt8Sda+XJsSrr2CVaUI5zmeeUqcck6evX3pukezzNyOXYHdlVfTZle6SZlAPZkj7tWXRx8AYlj4aLCakWW0fe3+TeqvPsDTurscLkc51133JcyfvWzPA15VxfSYXLdo/sk4kihpPmYl7WQWtj7XilPiM8nbDJ8uQ7f+uu7dWyFYmO7EuZ3i36W7aNNiY00+aW00HC/+CoxIj65hoiuj12rAYiOhPASma+O4fyDcMwDMMIpJOiEgc7YSOi2cy8koimAbgWwHsBfAnAS5h5a6yTsqCRzsno0fOSSvQLKYRLwDKX2X3R5pB5OSnLwldxLa/7C1mBpiHdT0upkoxIXRNmQFHsDXGtLlcymulxiAMmn75y6SdfR1yyTOnKXgu62C6lyFOnPSNJ37ju/oZ5NelAERIBF7TxoUWgzeO6FUd1QLrpdV7XaaacvKOza0wZMz5JSwd8LoEHi3gnhrRxq52wbX7dQu/qTvzFDW1xwhYsOWHmlfH/6xD5NzkVwCEA7o4nJnMA3EFEM+R5RLSIiJYQ0ZJ9+9IfMsMwDMMwcmK4iU+bCJKcENFYAF3MPBCnrwXwKWb+vcizFBmSE1/39XkFuZIrn2dPPjxJ37b+kWrdFJ0WTQ9DrqDGxEG/pE7IXlGG772+buZJSfqmgUeT9Fplf1cL0KX1uVafmsB7DqujrFWFXFluFPvwea2AXVZHY0RANrni0nRR5D5xn6JjJOszTuxtSz2j7hQJSF6SKe25mCl0c1YObEzSL5r+zCR9w7r7krTUdZHlyPGv7aH3Ku3qUudm30S+QQVdJGIu0lAXaZbmqM1FlydNF89lPMs+k8+or4TXpW+ktEQ6xnN5ljX9DJf6ZNVLIvt7Sm9V6qIFGtWu+dCRxybpox65b//M0N892phrteRk06tP9X7MJl1+Y0e6r58O4PL4YRgB4MdyYmIYhmEYRknoIIXYXAL/hTJCSE580SQbLoToq0hU1+IZ1/e9abk6kvoZx02quu2/Z1PVEZ1vCHHt/lw0zV1WrJVraToQvnvWIxTJkMuqTbOskUhdlDGKX5Q89vRdpH8u41xac60Tq0JNsqdJWuQ7gRRLN62cEGlPVhkuVit9isSq6LecrxRPO55WzuuFxPTnq2/1un7RyHGpOVd0ea/4BOh0ede49Ic2niXzxk9L0isG1ifpvN6VrZacbPx7f8nJ5N+0R3JSislJyLaONiC1F77LuWmi90Z1kKSVqZnfZp1Xf1yKZbV6lcFRXNaLUv6AbN9TnYT4buuEKCL7Tih3iO0eOVFxMV2VpN2hy9ahVoa8D83cOMs0M8qT3R4ukyiXH8msH46QsvPaugh5x2jl1xzPQTGzyLIbXculPVwiZLdjQiWfBYlL5GKXe9Xe9doCoeWTk79rYnLy287c1jEMwzAMowPg9q9dnQmanBDRBADfBXAsognnWwE8DOBnAOYDWArg9cycrrEZ4zuVcxGbuih9+jpz08rRqCi9yXrtc9j008oeJ7ZDJC4m13mRx2pnWJwZsrILMdvWJCdS2XVAKLtKackOxQ2+y8o7DW28ubSMi1lxrcl1tvKqdCoot0ZcpAMuUZK1+8oaCy7t4Tsm1O1chzhaLlJnrT3ykGiEbB+FXEviKy1xKjMjj/b9K2c+J0lfsfr21LI1x4AuTHAw49YkMGWQaAPoKJ2TUFPirwH4PTMfDeB4AA8C+CiA65j5CADXxX8bhmEYhtFGeNj/4wIRnUFEDxPRY0SU+ptPRK8nogeI6H4i+nFmmc3qnBBRP4C7ABzKohAiehjAQmZeTUQzAdzAzEc1KitEITYvZbz2a96Ul7zbqVVO6+opor+1YIJ5Bzjz1Y1wObeTORDvqQz4OvvzRQv1kTfafRQReNClTG28tlrnZN3p/jon065rrHNCRN0AHgHwtwBWALgNwNnM/IDIcwSAnwN4ETNvJqJpsW80lRDJySEA1gP4PhHdSUTfjX2dTGfm1XGeNYjMjQ3DMAzDaCMFSU5OAvAYMz/BzEMAfgrgzLo8bwfwzYqKR9bEBAjTORkB4NkA3svMi4noa6jbwmFmJqLMmVqQ9MNhj1tddTqc67LK18qvmP5qTpF8KcJ9t0QrpVldiqLxXeXJ+mpmy74SCi2Y4Pi5pyXp4QwdJhfLIZe+kWjBNFnRR5AWQ0NinLvoNUhCnhcfmrUEAnTrCqkX4GJB5WIVs2jWC5L0d1b9xevctGu6WCnJ+rrcq8RXWuJrzu8iLclDuqFJS2S7dzuYEks0M2HZxiOVoITsoA/ZErgQQc1sAMvF3ysAnFyX50gAIKK/AOgGcGGWT7QQyckKACuYeXH89y8RTVbWxts5iP9PnSFJ9/XDw9vTshiGYRiGkRPNSE7kb3X8WdTEpUcAOALAQgBnA/hObFDT8ISmYOY1RLSciI5i5ocBnA7ggfhzLoDPx///Wjn/IgAXAcD4sYcmU0mpQa3NmrVVoVpX5VyXWXSP0O4eQvpKcJTihrniztl3nqzVUbp7lisDbYW6cHrV3fKf1ifbfzX3p60otfpIfPNX+nPCqLHJsfX7tu73fT2anw7N6ZNEW3XKfpJ5pNt5lyB5sn+ktGTb8uuTdK/iwK1ar3Q/PFr4AZfVpDx3vLgnadnVXRMCvjqGZP+N6q6uyF2sG6RFmSo5EeV3ea5eU8tzyKO5EtfoFi785V3INqstPz0g5cVrbqnWU9xrj0e7an5t5DtAOmgckO0oGqfG6Zgw2wjxlzJhdPVZ3uRgNSglC7KNfSxnXKQ18p7k8ekidIaL+3qJtPTZg6p0SoZ76BH3JyWyUlK7o8CgiFnwsL/kRP5WK6wEMFf8PSc+JlkBYDEz7wHwJBE9gmiycptWaGhsnRMQmRL3AHgCwPmIpDE/BzAPwFOITIk3NSonRCHWcKOTt2NaSYjiqYbc7pkw70VJupXKwM1S5r460KmMuYNJ0bloilCIDbluqxViVz3/NO+bnnXz9VkKsSMQKcSejmhSchuAc5j5fpHnDERKsucS0RQAdwI4gZk3ppUJBPo5Yea7ACxI+er0kHINwzAMw8gXLkDnhJn3EtF7AFyNSJ/kYma+n4g+BWAJM18Rf/cSInoAwD4A/9JoYgKUxH29r+TEV6Gzk8lrdVSmVZZvDB1fQlZHRUgKpLLilmV/TNJp2z1l6++JiuMp34iyByK+sWF8yXLnLynT811m2iU50d4rrZacrDj5Rd43PWfxH9vivj7UCZthGIZhGEau5O6+npn/SkTvBfBuROKb3zLzh0MrKjmYVgZ53WuZ2qxoXYuQFZGvKa8L8n7TTI81F/i+uJjx+kqGNDfdB6u0RFK0S/LKOHbps7Ka+5eNVkpLJGXR12pGIbZdhAb+q7ivP4uIegCMIaLTEDlgOZ6ZdxPRtMZFGIZhGIZRNCXQ4nCm6clJ7L7+FADnAUDsGW6IiN4F4PPMvDs+nukJLsS5mMu+r7bPqDlakrjk0VasaXPUELNiacbYrBleo/Lz2tvO0gnSTDl9Vxfnz3p+kv7+qpu9zp3dNzlJrx6sGpPJ8SdNKeXqVRsHMo80D9bGZUViornA99WtkmNPC96ntfG88dU1xLJt1UfWZfz7UqTOmK+1la+UwaXuXQ4mwz4uz6W5bLt0JjRke7gExpM0G6xQc3/g0hpyPO+VzuocznXpe+33yMV8vRV0kuSkCPf1RwL4GyJaTEQ3EtGJudTUMAzDMIym4WHy/rSLkMB/CwDcAuAFwn39NgCvBnA9gPcBOBHAz1AXHLAeX2udb0+rOrv64ObqilmGfVfrXXsPSdrXAViraJVFANC6lVjR1jouaPddtLVMVvmaFCXk+hLfe5JOtrbsqnpyLkISkZU/JMBaEYRcy+de2iUfKfrd0Kq+KoN1pxbwsNXWOk8e/7fet33I3de2ZYYSonOS5r7+o/Hxy+LJyK1ENAxgCiIpS0LsAncRAPT1zsCYngkAgI07tiV5tFZ817rrlW+yeebkQ5L0PRufzMzvG3NB5q/0qE8cDQA4TtTx/s1PJWm5fSTF81odXR7+drx0fBVii5igSa+O0rtniMMrzbur7Ku0c+V5WqyekInKiICtmZ3KlqHWBi6jacGUI5L0HRsfq56bMRZdxmrRDstkOYf0z0zST2xdnaTleJU/SnKrQ74TNEWAtKO+E4aQZ6ddE5KsPL7bOvJ4XtuUcgtXeqrWnvvte9rnFVZyUGzrMPMaAMuJ6Kj4UMV9/f8BOA0AiOhIRN5jN6ScfxEzL2DmBZWJiWEYhmEYxcBM3p92UYT7+u0ALgZwAoAhAB9i5j8qRQAAekbNSSrhMhPWZtZaZE6JrwjfxYxvypjxSXqDkPykUYQoMy9xt5ZHxu3YKbbOmu0fGa9ln1D2kzEnXLYK5KpUbunJPtOUheWKksQVZB+7KM9pdfNZ4br0jSZFkXVUo3KLumjPhVYH2U6y31y2UCV5bGO4RFHuaVFU5PpyfOMgadvLNfFvYkmHvFe5StfeZVnlNcJ3C0T2yVgRV0kqYWvjSXvutPpUoirL8zTFc83Iojauk4if5KDkrtVRjVYs614S9/WPHfNS7yF/+ANXd9y2TiP39W8OKdcwDMMwjHwZbqMkxJeOdF9v6HSCA6aymUN2GnnpohjloxOeX4k9y2G0WnLy8NEv8+6kox76XedJTgzDMAzD6Aw6SSE2xAnbUYjMhCscCuATAGYD+HtE+iaPAzifmbc0cw3fWfkIz31fjRnjJibptYObk7RWio9GfBGWJ0WabwL5BXmr9Gev0GHRTIl969gVsM/ugrb33KqVrtQ70Cx6xs05NUkPK/ddhHl0XlTMlqXJsqZvofVxu1byeV037cwiglFq+PZxiONG32tVdNZcrP3SdFWA2rYsOoxGGekk4VaItc7DzHwCM58A4DkAdgC4HMC1AI5l5uMAPALgY3lU1DAMwzCM5ukkJ2x5beucDuBxZn4KwFPi+C0AzvIpSDaF76xcrqykbwf2tLRYI6QlLl1TY8Uw3NiKQVvRasjra/4qFkw9MklLvxF5SQ1YWdc0uyIaLdprJ1Xba1ix1HKxJKnRkncYNy4hB6aNnZCkN+xMt8IKWYj4rAS1e5LSksEVNybpMdINvqcbfknRryZZvpSYVJASUIk2tvul0zjhV6ToBSN7WgFqpLX3zLFVSe7KwY1JWrPWqamX5/V9LfBcpNVamS6WVZJKn0splUS7puwb3zE/XUjR5e+CL2XRJeokhdi8JidvBPCTlONvRe3WTyrSY6jcNkhzaAbU/khL5MCTaT3eQT7iUlIeljR8B6Z8OOTLe5+45pL1jzRdvstDI9vGRXwtt22GRD9UyhkWV9KcoWmmkZrpovS66ELNjzSlv7DWbd+SpH23lrqVSYC8r8oLWZpED3jeh5yUyQnJDmW7R3s5a1t3L55+XJK+du09SbpVWw2+E2w5Dorw2iqZOqY/Sa/fsTVJdztsI2jXSjPhlxMSX+eB0g2Ar/m3i5m8XBD2KV5QJbI9atoG1bbRFg6VcSa/H3KY1Gjx1IYdJiraQlX+BvWItOYOYVJvX5LeuHMg87pF0U6/Jb4ERyOKoxG/EsAv6o5/HMBeAJcq5y0ioiVEtGRoT2PfIIZhGIZhhMHs/2kXeUhOXgbgDmZeWzlAROcBeAWA07WYOsx8EYCLAGBq/1FJnv6e6ux7lYwWK86Vqz85+9aQVdCiQ2qz9a4M1/RA7cxZCqYr19Kc/bhsV0iFXLnNMCCcHO1UVkTaPXUrrpc15Oq8V1mJyTrLlXfaCnRWbzUi8LJ91Qi4A4rjpkmjq9FO5arDdyUo6UK6BEZKMdZsTxfjyjGnrdzG9/QmaVnntIdBOqyquY4iIdSQEiZtu0cq00qJpRxDclxet+7e1GtpSsGjFAmMpqDoI/3Ka82nbQfKseqy7bdjb/r40yQqviv+yviW15wpJKnLB6qOt6WEZI/oG9kHLtIulzaW7SQlbiNF+ZqbeHndXlTPlWRt8UzrrbbvHiE90qQccvvohImHJunF6x9O0lkhSIDaqMtSIiyV+ruULVQtDIShk8fk5GyILR0iOgPAhwGcysx+MmrDMAzDMAqhk3ROQt3XjwWwDFHU4a3xsccAjAJQ2Si9hZnf2agcc8J2cGGOmyIqK6u8dDZcgpqZA7fOw/d5KYvypZFNq52w3TnvTO8h8axlv+48J2zMvB3A5LpjhwfVyDAMwzCM3OmktWDHe4gNWSXIPcHvT6quIt+y4YYkrQW90/Q2NK32Cr6BByWz+6rzQGna6hLgTKKZ6WrI/C6r86z7Gif0MXytbPJC239/0fRnJunr16brW2h6Ny7WOpK8rVy060jdEs2Bmzzua4kj+3uMuJbmYC9vfJ8jlzE83sHyxBep1zCyq1qHLMeGLtISqfvRO7Ka3izMqccqfRPiJE1ed7e4D1/rNh8JaojTySmKPpAL7xbPyDfFsyPJS6+nKDppWydoctLAS+wNAP4HwGhEFjv/yMy3hlzLMAzDMIzm6SRT4twC/xFRN4CVAE4G8B0AX2Xm3xHRywF8mJkXauf2jJqTVKIIHQRthu4yc3dZZWmz5TTLoBDHaH1C4iAtBXzL1HyIuLR9Hu73ZXtpIc9deNr46Un6qW1rG+TcH01CovmskRZX2opIs/DIYt74aUl62bZ1qXlCQg4M7avWRetjTYpSRLgFSTv0I/K6pks5mp8RH6mBr4ShlTpdsg1qHOClONRrdK5PLX0dzmnvO/nb5yuB0a7rYvUj87da52TxrNd4D4iTV13WeTondSReYomIAYyPj/cDWNXoxJAJksuDqB13ua7moVJeN0v07esVVmNnjWlm9njRHgjNe6ILLvmzXjrz+qo/xk9sXe11fYnvhEQiJyQ1LxeSL5dqv+4VJouayH/IwclWGtqEROL7hGhmvBI5SdS2e8bOPiXzWi5ejDXasQV+5MQ5SfrhzStS88waNylJS5cGEpe6a2b+Ps+d74REK7uIiaA0C3eZkLiQVU+XLaZdyhZTyG+Ni9POImJa5UlZ6uFCnpMT6SX2nwBcTURfRuTo7fk5XscwDMMwDE8OGp2TCsJLbCXI37sA/DMz/4qIXg/gewBerJ0/p29KkpaOhTQmC1fA0hnOgOLMSnOkI2NvaIyoUXxNd9ikbf00O0ufKOp49LjqKu++rdWwRbv2pTuG0iQk6urZYaXrsiqTSEdccoVRSUvphAsucT18XexrcT2k2Ldmi0eUozlNk1eS8Zb21rj4rm7NVZQV5fgZKxWwxepPlqFtr8h2km7npSM12X/aFqSUlmxfeVOSltIVzdHYXgcng3L7S5K1Feay8ndRSJTSEq1MTVoix9bCaccm6T96Kk9rzuoklS0hKX1xeaPIPCGKvS7PkRZTSKuP7J8at+9yyytjK1HrM7l9KemqCX+RLhl1UQSW+aWUZl9NTKFhkb+aPspBWtcKOknnJNh9fUy9l9hzAVwWp38B4KT6E6T7+oFdG+u/NgzDMAwjR4ab+LSLXBRiieinAK5m5u/Hfz8I4F3MfAMRnQ7gi8z8HO38vJywFRGMzGW15rOXezA7SKrcu68SblZ5QDFtKaUJ8jnR6lykIqKmWJlXNFoXOsGBmybR0QhxcCYJae8spftOdFpYxLNZaScXXSYXZgREHHYZBy7Rk1utEHvTjNd5d8cpa37RmQqxsZfYvwXwDnH47QC+RkQjAOwCsCj0OoZhGIZhNM9wZ8xtAeQwOVG8xP4ZgCopCUWbkRYRut13jzfPvEDxppxFk7aCkvoYWQ6otDLq05KQlWZIe/s6y0q7d+1etSCHIboXLkhJhIsDt3bj61jLd3xoujM+puP1ZLko8HWE5+tIrQhcpMwu+SWVdsrrnnylJRpaHeTYqnEr0Ubp13BbXcD50fEeYg3DMAzDyIZtclIsmgt4X/KagWt7xml1872OXL1rIe7Ltg+d1a67PcOH+95dSHsMOzhOChk3WZKivHpSrrZ970Me1yQRnSBFKZoQaYkLPhIvbdyU682Qj6QvRPcvRKoaon9YFql3OWrhRqj7+n8G8DZE/XAvgPOZeVf83dcBvJWZxzUooq1IxUxtq0GaLW/ZXXUyJH/E9moxZuLyc1P6lJMyxcxVIh9EadIo42oUEfE0qxTNvLKVaNs3Wt1cvExKMf+4kdWJ5MadA871KkIMr5mtanFwpHm5HNtafVo1UclrsuiyHZNXbB1t8dTXZPnavS6YckSSfmDrsiRda77stw3lssU519MNhFZ+EVvyFWSbTRLv8w07tu2fuQEufabF7imLvKKTJCdNmxIT0WwA7wOwgJmPBdCNyBEbiGgBgIkNTjcMwzAMw0gldFtnBIBeItoDYAyAVXGMnS8BOAfAqwPLT79oASbDmrjdZ9VbT0USEaJk6aIUmXV9QI8Q67uSyGM13y5piURGhd03XO17F8VTzfxerkabFfkXIYbXxrY2JrK2JhvhK0XxkXrktXXh0jcu0gxfqaPMMdCkNEa7yu0bHs3M4zsmXd5VKzylJb7l+6Ddt3Q77ystkbiMCW0btCzba520rdO05ISZVwL4MoBlAFYD2MrM1wB4D4ArmLn5oCmGYRiGYeRKJzlha1pyQkQTAZwJ4BAAWwD8goj+AcDrACzMo3IaeUlL8lIkzdrLLYsyVBohK4l2U6RiatkoWpFPI+QJcZGilGVF2Qy+0hKX483SrnbshP4rUp+l0+gknZOQbZ0XA3iSmdcDABFdBuA/APQCeCxWGhxDRI8x8+H1JxPRIsTO2ai7H11dY+uzGIZhGIaRE8OdMzcJmpwsA/BcIhoDYCeA0wF8hZn/u5KBiAbTJiYAwMwXAbgIAGZPfEYyAV+3fUs1T0DlNM6b9bwkfcmqv+ZSpraX26y79veJleW31tycmme4JthUudYvWSv1vCyEQu7aV2/J14zQp89dLDCWCJ0CX7ICqTUiL0djmhRlTIukKHlZQclynj5pXpJ+YNOy1Dz9o6uLri27qtZ+zeqh+d5HK52wFdHGaeWEXCcrVIAr2jtMe6+UJWzJQeGEjZkXE9EvAdwBYC+AOxFPNgzDMAzDKBflWsY2JpfAf6H0jzssqcTTx89NjmurxS7Fz4TLnRQRzK1Zl+S+ddQIqbvvCi6PFYAMYjckVi+aBEPzORFSF7nCecGUo5P0TevuT81fZNgBbS3jK3EL8amhtWWI5MTFL8mOnP2itHKFqrXNiVOPTNLSikZzZ+4zPk6aelSSXrz+4dQ8ed23r1OzHoewFLLM7iatLmVIhb1cPW/zzsHMa4ZIEWU50h/STgcLP62/Wx3477IZ53gPj9es+XFbxC2lmJyMFFGJ218bYMqY8UnaRWG00+PfZFGEuDatvBClz4m9VV9/2kvKl2a3aYpA64MJyrZBJ1LZ7nExO9YcyLWLvLYL0jjQ3y9F0CmRnFs9OfnlzDd5N8ZZqy/tzKjEhmEYhmGUn/JO0/Yn1H39+wG8HdGi5jvM/F9ENAnAzwDMB7AUwOuZuWH4x7I1mCYt0ZSdDvTVTF79k1VOyAonL2mJxMXxWqvQrt7p0hJJRWLiYnZcBmmJJG9pieRAf78UQZmlJe2kk0ZSiPv6YxFNTE4CcDyAVxDR4QA+CuA6Zj4CwHXx34ZhGIZhtJFh8v+4QERnENHDRPQYEam/+UT0WiLiOMRNQ0IkJ08HsJiZd8QXvRHAaxA5ZlsY5/kBgBsAfKSZC4ToOhSxT5uHM58iFNeKVmrNO1xAEX1TFlO9ViN1HaR5eSevtkMCCR6I+hlFhOsoA756IRXDA18HigfruyGNIkyJ45A13wTwtwBWALiNiK5g5gfq8vUBeD+AxS7lNi05AXAfgL8hosmxr5OXA5gLYLpwXb8GwPSAaxiGYRiGkQPcxMeBkwA8xsxPMPMQgJ8iElLU82kAXwDgtCcb4ufkQSL6AoBrAGwHcBeAfXV5mIianqy6nKhZaQw7rJra4TAnRALkYvrsUnffOsgVue+10vKPGVE1wxsY2pmk+3p6U4+7IK1p4GBeLs1ANf2hscIiZIfQcShy9aWtJrXj+3JyIJfXuXlcS56nSVHGzz0tSUt9j3ZJS4p0QMYH6HrfVy/ER2KimueLtK9bh5CxVRbpTUEeYmcDWC7+XgHgZJmBiJ4NYC4z/5aI/sWl0BDJCZj5e8z8HGY+BcBmAI8AWEtEM+MKzQSwLu1cIlpEREuIaMnw8IGj1GcYhmEYZaSZwH/ytzr+LPK5JhF1AfgKgA/6nBdqrTONmdcR0TxE+ibPRRQI8FwAn4///3XaudJ9/Qjh58SXLZrjHcVRm0Tbvy3TOsVlti6lREXU3VcqoVFxgLRTWQHt2te8e3TpCE9ziiSRTrOkVELL4+KyPo+211aTUrdE1kv2TRFSA997CmmDiu8SzRJHSku2Lb8+SefhvC2USb19SXrjzoFcy6YWuhwvywq/QrP1kQ7eXKSLGi4SeBfK0JZAc/WQv9UKKxGpdFSYEx+r0AfgWAA3xO+rGQCuIKJXMvMSrdBQPye/IqLJAPYAeDczbyGizwP4ORFdAOApAK8PvEZDtMYO8agpz3TZ+nGpTx6ExJhxEU+OVRxbyfwhzo0q5cg2lYSYY+5wmJBIZFvK+5DHXZQPWzVpGVK8s5ZN1BxSZpZ5sBwfIUqzvnGSXO7DZULi4tk67Vq+sZ80h4G+W74u74yitwybHZeyzeQWbo13Vs/n29djrsvW+wHCbQCOIKJDEE1K3gjgnMqXzLwVwJTK30R0A4APNZqYAIGTE2be703AzBsRBQE0DMMwDKMkFKFzwsx7ieg9AK4G0A3gYma+n4g+BWAJM1/RTLml8BAbYgor8Y0J4jIr10SCviuMZnEp28V1u8vKR1u5+rrm1lYPlaN9o6qKryHO02Tb+G5tuazUpILu0HD1vrOiUPviMp57lPEs21pGwNVi6OzzbCcp5Zo4ujrO1u/Ympo/79Wzy/j3jXjsshp2kaj4xobRpBgalTwyjouLhFBbsXsrwjtITLuUZ9Alhs0Iz/dKZSzI8lykGbJsKbGSMXq08ay9+7T+0+InSdpp4l7UlZn5KgBX1R37hJJ3oUuZpZicGIZhGIZRLJ3k+Sd39/Xx8fcCeDci0+LfMvOHG5XjO5N0WQH4RlANuZZP/X33aF1WCVIp2Ld8lzx7PXVBslZumhKzL0XoW8jjg8KcOa+2bPY8bTzLtpbtmpcOiZQIaKvLELLq6dvHUlriEvE4ROE3xCGaz3gacnj+itB1054LiSaxdOk3Xx2zNOVUl/vW3kcu41l79+X9u9AquIMUXpqenNS5rx8C8HsiuhKR1u6ZAI5n5t1ENC2XmhqGYRiG0TTlmy7pFOG+fgGAzzPzbgBg5lQ/JyFIs9E9w+V1wFQ5rmnP+yLPbGVI8LxL991rbiUh99psn7hIcYqU3LSaVjk5LNqix4U89NFCJDR53UdeeiwuhFglZpXn+y7ulGfKlU6anBThvv7I+PhiIrqRiE7Mo6KGYRiGYTQPN/FpF0W4rx8BYBIih2wnIvJ5cig3iDnvu+LMK/CTy4xa09bW3Mqn+S9ocOuZdZT4tlMRK9Q8gpDNHTc1ST+5dXWSLsJPgoY2DlysDNQyFU39rHJcLIeKCIIpx7bcWz9y4pwk/fDmFV7XystKzgeXtvGVoszqm5ykVw5sbLpumhOvvJ9NzXeQfEaLCCDo4rvF5dxhj3N9fUxpEmd4SpwPhKCSBbmvL4RQPyffA/A9ACCi/0TkU/9oAJfFk5FbiWgYkQOW9fLc2AXuIgCg7n50dY1FUbgMWm0CoW07aIqqzRKi2BVSpi95vNSWbluTpMsgNnVR5HOaPCqmg1kU6YwK0H8gtbEtJyS+IvG8FNF98J3EuUxUtAmJNNsvQgG5WVyeyyIiGrt4NNbGWbPb0SFtLZ9vl4m0dq7LBEmqH8i2b2dk6U6aUhXhvn4YwGkArieiIwH0ANhQf25e7usNwzAMw8jmoJmcIN19/cUALiai+xBZ8ZzbaEsnlBBRW16iVZtZ+RGiwNtKRWBJkaaDRdxFXu3kIl0sE7419N3u8XUaWIQkrBMIUXRv1TMeIuVzqZWv+kEr6KRxV4T7+iEAbw4p1zAMwzCMfDlodE7ywleCURO0LSfFJE1ZTLowHxBOuTRlwizHRSEzV5drSkICnGnH81AKmze+6vpm2TY/S/O8VlLafWj70C5Ks7J/uqmapx0rKNlO2j2NHzUmSUt397PGTUrSqwY3eV1XK7NIfMe5puwaYnqsoSmMhihepyGDdso+lu8yLbBnCGM8XetLNAlJkdKSkPHpcq9FKB3nycG0rWMYhmEYRgfQSds6lLWPHOuQvALAOmY+Nj42CcDPAMwHsBTA65l5MxG9CcBHEC1UBgC8i5nvzqqEKcQahlFW8pKiHIgU6VCvnopUogxO6fJi79DKlm60fPZpb/K+7Y8/dWlbNoNcbB8vAXBG3bGPAriOmY8AcF38NwA8CeBUZn4mgE8jtsYxDMMwDKO9DDfxaReZ2zrMfBMRza87fCaAhXH6BwBuAPARZr5Z5LkFwBw40MrZbBHXyrtMl2lqGWb9zZKXM6OyjRuZp9tj7zkv9/W+5bsQ4mSriPrkXbb0W6JZ4mi6KGNnn5KkNT2Jovs2i6Kvo5UpfXzkpXOVhw6H5pDNV8/F189JGcKsAJ31u9Gszsl0Zq649lwDYHpKngsA/M6lsOMnH5qk7928NElrnSh/3EgME5fB+5wpRyTpuzY94XWuBmlKb/HxLqEc6XKdZ005PEkv31H1Xbd9T1WJbadQyArxrOriWMvXI6Psn7SopRNGVR3ubdw50KCmEb4vOpeXjvbi0MaT1scSmUcja2ImSzikf2aS9vWkO3VMf5Lesbc6VjRFSO1HbOG0Y5P09WvvTdIuDqy0fnj6pHlJ+vFt1fvKw4Gbz6QQ8I+QLSck21felKTlBEaOLdkGUonSRSG2omAtzXI152Za//V4OhoLQd6r9pzKPPJ58fnBlmNP9rGLA0U5PvYFvPPHCOViea/y3bBbHB+nGFYYOsEKsczMRFTzviSi0xBNTl6onSc9xM4bfzimjpkRWhXDMAzDMBQ6yVonUyEWAOJtnSuFQuzDABYy82oimgngBmY+Kv7uOACXA3gZMz/iUom8FGLLpux0oJD39oImrQkRs3ba9oMvmoliEffdLtIkBRLZH5N6+5K0i/StaIpUmj0QYrq0mk55LlqtEPuJ+f4KsZ9aWl6F2DSuAHBunD4XwK8BIHZjfxmAt7hOTAzDMAzDKJ5hsPenXWRu6xDRTxApv04hohUAPgng84iiDV8A4CkAr4+zfwLAZADfilfHe5l5QdY1powZn6SfPq6qQ/undQ9k3oCvoqIkL0dIeTue0uor90uHeVik0+88L4dAITotnJIeN1Ls1+6r7oNrK+bjJh+SpO/Z+GSSDpG0yLaZOLqqFLl+x9b96tsMPm2v3UeN7g6nj0+Zv3ZvvXqu1D9ZM7i5YV1agazngqlHJunb1jdez8j+KIO0ROqWaEqzfXMWJmmp4+AztuS76c9TTk7SL9ywOPPcvJQ+Xc50eQ/66rulcUh/VQVgaLj6/pAO9bTy8pJCHTFhdpJ+bMvKJK3dRREO8JqhbJKjRrhY65ytfHV6St63AXhbaKUMwzAMw8iXTtoULIWH2EExk/zL+gdT84RIP+RsUbqjl1YMGi4z7Z2KZvqhsbXFSLGKluHoNcYpdewXKxNpASRX+xJtte3iil3DZSWmacRXcu8Rx1wChD2weVmSlnXvFe6kfVcjk4XOwtrtW5J0l2JxI/ettRXfolkvSNIXr7nFuS493VXrA6n5L1eiLgHnZG9IyY02PiTSNbe0BOv1dE/uYtnVP7pqrXX7hkczy8wDzYJM4mJmSkoeiZSWDKy4IUlL6YqPZE1eZeGmJZl5JPI9wewnudHyaPc94Ck11vQds9zELx+oWjC6WNzI+o6oeTf5/VTLc5cOrE3No1kWliUIYDu3aXxx0jkhoouJaF0cabhybBIRXUtEj8b/T4yP9xPRb4jobiK6n4jOL6ryhmEYhmG4wU182oWrtc4pAAYB/FBY7HwRwCZm/jwRfRTARGb+CBH9K4D+OD0VwMMAZsTRilNpl/t6l0B6csUl8Zl1a3u3xwh/Dw9sWoYsfDXQXfZ3faUoeViwSImAXG21chD4rqRdJAJ57KcXgeaTpAgrkHZYOIXoVWiE3Id2rq8DNx9GZEgr6+vl4kPHpV1Dxnyzbezb33Kcy3aSz8LrZ56UpH+++tamr6v5odHqs3vX8pZawnxo/tneg+3LS3/SFmsdp20dHy+xiMZZH0UjdRyATQAayu59FTdDXqq15wqTTKXM6WMmJOm1O7aklpk1IDUHXg8qExLt/jSFWJftrP+ZujBJv2Pd9Ul6lBBDDu3LdtLk6+Qqi5DXcpiyn+gTcVz+kGsvF/VFLeqjbdWkoYnJa172Kc7sXMscUpxvjeyqjlvmPSJdvcIoTwd4fQ4Tzzwm/BJWJtu+CvI1P8zKM+gymde211wcuGXVV5sATBBbZXIrRKZHiPfUE2JCorXH7HHV6M3LBzak5pnTNyVJrxB5XPrBd8uwgvb8ae8DOT565ftOPBfahEQin2npYE3WZmpv1bhDRvQeoby7W00nbeuE6JxoXmK/gcjUeBWAPgBvYG5jbxiGYRiG0UFTE8dtHSDVEdsWZp4gvt/MzBOJ6CwALwDwAQCHAbgWwPHMvE0rO69tnbxMZ4ukCBG0ceBThnHjIrI2/Mly4GZO2CIqUs0QN/xlcqwItN4J2/vnv9H7tr+29Kcd5YQNANbG3mER/78uPn4+gMs44jFEkYqPrj+ZiBYR0RIiWjI8vD2gGoZhGIZhZMFN/GsXIds6FS+xn4fwEgtgGSIfKH8ioukAjgLwRP3JzHwRgIuA/CQnNYHa5LXyKDwnilj15hFMqxF5OZl7yYzjAQB/WHtPciyv9ggJBqlRpjEk+5UUZeii69tuaYlvAMpOQXPglrcb/E7HR2KS17OQZdbcaXSS3M1pcuLpJfbTAC4honsRjZGPMHO6NpVhGIZhGC2hkxRinXVOimSkkJxotdE0/F1WyX0Fh6suUh9Ac+oj8bVQ8LXA0NCum2WN4bsa0a7je9yFos1S06wt8pIIaA7qQlZ8Wl+VVb+rldKuonVwKlKUEAlKEf2Ul5m1i3uDtPKL1r9ykRRrrgh83de3WufkXfNf791g31768/KaEhuGYRiG0dl0kuSk1JMTXy11TfdiUJGW+Eoc8uhWF0mIROaRPjikjf4p056RpP+8oer+X/O9IG30Xe5P1nm3Q521vqqs4uR9yNW46qdAuU6Xomfi4tLaJRCXi2MmSVbAQ6DW50KlPVyCOGqrdLkynjl2YpJeOVgNguayWtXafqfSP0Wswiv9KcePvD+pnKdJTKXEaEi0UxHSHdkP2rvKRbrXpUgaKxITFz0UbWmb131r40OOS1/fS93Ctf6wg2v9yjMrn1cX6a2UbIxwkHZJaYl0lCn9Uvk6qWxnsL9OJXNbh4guBvAKAOuEGfHrAFwI4OkATmLmJXXnzAPwAIALmfnLWZWYOO7wpBKnTT4mOX7F6tuzb0CkfZ0u5RWVWMZpySNaqosMzaXuRYh0XcrM6hPZXoN7qg+tpvAmI4A+KiKA5hWVuG9UddvPJYaNb/nNRiXWnLO5OKGSMVXkpEVzppUXvs/jSVOPStKL1z9cRJUKQ5swysjBMhaOHAfNjl1tolLEFqdvX8qYSVt2pVtgyvw9iufiLA6fMCtJ93ZVF073blqaea7mLdmXuYrzOZcFnlxwtnpb5+3zX+e9xv7O0l+U1pT4EgBn1B27D8BrANy0X+6IrwD4XfPVMgzDMAwjTzrJlNg1ts58CAds4vgNAD4kJSdE9CpETti2Axh0kZy0K7aOYbhS9ArUMHwxs+POp9WSk7fOP8v7FXTx0l+WVnLiDBGNQxRf5z/yLNcwDMMwjDA6SXKSt0LshQC+ysyDpOyXp+G7ssxrJSr3fWVAO7l/PGNcdb9eKkpJRUEXk7gKvlE/JZrymWbKJq8/SdGLcVE6dgl25kNept0h++zafUvT2Z2KCa7Wb3I/W5Yj9ViaHbshytuaObC2D+6i66Jda8GUI5L07RsebbrOPviaYrvoA7koTPv2iXzHyEB9G3aokT32K0OiOW+bOO/0JK0p1Et9CxdlV0159LjJhyTp+4TOh8v7rFmdOJdfFu09O0PoX8nAfC7XGuMwJqQ+zGNbVqWWI5VyW80B54TNg5MBnEVEXwQwAcAwEe1i5m/UZySiRQAWAUBXdz+6usbWZzEMwzAMIyc6KZ5brpMTZk6m8kR0ISKdk/0mJnHexH39yIJ1TlxWOCMVyYmUMsjjmhQDDiaw1fP8bluuAPbsy54Dy9I370q3QnEJN59350hb+xCLGymxYo92r8+vmdr6Srnk+BhwMKXMkqxJtHt1YY8S1qF3pAgfv6+6knZpS+34A1uXZeZxKccHTXIZgoslh++VpIl7s47xtGtKacnmZdclaSldkaa7Eq3NXEzyH9u2KjOPhkv5PqhjVdzflt1VKyKX34Wad4A4Qzv3ia2rU8txMWFuBZ0zNXHQOYld1/8VwFFEtIKILiCiV8du7J8H4LdEdHXRFTUMwzAMo3mGwd6fdlEK9/WjR89LKuGy9yj3yuWK1mU1UoTrY20vtyJdcXFOJNFW8nLPWvZb0e7DfdssKwify36+Vl5ewQxDxoF2rqZf0Gz/ZIUBaITUM5F6B773qrny7jTrpbyu6eKnQ/M34zsOKjoZIQFNy2bRI/VM5Pj28Tni++y6OCH0ve7YkdV32C4hddSkIkdPnJukH9q8PEm32lrn7Ke9ynv4/+Sp/zt43df7PrQh8WA0J1dymyZtggHoHiGHlAfLR3ynekeV9S1gQuKiVOoT9wLI/vH07b+8JiQS7cXk4qRJdYImJ2UeCuEaIfetxcGRnji1SaKsuRZbROt7l/bLe0LiK54PUYit9bScPT6Y0ycWLmL+tLq5tLvcvik64rGvUzN5T81u6/hOKmTufjHZ1hzFaUjvw5oiv/YelxOSdnIwK8QahmEYhlFCDqjYOor7+i8B+HsAQwAeB3A+M2+Jv/sYgAsA7APwPmYurT6KXN1qs35tVZP3at5lNVDE9o2vwmMeQ3u4AElIXoS4tJZjYl+J1ijauNG21EL6OKT9msW3vi7PUV6xUGTdDu2fmaQ1xclmcWn3IqQoIf3dji09TVriYjZ9IMTHaaffEl+adV9/LYBjmfk4AI8A+BgAENExAN4I4BnxOd8iom4YhmEYhtFWhpv4tItMyQkz3xS7r5fHrhF/3gLgrDh9JoCfMvNuAE8S0WMATkJk7VM6fFfwLrNrH4pWDCzCuV0eCsUhZrHe1xLpVq4ZOsFlfSfU8UDiSSEtybvtfZ/LonVRyoRL++Zl3isVZTW9FKnP1GqKet8S0RkAvgagG8B3mfnzdd9/AMDbAOwFsB7AW5n5qUZl5uG+/q2oBvmbDUBq/qyIjxmGYRiG0UaKMCWOd0e+CeBlAI4BcHa8iyK5E8CCeLfllwC+mFVukEIsEX0c0Uzo0pByOoW8necUvVr1Ld/FBXce5tc1liwFS06KbuNmXXD7EuKqv1XO9VpJJ0p9iqxnyHPZSilKEa4c0tDcDxQxbmSZLuE42qm7UtA2zUkAHmPmJwCAiH6KaBflgUoGZr5e5L8FwJuzCm1ackJE5yFSlH0TV2VFKwHMFdnmxMfSzl9EREuIaMnwsJ9Jl2EYhmEYxSN/q+PPorosvjsmF6C626LSlOQk3l/6MIBTmVk6QrgCwI+J6CsAZgE4AsCtaWVI9/UjPN3X5zX7djlzYu+4JC0DuElatYoL0SE5dtL8JH2vCNAl21JKNPYpwQTzcIg2sku4/h+u+jxpl06Ihkt9NGlJs3o1vmPbRVqiUYSOke/4aPbZ0cakFgSzlWj3JNty9rjJSXr5wIbgsqU+nPQf4vt+9JWiyPfjFvF+1K6qBcr0sfrR2kCTYsrxId89If6yJLIOmn8cbYy2mmasdeRvdShE9GYACwCcmpXXxZT4JwAWApgSu6z/JCLrnFEAro1fwrcw8zuZ+X4i+jkicc5eAO9m6YUoJ3y9Aoa8pLQJiaRVL8EQsb2ckEhq2tKhXfMwoa6J4xJcWnMU7dW02Umzdl5eCs2+Zfreh+/4yKP/5TVrJtsiTysDnrn8MPtMSHzjOuWFy0TF5f2o0awZsta+2kKhxsR/OJ8JiYa2ZVOEI8lmKMjPidOOCRG9GMDHEQk1Mt25u1jrnJ1y+HsN8n8WwGezyjUMwzAMo3UUZK1zG4AjiOgQRJOSNwI4R2YgomcB+H8AzmDmdS6FdoyHWF8RoEsXtEo5q2jaVfPJvX1JWkZvzkLGnRn2XPHlHb/GlU4YHdoWQiePbV/Kdq9BUbebPK8IWqk0K+NCpcVLa5US+oFIEfIbZt5LRO8BcDUiU+KL412UTwFYwsxXAPgSgHEAfhHvtixj5lc2KrdjJieGYRiGYTRPUR5imfkqAFfVHfuESL/Yt8wi3Ncfh0h8Mx7RRO1EZg62nZLSkrz0SWqC6ikrnCljxifpjTu2VeugBHZLCyzoGzhPQyq9aeI5qQwno6ZOGDU2Sa/dviVJy1WIFohLu1cfaYlEBt/aINrUBXnXMtqzL5okTiq0DQmpzl6R1iKbyracMLra3ln36KuYKtGuL/tSRvGWSoDy+G5xXAYwlOPMZQU/t29Kkl4h9CoKdeony3NQjnW5ply97xSr95D3jaznHNFOmv5J5VryuZfjUNbluMmHJOnHtq1K0lLy4Ct91owBNCnK2NmnJGmXvtTqo0WXr/SbFqVZk6TK/p42dkKSXjO4ObOO8pmSaU2ZVv5eaM99W0L8xnRSbJ283dePAPAjAO9k5mcgUqRtfbANwzAMwzBqYGbvT7vI2339SwDcw8x3x/k2ulRCrlZluG+XcO0uq07NjEtbZUmktKTGEkZ02rOmHJak79zweLXMOE+ICbBESgrk9WvSIr/U4JfSEolcYat1k9Ig0WaadU/W/cr6vnPWC5P0/6z6c2p52h70Yf2zkvSjW1Ld6ahMHF1dFUrHSTuVVZuLabDsn00eUqUJDubqLm6x5fX7hHRqpLLik9KSfiHpkcHRfFd5LlYossye7urquVnTzhppmoMpscsLV1u9SzQJzHjR9gPiHSbH0AoPax3Zr1o/3adY40l8rWO2OFjiSGnJ9pU3JWkXXRRfC5ZKG88YNzE5Jp8FzVJG9o02trURUWuWXa2vJvXZqEhL8nDBkAcHmuQkC+m+/kgATERXE9EdRPThHMo3DMMwDCMQbuJfuyCXVUQsObmyonMijn8ckUOV1zAzE9GHALwbwIkAdgC4DsC/MfN1jcr3dcJWUweRLpvTJSOdTtG271KkJe3o47x0lXz9YWiSk7KOc98Vqm/baPo7vhzo74xOCCZYtEXbB2ZVpUpfXVWVKskr7R1a2VIVlFNmn+59ozetvK4tajJNW+sI9/WnC/f1KwDcxMwb4jxXAXg2oklK/fmLACwCAOruR1fX2PosTpThwR7X05ukXWIrdBp5v0jLPCGRlMGEs4KvAz5JiIOu8rSAG74ic9+2KcKr6IFIEabHlUVNXu+Pop/vr4gJSVnopHHX1LaOcF//yjr39VcDeCYRjYmVY0+FCP4jYeaLmHkBMy9odmJiGIZhGIYbRUQlLoq83ddvjuPq3IZoknYVM/828xrK8RCzylYy2AZpSUg0Wu9rNRknRqtDlpOlevISvxYhxpXKj1KBO2QrpYJmIqvFPZLktXWmlZPb2MqpnHagjae8tsIq5fgaAEh8x0ERz4gmRembs9CrbpU8Ltut2n0U8S6Rz7qmdJzH+yAPOkkhtgj39T9CZE5sGIZhGEZJaKdpsC+l8BBb485cOtZSZ8XCSZT8xsHZmW80Wk0BTs6cJ/dWHe+s37F1v2u5XN9F+qFJMFycK2krKG0lIfPL684dW42m+tS2tUqtG9f5qP45ybG7Nz6Ren2tDCh1dDGJluVripOyv/cMV1c4momqZu4uTeKzDDhdVoKkjJwas1wxDmTb9KJ6T1InSruupEc6AHNY3foqpGaFIHBZ6co2GOGwQnUp0+m6yrjUnlNZz14H6WGlHDmWZPvKZ117Fmr7w+8ZcXmvyDyyv7WxIqUlAytuSNJSupIV1Vm2rxZJXZOWSEmni1mxJr2Uv0EuTjZ7THLiTSkmJ4ZhGIZhFEs7TYN9yTQlVtzXfxrAmYjc068DcB4zryKiNwH4CKIJ5ACAd1UcsjUixJS4DLQqyJrmTK7TGq+T9QyM/ekU03CjvORh0ePrkLOdztAqtNqU+MRZp3i/cm9bdVNbTImbdV//JWY+jplPAHAlgEqAnycBnMrMzwTwaQAX5VRPwzAMwzACOBjc10sfvWMRT1SZ+WZx/BYAc+CAi+RBm/GGrMLzknhoHZhmVeR7nZo6yvsOsKDJC21PWtOTqdx7lp5BI6TW+2jh+tzXv0zICiprT7yerD53sfrw7WHfIG8SFyshiW8ftkNy5htcMS8JqK9lmiRNZ60MhNyTpEYHx8MvSsj4aZe0pCzS4oNC54SIPgvgHwBsBXBaSpYLUHVr3xCXF4E2qEJeNHm9gFyUy5pFK6MMWtfaj56smaxnZWIRohAmzw0pJ+QllXd/q30s0r4vN98JiXbdIl7meYxc3/ZwyVPElmzIj3f7n/B0Qu5Jot1f1kRFO087XgYz3rL0ZRl+N1xpOrYOM3+cmecCuBTAe+R3RHQaosnJR7TziWgRES0hoiXDw9u1bIZhGIZh5EAnOWELiq0TfzcPkbO1irLscQAuB/AyZn7EpRKaQqzv6kjOkOV9Fa2k5zMzz0vcLttDXn+vuH5eztnyiidSoQgFyhAT7RBT71ahbUP5mnKG0I4VqO/YK0JpvGiF36x3gozarkXebRcuEmpfNwYaFSmKr8Ks7zaldm4R74BWK8QeN+N53rdxz5q/dk5sHSI6gpkfjf88E8BD8fF5AC4D8BbXiYlhGIZhGMVTpnhhWTTrvv7lRHQUIlPipwC8M87+CQCTAXwrVoTcy8wLsq4xYXQ1tk5/z7gkLZ18ydXFkFS+FLNvF6dLUqLSldMqa7wI/Ldx58B+38tpp4u0RJumyvpKtPvOa8WnOZ/z3b+s5J42pj85tnNftezNOwdTz5sxbmKSXjO4OUmHrOSls65+4Zhp446qrneI5Mmn7V1cbWtO2OR4kvnl8zKtt9reywc2iDLT0SRJso1DVpTy3CliLEgHhhV8JXVFrIa1/tMkHof0z0jSywfWJ2npYFL2s/ZOqPSnlJYcPmFWkn5sy6okrfVlmkK6K1o7aWNbe041aYnqxE65bkViIvVQ5h7+d0l6ww5pp5HOyK7qc79vOHtsac/CpN6+JL1JvPO1Fg6RmOdJJ/k5ydV9PTO/DcDbQitlGIZhGEa+HFCSk1awU6yOnjthZpKWkhN1ttnk6j06Nd1VuC/bM7TXfcvW8msm1NpKoyYUQEB9asrPoc227K4qQA85SDy0fXbt/lyQ50rrAxcLGTfLD/e6aRKomrYmv9aW0sU9nL7yD9G1CXle5Ll7lbo1SytNNjWJytBwte1dXMZrVH5Iakxuu3rSM2eU0QzamVoASBdz/hp3/orpfVaNpbRk+WPVuLKaLoosT4akcEEbT1uVsBUa7XRZL+kkyYmTtQ4RXUxE64joPnHs00R0DxHdRUTXENGs+Hg/Ef2GiO4movuJ6PyiKm8YhmEYhhvDzN6fduFqrXMKgEEAPxRWOeMrztiI6H0AjmHmdxLRvwLoZ+aPENFUAA8DmMHM6gZfiPv6oldKLuW3arUWYklSJhfjZapLM7xy5nOS9BWrb29jTYpBWsgM7atKAVwC70k6Z43WevJ2+FbEO8jlOc3rWc5jDLm4wC/D74Wk1dY6R0x9jvdtP7r+9vJa6/h4iY3/76NIZjcOwCYAhcm0ip4w5OXgKQ9C7jXkxZHXi1TzoJrHdYowIdXaVU5ItJdz3i/BEA/JvnXxVULNa/xX+t9lEhTiZbgIfMeuyz1Wcrh4M86rD+S1XNrVt+21dsqj/i5eZoswJS6L91cXDhqdE8VL7DcAXAFgFYA+AG9gDlAOMAzDMAwjmE7SOQmanDDzxwF8nIg+hshL7CcBvBTAXQBeBOAwANcS0Z/qJC1ByNWqVDyUW1QuK6uQmD4SH4deIcqoIcj7GC9MZwd2pyux1UgBRDt1i7RsDxdnWRXT38mjq2Z4qwY3JemQWf2wp5t1rV99Y+6EOJVKQ1XC1eI3ifrKPPL6PQ4xkCTySr735CvdkPnTytfqKBVNpZmmVDzMK06RhqYoLtFMZ7W+SivFZWy7mPi7KHv7PgshEiOXczUHkxW0K2pSlAnzXpSkXSSQvpJzlzhZ7aST5ARNu6+v41IAr43T5wO4jCMeQxSp+Oj6E8x9vWEYhmG0jk5yXx8S+C/VSyyAZQBOB/AnIpoO4CgAT9Sfz8wXAbgIAHpGzUlagJSVec25osHkzHpIMTeWs1bpfKtHpLtELmna3Deq6mBNShlkHbqoOsdLMxmT99TlIHnYJ2a3crUgHf/IOkpJhVw1SQmTvI8tirMzuQqXbSYdF8n7lvXf7aCnUGmb2b1TkmNrd2xJLU+irWpCJBV9wnHeNmEWOKV3fJKWTsFkH8r2kAqjMs/0DOdiss7a6lAerzFNVvp4j3LuCRMPTdKL1z+cpKUDtI07hfM5Th/bw4rZr+wHOUY1p1jafaX1v4u5szxPkxq4PIOSWulidXzI0jXJkJQ4yP6R9Irnfeuu9MVZ5VrynSXrLqVgM8ZWHaBJU335bpDOBrco19See81h2bSxE1LL1KSnsv41bSzMkGWfyDFdaVdZL2kaLCVM8l0mpSVblv0xSbvoosj6ThxddRAqHa/JsTVZeX9I6WU79T46KfCf0+TE00vspwFcQkT3IhrrH2HmDfuXahiGYRiGsT9OpsRFE2JKXAbyNgs80NH2x31bLqTdXXQQupVwAdpq21dfxacuRQfdc9Gbmq64Jy8TUx0kVhJft+J5jd0iLTzaZT1S4xiyA96JLqbHGi7jwOX90WpT4jmTjvXujBWb7iuvKbFhGIZhGJ1NGYQRrhzUkpNmV7p50Un28XlShiBYIVYondZX7R7nRsTrZ56UpH+++tY21qR4SZwL7XBeqemWhEhRJL7vlVZLTmZOOMa7qVdveaAtkhNXD7EXA3gFgHUVD7Hiuw8C+DKAqVK3hIhOBPBXAG9k5l82Kt9lcuJioqUVMrZndJLW4rSUFZfB7vuQt/vFVPQ2WCdPJEIowlPrwdqWZaCiMLrNM46LEUZeExWJ9s5r9eRkxoSnez/Ga7Y82JbJiasp8SUAzqg/SERzAbwEkYWOPN4N4AsArgmsn2EYhmEYOcDM3p920bT7+pivAvgwgF/XHX8vgF8BODGodgJthe3iUrhoaYnP6tJ3JSrvW3OV/ooZz07SV627K0lrInxfaYkmefK9lzEjRwGojUSsmae6oIlrSXzjGxNEljl3/LQkvWzbuqbrmQcuMUw0hdUQ6UeZpCUuIQo0s+wy3UcjKs8IUJWYHDNpXnLsgU3L9jsHaJ+ES45L3wjMvnWu5HdxJhfifsDFDb4vZVEKbqffEl9C/JycCWAlM99NtV4QZwN4NSJ39rlNTgzDMAzDaJ4y6Ji60tTkhIjGAPhXRFs69fwXIt8mw3LSUhQhboflSuxNM05O0j9c9dckrSlvuihZVVZxmuMkX1427fgkffPWR5P0b9bckXluyEpCSktcVvDatXbs2Q2gdnXou9rS8FX01O7joSOrKlVPf/T+JC3vaUJv1RnTZuHQTo4n6SgqK5CeywrSJcCaJi3R6vtusRL8llghdnuuhuW1+oRjLRddCZ/Vs4uTPhepoIuCsByjlXHbDFqojZ7u6ntFjo+0az2oSEskY4R0UzqU3L6n+uz2invKS5rsEkpEwzcIX6VElzGjHZeO1DYKR2oavlKUduvzZVEWCY4LzUpODgNwCICK1GQOgDuI6CQACwD8ND4+BZGztr3M/H+yACJaBGARAFB3P7q6xjZZFcMwDMMwsugkyYmzKXGsc3JlvbVO/N1SAAvqPcES0SXxOQ2tdQ6ZfHxSCel6Wbo0lnR76hdIQvZINTRX1xXJke9stUuROLkEDpO0y4Q0yxmTdJS1QTjK0lrp74VOjZQS5eWETaKV4ms15bMq1FaCLlIqjZFKmAZtxazVQYZV0EIUhDjPk+WHSCgqNKvH0Ez+MYou1hETZifppQNrk7RcSfs4AZTjZ25fNfTD8oF0x9uy7HEiTIP2Ps0LFxcBsu+7ldALWUwWIRK2ivetyzOSl6M4KUXpm7MwtQ7yWoeLMfHolpXV/C221ukfd5j3T93WwcfLa60Tu6//K4CjiGgFEV1QbLUMwzAMw8iTTrLWKYUTNhn4z8UldMjstwit9iwfLEX4mXC5j7zuNcRhmaRypu++bEgb5IWLzk5N0MUc9p6blcQA+fn2Kdr6Je8+bKXkROuTvPzN+FinaOT1rOXFKCUAns8z4tK+vs+r771KqebAihuStNRFkXUYJSSEUseo1ZKTcWMO8e7WwR1PHrzu631FajW5Pc/tVqKEaiLBecKcdJ8we105sNH5mi5mqy7ifC1ysfZgyzJn901O0rLuLvEfZP/4bhWl9Y62beVTBuD/gy0Zrbwsumotz1LPHVbaRttKyXrxam2qKaa6sENMSLTJpRxnMtp07f2lm+xKZCtNcYhto22tNkvI9p6WW3sGa5TuHdwbaM/sVBG9dtXgpobluEwYDp8wK0k/sXV1kpZjT3vuxo6sTmTl1o/LBHfKmOp9bBRRqLV6akYFPvguxuR9a1GDXcaQNtFzUZqV7xj5m9JquPBlXH6UYnJiGIZhGEaxdJK1Tua2TprreiK6EMDbAayPs/0rM18Vf/cxABcA2Afgfcx8dVYlQmLrmHvtziOv2DpF972veL6sMWzyMm/shOjbvn2gSdBc8ksF4XK2Rvv4wKxTkvRXVt3UxprohDyvLu8GKUUZN+fU1Gu1eltn9Oh53kN1165lpVWIvQQprusBfJWZT4g/lYnJMQDeCOAZ8Tnfil3ZG4ZhGIbRRriJf+0ic1ungev6NM4E8FNm3g3gSSJ6DMBJiCx9VOS+5ZZdVVNiTQ9khnDTLSU/a7dvSc2v6VVoLrC1c+WMV5oPy73k7Sl7/SFmrlKRSraH5vDIRflL6mq4mCRre9UuJsxy774iJTls/Mzk2EObl2eWN1XRY3BRSPTtV81FutO1RP3lfv1OYSLbRdVyKv0pryn3/6UDLZd+knWZJMwtd4oVviY50drsqIlzkrQ0gdTaz1dROw+T6xB8TbT3DFfbTzPTlX0vpTHy/lzMxCvlyHeKJumRbSOVsbVnzQVfyYLM/1VFWuJi4p7Vz1LyuqcmFEb2c9EjztXa3UVCKE2Dlw+uT9KyT6S0ZHDFjUk6r2CCzVCUAQwRnQHgawC6AXyXmT9f9/0oAD8E8BwAGwG8gZmXNirTNfBfGu8honuI6GIiqswWZgOQT8CK+JhhGIZhGAcY8e7INwG8DMAxAM6Od1EkFwDYzMyHI4rJ94XMcl1mUvUO2IhoOoANiCa2nwYwk5nfSkTfAHALM/8ozvc9AL/LcsIWonMSsm/osnqRs+hnTHxakr5309Ik3eyKLmQP3/e+XeroUp889CrKoLvQSn2VrPLL4CyvrKbBZb1mM3R7Six9CNErCgns2Qltr0mVinAvP88hWKjURRk55dCW6nOMbOK3dk+GXgwRPQ/Ahcz80vjvjwEAM39O5Lk6zvNXIhoBYA2AqdxgAtKU5ISZ1zLzPmYeBvAdRFs3ALASwFyRdU58LO2GFhHREiJaMjy8PS2LYRiGYRg5wU18HHDZMUnyMPNeAFsBTEYjHD3EzQdwn/h7pkj/MyI9EyBShL0bwChEsXeeANDt4YlukafnOuf8RZbd6fnLVJey5S9TXexerW3KmL9MdSlbft+yy/hBFANvifgsqvv+LER6JpW/3wLgG3V57gMwR/z9OIApDa/rULGfAFgNYA+iGdEFAP4XwL0A7gFwRd1k5ePxhR8G8DLPRlhSVP4iy+70/GWqS9nyl6kudq/WNmXMX6a6lC2/b9md+AHwPABXi78/BuBjdXmuBvC8OD0CkVoINSrXxVrn7JTD32uQ/7MAPptVrmEYhmEYHc9tAI4gokMQqXG8EcA5dXmuAHAuIsvdswD8keOZioZ5iDUMwzAMoymYeS8RvQeRdKQbwMXMfD8RfQqR5OgKRAKN/43di2xCNIFpSNkmJxcVmL/Isjs9f5nqUrb8ZapL0fnLVJey5S9TXcqWv0x1KVt+37I7Eo4csV5Vd+wTIr0LwOt8yixFVGLDMAzDMIwKIU7YDMMwDMMwcscmJ4ZhGIZhlIq26ZwQ0dGIYvFUnLWsBHAFMz/oeP4PmfkflO96ECncrGLmPxDROQCeD+BBABcxc/OhcA3DMAzDKJS26JwQ0UcAnA3gp4h8pwCRN9k3InLoVh806Ir6IgCcBuCPAMDMr6zLfymiidcYAFsAjANwGYDTEd3zuTneTksgomnMnO4PuSTEbokvAPBqALPiwysB/BrA91wmhUT0CDMfqXx3KIB/A7AKwOcRxWh4HqJJ579wRiCpZiCiycy8Me9yRfml6ddG93qg9W3R/WoYRiBtctryCICRKcd7ADyacvwOAD8CsBDAqfH/q+P0qSn57xHOXtYi9lKLaFJzT0H3NLnBd/2IXrgPITKj2ojopft5ABNS8k+q+0wGsBTARACT6vIuAHB93D5zAVyLyDXwbQCelVJ2N4B3IIqJ9IK67/4tJf97EHvyA3A4gJsQTfgWA3hmXd6fAPg2gOcimmzOidPfBvCzlLIHAGyLPwPxZ1/leEr+mwC8C8BHEXkc/GB8zxcgspuvzz8ewOcQOQ08p+67b6Xk/7y41wWIPBw/BuApZZwV1q++fduCey1N3xZ9r3XnTgfw7Pgz3eN98MqM70eI9Li4XvuNgbpz0t6Z+3nZRPQeJfH3aXF7qk4xEb0bTwbwmvhzMjKcZNWd/48NvptXeR4QeRs/C8CxrmX7fBC9o14L4Jgi+jWrb5vpV/sobdmWi0Yv86elHH8agIdTjnchcpN/LYAT4mNPNCj/vvgBnRi/DCfFx0cDeDAlf9Ev9qsBfATADHFsRnzsmpT8wwCerPvsif9/oi7vrYiiQZ6NKHbBWfHx0wH8NaXs7wL4MYB/AnA7gK+I7+5IyX+/SP8WwKvj9EIAf6nL+0iDPtnvOwBfRxRGe7o49mSDMu4U6WXad+LYr+K+ehUiJ0C/AjCqwb3eK9LXAzgxTh+JFE+PRfarb9+24F5L07dF32v83QkAbkE02fxD/HkoPvbsuryvqfu8FlFgs9cAeE1K2echmsg+EvfvEwCui/v47JT8pyGSMG8AcA2A+eK7tPu9G8DEOP0vAG5GJJW6FsDnUvK/BNH763eI3g/fBfD7+NhLUvJ/oO7zwbhuHwDwgbq8H0U0vh8C8Lb4/+8BuL8+b5z/mXEbL0dkhjtRPg8p+a9H9V38lrhNv4vIg/l7Q/rVt299+9U+jT/tuShwhngYLoo/lYfhjAbnzQHwCwDfQN0LrC7fP8cD4ykA74sHyHfiAfvJlPxFv9j3m3A1+i5+2H8PIZmA8mKH/w/2PSI9Im77yxDFQ0rL/7BI36aVFf99CyJb9i5xrAvAGwAsVur/HETbc++L8zaadN4et/GJiF6GC+Ljh9fXJT5+V93fHwfwF0QSi7R+fRDxygdRdO3UPm9Fv/r2bQvutVV9e1Jd3x6RMs4KvdfKNQCcnHL8uQDurju2B8CVAC4G8P34MxD/f3FKGfcCmIIo/tg2AIfFx6cr4/g2AM+I02cBeBTAc9PGQXxMxkFbAqA3To9Qyn8QYsIjjh+C9MXcAICfAfgEgE/Gn82VdF3e+wH0xn0zgCgSLQCMlfUU+f+M6PdhAoAPxecf5nivtyGWYCPa0k99J7j2q2/f+varfRp/2nfh6GX1XEQz0dfGaacggQD+DsB/ZuSZBWBWnJ4QP9QnKXnvqvs77xf7NQA+jNpV5HREK+w/KHWqTMS+AqAPyosdkTvglyD64XgKwKvi46cifaL0UMqxT8b3m7al9lkAlwA4FMC/IpK4PA3A+QCurMs7H9FLaz2i1cOjANbFxw7JGAvvA/AnRErMWr7TEcVsehDACxFNIivXeJXST111x85D9MJ7KiX/e+O+ehGACwF8LW7H/wDwv63sV9++bcG9Vvp2Xdy3j7Swb89s5b3G5+z3LIjvHqv7+0REC6B3iWNPNjj/LpFeVfdd2g9q/WToGXFbvQrp76ebEW+bIJoMV6Qoo5E+IXgUYjtCHO+pv9f4+Lx4DH8BwJj4mPZ+qmyxd8d9KSe3aXWpv9fT4vo9V7nXOwHMjtPXAxgtrnd/Sn7nfvXtW99+tU/jT9srUIZP0S87RNtLX0AkPtyMSD/hwfhY1j7zKxGtWtco3x+PaHvhdwCOjuuyJa7781Py/wgp0ilEItc9yjXOQ6RjsgHRquEBAP8JoL9BvSfHnx959MNMABs9++7K+r4T330RwItTjp+hvaQQbVf9LH7p3YvI6+EipO/3F9avDfp2c9y39fpCzdzraR732oMoNsbfxv36JgDfAvDuBvn/oVInRCL3JwD8Y0b5lfxq+YH9eoe413ek1SXO/3VE25hvQGTp9/w4/VvURVyN83cBeD+iH8iT0HjSeQWibeRvIJIs/X8AXoBokXB1Sv4lEFuH8bE5iKQAAyn5j0O0tfPD+PM4opX+EtRtW8f5PxaPgY8giolyTpy+E3UB3OrOOxPRouYs7X4RLWx+jEhx+ieIts7fhGhr5+cp+e9G3Xslvp9HkfJuiPv1fgCfitvz5rgdrwXwodB+9elb3361T+OPeYgFQERfRKQj8Ie642cA+G9mPiLlnIWIFPiORCQuXQ7g/xCJ+vam5D8a0QvlFmYelNdg5t8r+WcjmhTsQyQivC8tPxE9HZGkaLFj2ScBYGa+jYiOQfRSf4gjF8Rp7SPzPyPO/2B9/hSrKiCawGlWVYXmT7mPFyJ6udzHzNc0yhvn/5s4/71p+YnoZETttpWIxiDaX382opflfzLz1pT8DzLztjj/hXH+2xvkr5Tfi+hHJLV8InofgMuZeXnWfTWZv2IB14tIKXcsgMuhWMClWMy55nct/1BE+/5zET0fjwD4MTNvU+p/KCIJbSX/w43yx+e8DOnuDlKfk/ic2YgsjRYw86FKnvGIJl2M6IfsDEQLgGUAPs3Mq+vyvxjAema+u+54P4D3cBRstf4a3YikbpX30wpEP5BblDodg2jCXH+vD2j3Gp83FtE4PpmZT0n5fgQiyR8D+CWi5+mc+F6/yczb6/Kfg+jH/5a64/MA/Dszvz3lGv1xmfJef83MDyl19u7X+LxZAP4LSt+m9OtLEUmYnwLwmfp+NTJo9+yo7B8A54fmRyTWfhjR5GUphJga6aJK5/xx3oc8yv4kohX7EkSz/D8C+HdE1hIfd8h/nZYf/lZVvvnv9Mx/q0i/HdFK85OIVnsfzcj/tvh6jfLfj+r23kWIfpReGJ9zWSvzI/pBX4Vo++QfEe/tNxinMv+7kGL1UZffywKuyPyIxvw1iJQ8bwbwTUTbjw8AWJhS9vt98tvn4PoAmOaZ38dqy6ts+4i2a3cFyv5BA8Vb1/yIxMjj4vR8RD/074//vjMkf5NldyNa0W4DMD4+3qv8aDjnh79VVdH57xTp21CrjJemG+Sb/0GRrp803tXK/IgmUl2IVsvfQ6T383tEWyV9affqmd/XAq6w/JUxGafHALghTs9rNOZd88ff9SNSkn8QGWbi8Dcp98rf6APgd6H5UWuteHbdd2nWis7WjT5542MzEJmnfxPR9uGFcf/9HMBMx/z3NMhfb84/CY3N+Z3z+5Ztn8afskUlbgtEdI/2FSIFx6D8iHQiBgGAmZfGW0K/JKKnxeeE5Pctey8z7wOwg4ge51iszcw7iWg4JD8zDwP4KhH9Iv5/LRp4IS46P4AuIpqI6EeYmHl9XM52Itpv662J/PcR0fnM/H0AdxPRAmZeQkRHItLyb2V+jtvnGgDXENFIVM2QvwxgamD+7yH6Me1GpDD+CyJ6ApGi4k9T6l50/hGItmdGIfInAWZeFt9HGr75f45IqngaM68BACKagWj75eeIJnX1eRfW5T03JW+j/Gllg4ierdSREJnG1h70zI9IH+VRRErIbyWisxBNJHYjav+s/K9tkN8nLxDpqPwW0YLgegCXAng5IuXf/0G0HZOV/+8a5N+AaJtFMhuRFJcRKf43m9+3bKMR7Z4dleGDSIx8AiIrFPmZjxQLgyby/xHxSl8cG4FIWW1fSP4myl6Mqoa91JzvR/o2kFf+unMzraqKzI9o1fIEYj8iiFdSiH6c7sohfz+il+PjcTvtic+7EcDxrcwPRQIQfzcm5ZhX/vi4swVckfkRbdPcg8g9wEOIt1IRTahuCs0ff+dsJu6Tt8n8+xA959enfHbmkP+uur+zrBWd8zdR9p0iXW8+n/YM+ub3Nef3cevgVbZ9Gn/aXoEyfBCt2l6ofPfjHPLPQZ22vfjuBSH5myh7lJJ3Cuo8vjaTvxM+iET7qvmrb35EouvjEfn1yNyPLiI/gCM928Arf9k+iMxpzwJwdEH5nc3EffI2mf8+AEco9VyeQ35fa0Xn/E2UfbdIf6buu7StVa/88XFnc37f/L5l26dBu7e7Avaxj33sU7YPas3EN6HWTHxis3mbzH8WgKOUer4qh/xeptk++Zso+1OIdejqjh8O4Jeh+evyZJrzN5vft2z77P8xU2LDMAwPhB5QrnkPtvxlqEtsol9x05Brft+yjVpscmIYhuEBES1j5nl55z3Y8pepLkXn9y3baGzpYBiGcVDiY5FXtLVfJ+cvU12Kzt+EFafRAJucGIZh7M90RB4+N9cdJ0SO3JrNe7DlL1Ndis7vW7bRAJucGIZh7M+ViBQt76r/gohuCMh7sOUvU12Kzu9bttEA0zkxDMMwDKNUdLW7AoZhGIZhGBKbnBiGYRiGUSpscmIYhmEYRqmwyYlhGIZhGKXCJieGYRiGYZSK/x/XUi6DZAmmjAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,5))\n",
    "sns.heatmap(estimator.W)\n",
    "plt.title(\"Similarity between intents on CLINC-150\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ad060f5-abea-46f8-b521-a00c63792615",
   "metadata": {},
   "source": [
    "While this dataset has over 150 classes, we can find some pairs with high similarity (the red points the matrix).\n",
    "\n",
    "Parsing this figure is hard with such an high amount of classes so we will need to dig down.\n",
    "\n",
    "As a sanity check, we can compute the entropy of each row of the W matrix. We would expect the `oos` class to be the most \"confused\" and hence have the highest entropy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "15cd184e-a659-4e8f-80a9-17f3d05d18ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class that is the most confused (highest entropy): oos\n"
     ]
    }
   ],
   "source": [
    "from scipy.stats import entropy\n",
    "import pandas as pd\n",
    "entropy_per_class = entropy(estimator.W / estimator.W.sum(-1)[:, None], axis=-1)\n",
    "print(\"Class that is the most confused (highest entropy):\", class_names[np.argmax(entropy_per_class)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f23770ca-7bd7-4e4f-9f77-0b1dd1db9395",
   "metadata": {},
   "outputs": [],
   "source": [
    "pairs = list(zip(*np.unravel_index(np.argsort(estimator.W, axis=None), estimator.W.shape)))[::-1]\n",
    "pairs = [(i,j) for i,j in pairs if i != j]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "256abe2b-70f4-43df-aaaa-76b81af510b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Most similar pairs\n",
      "                                     Intent pair  Similarity\n",
      "0             change_user_name <> change_ai_name       0.552\n",
      "1                     recipe <> ingredients_list       0.492\n",
      "2                 what_is_your_name <> user_name       0.412\n",
      "3                                      yes <> no       0.388\n",
      "4                               timezone <> time       0.384\n",
      "5                        pto_balance <> pto_used       0.300\n",
      "6  confirm_reservation <> restaurant_reservation       0.292\n",
      "7          change_user_name <> what_is_your_name       0.284\n",
      "8          shopping_list_update <> shopping_list       0.276\n",
      "9                    calendar <> calendar_update       0.272\n"
     ]
    }
   ],
   "source": [
    "print(\"Most similar pairs\")\n",
    "lst = []\n",
    "for idx, (i,j) in enumerate(pairs[::2][:10]):\n",
    "    lst.append({\"Intent pair\" : f\"{class_names[i]} <> {class_names[j]}\", \"Similarity\": estimator.W[i,j]})\n",
    "print(pd.DataFrame(lst))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9740a99-8331-4fca-aa7f-26a7a36c4226",
   "metadata": {},
   "source": [
    "### Analysis\n",
    "\n",
    "By looking at the top-10 most similar pairs, we get some good insights on the dataset. While this does not 100% indicates that the classifier trained downstream will have issues with this pair, we know that these intents are similar. In consequence, the classifier might not be able to separate the pair easily.\n",
    "\n",
    "We can further investigate and look at the samples that contribute the most to this decision. Let's take the most similar pairs, `change_user_name <> change_ai_name` and `recipe <> ingredients_list` ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ca48a058-1b90-4e89-836f-d0c8f83e18e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "change_user_name <> change_ai_name\n",
      "\t i need to change my username\n",
      "\t change my name to master\n",
      "\t please change my name to loverboy\n",
      "\t can i change my nickname\n",
      "\t change the name that you call me\n",
      "\t please stop calling me dave, my name's bob\n",
      "\t is there a way i can change the name you use to address me\n",
      "\t i need to change my name to something else for you to call me\n",
      "\t can i change what you call me\n",
      "\t i'd like you to refer to me as bob\n",
      "change_ai_name <> change_user_name\n",
      "\t i want to start calling you marie\n",
      "\t i want to start calling you bob\n",
      "\t i'm calling you rupert from now on\n",
      "\t from now, your new name will be lord vader\n",
      "\t how can i change your name\n",
      "\t if i may, i'd like to call you slave\n",
      "\t i want to change the name i call you to juan\n",
      "\t i want to change your name to another name\n",
      "\t rename yourself to betty\n",
      "\t you need to respond to a new name called joey\n",
      "-----------------------------\n",
      "recipe <> ingredients_list\n",
      "\t how do i make the perfect omelette\n",
      "\t please show me a recipe for chili\n",
      "\t can you search for the best new york style cheesecake recipes please\n",
      "\t how do you create pork chili\n",
      "\t what is a good chili recipe\n",
      "\t how do i make lassagna\n",
      "\t please find me a recipe for spaghetti now\n",
      "\t can you give me a recipe for german chocolate cake\n",
      "\t find a recipe for german chocolate cake\n",
      "\t can you instruct me on how to make german chocolate cake\n",
      "ingredients_list <> recipe\n",
      "\t what do i need to make a cajun chili\n",
      "\t what do you put in chili\n",
      "\t what ingredients are in sushi\n",
      "\t i want to make a meat stew what ingredients do i need\n",
      "\t show me the recipe for broccoli cheddar soup\n",
      "\t what do you mix to make a chocolate cake\n",
      "\t what do you need to make general tsao's chicken\n",
      "\t what ingredients go in mashed potato\n",
      "\t what would i need if i wanted to make chicken marsala\n",
      "\t what ingredients do i need to make spaghetti\n"
     ]
    }
   ],
   "source": [
    "def show_most_confused(source_intent, target_intent, estimator, class_names):\n",
    "    print(f\"{class_names[source_intent]} <> {class_names[target_intent]}\")\n",
    "    closest_to_second = np.argsort([sample.sample_probability_norm[target_intent] for sample in estimator.similarity_arrays[source_intent].values()])[::-1][:10]\n",
    "    dataset_indices = estimator.class_indices[source_intent][closest_to_second]\n",
    "    for di in dataset_indices:\n",
    "        print(\"\\t\", ds[int(di)][\"text\"])\n",
    "        \n",
    "first_pair = pairs[0]    \n",
    "show_most_confused(first_pair[0], first_pair[1], estimator, class_names)\n",
    "show_most_confused(first_pair[1], first_pair[0], estimator, class_names)\n",
    "\n",
    "print(\"-----------------------------\")\n",
    "second_pair = pairs[2]    \n",
    "show_most_confused(second_pair[0], second_pair[1], estimator, class_names)\n",
    "show_most_confused(second_pair[1], second_pair[0], estimator, class_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb42e119-3e88-4b35-906e-7e49957b7e28",
   "metadata": {},
   "source": [
    "From the top-5 most confused examples per pair, we can see that the sentences are quite similar.\n",
    "While a human could easily separate the two intents, we see that the sentences are made of the same words which might confuse the classifier.\n",
    "\n",
    "Some sentences could be seen as mislabelled. For example, in `ingredients_list` we have the sentence `\"show me the recipe for broccoli cheddar soup\"` which could be from the `recipe` intent.\n",
    "\n",
    "Of course, these features come from a model that was not trained to separate these classes, they come from a general-purpose language model. The goal of this analysis is to give insights to the data scientist before they train an expensive model. If we were to train a model on this dataset, the model could probably handle the confusion between `ingredients_list` and `recipe` by fitting the word *ingredients* for the former and *recipe* for the latter.\n",
    "\n",
    "\n",
    "## Conclusion\n",
    "\n",
    "In this tutorial, we covered how to conduct a data-driven analysis for CLINC-150, an intent classification dataset. By using sentence embedding and the `spectral_metric` library, we found the intents that would be the most likely to be confused and a potential mislabelling problem between `ingredients_list` and `recipe`.\n",
    "\n",
    "Following our analysis, we could take the following actions:\n",
    "1. Upweight the intents `change_user_name` and `change_ai_name` during training for the model to better learn to separate them.\n",
    "2. Merge `recipe` and `ingredients_list` as they are quite similar in intention.\n",
    "3. Analyse other overlapping sentences to find mislabelled sentences.\n",
    "\n",
    "If you have any questions, suggestions or ideas for this library please reach out:\n",
    "\n",
    "1. frederic.branchaud.charron@gmail.com\n",
    "2. [@Dref360 on Github](https://github.com./Dref360)\n",
    "\n",
    "\n",
    "If you have a dataset that you think would be a good fit for this analysis let me know too!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e109f068-dd50-4488-8861-b4784ac79989",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}