diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..19aec8e
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1 @@
+English/RL\ Tutorial\ 2/Deep\ Q\ Learning-Update.ipynb 
diff --git a/English/RL Tutorial 2/.ipynb_checkpoints/Deep Q Learning-checkpoint.ipynb b/English/RL Tutorial 2/.ipynb_checkpoints/Deep Q Learning-checkpoint.ipynb
deleted file mode 100644
index bad4559..0000000
--- a/English/RL Tutorial 2/.ipynb_checkpoints/Deep Q Learning-checkpoint.ipynb	
+++ /dev/null
@@ -1,1482 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Reinforcement Learning Tutorial -2: DQN"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### MD Muhaimin Rahman\n",
-    "contact: sezan92[at]gmail[dot]com"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "In our last tutorial, I tried to explain "
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "In this project, I have worked on Deep Q Learning . This is the first part which solves the problem of [Cart Pole](https://github.com/openai/gym/wiki/CartPole-v0) by ```Open AI``` Gym. The concept is based on the ground breaking [work](https://arxiv.org/pdf/1312.5602.pdf) of Deep Mind. The CartPole video is available [here](https://youtu.be/IKobsp1WszI). Part 2 of this notebook is on Mountain Car Problem which can be accessed from [here](https://github.com/sezan92/DQL/blob/master/Deep%20Q%20Learning%202.ipynb) . "
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Rule"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The rule of the game is simple . The cart must balance the pole i.e. keep it within a specific angle. For each step within the angle the cart will get 1 points. The cart will be considered winner if it gets 200 points. \n",
-    "** Episode will terminate if **\n",
-    "* Pole Angle is more than ±12°\n",
-    "* Cart Position is more than ±2.4 (center of the cart reaches the edge of the display)\n",
-    "* Episode length is greater than 200\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![CartPole](Cartpole.png)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Importing Packages"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "metadata": {
-    "collapsed": true
-   },
-   "outputs": [],
-   "source": [
-    "import gym\n",
-    "import numpy as np\n",
-    "from collections import deque\n",
-    "import random\n",
-    "from keras import Sequential\n",
-    "from keras.layers import Dense\n",
-    "from keras.optimizers import Adam\n",
-    "import matplotlib.pyplot as plt\n",
-    "%matplotlib inline"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Setting up Environment"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[2018-03-06 22:30:47,690] Making new env: CartPole-v0\n"
-     ]
-    }
-   ],
-   "source": [
-    "env = gym.make('CartPole-v0')"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Hiperparameters"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {
-    "collapsed": true
-   },
-   "outputs": [],
-   "source": [
-    "legal_actions=env.action_space.n\n",
-    "actions = [0,1]\n",
-    "gamma =0.95\n",
-    "lr =0.5\n",
-    "num_episodes =1000\n",
-    "epsilon =1\n",
-    "epsilon_decay =0.995\n",
-    "memory_size =1000\n",
-    "batch_size=100\n",
-    "show=False"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Change show into ```True``` for visualization "
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### The Concept of Experience Replay"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "One of the challenges of Deep Learning based Q Learning algorithms is that, there is a high chance that every state of an agent is related to previous states. Which may not be the case always. Specially when we are giving actions randomly. So we need random states with random actions.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Here comes Experience Replay, which is nothing. Let the agent take random actions for , say 1000 times. Save those data into memory. While training the agent, feed the data randomly to the architecture. Also update the memory with new data while training goes on."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "So our steps are as following\n",
-    "* Initialize Memory\n",
-    "* Define Deep Learning Architecture\n",
-    "* Save N number of Random states with their rewards , actions and resulting states\n",
-    "* Start Episode\n",
-    "* Take some actions based on $\\epsilon$ greedy policy\n",
-    "* Save those states,actions, rewards and new states in the memory\n",
-    "* Take random minibatch from the memory and train the architecture based on the minibatch\n",
-    "* Start New Episode\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "For our memory, we will use special data structure of python ```deque``` . It is a special type of list which has a limit of its elements. The moment you append another element it will remove the first element of it. It is working like a open tube with two open ends"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {
-    "collapsed": true
-   },
-   "outputs": [],
-   "source": [
-    "memory=deque(maxlen=memory_size)\n",
-    "s=env.reset()\n",
-    "s = s.reshape((1,-1))\n",
-    "a=env.action_space.sample()\n",
-    "new_s,r,d,_ =env.step(a)\n",
-    "new_s = new_s.reshape((1,-1))\n",
-    "experience=(s,r,a,new_s)\n",
-    "memory.append(experience)\n",
-    "s = new_s\n",
-    "for _ in range(memory_size):\n",
-    "    a=env.action_space.sample()\n",
-    "    new_s,r,d,_ =env.step(a)\n",
-    "    new_s = new_s.reshape((1,-1))\n",
-    "    if show:\n",
-    "        env.render()\n",
-    "    if d:\n",
-    "        r=-100\n",
-    "        experience =(s,r,a,new_s)\n",
-    "        s=env.reset()\n",
-    "        s = s.reshape((1,-1))\n",
-    "    else:    \n",
-    "        experience =(s,r,a,new_s)\n",
-    "    memory.append(experience)\n",
-    "    s = new_s\n",
-    "env.close() "
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Deep Q Learning"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Vanilla Q learning works best where the state and actions of the agent is discrete . Like the following diagram , which is taken from [A painless Q Learning Tutorial](http://mnemstudio.org/path-finding-q-learning-tutorial.htm) "
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![Painless](q_matrix5.gif)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Here the actions[along rows] are discrete as 0-5 and states [along columns] are also 0-5 . For state 0 the maximum Q value is 80 which is in column 4. So the best action will be 4. But in our case, the Cartpole game, the states aren't discrete. We had to discretize in the previous [version](https://github.com/sezan92/CartPoleRL/blob/master/CartPoleQLearning.ipynb)."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Other than discretization , we cannot make the infinite states. So the best way is to train a neural network ,which given a state , gives output of Q value for each action. The agent will choose the action with maximum Q value and it will go on."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![DQL](DQL.png)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Our Model"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "_________________________________________________________________\n",
-      "Layer (type)                 Output Shape              Param #   \n",
-      "=================================================================\n",
-      "dense_1 (Dense)              (None, 1, 20)             100       \n",
-      "_________________________________________________________________\n",
-      "dense_2 (Dense)              (None, 1, 20)             420       \n",
-      "_________________________________________________________________\n",
-      "dense_3 (Dense)              (None, 1, 2)              42        \n",
-      "=================================================================\n",
-      "Total params: 562\n",
-      "Trainable params: 562\n",
-      "Non-trainable params: 0\n",
-      "_________________________________________________________________\n"
-     ]
-    }
-   ],
-   "source": [
-    "model = Sequential()\n",
-    "model.add(Dense(20,activation='relu',input_shape=(1,4)))\n",
-    "model.add(Dense(20,activation='relu'))\n",
-    "model.add(Dense(2,activation='linear'))\n",
-    "model.compile(loss='mse',optimizer=Adam(lr=0.01),)\n",
-    "model.summary()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Training"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Episode 0, Failed! Reward 16\n",
-      "Episode 1, Failed! Reward 14\n",
-      "Episode 2, Failed! Reward 21\n",
-      "Episode 3, Failed! Reward 34\n",
-      "Episode 4, Failed! Reward 12\n",
-      "Episode 5, Failed! Reward 13\n",
-      "Episode 6, Failed! Reward 47\n",
-      "Episode 7, Failed! Reward 35\n",
-      "Episode 8, Failed! Reward 11\n",
-      "Episode 9, Failed! Reward 88\n",
-      "Episode 10, Failed! Reward 24\n",
-      "Episode 11, Failed! Reward 9\n",
-      "Episode 12, Failed! Reward 17\n",
-      "Episode 13, Failed! Reward 20\n",
-      "Episode 14, Failed! Reward 23\n",
-      "Episode 15, Failed! Reward 19\n",
-      "Episode 16, Failed! Reward 14\n",
-      "Episode 17, Failed! Reward 24\n",
-      "Episode 18, Failed! Reward 19\n",
-      "Episode 19, Failed! Reward 12\n",
-      "Episode 20, Failed! Reward 20\n",
-      "Episode 21, Failed! Reward 20\n",
-      "Episode 22, Failed! Reward 23\n",
-      "Episode 23, Failed! Reward 22\n",
-      "Episode 24, Failed! Reward 24\n",
-      "Episode 25, Failed! Reward 14\n",
-      "Episode 26, Failed! Reward 15\n",
-      "Episode 27, Failed! Reward 38\n",
-      "Episode 28, Failed! Reward 61\n",
-      "Episode 29, Failed! Reward 44\n",
-      "Episode 30, Failed! Reward 23\n",
-      "Episode 31, Failed! Reward 18\n",
-      "Episode 32, Failed! Reward 55\n",
-      "Episode 33, Failed! Reward 16\n",
-      "Episode 34, Failed! Reward 38\n",
-      "Episode 35, Failed! Reward 29\n",
-      "Episode 36, Failed! Reward 15\n",
-      "Episode 37, Failed! Reward 42\n",
-      "Episode 38, Failed! Reward 30\n",
-      "Episode 39, Failed! Reward 27\n",
-      "Episode 40, Failed! Reward 59\n",
-      "Episode 41, Failed! Reward 21\n",
-      "Episode 42, Failed! Reward 14\n",
-      "Episode 43, Failed! Reward 22\n",
-      "Episode 44, Failed! Reward 48\n",
-      "Episode 45, Failed! Reward 12\n",
-      "Episode 46, Failed! Reward 15\n",
-      "Episode 47, Failed! Reward 12\n",
-      "Episode 48, Failed! Reward 30\n",
-      "Episode 49, Failed! Reward 33\n",
-      "Episode 50, Failed! Reward 44\n",
-      "Episode 51, Failed! Reward 41\n",
-      "Episode 52, Failed! Reward 10\n",
-      "Episode 53, Failed! Reward 19\n",
-      "Episode 54, Failed! Reward 20\n",
-      "Episode 55, Failed! Reward 18\n",
-      "Episode 56, Failed! Reward 31\n",
-      "Episode 57, Failed! Reward 68\n",
-      "Episode 58, Failed! Reward 22\n",
-      "Episode 59, Failed! Reward 17\n",
-      "Episode 60, Failed! Reward 65\n",
-      "Episode 61, Failed! Reward 23\n",
-      "Episode 62, Failed! Reward 70\n",
-      "Episode 63, Failed! Reward 69\n",
-      "Episode 64, Failed! Reward 54\n",
-      "Episode 65, Failed! Reward 42\n",
-      "Episode 66, Failed! Reward 31\n",
-      "Episode 67, Failed! Reward 93\n",
-      "Episode 68, Failed! Reward 48\n",
-      "Episode 69, Failed! Reward 26\n",
-      "Episode 70, Failed! Reward 59\n",
-      "Episode 71, Failed! Reward 12\n",
-      "Episode 72, Failed! Reward 19\n",
-      "Episode 73, Failed! Reward 44\n",
-      "Episode 74, Failed! Reward 33\n",
-      "Episode 75, Failed! Reward 37\n",
-      "Episode 76, Failed! Reward 56\n",
-      "Episode 77, Failed! Reward 32\n",
-      "Episode 78, Failed! Reward 25\n",
-      "Episode 79, Failed! Reward 43\n",
-      "Episode 80, Failed! Reward 94\n",
-      "Episode 81, Failed! Reward 55\n",
-      "Episode 82, Failed! Reward 28\n",
-      "Episode 83, Failed! Reward 32\n",
-      "Episode 84, Failed! Reward 54\n",
-      "Episode 85, Failed! Reward 13\n",
-      "Episode 86, Failed! Reward 57\n",
-      "Episode 87, Failed! Reward 43\n",
-      "Episode 88, Failed! Reward 57\n",
-      "Episode 89, Failed! Reward 44\n",
-      "Episode 90, Failed! Reward 32\n",
-      "Episode 91, Failed! Reward 32\n",
-      "Episode 92, Failed! Reward 18\n",
-      "Episode 93, Failed! Reward 46\n",
-      "Episode 94, Failed! Reward 76\n",
-      "Episode 95, Failed! Reward 12\n",
-      "Episode 96, Failed! Reward 63\n",
-      "Episode 97, Failed! Reward 88\n",
-      "Episode 98, Failed! Reward 27\n",
-      "Episode 99, Failed! Reward 72\n",
-      "Episode 100, Failed! Reward 70\n",
-      "Episode 101, Failed! Reward 47\n",
-      "Episode 102, Failed! Reward 93\n",
-      "Episode 103, Failed! Reward 57\n",
-      "Episode 104, Failed! Reward 28\n",
-      "Episode 105, Failed! Reward 86\n",
-      "Episode 106, Failed! Reward 53\n",
-      "Episode 107, Failed! Reward 113\n",
-      "Episode 108, Failed! Reward 48\n",
-      "Episode 109, Failed! Reward 33\n",
-      "Episode 110, Failed! Reward 61\n",
-      "Episode 111, Failed! Reward 105\n",
-      "Episode 112, Failed! Reward 96\n",
-      "Episode 113, Failed! Reward 64\n",
-      "Episode 114, Failed! Reward 30\n",
-      "Episode 115, Failed! Reward 48\n",
-      "Episode 116, Failed! Reward 76\n",
-      "Episode 117, Failed! Reward 149\n",
-      "Episode 118, Failed! Reward 135\n",
-      "Episode 119, Failed! Reward 41\n",
-      "Episode 120, Failed! Reward 120\n",
-      "Episode 121, Failed! Reward 77\n",
-      "Episode 122, Failed! Reward 99\n",
-      "Episode 123, Failed! Reward 73\n",
-      "Episode 124, Failed! Reward 183\n",
-      "Episode 125, Failed! Reward 135\n",
-      "Episode 126, Failed! Reward 120\n",
-      "Episode 127, Failed! Reward 98\n",
-      "Episode 128, Failed! Reward 124\n",
-      "Episode 129, Failed! Reward 67\n",
-      "Episode 130, Failed! Reward 150\n",
-      "Episode 131, Failed! Reward 71\n",
-      "Episode 132, Failed! Reward 121\n",
-      "Episode 133, Failed! Reward 29\n",
-      "Episode 134, Failed! Reward 65\n",
-      "Episode 135, Failed! Reward 97\n",
-      "Episode 136, Failed! Reward 93\n",
-      "Episode 137, Failed! Reward 175\n",
-      "Episode 138, Failed! Reward 37\n",
-      "Episode 139, Failed! Reward 159\n",
-      "Episode 140, Failed! Reward 40\n",
-      "Episode 141, Failed! Reward 64\n",
-      "Episode 142, Failed! Reward 92\n",
-      "Episode 143, Failed! Reward 42\n",
-      "Episode 144, Failed! Reward 80\n",
-      "Episode 145, Failed! Reward 96\n",
-      "Episode 146, Failed! Reward 34\n",
-      "Episode 147, Failed! Reward 69\n",
-      "Episode 148, Failed! Reward 40\n",
-      "Episode 149, Failed! Reward 17\n",
-      "Episode 150, Failed! Reward 65\n",
-      "Episode 151, Failed! Reward 42\n",
-      "Episode 152, Failed! Reward 62\n",
-      "Episode 153, Failed! Reward 98\n",
-      "Episode 154, Passed! Reward 200\n",
-      "Episode 155, Failed! Reward 108\n",
-      "Episode 156, Failed! Reward 87\n",
-      "Episode 157, Failed! Reward 112\n",
-      "Episode 158, Failed! Reward 121\n",
-      "Episode 159, Failed! Reward 22\n",
-      "Episode 160, Failed! Reward 115\n",
-      "Episode 161, Failed! Reward 170\n",
-      "Episode 162, Failed! Reward 103\n",
-      "Episode 163, Passed! Reward 200\n",
-      "Episode 164, Failed! Reward 178\n",
-      "Episode 165, Failed! Reward 146\n",
-      "Episode 166, Failed! Reward 107\n",
-      "Episode 167, Failed! Reward 140\n",
-      "Episode 168, Failed! Reward 55\n",
-      "Episode 169, Failed! Reward 143\n",
-      "Episode 170, Failed! Reward 146\n",
-      "Episode 171, Failed! Reward 173\n",
-      "Episode 172, Failed! Reward 116\n",
-      "Episode 173, Failed! Reward 34\n",
-      "Episode 174, Failed! Reward 80\n",
-      "Episode 175, Failed! Reward 128\n",
-      "Episode 176, Failed! Reward 17\n",
-      "Episode 177, Failed! Reward 15\n",
-      "Episode 178, Failed! Reward 138\n",
-      "Episode 179, Failed! Reward 121\n",
-      "Episode 180, Failed! Reward 27\n",
-      "Episode 181, Failed! Reward 131\n",
-      "Episode 182, Failed! Reward 121\n",
-      "Episode 183, Failed! Reward 126\n",
-      "Episode 184, Failed! Reward 136\n",
-      "Episode 185, Failed! Reward 33\n",
-      "Episode 186, Failed! Reward 30\n",
-      "Episode 187, Failed! Reward 133\n",
-      "Episode 188, Failed! Reward 169\n",
-      "Episode 189, Failed! Reward 16\n",
-      "Episode 190, Failed! Reward 178\n",
-      "Episode 191, Failed! Reward 125\n",
-      "Episode 192, Failed! Reward 127\n",
-      "Episode 193, Failed! Reward 137\n",
-      "Episode 194, Failed! Reward 161\n",
-      "Episode 195, Failed! Reward 131\n",
-      "Episode 196, Failed! Reward 113\n",
-      "Episode 197, Failed! Reward 83\n",
-      "Episode 198, Failed! Reward 108\n",
-      "Episode 199, Failed! Reward 184\n",
-      "Episode 201, Failed! Reward 161\n",
-      "Episode 202, Failed! Reward 170\n",
-      "Episode 203, Failed! Reward 172\n",
-      "Episode 204, Failed! Reward 16\n",
-      "Episode 205, Failed! Reward 78\n",
-      "Episode 206, Failed! Reward 126\n",
-      "Episode 207, Failed! Reward 154\n",
-      "Episode 208, Failed! Reward 164\n",
-      "Episode 209, Failed! Reward 99\n",
-      "Episode 210, Failed! Reward 159\n",
-      "Episode 211, Failed! Reward 118\n",
-      "Episode 212, Failed! Reward 156\n",
-      "Episode 213, Failed! Reward 139\n",
-      "Episode 214, Failed! Reward 153\n",
-      "Episode 215, Failed! Reward 161\n",
-      "Episode 216, Failed! Reward 18\n",
-      "Episode 217, Failed! Reward 12\n",
-      "Episode 218, Failed! Reward 130\n",
-      "Episode 219, Failed! Reward 119\n",
-      "Episode 220, Failed! Reward 122\n",
-      "Episode 221, Failed! Reward 136\n",
-      "Episode 222, Failed! Reward 135\n",
-      "Episode 223, Failed! Reward 24\n",
-      "Episode 224, Failed! Reward 159\n",
-      "Episode 225, Failed! Reward 142\n",
-      "Episode 226, Failed! Reward 147\n",
-      "Episode 227, Failed! Reward 138\n",
-      "Episode 228, Failed! Reward 147\n",
-      "Episode 229, Failed! Reward 147\n",
-      "Episode 230, Failed! Reward 150\n",
-      "Episode 231, Failed! Reward 150\n",
-      "Episode 232, Failed! Reward 129\n",
-      "Episode 233, Failed! Reward 150\n",
-      "Episode 234, Failed! Reward 149\n",
-      "Episode 235, Failed! Reward 151\n",
-      "Episode 236, Failed! Reward 116\n",
-      "Episode 237, Failed! Reward 108\n",
-      "Episode 238, Failed! Reward 126\n",
-      "Episode 239, Failed! Reward 135\n",
-      "Episode 240, Failed! Reward 116\n",
-      "Episode 241, Failed! Reward 15\n",
-      "Episode 242, Failed! Reward 108\n",
-      "Episode 243, Failed! Reward 24\n",
-      "Episode 244, Failed! Reward 14\n",
-      "Episode 245, Failed! Reward 13\n",
-      "Episode 246, Failed! Reward 104\n",
-      "Episode 247, Failed! Reward 61\n",
-      "Episode 248, Passed! Reward 200\n",
-      "Episode 249, Failed! Reward 126\n",
-      "Episode 250, Failed! Reward 159\n",
-      "Episode 251, Failed! Reward 175\n",
-      "Episode 252, Failed! Reward 190\n",
-      "Episode 253, Failed! Reward 127\n",
-      "Episode 255, Failed! Reward 137\n",
-      "Episode 256, Failed! Reward 154\n",
-      "Episode 257, Failed! Reward 141\n",
-      "Episode 258, Failed! Reward 156\n",
-      "Episode 259, Failed! Reward 105\n",
-      "Episode 260, Failed! Reward 143\n",
-      "Episode 261, Failed! Reward 25\n",
-      "Episode 262, Failed! Reward 156\n",
-      "Episode 263, Failed! Reward 175\n",
-      "Episode 264, Failed! Reward 148\n",
-      "Episode 265, Failed! Reward 169\n",
-      "Episode 266, Failed! Reward 183\n",
-      "Episode 267, Failed! Reward 180\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Episode 268, Failed! Reward 190\n",
-      "Episode 269, Failed! Reward 174\n",
-      "Episode 270, Failed! Reward 154\n",
-      "Episode 271, Failed! Reward 154\n",
-      "Episode 272, Failed! Reward 181\n",
-      "Episode 273, Failed! Reward 167\n",
-      "Episode 274, Failed! Reward 171\n",
-      "Episode 275, Failed! Reward 166\n",
-      "Episode 276, Failed! Reward 168\n",
-      "Episode 277, Failed! Reward 159\n",
-      "Episode 278, Failed! Reward 149\n",
-      "Episode 279, Failed! Reward 147\n",
-      "Episode 280, Failed! Reward 142\n",
-      "Episode 281, Failed! Reward 31\n",
-      "Episode 282, Failed! Reward 140\n",
-      "Episode 283, Failed! Reward 172\n",
-      "Episode 284, Failed! Reward 174\n",
-      "Episode 285, Failed! Reward 161\n",
-      "Episode 286, Failed! Reward 173\n",
-      "Episode 287, Failed! Reward 165\n",
-      "Episode 288, Failed! Reward 184\n",
-      "Episode 289, Passed! Reward 199\n",
-      "Episode 290, Failed! Reward 188\n",
-      "Episode 291, Failed! Reward 99\n",
-      "Episode 292, Failed! Reward 187\n",
-      "Episode 294, Failed! Reward 182\n",
-      "Episode 295, Passed! Reward 200\n",
-      "Episode 296, Failed! Reward 171\n",
-      "Episode 297, Failed! Reward 185\n",
-      "Episode 298, Failed! Reward 180\n",
-      "Episode 299, Failed! Reward 194\n",
-      "Episode 300, Failed! Reward 177\n",
-      "Episode 301, Failed! Reward 188\n",
-      "Episode 302, Failed! Reward 177\n",
-      "Episode 303, Failed! Reward 178\n",
-      "Episode 304, Failed! Reward 191\n",
-      "Episode 305, Passed! Reward 200\n",
-      "Episode 306, Failed! Reward 179\n",
-      "Episode 307, Passed! Reward 200\n",
-      "Episode 308, Failed! Reward 169\n",
-      "Episode 309, Failed! Reward 166\n",
-      "Episode 310, Failed! Reward 165\n",
-      "Episode 311, Failed! Reward 144\n",
-      "Episode 312, Failed! Reward 160\n",
-      "Episode 313, Failed! Reward 170\n",
-      "Episode 314, Failed! Reward 170\n",
-      "Episode 315, Failed! Reward 182\n",
-      "Episode 316, Failed! Reward 157\n",
-      "Episode 317, Failed! Reward 176\n",
-      "Episode 318, Passed! Reward 200\n",
-      "Episode 319, Passed! Reward 200\n",
-      "Episode 320, Passed! Reward 200\n",
-      "Episode 321, Passed! Reward 200\n",
-      "Episode 322, Passed! Reward 200\n",
-      "Episode 323, Passed! Reward 200\n",
-      "Episode 324, Failed! Reward 176\n",
-      "Episode 325, Failed! Reward 172\n",
-      "Episode 326, Passed! Reward 200\n",
-      "Episode 327, Failed! Reward 167\n",
-      "Episode 328, Failed! Reward 193\n",
-      "Episode 329, Failed! Reward 183\n",
-      "Episode 330, Failed! Reward 169\n",
-      "Episode 331, Failed! Reward 174\n",
-      "Episode 332, Failed! Reward 169\n",
-      "Episode 333, Failed! Reward 167\n",
-      "Episode 334, Failed! Reward 194\n",
-      "Episode 335, Failed! Reward 175\n",
-      "Episode 336, Failed! Reward 174\n",
-      "Episode 337, Failed! Reward 170\n",
-      "Episode 338, Failed! Reward 156\n",
-      "Episode 339, Failed! Reward 162\n",
-      "Episode 340, Failed! Reward 161\n",
-      "Episode 341, Passed! Reward 200\n",
-      "Episode 342, Failed! Reward 158\n",
-      "Episode 343, Failed! Reward 160\n",
-      "Episode 344, Passed! Reward 200\n",
-      "Episode 345, Failed! Reward 153\n",
-      "Episode 346, Failed! Reward 144\n",
-      "Episode 347, Failed! Reward 145\n",
-      "Episode 348, Failed! Reward 145\n",
-      "Episode 349, Failed! Reward 140\n",
-      "Episode 350, Failed! Reward 143\n",
-      "Episode 351, Failed! Reward 136\n",
-      "Episode 352, Failed! Reward 145\n",
-      "Episode 353, Failed! Reward 149\n",
-      "Episode 354, Failed! Reward 152\n",
-      "Episode 355, Failed! Reward 144\n",
-      "Episode 356, Failed! Reward 139\n",
-      "Episode 357, Failed! Reward 168\n",
-      "Episode 358, Passed! Reward 200\n",
-      "Episode 359, Passed! Reward 200\n",
-      "Episode 360, Failed! Reward 149\n",
-      "Episode 361, Failed! Reward 136\n",
-      "Episode 362, Failed! Reward 143\n",
-      "Episode 363, Failed! Reward 140\n",
-      "Episode 364, Failed! Reward 133\n",
-      "Episode 365, Failed! Reward 130\n",
-      "Episode 366, Failed! Reward 117\n",
-      "Episode 367, Failed! Reward 137\n",
-      "Episode 368, Failed! Reward 125\n",
-      "Episode 369, Failed! Reward 136\n",
-      "Episode 370, Failed! Reward 141\n",
-      "Episode 371, Failed! Reward 132\n",
-      "Episode 372, Failed! Reward 134\n",
-      "Episode 373, Failed! Reward 123\n",
-      "Episode 374, Failed! Reward 139\n",
-      "Episode 375, Failed! Reward 144\n",
-      "Episode 376, Failed! Reward 147\n",
-      "Episode 377, Failed! Reward 149\n",
-      "Episode 378, Failed! Reward 147\n",
-      "Episode 379, Failed! Reward 127\n",
-      "Episode 380, Failed! Reward 139\n",
-      "Episode 381, Failed! Reward 175\n",
-      "Episode 382, Passed! Reward 200\n",
-      "Episode 383, Passed! Reward 200\n",
-      "Episode 384, Passed! Reward 200\n",
-      "Episode 385, Failed! Reward 144\n",
-      "Episode 386, Failed! Reward 136\n",
-      "Episode 387, Failed! Reward 32\n",
-      "Episode 388, Failed! Reward 138\n",
-      "Episode 389, Failed! Reward 133\n",
-      "Episode 390, Failed! Reward 134\n",
-      "Episode 391, Failed! Reward 141\n",
-      "Episode 392, Failed! Reward 135\n",
-      "Episode 393, Failed! Reward 135\n",
-      "Episode 394, Failed! Reward 152\n",
-      "Episode 395, Failed! Reward 146\n",
-      "Episode 396, Failed! Reward 150\n",
-      "Episode 397, Failed! Reward 131\n",
-      "Episode 398, Failed! Reward 138\n",
-      "Episode 399, Failed! Reward 142\n",
-      "Episode 400, Failed! Reward 157\n",
-      "Episode 401, Failed! Reward 137\n",
-      "Episode 402, Failed! Reward 146\n",
-      "Episode 403, Failed! Reward 138\n",
-      "Episode 404, Failed! Reward 133\n",
-      "Episode 405, Failed! Reward 137\n",
-      "Episode 406, Failed! Reward 132\n",
-      "Episode 407, Failed! Reward 139\n",
-      "Episode 408, Failed! Reward 133\n",
-      "Episode 409, Failed! Reward 147\n",
-      "Episode 410, Failed! Reward 149\n",
-      "Episode 411, Failed! Reward 151\n",
-      "Episode 412, Failed! Reward 152\n",
-      "Episode 413, Failed! Reward 154\n",
-      "Episode 414, Failed! Reward 147\n",
-      "Episode 415, Failed! Reward 152\n",
-      "Episode 416, Failed! Reward 150\n",
-      "Episode 417, Failed! Reward 167\n",
-      "Episode 418, Failed! Reward 157\n",
-      "Episode 419, Failed! Reward 153\n",
-      "Episode 420, Failed! Reward 157\n",
-      "Episode 421, Failed! Reward 166\n",
-      "Episode 422, Failed! Reward 152\n",
-      "Episode 423, Failed! Reward 166\n",
-      "Episode 424, Failed! Reward 143\n",
-      "Episode 425, Failed! Reward 165\n",
-      "Episode 426, Failed! Reward 155\n",
-      "Episode 427, Failed! Reward 153\n",
-      "Episode 428, Failed! Reward 143\n",
-      "Episode 429, Failed! Reward 148\n",
-      "Episode 430, Failed! Reward 150\n",
-      "Episode 431, Failed! Reward 146\n",
-      "Episode 432, Failed! Reward 139\n",
-      "Episode 433, Failed! Reward 152\n",
-      "Episode 434, Failed! Reward 155\n",
-      "Episode 435, Failed! Reward 163\n",
-      "Episode 436, Failed! Reward 159\n",
-      "Episode 437, Failed! Reward 155\n",
-      "Episode 438, Failed! Reward 152\n",
-      "Episode 439, Failed! Reward 153\n",
-      "Episode 440, Failed! Reward 170\n",
-      "Episode 441, Failed! Reward 159\n",
-      "Episode 442, Failed! Reward 151\n",
-      "Episode 443, Failed! Reward 168\n",
-      "Episode 444, Failed! Reward 162\n",
-      "Episode 445, Failed! Reward 153\n",
-      "Episode 446, Failed! Reward 148\n",
-      "Episode 447, Failed! Reward 155\n",
-      "Episode 448, Failed! Reward 156\n",
-      "Episode 449, Failed! Reward 145\n",
-      "Episode 450, Failed! Reward 155\n",
-      "Episode 451, Failed! Reward 162\n",
-      "Episode 452, Failed! Reward 151\n",
-      "Episode 453, Failed! Reward 164\n",
-      "Episode 454, Failed! Reward 156\n",
-      "Episode 455, Failed! Reward 165\n",
-      "Episode 456, Failed! Reward 161\n",
-      "Episode 457, Failed! Reward 151\n",
-      "Episode 458, Failed! Reward 163\n",
-      "Episode 459, Failed! Reward 156\n",
-      "Episode 460, Failed! Reward 157\n",
-      "Episode 461, Failed! Reward 156\n",
-      "Episode 462, Failed! Reward 157\n",
-      "Episode 463, Failed! Reward 173\n",
-      "Episode 464, Failed! Reward 150\n",
-      "Episode 465, Failed! Reward 150\n",
-      "Episode 466, Failed! Reward 168\n",
-      "Episode 467, Failed! Reward 194\n",
-      "Episode 468, Failed! Reward 167\n",
-      "Episode 469, Failed! Reward 163\n",
-      "Episode 470, Failed! Reward 145\n",
-      "Episode 471, Failed! Reward 132\n",
-      "Episode 472, Failed! Reward 127\n",
-      "Episode 473, Failed! Reward 123\n",
-      "Episode 474, Failed! Reward 135\n",
-      "Episode 475, Failed! Reward 132\n",
-      "Episode 476, Failed! Reward 124\n",
-      "Episode 477, Failed! Reward 129\n",
-      "Episode 478, Failed! Reward 138\n",
-      "Episode 479, Failed! Reward 130\n",
-      "Episode 480, Failed! Reward 134\n",
-      "Episode 481, Failed! Reward 138\n",
-      "Episode 482, Failed! Reward 135\n",
-      "Episode 483, Failed! Reward 127\n",
-      "Episode 484, Failed! Reward 124\n",
-      "Episode 485, Failed! Reward 130\n",
-      "Episode 486, Failed! Reward 145\n",
-      "Episode 487, Failed! Reward 131\n",
-      "Episode 488, Failed! Reward 137\n",
-      "Episode 489, Failed! Reward 139\n",
-      "Episode 490, Failed! Reward 127\n",
-      "Episode 491, Failed! Reward 131\n",
-      "Episode 492, Failed! Reward 131\n",
-      "Episode 493, Failed! Reward 131\n",
-      "Episode 494, Failed! Reward 136\n",
-      "Episode 495, Failed! Reward 154\n",
-      "Episode 496, Failed! Reward 136\n",
-      "Episode 497, Failed! Reward 134\n",
-      "Episode 498, Failed! Reward 138\n",
-      "Episode 499, Failed! Reward 129\n",
-      "Episode 500, Failed! Reward 141\n",
-      "Episode 501, Failed! Reward 144\n",
-      "Episode 502, Failed! Reward 146\n",
-      "Episode 503, Failed! Reward 142\n",
-      "Episode 504, Failed! Reward 157\n",
-      "Episode 505, Failed! Reward 156\n",
-      "Episode 506, Failed! Reward 157\n",
-      "Episode 507, Failed! Reward 147\n",
-      "Episode 508, Failed! Reward 146\n",
-      "Episode 509, Failed! Reward 149\n",
-      "Episode 510, Failed! Reward 139\n",
-      "Episode 511, Failed! Reward 142\n",
-      "Episode 512, Failed! Reward 145\n",
-      "Episode 513, Failed! Reward 148\n",
-      "Episode 514, Failed! Reward 139\n",
-      "Episode 515, Failed! Reward 142\n",
-      "Episode 516, Failed! Reward 140\n",
-      "Episode 517, Failed! Reward 138\n",
-      "Episode 518, Failed! Reward 145\n",
-      "Episode 519, Failed! Reward 143\n",
-      "Episode 520, Failed! Reward 157\n",
-      "Episode 521, Failed! Reward 148\n",
-      "Episode 522, Failed! Reward 131\n",
-      "Episode 523, Failed! Reward 148\n",
-      "Episode 524, Failed! Reward 132\n",
-      "Episode 525, Failed! Reward 131\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Episode 526, Failed! Reward 129\n",
-      "Episode 527, Failed! Reward 126\n",
-      "Episode 528, Failed! Reward 132\n",
-      "Episode 529, Failed! Reward 131\n",
-      "Episode 530, Failed! Reward 144\n",
-      "Episode 531, Failed! Reward 145\n",
-      "Episode 532, Failed! Reward 133\n",
-      "Episode 533, Failed! Reward 139\n",
-      "Episode 534, Failed! Reward 143\n",
-      "Episode 535, Failed! Reward 139\n",
-      "Episode 536, Failed! Reward 146\n",
-      "Episode 537, Failed! Reward 150\n",
-      "Episode 538, Failed! Reward 137\n",
-      "Episode 539, Failed! Reward 149\n",
-      "Episode 540, Failed! Reward 163\n",
-      "Episode 541, Failed! Reward 150\n",
-      "Episode 542, Failed! Reward 158\n",
-      "Episode 543, Failed! Reward 153\n",
-      "Episode 544, Failed! Reward 141\n",
-      "Episode 545, Failed! Reward 160\n",
-      "Episode 546, Failed! Reward 149\n",
-      "Episode 547, Failed! Reward 163\n",
-      "Episode 548, Failed! Reward 140\n",
-      "Episode 549, Failed! Reward 149\n",
-      "Episode 550, Failed! Reward 157\n",
-      "Episode 551, Failed! Reward 148\n",
-      "Episode 552, Failed! Reward 145\n",
-      "Episode 553, Failed! Reward 132\n",
-      "Episode 554, Failed! Reward 135\n",
-      "Episode 555, Failed! Reward 132\n",
-      "Episode 556, Failed! Reward 136\n",
-      "Episode 557, Failed! Reward 149\n",
-      "Episode 558, Failed! Reward 141\n",
-      "Episode 559, Failed! Reward 156\n",
-      "Episode 560, Failed! Reward 151\n",
-      "Episode 561, Failed! Reward 145\n",
-      "Episode 562, Failed! Reward 146\n",
-      "Episode 563, Failed! Reward 152\n",
-      "Episode 564, Failed! Reward 161\n",
-      "Episode 565, Failed! Reward 144\n",
-      "Episode 566, Failed! Reward 164\n",
-      "Episode 567, Failed! Reward 142\n",
-      "Episode 568, Failed! Reward 146\n",
-      "Episode 569, Failed! Reward 148\n",
-      "Episode 570, Failed! Reward 148\n",
-      "Episode 571, Failed! Reward 150\n",
-      "Episode 572, Failed! Reward 143\n",
-      "Episode 573, Failed! Reward 152\n",
-      "Episode 574, Failed! Reward 152\n",
-      "Episode 575, Failed! Reward 149\n",
-      "Episode 576, Failed! Reward 145\n",
-      "Episode 577, Failed! Reward 139\n",
-      "Episode 578, Failed! Reward 135\n",
-      "Episode 579, Failed! Reward 138\n",
-      "Episode 580, Failed! Reward 136\n",
-      "Episode 581, Failed! Reward 164\n",
-      "Episode 582, Failed! Reward 160\n",
-      "Episode 583, Failed! Reward 154\n",
-      "Episode 584, Failed! Reward 163\n",
-      "Episode 585, Failed! Reward 149\n",
-      "Episode 586, Failed! Reward 150\n",
-      "Episode 587, Failed! Reward 154\n",
-      "Episode 588, Failed! Reward 157\n",
-      "Episode 589, Failed! Reward 142\n",
-      "Episode 590, Failed! Reward 159\n",
-      "Episode 591, Failed! Reward 143\n",
-      "Episode 592, Failed! Reward 163\n",
-      "Episode 593, Failed! Reward 145\n",
-      "Episode 594, Failed! Reward 152\n",
-      "Episode 595, Failed! Reward 187\n",
-      "Episode 596, Failed! Reward 174\n",
-      "Episode 597, Failed! Reward 159\n",
-      "Episode 598, Failed! Reward 161\n",
-      "Episode 599, Failed! Reward 159\n",
-      "Episode 600, Failed! Reward 140\n",
-      "Episode 601, Failed! Reward 174\n",
-      "Episode 602, Failed! Reward 170\n",
-      "Episode 603, Passed! Reward 200\n",
-      "Episode 604, Passed! Reward 199\n",
-      "Episode 605, Failed! Reward 180\n",
-      "Episode 606, Failed! Reward 155\n",
-      "Episode 607, Failed! Reward 181\n",
-      "Episode 608, Failed! Reward 184\n",
-      "Episode 609, Failed! Reward 178\n",
-      "Episode 610, Failed! Reward 162\n",
-      "Episode 611, Failed! Reward 169\n",
-      "Episode 612, Failed! Reward 168\n",
-      "Episode 613, Failed! Reward 156\n",
-      "Episode 614, Failed! Reward 146\n",
-      "Episode 615, Failed! Reward 150\n",
-      "Episode 616, Failed! Reward 163\n",
-      "Episode 617, Failed! Reward 163\n",
-      "Episode 618, Failed! Reward 172\n",
-      "Episode 619, Failed! Reward 160\n",
-      "Episode 620, Failed! Reward 166\n",
-      "Episode 621, Failed! Reward 153\n",
-      "Episode 622, Failed! Reward 163\n",
-      "Episode 623, Failed! Reward 149\n",
-      "Episode 624, Failed! Reward 159\n",
-      "Episode 625, Failed! Reward 162\n",
-      "Episode 626, Failed! Reward 150\n",
-      "Episode 627, Failed! Reward 155\n",
-      "Episode 628, Failed! Reward 173\n",
-      "Episode 629, Failed! Reward 162\n",
-      "Episode 630, Failed! Reward 157\n",
-      "Episode 631, Failed! Reward 166\n",
-      "Episode 632, Failed! Reward 150\n",
-      "Episode 633, Failed! Reward 144\n",
-      "Episode 634, Failed! Reward 161\n",
-      "Episode 635, Failed! Reward 166\n",
-      "Episode 636, Failed! Reward 155\n",
-      "Episode 637, Failed! Reward 155\n",
-      "Episode 638, Failed! Reward 155\n",
-      "Episode 639, Failed! Reward 151\n",
-      "Episode 640, Failed! Reward 170\n",
-      "Episode 641, Failed! Reward 164\n",
-      "Episode 642, Failed! Reward 162\n",
-      "Episode 643, Failed! Reward 142\n",
-      "Episode 644, Failed! Reward 155\n",
-      "Episode 645, Failed! Reward 149\n",
-      "Episode 646, Failed! Reward 147\n",
-      "Episode 647, Failed! Reward 175\n",
-      "Episode 648, Failed! Reward 158\n",
-      "Episode 649, Failed! Reward 143\n",
-      "Episode 650, Failed! Reward 151\n",
-      "Episode 651, Failed! Reward 166\n",
-      "Episode 652, Failed! Reward 171\n",
-      "Episode 653, Failed! Reward 171\n",
-      "Episode 654, Failed! Reward 178\n",
-      "Episode 655, Failed! Reward 174\n",
-      "Episode 656, Failed! Reward 167\n",
-      "Episode 657, Failed! Reward 147\n",
-      "Episode 658, Failed! Reward 145\n",
-      "Episode 659, Failed! Reward 146\n",
-      "Episode 660, Failed! Reward 155\n",
-      "Episode 661, Failed! Reward 153\n",
-      "Episode 662, Failed! Reward 182\n",
-      "Episode 663, Failed! Reward 177\n",
-      "Episode 664, Failed! Reward 158\n",
-      "Episode 665, Failed! Reward 156\n",
-      "Episode 666, Failed! Reward 154\n",
-      "Episode 667, Failed! Reward 157\n",
-      "Episode 668, Failed! Reward 158\n",
-      "Episode 669, Failed! Reward 151\n",
-      "Episode 670, Failed! Reward 162\n",
-      "Episode 671, Failed! Reward 180\n",
-      "Episode 672, Failed! Reward 153\n",
-      "Episode 673, Failed! Reward 159\n",
-      "Episode 674, Failed! Reward 164\n",
-      "Episode 675, Failed! Reward 157\n",
-      "Episode 676, Failed! Reward 177\n",
-      "Episode 677, Failed! Reward 160\n",
-      "Episode 678, Failed! Reward 156\n",
-      "Episode 679, Failed! Reward 154\n",
-      "Episode 680, Failed! Reward 148\n",
-      "Episode 681, Failed! Reward 153\n",
-      "Episode 682, Failed! Reward 149\n",
-      "Episode 683, Failed! Reward 158\n",
-      "Episode 684, Failed! Reward 145\n",
-      "Episode 685, Failed! Reward 150\n",
-      "Episode 686, Failed! Reward 145\n",
-      "Episode 687, Failed! Reward 152\n",
-      "Episode 688, Failed! Reward 153\n",
-      "Episode 689, Failed! Reward 173\n",
-      "Episode 690, Failed! Reward 172\n",
-      "Episode 691, Failed! Reward 161\n",
-      "Episode 692, Failed! Reward 164\n",
-      "Episode 693, Failed! Reward 156\n",
-      "Episode 694, Failed! Reward 159\n",
-      "Episode 695, Failed! Reward 147\n",
-      "Episode 696, Failed! Reward 145\n",
-      "Episode 697, Failed! Reward 159\n",
-      "Episode 698, Failed! Reward 166\n",
-      "Episode 699, Failed! Reward 168\n",
-      "Episode 700, Failed! Reward 154\n",
-      "Episode 701, Failed! Reward 160\n",
-      "Episode 702, Failed! Reward 138\n",
-      "Episode 703, Failed! Reward 137\n",
-      "Episode 704, Failed! Reward 177\n",
-      "Episode 705, Failed! Reward 150\n",
-      "Episode 706, Failed! Reward 143\n",
-      "Episode 707, Failed! Reward 160\n",
-      "Episode 708, Passed! Reward 200\n",
-      "Episode 709, Failed! Reward 169\n",
-      "Episode 710, Failed! Reward 178\n",
-      "Episode 711, Failed! Reward 150\n",
-      "Episode 712, Failed! Reward 143\n",
-      "Episode 713, Failed! Reward 147\n",
-      "Episode 714, Failed! Reward 113\n",
-      "Episode 715, Failed! Reward 135\n",
-      "Episode 716, Failed! Reward 141\n",
-      "Episode 717, Failed! Reward 171\n",
-      "Episode 718, Passed! Reward 200\n",
-      "Episode 719, Failed! Reward 167\n",
-      "Episode 720, Failed! Reward 185\n",
-      "Episode 721, Failed! Reward 151\n",
-      "Episode 722, Failed! Reward 149\n",
-      "Episode 723, Failed! Reward 155\n",
-      "Episode 724, Failed! Reward 166\n",
-      "Episode 725, Failed! Reward 161\n",
-      "Episode 726, Failed! Reward 169\n",
-      "Episode 727, Failed! Reward 153\n",
-      "Episode 728, Failed! Reward 157\n",
-      "Episode 729, Failed! Reward 149\n",
-      "Episode 730, Failed! Reward 154\n",
-      "Episode 731, Failed! Reward 139\n",
-      "Episode 732, Failed! Reward 130\n",
-      "Episode 733, Failed! Reward 157\n",
-      "Episode 734, Failed! Reward 147\n",
-      "Episode 735, Failed! Reward 137\n",
-      "Episode 736, Failed! Reward 139\n",
-      "Episode 737, Failed! Reward 152\n",
-      "Episode 738, Failed! Reward 149\n",
-      "Episode 739, Failed! Reward 162\n",
-      "Episode 740, Failed! Reward 153\n",
-      "Episode 741, Failed! Reward 140\n",
-      "Episode 742, Failed! Reward 149\n",
-      "Episode 743, Failed! Reward 155\n",
-      "Episode 744, Failed! Reward 167\n",
-      "Episode 745, Failed! Reward 155\n",
-      "Episode 746, Failed! Reward 149\n",
-      "Episode 747, Failed! Reward 138\n",
-      "Episode 748, Failed! Reward 132\n",
-      "Episode 749, Failed! Reward 158\n",
-      "Episode 750, Failed! Reward 143\n",
-      "Episode 751, Failed! Reward 148\n",
-      "Episode 752, Failed! Reward 129\n",
-      "Episode 753, Failed! Reward 140\n",
-      "Episode 754, Failed! Reward 148\n",
-      "Episode 755, Failed! Reward 135\n",
-      "Episode 756, Failed! Reward 138\n",
-      "Episode 757, Failed! Reward 154\n",
-      "Episode 758, Failed! Reward 125\n",
-      "Episode 759, Failed! Reward 131\n",
-      "Episode 760, Failed! Reward 147\n",
-      "Episode 761, Failed! Reward 137\n",
-      "Episode 762, Failed! Reward 132\n",
-      "Episode 763, Failed! Reward 144\n",
-      "Episode 764, Failed! Reward 139\n",
-      "Episode 765, Failed! Reward 145\n",
-      "Episode 766, Failed! Reward 127\n",
-      "Episode 767, Failed! Reward 123\n",
-      "Episode 768, Failed! Reward 131\n",
-      "Episode 769, Failed! Reward 124\n",
-      "Episode 770, Failed! Reward 148\n",
-      "Episode 771, Failed! Reward 137\n",
-      "Episode 772, Failed! Reward 141\n",
-      "Episode 773, Failed! Reward 135\n",
-      "Episode 774, Failed! Reward 127\n",
-      "Episode 775, Failed! Reward 146\n",
-      "Episode 776, Failed! Reward 136\n",
-      "Episode 777, Failed! Reward 151\n",
-      "Episode 778, Failed! Reward 117\n",
-      "Episode 779, Failed! Reward 136\n",
-      "Episode 780, Failed! Reward 130\n",
-      "Episode 781, Failed! Reward 123\n",
-      "Episode 782, Failed! Reward 119\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Episode 783, Failed! Reward 126\n",
-      "Episode 784, Failed! Reward 154\n",
-      "Episode 785, Failed! Reward 183\n",
-      "Episode 786, Failed! Reward 125\n",
-      "Episode 787, Failed! Reward 118\n",
-      "Episode 788, Failed! Reward 143\n",
-      "Episode 789, Failed! Reward 113\n",
-      "Episode 790, Failed! Reward 121\n",
-      "Episode 791, Failed! Reward 136\n",
-      "Episode 792, Failed! Reward 140\n",
-      "Episode 793, Failed! Reward 157\n",
-      "Episode 794, Failed! Reward 136\n",
-      "Episode 795, Failed! Reward 126\n",
-      "Episode 796, Failed! Reward 134\n",
-      "Episode 797, Failed! Reward 149\n",
-      "Episode 798, Failed! Reward 164\n",
-      "Episode 799, Failed! Reward 176\n",
-      "Episode 800, Failed! Reward 168\n",
-      "Episode 801, Failed! Reward 156\n",
-      "Episode 802, Failed! Reward 163\n",
-      "Episode 803, Failed! Reward 162\n",
-      "Episode 804, Failed! Reward 172\n",
-      "Episode 805, Failed! Reward 171\n",
-      "Episode 806, Failed! Reward 154\n",
-      "Episode 807, Failed! Reward 161\n",
-      "Episode 808, Failed! Reward 149\n",
-      "Episode 809, Failed! Reward 155\n",
-      "Episode 810, Failed! Reward 137\n",
-      "Episode 811, Failed! Reward 153\n",
-      "Episode 812, Failed! Reward 169\n",
-      "Episode 813, Failed! Reward 147\n",
-      "Episode 814, Failed! Reward 146\n",
-      "Episode 815, Failed! Reward 138\n",
-      "Episode 816, Failed! Reward 159\n",
-      "Episode 817, Failed! Reward 146\n",
-      "Episode 818, Failed! Reward 151\n",
-      "Episode 819, Failed! Reward 145\n",
-      "Episode 820, Failed! Reward 141\n",
-      "Episode 821, Failed! Reward 144\n",
-      "Episode 822, Failed! Reward 160\n",
-      "Episode 823, Failed! Reward 177\n",
-      "Episode 824, Failed! Reward 148\n",
-      "Episode 825, Failed! Reward 143\n",
-      "Episode 826, Failed! Reward 137\n",
-      "Episode 827, Failed! Reward 156\n",
-      "Episode 828, Failed! Reward 168\n",
-      "Episode 829, Failed! Reward 148\n",
-      "Episode 830, Failed! Reward 156\n",
-      "Episode 831, Failed! Reward 149\n",
-      "Episode 832, Failed! Reward 138\n",
-      "Episode 833, Failed! Reward 151\n",
-      "Episode 834, Failed! Reward 158\n",
-      "Episode 835, Failed! Reward 163\n",
-      "Episode 836, Failed! Reward 151\n",
-      "Episode 837, Failed! Reward 173\n",
-      "Episode 838, Failed! Reward 179\n",
-      "Episode 839, Failed! Reward 150\n",
-      "Episode 840, Failed! Reward 162\n",
-      "Episode 841, Failed! Reward 171\n",
-      "Episode 842, Failed! Reward 159\n",
-      "Episode 843, Failed! Reward 165\n",
-      "Episode 844, Failed! Reward 149\n",
-      "Episode 845, Passed! Reward 196\n",
-      "Episode 846, Failed! Reward 179\n",
-      "Episode 847, Failed! Reward 149\n",
-      "Episode 848, Failed! Reward 171\n",
-      "Episode 849, Failed! Reward 162\n",
-      "Episode 850, Failed! Reward 170\n",
-      "Episode 851, Passed! Reward 200\n",
-      "Episode 852, Failed! Reward 192\n",
-      "Episode 853, Failed! Reward 185\n",
-      "Episode 854, Passed! Reward 198\n",
-      "Episode 855, Failed! Reward 164\n",
-      "Episode 856, Failed! Reward 156\n",
-      "Episode 857, Passed! Reward 197\n",
-      "Episode 858, Passed! Reward 200\n",
-      "Episode 859, Failed! Reward 177\n",
-      "Episode 860, Passed! Reward 200\n",
-      "Episode 861, Passed! Reward 200\n",
-      "Episode 862, Passed! Reward 200\n",
-      "Episode 863, Passed! Reward 200\n",
-      "Episode 864, Passed! Reward 200\n",
-      "Episode 865, Passed! Reward 200\n",
-      "Episode 866, Passed! Reward 200\n",
-      "Episode 867, Passed! Reward 200\n",
-      "Episode 868, Passed! Reward 200\n",
-      "Episode 869, Passed! Reward 200\n",
-      "Episode 870, Passed! Reward 200\n",
-      "Episode 871, Passed! Reward 200\n",
-      "Episode 872, Passed! Reward 200\n",
-      "Episode 873, Passed! Reward 200\n",
-      "Episode 874, Passed! Reward 200\n",
-      "Episode 875, Passed! Reward 200\n",
-      "Episode 876, Passed! Reward 200\n",
-      "Episode 877, Passed! Reward 200\n",
-      "Episode 878, Passed! Reward 200\n",
-      "Episode 879, Passed! Reward 200\n",
-      "Episode 880, Passed! Reward 200\n",
-      "Episode 881, Passed! Reward 200\n",
-      "Episode 882, Passed! Reward 200\n",
-      "Episode 883, Passed! Reward 200\n",
-      "Episode 884, Passed! Reward 200\n",
-      "Episode 885, Passed! Reward 200\n",
-      "Episode 886, Passed! Reward 200\n",
-      "Episode 887, Passed! Reward 200\n",
-      "Episode 888, Passed! Reward 200\n",
-      "Episode 889, Passed! Reward 200\n",
-      "Episode 890, Passed! Reward 200\n",
-      "Episode 891, Passed! Reward 200\n",
-      "Episode 892, Passed! Reward 200\n",
-      "Episode 893, Passed! Reward 200\n",
-      "Episode 894, Passed! Reward 200\n",
-      "Episode 895, Passed! Reward 200\n",
-      "Episode 896, Passed! Reward 200\n",
-      "Episode 897, Passed! Reward 200\n",
-      "Episode 898, Passed! Reward 200\n",
-      "Episode 899, Passed! Reward 200\n",
-      "Episode 900, Passed! Reward 200\n",
-      "Episode 901, Passed! Reward 200\n",
-      "Episode 902, Passed! Reward 200\n",
-      "Episode 903, Passed! Reward 200\n",
-      "Episode 904, Passed! Reward 200\n",
-      "Episode 905, Passed! Reward 200\n",
-      "Episode 906, Passed! Reward 200\n",
-      "Episode 907, Passed! Reward 200\n",
-      "Episode 908, Passed! Reward 200\n",
-      "Episode 909, Passed! Reward 200\n",
-      "Episode 910, Passed! Reward 200\n",
-      "Episode 911, Passed! Reward 200\n",
-      "Episode 912, Passed! Reward 200\n",
-      "Episode 913, Passed! Reward 200\n",
-      "Episode 914, Passed! Reward 200\n",
-      "Episode 915, Passed! Reward 200\n",
-      "Episode 916, Passed! Reward 200\n",
-      "Episode 917, Passed! Reward 200\n",
-      "Episode 918, Passed! Reward 200\n",
-      "Episode 919, Passed! Reward 200\n",
-      "Episode 920, Passed! Reward 200\n",
-      "Episode 921, Passed! Reward 200\n",
-      "Episode 922, Passed! Reward 200\n",
-      "Episode 923, Passed! Reward 200\n",
-      "Episode 924, Passed! Reward 200\n",
-      "Episode 925, Passed! Reward 200\n",
-      "Episode 926, Passed! Reward 200\n",
-      "Episode 927, Passed! Reward 200\n",
-      "Episode 928, Passed! Reward 200\n",
-      "Episode 929, Passed! Reward 200\n",
-      "Episode 930, Passed! Reward 200\n",
-      "Episode 931, Passed! Reward 200\n",
-      "Episode 932, Passed! Reward 200\n",
-      "Episode 933, Passed! Reward 200\n",
-      "Episode 934, Passed! Reward 200\n",
-      "Episode 935, Passed! Reward 200\n",
-      "Episode 936, Passed! Reward 200\n",
-      "Episode 937, Passed! Reward 200\n",
-      "Episode 938, Passed! Reward 200\n",
-      "Episode 939, Passed! Reward 200\n",
-      "Episode 940, Passed! Reward 200\n",
-      "Episode 941, Passed! Reward 200\n",
-      "Episode 942, Passed! Reward 200\n",
-      "Episode 943, Passed! Reward 200\n",
-      "Episode 944, Passed! Reward 200\n",
-      "Episode 945, Passed! Reward 200\n",
-      "Episode 946, Passed! Reward 200\n",
-      "Episode 947, Passed! Reward 200\n",
-      "Episode 948, Passed! Reward 200\n",
-      "Episode 949, Passed! Reward 200\n",
-      "Episode 950, Passed! Reward 200\n",
-      "Episode 951, Passed! Reward 200\n",
-      "Episode 952, Passed! Reward 200\n",
-      "Episode 953, Passed! Reward 200\n",
-      "Episode 954, Passed! Reward 200\n",
-      "Episode 955, Passed! Reward 200\n",
-      "Episode 956, Passed! Reward 200\n",
-      "Episode 957, Passed! Reward 200\n",
-      "Episode 958, Passed! Reward 200\n",
-      "Episode 959, Passed! Reward 200\n",
-      "Episode 960, Passed! Reward 200\n",
-      "Episode 961, Passed! Reward 200\n",
-      "Episode 962, Passed! Reward 200\n",
-      "Episode 963, Passed! Reward 200\n",
-      "Episode 964, Passed! Reward 200\n",
-      "Episode 965, Passed! Reward 200\n",
-      "Episode 966, Passed! Reward 200\n",
-      "Episode 967, Passed! Reward 200\n",
-      "Episode 968, Passed! Reward 200\n",
-      "Episode 969, Passed! Reward 200\n",
-      "Episode 970, Passed! Reward 200\n",
-      "Episode 971, Passed! Reward 200\n",
-      "Episode 972, Passed! Reward 200\n",
-      "Episode 973, Passed! Reward 200\n",
-      "Episode 974, Passed! Reward 200\n",
-      "Episode 975, Passed! Reward 200\n",
-      "Episode 976, Passed! Reward 200\n",
-      "Episode 977, Passed! Reward 200\n",
-      "Episode 978, Passed! Reward 200\n",
-      "Episode 979, Passed! Reward 200\n",
-      "Episode 980, Passed! Reward 200\n",
-      "Episode 981, Passed! Reward 200\n",
-      "Episode 982, Passed! Reward 200\n",
-      "Episode 983, Passed! Reward 200\n",
-      "Episode 984, Passed! Reward 200\n",
-      "Episode 985, Passed! Reward 200\n",
-      "Episode 986, Passed! Reward 200\n",
-      "Episode 987, Passed! Reward 200\n",
-      "Episode 988, Passed! Reward 200\n",
-      "Episode 989, Passed! Reward 200\n",
-      "Episode 990, Passed! Reward 200\n",
-      "Episode 991, Passed! Reward 200\n",
-      "Episode 992, Passed! Reward 200\n",
-      "Episode 993, Passed! Reward 200\n",
-      "Episode 994, Passed! Reward 200\n",
-      "Episode 995, Passed! Reward 200\n",
-      "Episode 996, Passed! Reward 200\n",
-      "Episode 997, Passed! Reward 200\n",
-      "Episode 998, Passed! Reward 200\n",
-      "Episode 999, Passed! Reward 200\n"
-     ]
-    }
-   ],
-   "source": [
-    "ep_list =[]\n",
-    "reward_list =[]  \n",
-    "for ep in range(num_episodes):\n",
-    "    s= env.reset()\n",
-    "    s=s.reshape((1,-1))\n",
-    "    rAll =0\n",
-    "    d = False\n",
-    "    j = 0\n",
-    "    for j in range(200):\n",
-    "        #time.sleep(0.01)\n",
-    "        #epsilon greedy. to choose random actions initially when Q is all zeros\n",
-    "        if np.random.random()< epsilon:\n",
-    "            a = np.random.randint(0,legal_actions)\n",
-    "            #epsilon = epsilon*epsilon_decay\n",
-    "        else:\n",
-    "            Q = model.predict(s.reshape(-1,s.shape[0],s.shape[1]))\n",
-    "            a =np.argmax(Q)\n",
-    "        new_s,r,d,_ = env.step(a)\n",
-    "        new_s = new_s.reshape((1,-1))\n",
-    "        rAll=rAll+r\n",
-    "        if show:\n",
-    "            env.render()\n",
-    "        if d:\n",
-    "            if rAll<195:\n",
-    "                r =-100\n",
-    "                experience = (s,r,a,new_s)\n",
-    "                memory.append(experience)\n",
-    "                print(\"Episode %d, Failed! Reward %d\"%(ep,rAll))\n",
-    "                #break\n",
-    "            elif rAll>195:\n",
-    "                print(\"Episode %d, Passed! Reward %d\"%(ep,rAll))\n",
-    "            ep_list.append(ep)\n",
-    "            reward_list.append(rAll)\n",
-    "            break\n",
-    "        \n",
-    "        experience = (s,r,a,new_s)\n",
-    "        memory.append(experience)\n",
-    "        if j==199:\n",
-    "            print(\"Reward %d after full episode\"%(rAll))\n",
-    "            \n",
-    "        s = new_s\n",
-    "    batches=random.sample(memory,batch_size)\n",
-    "    states= np.array([batch[0] for batch in batches])\n",
-    "    rewards= np.array([batch[1] for batch in batches])\n",
-    "    actions= np.array([batch[2] for batch in batches])\n",
-    "    new_states= np.array([batch[3] for batch in batches])\n",
-    "    Qs =model.predict(states)\n",
-    "    new_Qs = model.predict(new_states)\n",
-    "    for i in range(len(rewards)):\n",
-    "        if rewards[i]==-100:\n",
-    "            Qs[i][0][actions[i]]=Qs[i][0][actions[i]]+ lr*(rewards[i]-Qs[i][0][actions[i]])\n",
-    "        else:\n",
-    "            Qs[i][0][actions[i]]= Qs[i][0][actions[i]]+ lr*(rewards[i]+gamma*np.max(new_Qs[i])-Qs[i][0][actions[i]])\n",
-    "    model.fit(states,Qs,verbose=0)\n",
-    "    epsilon=epsilon*epsilon_decay\n",
-    "env.close()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Plot of Episodes vs Reward"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Deep Q Learning"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![DQL](Reward_vs_Episode_DL_lr_0.500000_eps_1000_CartPole.jpg)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Plot for Vanilla ```Q Learning```"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![QLearning](Reward_vs_Episode_QL_lr_0.500000_eps_1000_CartPole.jpg)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The above two pictures show that Deep Q Network reached the highest points faster than traditional Q Learning Algorithm"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Further Improvement"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "You can try and give some trials with different architectures, parameters and hyperparameters for improvement. "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "collapsed": true
-   },
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 2",
-   "language": "python",
-   "name": "python2"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 2
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython2",
-   "version": "2.7.15"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/English/RL Tutorial 2/Atari-breakout.jpg b/English/RL Tutorial 2/Atari-breakout.jpg
new file mode 100644
index 0000000..e50400a
Binary files /dev/null and b/English/RL Tutorial 2/Atari-breakout.jpg differ
diff --git a/English/RL Tutorial 2/Deep Q Learning.ipynb b/English/RL Tutorial 2/Deep Q Learning.ipynb
index 7690457..5e2e6cc 100644
--- a/English/RL Tutorial 2/Deep Q Learning.ipynb	
+++ b/English/RL Tutorial 2/Deep Q Learning.ipynb	
@@ -19,72 +19,253 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In our last tutorial, I tried to explain Q-learning algorithm. The biggest problem with Q-learning , is that,it takes discrete inputs and outputs discrete values. In the Mountain Car problem, we bypassed this issue by discretizing states which are actually continuous. But this can't be done always. Specially when, the states are multidimensional states or images. Deep learning comes here to solve this problem"
+    "In the last tutorial, I tried to explain Q-learning algorithm. The biggest problem with Q-learning is that,it only takes discrete inputs and outputs discrete values. In the Mountain Car problem, we solved this issue by discretizing states which are actually continuous. But this can't be done always. Specially when the states are multidimensional states or images. Deep learning comes here to solve this problem!"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In this project, I have worked on Deep Q Learning . This is the first part which solves the problem of [Cart Pole](https://github.com/openai/gym/wiki/CartPole-v0) by ```Open AI``` Gym. The concept is based on the ground breaking [work](https://arxiv.org/pdf/1312.5602.pdf) of Deep Mind. The CartPole video is available [here](https://youtu.be/IKobsp1WszI). Part 2 of this notebook is on Mountain Car Problem which can be accessed from [here](https://github.com/sezan92/DQL/blob/master/Deep%20Q%20Learning%202.ipynb) . "
+    "For example breakout game by atari"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Rule"
+    "![](Atari-breakout.jpg)"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The rule of the game is simple . The cart must balance the pole i.e. keep it within a specific angle. For each step within the angle the cart will get 1 points. The cart will be considered winner if it gets 200 points. \n",
-    "** Episode will terminate if **\n",
-    "* Pole Angle is more than ±12°\n",
-    "* Cart Position is more than ±2.4 (center of the cart reaches the edge of the display)\n",
-    "* Episode length is greater than 200\n"
+    "Here , the states are the actual image itself! It is 210x160x3 size RGB numpy array. How will you make discrete for $Q learning$ ? Will that be efficient ? ***NO***! . DQN comes us to save us!"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![CartPole](Cartpole.png)"
+    "## Intuition"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Importing Packages"
+    "### Deep Q Learning"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Q learning is a lookup table problem. i.e. You have the state , you just look at the table and see which action gives you best $Q$ value! That's it. But for continuous state - as mentioned above- you cannot make a lookup table! You need something like a regression model! Which will give you the Q values for given state and action! And the best regression model would be, Deep Neural Network!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "So, we will replace the Q table explained in the last tutorial with a Neural Network. i.e. some thing like the following picture"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![](Q2DQN.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "***But there is one little problem!***"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In our mountain car problem, we have three discrete actions. $0,1 & 2$ . Using the above architecture, we will have to calculate $Q$ value for each action . Because , you need to take the action with best $Q$ value. To get the action of the best $Q$ value, you need to know the $Q(state,action)$ for each state! So in our case, we will have to run same feed forward process three times!"
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "action_list =[0,1,2]\n",
+    "Q1 = model.predict(state,action_list[0])\n",
+    "Q2 = model.predict(state,action_list[1])\n",
+    "Q3 = model.predict(state,action_list[2])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "What if we have more 100 actions ? Will we feed forward 100 times! It is a bit inefficient!! Instead, we will use the following architecture."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![](NewDQN.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Meaning, our output layer will calculate $Q$ value for each action. As a result we can calculate Q values in one single forward pass each step! And then we will choose the action with maximum value"
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "action = np.argmax(Q(state))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Bellman Update Equation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In the Original Equation, the bellman update equation is \n",
+    "\\begin{equation}\n",
+    "Q(s_t,a) = Q(s_t,a) + \\alpha (Q'(s_{t+1},a)-Q(s_t,a))\n",
+    "\\end{equation}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "For DQN, we will use similar equation, using Gradient descent\n",
+    "\\begin{equation}\n",
+    "\\theta_Q \\gets \\theta_Q - \\alpha \\frac{\\partial}{\\partial \\theta}(Q'(s_{t+1},a)-Q(s_t,a))^2\n",
+    "\\end{equation}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If you are intelligent enough, then you may ask , why there is a squareed part in the gradient descent equation but not in the actual bellman update equation? The reason might be, Mean squared errors are more sensitive to sudden spikes in the target data, which makes it most popular metric for regression models!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### The Concept of Experience Replay"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "One of the problems in Reinforcment learning is relearning Problem. That is , suppose, in the course of trial and error, one state $s_t$ comes only once or twice, and never comes back. What will happen? There is a chance that the Agent will forget that experience after some time- like us! . So we need to make the agent keep some kind of track of that memory as well. This problem was solved in 1993- yes 26 years ago- by Long Ji lin. In his paper, ***Reinforcement Learning for Robots Using Neural Networks*** , he introduced the concept of Experience Replay. What he did was, he initialized a buffer of a certain size . He stored the experiences of the agent, i.e. state $s_t$, action $a$,next state $s_{t+1}$, reward $r$ . Before training the agent, he you just sample randomly from the buffer . It also helps randomizing the data , which in turn, helps to converge the model faster, as mentioned by Yoshua Bengio in his paper ***Practical Recommendations for Gradient-Based Training of Deep\n",
+    "Architectures***,2012 "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### The concept of $\\epsilon$-greedy Policy"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In the beginning of training, we will have to explore random actions. Because we dont know the value of each action for each state. So we will take some random actions. We will evaluate those actions and see which random action gives us the most rewards. We will try to increase those actions. It means, at first you just ***explore*** different actions , the more you take actions the less you explore and more use your previouse experience to guide you-known as ***exploit***.  This thing can be done using a technique -with another freaking out name- $\\epsilon$-greedy policy. "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The big idea is that, we will select a value of $\\epsilon$ , suppose $0.9$. Then we will generate a random floating point number . If the generated number is greater than $\\epsilon$ we will take action according to the DQN, otherwise a random action. After each episode , we will decrease the value of $\\epsilon$ . As a result , in the last episodes, the agent will take actions according to DQN model, not the random actions."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Set $\\epsilon$\n",
+    "- Generate random number $n_{rand}$\n",
+    "- if $n_{rand} < \\epsilon$ ***do***\n",
+    "- - take random action\n",
+    "- else ***do***\n",
+    "- - take action according to DQN"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Coding!!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Okay, let's start the most juicy part!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "***Importing Libraries***"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
-   "metadata": {
-    "collapsed": true
-   },
-   "outputs": [],
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/home/sezan92/anaconda/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
+      "  from ._conv import register_converters as _register_converters\n",
+      "Using TensorFlow backend.\n"
+     ]
+    }
+   ],
    "source": [
     "import gym\n",
     "import numpy as np\n",
     "from collections import deque\n",
     "import random\n",
-    "from keras import Sequential\n",
+    "import keras\n",
+    "from keras.models import Sequential\n",
     "from keras.layers import Dense\n",
     "from keras.optimizers import Adam\n",
     "import matplotlib.pyplot as plt\n",
-    "%matplotlib inline"
+    "from sklearn.preprocessing import OneHotEncoder"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Setting up Environment"
+    "***Initialization of Environment***"
    ]
   },
   {
@@ -93,198 +274,174 @@
    "metadata": {},
    "outputs": [
     {
-     "name": "stderr",
+     "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[2018-03-06 22:30:47,690] Making new env: CartPole-v0\n"
+      "\u001b[33mWARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'>. Please provide explicit dtype.\u001b[0m\n"
      ]
     }
    ],
    "source": [
-    "env = gym.make('CartPole-v0')"
+    "env = gym.make('MountainCar-v0')"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Hiperparameters"
+    "***Hyper parameters***"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- ```action_size``` number of actions\n",
+    "- ```actions``` the actions list\n",
+    "- ```gamma``` discount factor $\\gamma$\n",
+    "- ```lr``` learning rate $\\alpha$\n",
+    "- ```num_episodes``` number of episodes\n",
+    "- ```epsilon``` epsilon , to choose random actions for epsilon greedy policy \n",
+    "- ```epsilon_decay``` epsilon decay rate\n",
+    "- ```batch_size``` batch size for training"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {
-    "collapsed": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
-    "legal_actions=env.action_space.n\n",
-    "actions = [0,1]\n",
-    "gamma =0.95\n",
-    "lr =0.5\n",
+    "action_list = [0,1,2]\n",
+    "gamma =0.45\n",
+    "lr =0.001\n",
     "num_episodes =1000\n",
     "epsilon =1\n",
     "epsilon_decay =0.995\n",
     "memory_size =1000\n",
     "batch_size=100\n",
-    "show=False"
+    "show=False\n",
+    "action_size=env.action_space.n\n",
+    "state_size=env.observation_space.shape[0]"
    ]
   },
   {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Change show into ```True``` for visualization "
-   ]
-  },
-  {
-   "cell_type": "markdown",
+   "cell_type": "code",
+   "execution_count": 4,
    "metadata": {},
+   "outputs": [],
    "source": [
-    "### The Concept of Experience Replay"
+    "factor=[1,100]"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "One of the challenges of Deep Learning based Q Learning algorithms is that, there is a high chance that every state of an agent is related to previous states. Which may not be the case always. Specially when we are giving actions randomly. So we need random states with random actions.\n"
+    "Initializing Replay buffer for ***Experience Replay***"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Here comes Experience Replay, which is nothing. Let the agent take random actions for , say 1000 times. Save those data into memory. While training the agent, feed the data randomly to the architecture. Also update the memory with new data while training goes on."
+    "- ```memory``` a deque -which is a special type of list with limited memory- the replay buffer\n",
+    "- ```s``` current state\n",
+    "- ```a``` action\n",
+    "- ```new_s``` new state\n",
+    "- ```r``` reward\n",
+    "- ```d``` terminal\n",
+    "- ```experience``` tuple of state,reward,action,next state\n"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "So our steps are as following\n",
-    "* Initialize Memory\n",
-    "* Define Deep Learning Architecture\n",
-    "* Save N number of Random states with their rewards , actions and resulting states\n",
-    "* Start Episode\n",
-    "* Take some actions based on $\\epsilon$ greedy policy\n",
-    "* Save those states,actions, rewards and new states in the memory\n",
-    "* Take random minibatch from the memory and train the architecture based on the minibatch\n",
-    "* Start New Episode\n"
+    "***Psuedocode*** for experience replay"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "For our memory, we will use special data structure of python ```deque``` . It is a special type of list which has a limit of its elements. The moment you append another element it will remove the first element of it. It is working like a open tube with two open ends"
+    "- get initial state $s$\n",
+    "- for each iteration do\n",
+    "- - take a random action $a$\n",
+    "- - get next state $s_{next}$, reward $r$, terminal $d$ \n",
+    "- - $s \\gets s_{next}$\n",
+    "- - if environment is terminated do\n",
+    "- - - reward $\\gets$ -100\n",
+    "- - - reset environment\n",
+    "- - - add state,reward,action,next state into replay buffer\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {
-    "collapsed": true
-   },
+   "execution_count": 5,
+   "metadata": {},
    "outputs": [],
    "source": [
     "memory=deque(maxlen=memory_size)\n",
     "s=env.reset()\n",
     "s = s.reshape((1,-1))\n",
-    "a=env.action_space.sample()\n",
-    "new_s,r,d,_ =env.step(a)\n",
-    "new_s = new_s.reshape((1,-1))\n",
-    "experience=(s,r,a,new_s)\n",
-    "memory.append(experience)\n",
-    "s = new_s\n",
+    "s = s*factor\n",
     "for _ in range(memory_size):\n",
     "    a=env.action_space.sample()\n",
     "    new_s,r,d,_ =env.step(a)\n",
     "    new_s = new_s.reshape((1,-1))\n",
+    "    new_s = new_s*factor\n",
     "    if show:\n",
     "        env.render()\n",
     "    if d:\n",
     "        r=-100\n",
-    "        experience =(s,r,a,new_s)\n",
+    "        experience =(s,r,a,new_s,d)\n",
     "        s=env.reset()\n",
     "        s = s.reshape((1,-1))\n",
     "    else:    \n",
-    "        experience =(s,r,a,new_s)\n",
+    "        experience =(s,r,a,new_s,d)\n",
     "    memory.append(experience)\n",
     "    s = new_s\n",
-    "env.close() "
+    "env.close()  "
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Deep Q Learning"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Vanilla Q learning works best where the state and actions of the agent is discrete . Like the following diagram , which is taken from [A painless Q Learning Tutorial](http://mnemstudio.org/path-finding-q-learning-tutorial.htm) "
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![Painless](q_matrix5.gif)"
+    "***Model Definition***"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Here the actions[along rows] are discrete as 0-5 and states [along columns] are also 0-5 . For state 0 the maximum Q value is 80 which is in column 4. So the best action will be 4. But in our case, the Cartpole game, the states aren't discrete. We had to discretize in the previous [version](https://github.com/sezan92/CartPoleRL/blob/master/CartPoleQLearning.ipynb)."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Other than discretization , we cannot make the infinite states. So the best way is to train a neural network ,which given a state , gives output of Q value for each action. The agent will choose the action with maximum Q value and it will go on."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![DQL](DQL.png)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Our Model"
+    "Here , I have defined the model as a simple MLP neural network with 2 hidden layers of 100 nodes with ```relu``` activation function"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
+      "WARNING:tensorflow:From /home/sezan92/anaconda/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:1290: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.\n",
+      "Instructions for updating:\n",
+      "keep_dims is deprecated, use keepdims instead\n",
       "_________________________________________________________________\n",
       "Layer (type)                 Output Shape              Param #   \n",
       "=================================================================\n",
-      "dense_1 (Dense)              (None, 1, 20)             100       \n",
+      "dense_1 (Dense)              (None, 1, 100)            300       \n",
       "_________________________________________________________________\n",
-      "dense_2 (Dense)              (None, 1, 20)             420       \n",
+      "dense_2 (Dense)              (None, 1, 100)            10100     \n",
       "_________________________________________________________________\n",
-      "dense_3 (Dense)              (None, 1, 2)              42        \n",
+      "dense_3 (Dense)              (None, 1, 3)              303       \n",
       "=================================================================\n",
-      "Total params: 562\n",
-      "Trainable params: 562\n",
+      "Total params: 10,703\n",
+      "Trainable params: 10,703\n",
       "Non-trainable params: 0\n",
       "_________________________________________________________________\n"
      ]
@@ -292,10 +449,10 @@
    ],
    "source": [
     "model = Sequential()\n",
-    "model.add(Dense(20,activation='relu',input_shape=(1,4)))\n",
-    "model.add(Dense(20,activation='relu'))\n",
-    "model.add(Dense(2,activation='linear'))\n",
-    "model.compile(loss='mse',optimizer=Adam(lr=0.01),)\n",
+    "model.add(Dense(100,activation='relu',input_shape=(1,state_size)))\n",
+    "model.add(Dense(100,activation='relu'))\n",
+    "model.add(Dense(action_size,activation='linear'))\n",
+    "model.compile(loss='mse',optimizer=Adam(lr=lr),)\n",
     "model.summary()"
    ]
   },
@@ -303,1157 +460,1180 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Training"
+    "Here ,\n",
+    "- ```ep_list``` list of episodes\n",
+    "- ```reward_list``` list of rewards\n",
+    "- ```total_rewards``` totatl reward"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "***Psuedocode***"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- for each episode do\n",
+    "- - get initial state $s$\n",
+    "- -  $rewards_{total} \\gets 0 $\n",
+    "- - set terminal $d$ to false \n",
+    "- - for each step do\n",
+    "- - - choose action based on epsilon greedy policy\n",
+    "- - - get next state $s_{next}$, reward $r$, terminal $d$ doing the action\n",
+    "- - - $rewards_{total} \\gets rewards_{total}+r$\n",
+    "- - - if $d$ is $True $ \n",
+    "- - - - if $rewards_{total}<-199$ \n",
+    "- - - - - then give punishment $r \\gets -100$\n",
+    "- - - - - break \n",
+    "- - - $s \\gets s_{next}$\n",
+    "- - take random samples of $s,r,a,s_{next}$ from replay buffer\n",
+    "- - get $Q(s_{next})$ \n",
+    "- - $Q_{target} \\gets r+\\gamma max(Q(s_{next})) $\n",
+    "- - $loss \\gets \\frac{1}{N}\\sum(Q_{target}-Q(s))^2$\n",
+    "- - train the network using this loss"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Episode 0, Failed! Reward 16\n",
-      "Episode 1, Failed! Reward 14\n",
-      "Episode 2, Failed! Reward 21\n",
-      "Episode 3, Failed! Reward 34\n",
-      "Episode 4, Failed! Reward 12\n",
-      "Episode 5, Failed! Reward 13\n",
-      "Episode 6, Failed! Reward 47\n",
-      "Episode 7, Failed! Reward 35\n",
-      "Episode 8, Failed! Reward 11\n",
-      "Episode 9, Failed! Reward 88\n",
-      "Episode 10, Failed! Reward 24\n",
-      "Episode 11, Failed! Reward 9\n",
-      "Episode 12, Failed! Reward 17\n",
-      "Episode 13, Failed! Reward 20\n",
-      "Episode 14, Failed! Reward 23\n",
-      "Episode 15, Failed! Reward 19\n",
-      "Episode 16, Failed! Reward 14\n",
-      "Episode 17, Failed! Reward 24\n",
-      "Episode 18, Failed! Reward 19\n",
-      "Episode 19, Failed! Reward 12\n",
-      "Episode 20, Failed! Reward 20\n",
-      "Episode 21, Failed! Reward 20\n",
-      "Episode 22, Failed! Reward 23\n",
-      "Episode 23, Failed! Reward 22\n",
-      "Episode 24, Failed! Reward 24\n",
-      "Episode 25, Failed! Reward 14\n",
-      "Episode 26, Failed! Reward 15\n",
-      "Episode 27, Failed! Reward 38\n",
-      "Episode 28, Failed! Reward 61\n",
-      "Episode 29, Failed! Reward 44\n",
-      "Episode 30, Failed! Reward 23\n",
-      "Episode 31, Failed! Reward 18\n",
-      "Episode 32, Failed! Reward 55\n",
-      "Episode 33, Failed! Reward 16\n",
-      "Episode 34, Failed! Reward 38\n",
-      "Episode 35, Failed! Reward 29\n",
-      "Episode 36, Failed! Reward 15\n",
-      "Episode 37, Failed! Reward 42\n",
-      "Episode 38, Failed! Reward 30\n",
-      "Episode 39, Failed! Reward 27\n",
-      "Episode 40, Failed! Reward 59\n",
-      "Episode 41, Failed! Reward 21\n",
-      "Episode 42, Failed! Reward 14\n",
-      "Episode 43, Failed! Reward 22\n",
-      "Episode 44, Failed! Reward 48\n",
-      "Episode 45, Failed! Reward 12\n",
-      "Episode 46, Failed! Reward 15\n",
-      "Episode 47, Failed! Reward 12\n",
-      "Episode 48, Failed! Reward 30\n",
-      "Episode 49, Failed! Reward 33\n",
-      "Episode 50, Failed! Reward 44\n",
-      "Episode 51, Failed! Reward 41\n",
-      "Episode 52, Failed! Reward 10\n",
-      "Episode 53, Failed! Reward 19\n",
-      "Episode 54, Failed! Reward 20\n",
-      "Episode 55, Failed! Reward 18\n",
-      "Episode 56, Failed! Reward 31\n",
-      "Episode 57, Failed! Reward 68\n",
-      "Episode 58, Failed! Reward 22\n",
-      "Episode 59, Failed! Reward 17\n",
-      "Episode 60, Failed! Reward 65\n",
-      "Episode 61, Failed! Reward 23\n",
-      "Episode 62, Failed! Reward 70\n",
-      "Episode 63, Failed! Reward 69\n",
-      "Episode 64, Failed! Reward 54\n",
-      "Episode 65, Failed! Reward 42\n",
-      "Episode 66, Failed! Reward 31\n",
-      "Episode 67, Failed! Reward 93\n",
-      "Episode 68, Failed! Reward 48\n",
-      "Episode 69, Failed! Reward 26\n",
-      "Episode 70, Failed! Reward 59\n",
-      "Episode 71, Failed! Reward 12\n",
-      "Episode 72, Failed! Reward 19\n",
-      "Episode 73, Failed! Reward 44\n",
-      "Episode 74, Failed! Reward 33\n",
-      "Episode 75, Failed! Reward 37\n",
-      "Episode 76, Failed! Reward 56\n",
-      "Episode 77, Failed! Reward 32\n",
-      "Episode 78, Failed! Reward 25\n",
-      "Episode 79, Failed! Reward 43\n",
-      "Episode 80, Failed! Reward 94\n",
-      "Episode 81, Failed! Reward 55\n",
-      "Episode 82, Failed! Reward 28\n",
-      "Episode 83, Failed! Reward 32\n",
-      "Episode 84, Failed! Reward 54\n",
-      "Episode 85, Failed! Reward 13\n",
-      "Episode 86, Failed! Reward 57\n",
-      "Episode 87, Failed! Reward 43\n",
-      "Episode 88, Failed! Reward 57\n",
-      "Episode 89, Failed! Reward 44\n",
-      "Episode 90, Failed! Reward 32\n",
-      "Episode 91, Failed! Reward 32\n",
-      "Episode 92, Failed! Reward 18\n",
-      "Episode 93, Failed! Reward 46\n",
-      "Episode 94, Failed! Reward 76\n",
-      "Episode 95, Failed! Reward 12\n",
-      "Episode 96, Failed! Reward 63\n",
-      "Episode 97, Failed! Reward 88\n",
-      "Episode 98, Failed! Reward 27\n",
-      "Episode 99, Failed! Reward 72\n",
-      "Episode 100, Failed! Reward 70\n",
-      "Episode 101, Failed! Reward 47\n",
-      "Episode 102, Failed! Reward 93\n",
-      "Episode 103, Failed! Reward 57\n",
-      "Episode 104, Failed! Reward 28\n",
-      "Episode 105, Failed! Reward 86\n",
-      "Episode 106, Failed! Reward 53\n",
-      "Episode 107, Failed! Reward 113\n",
-      "Episode 108, Failed! Reward 48\n",
-      "Episode 109, Failed! Reward 33\n",
-      "Episode 110, Failed! Reward 61\n",
-      "Episode 111, Failed! Reward 105\n",
-      "Episode 112, Failed! Reward 96\n",
-      "Episode 113, Failed! Reward 64\n",
-      "Episode 114, Failed! Reward 30\n",
-      "Episode 115, Failed! Reward 48\n",
-      "Episode 116, Failed! Reward 76\n",
-      "Episode 117, Failed! Reward 149\n",
-      "Episode 118, Failed! Reward 135\n",
-      "Episode 119, Failed! Reward 41\n",
-      "Episode 120, Failed! Reward 120\n",
-      "Episode 121, Failed! Reward 77\n",
-      "Episode 122, Failed! Reward 99\n",
-      "Episode 123, Failed! Reward 73\n",
-      "Episode 124, Failed! Reward 183\n",
-      "Episode 125, Failed! Reward 135\n",
-      "Episode 126, Failed! Reward 120\n",
-      "Episode 127, Failed! Reward 98\n",
-      "Episode 128, Failed! Reward 124\n",
-      "Episode 129, Failed! Reward 67\n",
-      "Episode 130, Failed! Reward 150\n",
-      "Episode 131, Failed! Reward 71\n",
-      "Episode 132, Failed! Reward 121\n",
-      "Episode 133, Failed! Reward 29\n",
-      "Episode 134, Failed! Reward 65\n",
-      "Episode 135, Failed! Reward 97\n",
-      "Episode 136, Failed! Reward 93\n",
-      "Episode 137, Failed! Reward 175\n",
-      "Episode 138, Failed! Reward 37\n",
-      "Episode 139, Failed! Reward 159\n",
-      "Episode 140, Failed! Reward 40\n",
-      "Episode 141, Failed! Reward 64\n",
-      "Episode 142, Failed! Reward 92\n",
-      "Episode 143, Failed! Reward 42\n",
-      "Episode 144, Failed! Reward 80\n",
-      "Episode 145, Failed! Reward 96\n",
-      "Episode 146, Failed! Reward 34\n",
-      "Episode 147, Failed! Reward 69\n",
-      "Episode 148, Failed! Reward 40\n",
-      "Episode 149, Failed! Reward 17\n",
-      "Episode 150, Failed! Reward 65\n",
-      "Episode 151, Failed! Reward 42\n",
-      "Episode 152, Failed! Reward 62\n",
-      "Episode 153, Failed! Reward 98\n",
-      "Episode 154, Passed! Reward 200\n",
-      "Episode 155, Failed! Reward 108\n",
-      "Episode 156, Failed! Reward 87\n",
-      "Episode 157, Failed! Reward 112\n",
-      "Episode 158, Failed! Reward 121\n",
-      "Episode 159, Failed! Reward 22\n",
-      "Episode 160, Failed! Reward 115\n",
-      "Episode 161, Failed! Reward 170\n",
-      "Episode 162, Failed! Reward 103\n",
-      "Episode 163, Passed! Reward 200\n",
-      "Episode 164, Failed! Reward 178\n",
-      "Episode 165, Failed! Reward 146\n",
-      "Episode 166, Failed! Reward 107\n",
-      "Episode 167, Failed! Reward 140\n",
-      "Episode 168, Failed! Reward 55\n",
-      "Episode 169, Failed! Reward 143\n",
-      "Episode 170, Failed! Reward 146\n",
-      "Episode 171, Failed! Reward 173\n",
-      "Episode 172, Failed! Reward 116\n",
-      "Episode 173, Failed! Reward 34\n",
-      "Episode 174, Failed! Reward 80\n",
-      "Episode 175, Failed! Reward 128\n",
-      "Episode 176, Failed! Reward 17\n",
-      "Episode 177, Failed! Reward 15\n",
-      "Episode 178, Failed! Reward 138\n",
-      "Episode 179, Failed! Reward 121\n",
-      "Episode 180, Failed! Reward 27\n",
-      "Episode 181, Failed! Reward 131\n",
-      "Episode 182, Failed! Reward 121\n",
-      "Episode 183, Failed! Reward 126\n",
-      "Episode 184, Failed! Reward 136\n",
-      "Episode 185, Failed! Reward 33\n",
-      "Episode 186, Failed! Reward 30\n",
-      "Episode 187, Failed! Reward 133\n",
-      "Episode 188, Failed! Reward 169\n",
-      "Episode 189, Failed! Reward 16\n",
-      "Episode 190, Failed! Reward 178\n",
-      "Episode 191, Failed! Reward 125\n",
-      "Episode 192, Failed! Reward 127\n",
-      "Episode 193, Failed! Reward 137\n",
-      "Episode 194, Failed! Reward 161\n",
-      "Episode 195, Failed! Reward 131\n",
-      "Episode 196, Failed! Reward 113\n",
-      "Episode 197, Failed! Reward 83\n",
-      "Episode 198, Failed! Reward 108\n",
-      "Episode 199, Failed! Reward 184\n",
-      "Episode 201, Failed! Reward 161\n",
-      "Episode 202, Failed! Reward 170\n",
-      "Episode 203, Failed! Reward 172\n",
-      "Episode 204, Failed! Reward 16\n",
-      "Episode 205, Failed! Reward 78\n",
-      "Episode 206, Failed! Reward 126\n",
-      "Episode 207, Failed! Reward 154\n",
-      "Episode 208, Failed! Reward 164\n",
-      "Episode 209, Failed! Reward 99\n",
-      "Episode 210, Failed! Reward 159\n",
-      "Episode 211, Failed! Reward 118\n",
-      "Episode 212, Failed! Reward 156\n",
-      "Episode 213, Failed! Reward 139\n",
-      "Episode 214, Failed! Reward 153\n",
-      "Episode 215, Failed! Reward 161\n",
-      "Episode 216, Failed! Reward 18\n",
-      "Episode 217, Failed! Reward 12\n",
-      "Episode 218, Failed! Reward 130\n",
-      "Episode 219, Failed! Reward 119\n",
-      "Episode 220, Failed! Reward 122\n",
-      "Episode 221, Failed! Reward 136\n",
-      "Episode 222, Failed! Reward 135\n",
-      "Episode 223, Failed! Reward 24\n",
-      "Episode 224, Failed! Reward 159\n",
-      "Episode 225, Failed! Reward 142\n",
-      "Episode 226, Failed! Reward 147\n",
-      "Episode 227, Failed! Reward 138\n",
-      "Episode 228, Failed! Reward 147\n",
-      "Episode 229, Failed! Reward 147\n",
-      "Episode 230, Failed! Reward 150\n",
-      "Episode 231, Failed! Reward 150\n",
-      "Episode 232, Failed! Reward 129\n",
-      "Episode 233, Failed! Reward 150\n",
-      "Episode 234, Failed! Reward 149\n",
-      "Episode 235, Failed! Reward 151\n",
-      "Episode 236, Failed! Reward 116\n",
-      "Episode 237, Failed! Reward 108\n",
-      "Episode 238, Failed! Reward 126\n",
-      "Episode 239, Failed! Reward 135\n",
-      "Episode 240, Failed! Reward 116\n",
-      "Episode 241, Failed! Reward 15\n",
-      "Episode 242, Failed! Reward 108\n",
-      "Episode 243, Failed! Reward 24\n",
-      "Episode 244, Failed! Reward 14\n",
-      "Episode 245, Failed! Reward 13\n",
-      "Episode 246, Failed! Reward 104\n",
-      "Episode 247, Failed! Reward 61\n",
-      "Episode 248, Passed! Reward 200\n",
-      "Episode 249, Failed! Reward 126\n",
-      "Episode 250, Failed! Reward 159\n",
-      "Episode 251, Failed! Reward 175\n",
-      "Episode 252, Failed! Reward 190\n",
-      "Episode 253, Failed! Reward 127\n",
-      "Episode 255, Failed! Reward 137\n",
-      "Episode 256, Failed! Reward 154\n",
-      "Episode 257, Failed! Reward 141\n",
-      "Episode 258, Failed! Reward 156\n",
-      "Episode 259, Failed! Reward 105\n",
-      "Episode 260, Failed! Reward 143\n",
-      "Episode 261, Failed! Reward 25\n",
-      "Episode 262, Failed! Reward 156\n",
-      "Episode 263, Failed! Reward 175\n",
-      "Episode 264, Failed! Reward 148\n",
-      "Episode 265, Failed! Reward 169\n",
-      "Episode 266, Failed! Reward 183\n",
-      "Episode 267, Failed! Reward 180\n"
+      "Episode 0, Failed! Reward -200\n",
+      "Episode 1, Failed! Reward -200\n",
+      "Episode 2, Failed! Reward -200\n",
+      "Episode 3, Failed! Reward -200\n",
+      "Episode 4, Failed! Reward -200\n",
+      "Episode 5, Failed! Reward -200\n",
+      "Episode 6, Failed! Reward -200\n",
+      "Episode 7, Failed! Reward -200\n",
+      "Episode 8, Failed! Reward -200\n",
+      "Episode 9, Failed! Reward -200\n",
+      "Episode 10, Failed! Reward -200\n",
+      "Episode 11, Failed! Reward -200\n",
+      "Episode 12, Failed! Reward -200\n",
+      "Episode 13, Failed! Reward -200\n",
+      "Episode 14, Failed! Reward -200\n",
+      "Episode 15, Failed! Reward -200\n",
+      "Episode 16, Failed! Reward -200\n",
+      "Episode 17, Failed! Reward -200\n",
+      "Episode 18, Failed! Reward -200\n",
+      "Episode 19, Failed! Reward -200\n",
+      "Episode 20, Failed! Reward -200\n",
+      "Episode 21, Failed! Reward -200\n",
+      "Episode 22, Failed! Reward -200\n",
+      "Episode 23, Failed! Reward -200\n",
+      "Episode 24, Failed! Reward -200\n",
+      "Episode 25, Failed! Reward -200\n",
+      "Episode 26, Failed! Reward -200\n",
+      "Episode 27, Failed! Reward -200\n",
+      "Episode 28, Failed! Reward -200\n",
+      "Episode 29, Failed! Reward -200\n",
+      "Episode 30, Failed! Reward -200\n",
+      "Episode 31, Failed! Reward -200\n",
+      "Episode 32, Failed! Reward -200\n",
+      "Episode 33, Failed! Reward -200\n",
+      "Episode 34, Failed! Reward -200\n",
+      "Episode 35, Failed! Reward -200\n",
+      "Episode 36, Failed! Reward -200\n",
+      "Episode 37, Failed! Reward -200\n",
+      "Episode 38, Failed! Reward -200\n",
+      "Episode 39, Failed! Reward -200\n",
+      "Episode 40, Failed! Reward -200\n",
+      "Episode 41, Failed! Reward -200\n",
+      "Episode 42, Failed! Reward -200\n",
+      "Episode 43, Failed! Reward -200\n",
+      "Episode 44, Failed! Reward -200\n",
+      "Episode 45, Failed! Reward -200\n",
+      "Episode 46, Failed! Reward -200\n",
+      "Episode 47, Failed! Reward -200\n",
+      "Episode 48, Failed! Reward -200\n",
+      "Episode 49, Failed! Reward -200\n",
+      "Episode 50, Failed! Reward -200\n",
+      "Episode 51, Failed! Reward -200\n",
+      "Episode 52, Failed! Reward -200\n",
+      "Episode 53, Failed! Reward -200\n",
+      "Episode 54, Failed! Reward -200\n",
+      "Episode 55, Failed! Reward -200\n",
+      "Episode 56, Failed! Reward -200\n",
+      "Episode 57, Failed! Reward -200\n",
+      "Episode 58, Failed! Reward -200\n",
+      "Episode 59, Failed! Reward -200\n",
+      "Episode 60, Failed! Reward -200\n",
+      "Episode 61, Failed! Reward -200\n",
+      "Episode 62, Failed! Reward -200\n",
+      "Episode 63, Failed! Reward -200\n",
+      "Episode 64, Failed! Reward -200\n",
+      "Episode 65, Failed! Reward -200\n",
+      "Episode 66, Failed! Reward -200\n",
+      "Episode 67, Failed! Reward -200\n",
+      "Episode 68, Failed! Reward -200\n",
+      "Episode 69, Failed! Reward -200\n",
+      "Episode 70, Failed! Reward -200\n",
+      "Episode 71, Failed! Reward -200\n",
+      "Episode 72, Failed! Reward -200\n",
+      "Episode 73, Failed! Reward -200\n",
+      "Episode 74, Failed! Reward -200\n",
+      "Episode 75, Failed! Reward -200\n",
+      "Episode 76, Failed! Reward -200\n",
+      "Episode 77, Failed! Reward -200\n",
+      "Episode 78, Failed! Reward -200\n",
+      "Episode 79, Failed! Reward -200\n",
+      "Episode 80, Failed! Reward -200\n",
+      "Episode 81, Failed! Reward -200\n",
+      "Episode 82, Failed! Reward -200\n",
+      "Episode 83, Failed! Reward -200\n",
+      "Episode 84, Failed! Reward -200\n",
+      "Episode 85, Failed! Reward -200\n",
+      "Episode 86, Failed! Reward -200\n",
+      "Episode 87, Failed! Reward -200\n",
+      "Episode 88, Failed! Reward -200\n",
+      "Episode 89, Failed! Reward -200\n",
+      "Episode 90, Failed! Reward -200\n",
+      "Episode 91, Failed! Reward -200\n",
+      "Episode 92, Failed! Reward -200\n",
+      "Episode 93, Failed! Reward -200\n",
+      "Episode 94, Failed! Reward -200\n",
+      "Episode 95, Failed! Reward -200\n",
+      "Episode 96, Failed! Reward -200\n",
+      "Episode 97, Failed! Reward -200\n",
+      "Episode 98, Failed! Reward -200\n",
+      "Episode 99, Failed! Reward -200\n",
+      "Episode 100, Failed! Reward -200\n",
+      "Episode 101, Failed! Reward -200\n",
+      "Episode 102, Failed! Reward -200\n",
+      "Episode 103, Failed! Reward -200\n",
+      "Episode 104, Failed! Reward -200\n",
+      "Episode 105, Failed! Reward -200\n",
+      "Episode 106, Failed! Reward -200\n",
+      "Episode 107, Failed! Reward -200\n",
+      "Episode 108, Failed! Reward -200\n",
+      "Episode 109, Failed! Reward -200\n",
+      "Episode 110, Failed! Reward -200\n",
+      "Episode 111, Failed! Reward -200\n",
+      "Episode 112, Failed! Reward -200\n",
+      "Episode 113, Failed! Reward -200\n",
+      "Episode 114, Failed! Reward -200\n",
+      "Episode 115, Failed! Reward -200\n",
+      "Episode 116, Failed! Reward -200\n",
+      "Episode 117, Failed! Reward -200\n",
+      "Episode 118, Failed! Reward -200\n",
+      "Episode 119, Failed! Reward -200\n",
+      "Episode 120, Failed! Reward -200\n",
+      "Episode 121, Failed! Reward -200\n",
+      "Episode 122, Failed! Reward -200\n",
+      "Episode 123, Failed! Reward -200\n",
+      "Episode 124, Failed! Reward -200\n",
+      "Episode 125, Failed! Reward -200\n",
+      "Episode 126, Failed! Reward -200\n",
+      "Episode 127, Failed! Reward -200\n",
+      "Episode 128, Failed! Reward -200\n",
+      "Episode 129, Failed! Reward -200\n",
+      "Episode 130, Failed! Reward -200\n",
+      "Episode 131, Failed! Reward -200\n",
+      "Episode 132, Failed! Reward -200\n",
+      "Episode 133, Failed! Reward -200\n",
+      "Episode 134, Failed! Reward -200\n",
+      "Episode 135, Failed! Reward -200\n",
+      "Episode 136, Failed! Reward -200\n",
+      "Episode 137, Failed! Reward -200\n",
+      "Episode 138, Failed! Reward -200\n",
+      "Episode 139, Failed! Reward -200\n",
+      "Episode 140, Failed! Reward -200\n",
+      "Episode 141, Failed! Reward -200\n",
+      "Episode 142, Failed! Reward -200\n",
+      "Episode 143, Failed! Reward -200\n",
+      "Episode 144, Failed! Reward -200\n",
+      "Episode 145, Failed! Reward -200\n",
+      "Episode 146, Failed! Reward -200\n",
+      "Episode 147, Failed! Reward -200\n",
+      "Episode 148, Failed! Reward -200\n",
+      "Episode 149, Failed! Reward -200\n",
+      "Episode 150, Failed! Reward -200\n",
+      "Episode 151, Failed! Reward -200\n",
+      "Episode 152, Failed! Reward -200\n",
+      "Episode 153, Failed! Reward -200\n",
+      "Episode 154, Failed! Reward -200\n",
+      "Episode 155, Failed! Reward -200\n",
+      "Episode 156, Failed! Reward -200\n",
+      "Episode 157, Failed! Reward -200\n",
+      "Episode 158, Failed! Reward -200\n",
+      "Episode 159, Failed! Reward -200\n",
+      "Episode 160, Failed! Reward -200\n",
+      "Episode 161, Failed! Reward -200\n",
+      "Episode 162, Failed! Reward -200\n",
+      "Episode 163, Better! Reward -159\n",
+      "Episode 164, Failed! Reward -200\n",
+      "Episode 165, Failed! Reward -200\n",
+      "Episode 166, Failed! Reward -200\n",
+      "Episode 167, Failed! Reward -200\n",
+      "Episode 168, Failed! Reward -200\n",
+      "Episode 169, Failed! Reward -200\n",
+      "Episode 170, Failed! Reward -200\n",
+      "Episode 171, Failed! Reward -200\n",
+      "Episode 172, Failed! Reward -200\n",
+      "Episode 173, Failed! Reward -200\n",
+      "Episode 174, Failed! Reward -200\n",
+      "Episode 175, Failed! Reward -200\n",
+      "Episode 176, Failed! Reward -200\n",
+      "Episode 177, Failed! Reward -200\n",
+      "Episode 178, Failed! Reward -200\n",
+      "Episode 179, Failed! Reward -200\n",
+      "Episode 180, Failed! Reward -200\n",
+      "Episode 181, Failed! Reward -200\n",
+      "Episode 182, Failed! Reward -200\n",
+      "Episode 183, Failed! Reward -200\n",
+      "Episode 184, Failed! Reward -200\n",
+      "Episode 185, Failed! Reward -200\n",
+      "Episode 186, Failed! Reward -200\n",
+      "Episode 187, Failed! Reward -200\n",
+      "Episode 188, Failed! Reward -200\n",
+      "Episode 189, Failed! Reward -200\n",
+      "Episode 190, Failed! Reward -200\n",
+      "Episode 191, Failed! Reward -200\n",
+      "Episode 192, Failed! Reward -200\n",
+      "Episode 193, Failed! Reward -200\n",
+      "Episode 194, Failed! Reward -200\n",
+      "Episode 195, Failed! Reward -200\n",
+      "Episode 196, Failed! Reward -200\n",
+      "Episode 197, Failed! Reward -200\n",
+      "Episode 198, Failed! Reward -200\n",
+      "Episode 199, Failed! Reward -200\n",
+      "Episode 200, Failed! Reward -200\n",
+      "Episode 201, Failed! Reward -200\n",
+      "Episode 202, Failed! Reward -200\n",
+      "Episode 203, Failed! Reward -200\n",
+      "Episode 204, Failed! Reward -200\n",
+      "Episode 205, Failed! Reward -200\n",
+      "Episode 206, Failed! Reward -200\n",
+      "Episode 207, Failed! Reward -200\n",
+      "Episode 208, Failed! Reward -200\n",
+      "Episode 209, Failed! Reward -200\n",
+      "Episode 210, Failed! Reward -200\n",
+      "Episode 211, Failed! Reward -200\n",
+      "Episode 212, Failed! Reward -200\n",
+      "Episode 213, Failed! Reward -200\n",
+      "Episode 214, Failed! Reward -200\n",
+      "Episode 215, Failed! Reward -200\n",
+      "Episode 216, Failed! Reward -200\n",
+      "Episode 217, Failed! Reward -200\n",
+      "Episode 218, Failed! Reward -200\n",
+      "Episode 219, Failed! Reward -200\n",
+      "Episode 220, Failed! Reward -200\n",
+      "Episode 221, Failed! Reward -200\n",
+      "Episode 222, Failed! Reward -200\n",
+      "Episode 223, Failed! Reward -200\n",
+      "Episode 224, Failed! Reward -200\n",
+      "Episode 225, Failed! Reward -200\n",
+      "Episode 226, Failed! Reward -200\n",
+      "Episode 227, Failed! Reward -200\n",
+      "Episode 228, Failed! Reward -200\n",
+      "Episode 229, Failed! Reward -200\n",
+      "Episode 230, Failed! Reward -200\n",
+      "Episode 231, Failed! Reward -200\n",
+      "Episode 232, Failed! Reward -200\n",
+      "Episode 233, Failed! Reward -200\n",
+      "Episode 234, Failed! Reward -200\n",
+      "Episode 235, Failed! Reward -200\n",
+      "Episode 236, Failed! Reward -200\n",
+      "Episode 237, Failed! Reward -200\n",
+      "Episode 238, Failed! Reward -200\n",
+      "Episode 239, Failed! Reward -200\n",
+      "Episode 240, Failed! Reward -200\n",
+      "Episode 241, Failed! Reward -200\n",
+      "Episode 242, Failed! Reward -200\n",
+      "Episode 243, Failed! Reward -200\n",
+      "Episode 244, Failed! Reward -200\n",
+      "Episode 245, Failed! Reward -200\n",
+      "Episode 246, Failed! Reward -200\n",
+      "Episode 247, Failed! Reward -200\n",
+      "Episode 248, Failed! Reward -200\n",
+      "Episode 249, Failed! Reward -200\n",
+      "Episode 250, Failed! Reward -200\n",
+      "Episode 251, Failed! Reward -200\n"
      ]
     },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Episode 268, Failed! Reward 190\n",
-      "Episode 269, Failed! Reward 174\n",
-      "Episode 270, Failed! Reward 154\n",
-      "Episode 271, Failed! Reward 154\n",
-      "Episode 272, Failed! Reward 181\n",
-      "Episode 273, Failed! Reward 167\n",
-      "Episode 274, Failed! Reward 171\n",
-      "Episode 275, Failed! Reward 166\n",
-      "Episode 276, Failed! Reward 168\n",
-      "Episode 277, Failed! Reward 159\n",
-      "Episode 278, Failed! Reward 149\n",
-      "Episode 279, Failed! Reward 147\n",
-      "Episode 280, Failed! Reward 142\n",
-      "Episode 281, Failed! Reward 31\n",
-      "Episode 282, Failed! Reward 140\n",
-      "Episode 283, Failed! Reward 172\n",
-      "Episode 284, Failed! Reward 174\n",
-      "Episode 285, Failed! Reward 161\n",
-      "Episode 286, Failed! Reward 173\n",
-      "Episode 287, Failed! Reward 165\n",
-      "Episode 288, Failed! Reward 184\n",
-      "Episode 289, Passed! Reward 199\n",
-      "Episode 290, Failed! Reward 188\n",
-      "Episode 291, Failed! Reward 99\n",
-      "Episode 292, Failed! Reward 187\n",
-      "Episode 294, Failed! Reward 182\n",
-      "Episode 295, Passed! Reward 200\n",
-      "Episode 296, Failed! Reward 171\n",
-      "Episode 297, Failed! Reward 185\n",
-      "Episode 298, Failed! Reward 180\n",
-      "Episode 299, Failed! Reward 194\n",
-      "Episode 300, Failed! Reward 177\n",
-      "Episode 301, Failed! Reward 188\n",
-      "Episode 302, Failed! Reward 177\n",
-      "Episode 303, Failed! Reward 178\n",
-      "Episode 304, Failed! Reward 191\n",
-      "Episode 305, Passed! Reward 200\n",
-      "Episode 306, Failed! Reward 179\n",
-      "Episode 307, Passed! Reward 200\n",
-      "Episode 308, Failed! Reward 169\n",
-      "Episode 309, Failed! Reward 166\n",
-      "Episode 310, Failed! Reward 165\n",
-      "Episode 311, Failed! Reward 144\n",
-      "Episode 312, Failed! Reward 160\n",
-      "Episode 313, Failed! Reward 170\n",
-      "Episode 314, Failed! Reward 170\n",
-      "Episode 315, Failed! Reward 182\n",
-      "Episode 316, Failed! Reward 157\n",
-      "Episode 317, Failed! Reward 176\n",
-      "Episode 318, Passed! Reward 200\n",
-      "Episode 319, Passed! Reward 200\n",
-      "Episode 320, Passed! Reward 200\n",
-      "Episode 321, Passed! Reward 200\n",
-      "Episode 322, Passed! Reward 200\n",
-      "Episode 323, Passed! Reward 200\n",
-      "Episode 324, Failed! Reward 176\n",
-      "Episode 325, Failed! Reward 172\n",
-      "Episode 326, Passed! Reward 200\n",
-      "Episode 327, Failed! Reward 167\n",
-      "Episode 328, Failed! Reward 193\n",
-      "Episode 329, Failed! Reward 183\n",
-      "Episode 330, Failed! Reward 169\n",
-      "Episode 331, Failed! Reward 174\n",
-      "Episode 332, Failed! Reward 169\n",
-      "Episode 333, Failed! Reward 167\n",
-      "Episode 334, Failed! Reward 194\n",
-      "Episode 335, Failed! Reward 175\n",
-      "Episode 336, Failed! Reward 174\n",
-      "Episode 337, Failed! Reward 170\n",
-      "Episode 338, Failed! Reward 156\n",
-      "Episode 339, Failed! Reward 162\n",
-      "Episode 340, Failed! Reward 161\n",
-      "Episode 341, Passed! Reward 200\n",
-      "Episode 342, Failed! Reward 158\n",
-      "Episode 343, Failed! Reward 160\n",
-      "Episode 344, Passed! Reward 200\n",
-      "Episode 345, Failed! Reward 153\n",
-      "Episode 346, Failed! Reward 144\n",
-      "Episode 347, Failed! Reward 145\n",
-      "Episode 348, Failed! Reward 145\n",
-      "Episode 349, Failed! Reward 140\n",
-      "Episode 350, Failed! Reward 143\n",
-      "Episode 351, Failed! Reward 136\n",
-      "Episode 352, Failed! Reward 145\n",
-      "Episode 353, Failed! Reward 149\n",
-      "Episode 354, Failed! Reward 152\n",
-      "Episode 355, Failed! Reward 144\n",
-      "Episode 356, Failed! Reward 139\n",
-      "Episode 357, Failed! Reward 168\n",
-      "Episode 358, Passed! Reward 200\n",
-      "Episode 359, Passed! Reward 200\n",
-      "Episode 360, Failed! Reward 149\n",
-      "Episode 361, Failed! Reward 136\n",
-      "Episode 362, Failed! Reward 143\n",
-      "Episode 363, Failed! Reward 140\n",
-      "Episode 364, Failed! Reward 133\n",
-      "Episode 365, Failed! Reward 130\n",
-      "Episode 366, Failed! Reward 117\n",
-      "Episode 367, Failed! Reward 137\n",
-      "Episode 368, Failed! Reward 125\n",
-      "Episode 369, Failed! Reward 136\n",
-      "Episode 370, Failed! Reward 141\n",
-      "Episode 371, Failed! Reward 132\n",
-      "Episode 372, Failed! Reward 134\n",
-      "Episode 373, Failed! Reward 123\n",
-      "Episode 374, Failed! Reward 139\n",
-      "Episode 375, Failed! Reward 144\n",
-      "Episode 376, Failed! Reward 147\n",
-      "Episode 377, Failed! Reward 149\n",
-      "Episode 378, Failed! Reward 147\n",
-      "Episode 379, Failed! Reward 127\n",
-      "Episode 380, Failed! Reward 139\n",
-      "Episode 381, Failed! Reward 175\n",
-      "Episode 382, Passed! Reward 200\n",
-      "Episode 383, Passed! Reward 200\n",
-      "Episode 384, Passed! Reward 200\n",
-      "Episode 385, Failed! Reward 144\n",
-      "Episode 386, Failed! Reward 136\n",
-      "Episode 387, Failed! Reward 32\n",
-      "Episode 388, Failed! Reward 138\n",
-      "Episode 389, Failed! Reward 133\n",
-      "Episode 390, Failed! Reward 134\n",
-      "Episode 391, Failed! Reward 141\n",
-      "Episode 392, Failed! Reward 135\n",
-      "Episode 393, Failed! Reward 135\n",
-      "Episode 394, Failed! Reward 152\n",
-      "Episode 395, Failed! Reward 146\n",
-      "Episode 396, Failed! Reward 150\n",
-      "Episode 397, Failed! Reward 131\n",
-      "Episode 398, Failed! Reward 138\n",
-      "Episode 399, Failed! Reward 142\n",
-      "Episode 400, Failed! Reward 157\n",
-      "Episode 401, Failed! Reward 137\n",
-      "Episode 402, Failed! Reward 146\n",
-      "Episode 403, Failed! Reward 138\n",
-      "Episode 404, Failed! Reward 133\n",
-      "Episode 405, Failed! Reward 137\n",
-      "Episode 406, Failed! Reward 132\n",
-      "Episode 407, Failed! Reward 139\n",
-      "Episode 408, Failed! Reward 133\n",
-      "Episode 409, Failed! Reward 147\n",
-      "Episode 410, Failed! Reward 149\n",
-      "Episode 411, Failed! Reward 151\n",
-      "Episode 412, Failed! Reward 152\n",
-      "Episode 413, Failed! Reward 154\n",
-      "Episode 414, Failed! Reward 147\n",
-      "Episode 415, Failed! Reward 152\n",
-      "Episode 416, Failed! Reward 150\n",
-      "Episode 417, Failed! Reward 167\n",
-      "Episode 418, Failed! Reward 157\n",
-      "Episode 419, Failed! Reward 153\n",
-      "Episode 420, Failed! Reward 157\n",
-      "Episode 421, Failed! Reward 166\n",
-      "Episode 422, Failed! Reward 152\n",
-      "Episode 423, Failed! Reward 166\n",
-      "Episode 424, Failed! Reward 143\n",
-      "Episode 425, Failed! Reward 165\n",
-      "Episode 426, Failed! Reward 155\n",
-      "Episode 427, Failed! Reward 153\n",
-      "Episode 428, Failed! Reward 143\n",
-      "Episode 429, Failed! Reward 148\n",
-      "Episode 430, Failed! Reward 150\n",
-      "Episode 431, Failed! Reward 146\n",
-      "Episode 432, Failed! Reward 139\n",
-      "Episode 433, Failed! Reward 152\n",
-      "Episode 434, Failed! Reward 155\n",
-      "Episode 435, Failed! Reward 163\n",
-      "Episode 436, Failed! Reward 159\n",
-      "Episode 437, Failed! Reward 155\n",
-      "Episode 438, Failed! Reward 152\n",
-      "Episode 439, Failed! Reward 153\n",
-      "Episode 440, Failed! Reward 170\n",
-      "Episode 441, Failed! Reward 159\n",
-      "Episode 442, Failed! Reward 151\n",
-      "Episode 443, Failed! Reward 168\n",
-      "Episode 444, Failed! Reward 162\n",
-      "Episode 445, Failed! Reward 153\n",
-      "Episode 446, Failed! Reward 148\n",
-      "Episode 447, Failed! Reward 155\n",
-      "Episode 448, Failed! Reward 156\n",
-      "Episode 449, Failed! Reward 145\n",
-      "Episode 450, Failed! Reward 155\n",
-      "Episode 451, Failed! Reward 162\n",
-      "Episode 452, Failed! Reward 151\n",
-      "Episode 453, Failed! Reward 164\n",
-      "Episode 454, Failed! Reward 156\n",
-      "Episode 455, Failed! Reward 165\n",
-      "Episode 456, Failed! Reward 161\n",
-      "Episode 457, Failed! Reward 151\n",
-      "Episode 458, Failed! Reward 163\n",
-      "Episode 459, Failed! Reward 156\n",
-      "Episode 460, Failed! Reward 157\n",
-      "Episode 461, Failed! Reward 156\n",
-      "Episode 462, Failed! Reward 157\n",
-      "Episode 463, Failed! Reward 173\n",
-      "Episode 464, Failed! Reward 150\n",
-      "Episode 465, Failed! Reward 150\n",
-      "Episode 466, Failed! Reward 168\n",
-      "Episode 467, Failed! Reward 194\n",
-      "Episode 468, Failed! Reward 167\n",
-      "Episode 469, Failed! Reward 163\n",
-      "Episode 470, Failed! Reward 145\n",
-      "Episode 471, Failed! Reward 132\n",
-      "Episode 472, Failed! Reward 127\n",
-      "Episode 473, Failed! Reward 123\n",
-      "Episode 474, Failed! Reward 135\n",
-      "Episode 475, Failed! Reward 132\n",
-      "Episode 476, Failed! Reward 124\n",
-      "Episode 477, Failed! Reward 129\n",
-      "Episode 478, Failed! Reward 138\n",
-      "Episode 479, Failed! Reward 130\n",
-      "Episode 480, Failed! Reward 134\n",
-      "Episode 481, Failed! Reward 138\n",
-      "Episode 482, Failed! Reward 135\n",
-      "Episode 483, Failed! Reward 127\n",
-      "Episode 484, Failed! Reward 124\n",
-      "Episode 485, Failed! Reward 130\n",
-      "Episode 486, Failed! Reward 145\n",
-      "Episode 487, Failed! Reward 131\n",
-      "Episode 488, Failed! Reward 137\n",
-      "Episode 489, Failed! Reward 139\n",
-      "Episode 490, Failed! Reward 127\n",
-      "Episode 491, Failed! Reward 131\n",
-      "Episode 492, Failed! Reward 131\n",
-      "Episode 493, Failed! Reward 131\n",
-      "Episode 494, Failed! Reward 136\n",
-      "Episode 495, Failed! Reward 154\n",
-      "Episode 496, Failed! Reward 136\n",
-      "Episode 497, Failed! Reward 134\n",
-      "Episode 498, Failed! Reward 138\n",
-      "Episode 499, Failed! Reward 129\n",
-      "Episode 500, Failed! Reward 141\n",
-      "Episode 501, Failed! Reward 144\n",
-      "Episode 502, Failed! Reward 146\n",
-      "Episode 503, Failed! Reward 142\n",
-      "Episode 504, Failed! Reward 157\n",
-      "Episode 505, Failed! Reward 156\n",
-      "Episode 506, Failed! Reward 157\n",
-      "Episode 507, Failed! Reward 147\n",
-      "Episode 508, Failed! Reward 146\n",
-      "Episode 509, Failed! Reward 149\n",
-      "Episode 510, Failed! Reward 139\n",
-      "Episode 511, Failed! Reward 142\n",
-      "Episode 512, Failed! Reward 145\n",
-      "Episode 513, Failed! Reward 148\n",
-      "Episode 514, Failed! Reward 139\n",
-      "Episode 515, Failed! Reward 142\n",
-      "Episode 516, Failed! Reward 140\n",
-      "Episode 517, Failed! Reward 138\n",
-      "Episode 518, Failed! Reward 145\n",
-      "Episode 519, Failed! Reward 143\n",
-      "Episode 520, Failed! Reward 157\n",
-      "Episode 521, Failed! Reward 148\n",
-      "Episode 522, Failed! Reward 131\n",
-      "Episode 523, Failed! Reward 148\n",
-      "Episode 524, Failed! Reward 132\n",
-      "Episode 525, Failed! Reward 131\n"
+      "Episode 252, Failed! Reward -200\n",
+      "Episode 253, Failed! Reward -200\n",
+      "Episode 254, Failed! Reward -200\n",
+      "Episode 255, Failed! Reward -200\n",
+      "Episode 256, Failed! Reward -200\n",
+      "Episode 257, Failed! Reward -200\n",
+      "Episode 258, Failed! Reward -200\n",
+      "Episode 259, Failed! Reward -200\n",
+      "Episode 260, Failed! Reward -200\n",
+      "Episode 261, Failed! Reward -200\n",
+      "Episode 262, Failed! Reward -200\n",
+      "Episode 263, Failed! Reward -200\n",
+      "Episode 264, Failed! Reward -200\n",
+      "Episode 265, Failed! Reward -200\n",
+      "Episode 266, Failed! Reward -200\n",
+      "Episode 267, Failed! Reward -200\n",
+      "Episode 268, Failed! Reward -200\n",
+      "Episode 269, Failed! Reward -200\n",
+      "Episode 270, Failed! Reward -200\n",
+      "Episode 271, Failed! Reward -200\n",
+      "Episode 272, Failed! Reward -200\n",
+      "Episode 273, Failed! Reward -200\n",
+      "Episode 274, Failed! Reward -200\n",
+      "Episode 275, Failed! Reward -200\n",
+      "Episode 276, Failed! Reward -200\n",
+      "Episode 277, Failed! Reward -200\n",
+      "Episode 278, Failed! Reward -200\n",
+      "Episode 279, Failed! Reward -200\n",
+      "Episode 280, Failed! Reward -200\n",
+      "Episode 281, Failed! Reward -200\n",
+      "Episode 282, Failed! Reward -200\n",
+      "Episode 283, Failed! Reward -200\n",
+      "Episode 284, Failed! Reward -200\n",
+      "Episode 285, Failed! Reward -200\n",
+      "Episode 286, Failed! Reward -200\n",
+      "Episode 287, Failed! Reward -200\n",
+      "Episode 288, Failed! Reward -200\n",
+      "Episode 289, Failed! Reward -200\n",
+      "Episode 290, Failed! Reward -200\n",
+      "Episode 291, Failed! Reward -200\n",
+      "Episode 292, Failed! Reward -200\n",
+      "Episode 293, Failed! Reward -200\n",
+      "Episode 294, Failed! Reward -200\n",
+      "Episode 295, Failed! Reward -200\n",
+      "Episode 296, Failed! Reward -200\n",
+      "Episode 297, Failed! Reward -200\n",
+      "Episode 298, Failed! Reward -200\n",
+      "Episode 299, Failed! Reward -200\n",
+      "Episode 300, Failed! Reward -200\n",
+      "Episode 301, Failed! Reward -200\n",
+      "Episode 302, Failed! Reward -200\n",
+      "Episode 303, Failed! Reward -200\n",
+      "Episode 304, Failed! Reward -200\n",
+      "Episode 305, Failed! Reward -200\n",
+      "Episode 306, Failed! Reward -200\n",
+      "Episode 307, Failed! Reward -200\n",
+      "Episode 308, Failed! Reward -200\n",
+      "Episode 309, Failed! Reward -200\n",
+      "Episode 310, Failed! Reward -200\n",
+      "Episode 311, Failed! Reward -200\n",
+      "Episode 312, Failed! Reward -200\n",
+      "Episode 313, Failed! Reward -200\n",
+      "Episode 314, Failed! Reward -200\n",
+      "Episode 315, Failed! Reward -200\n",
+      "Episode 316, Failed! Reward -200\n",
+      "Episode 317, Failed! Reward -200\n",
+      "Episode 318, Failed! Reward -200\n",
+      "Episode 319, Failed! Reward -200\n",
+      "Episode 320, Failed! Reward -200\n",
+      "Episode 321, Failed! Reward -200\n",
+      "Episode 322, Failed! Reward -200\n",
+      "Episode 323, Failed! Reward -200\n",
+      "Episode 324, Failed! Reward -200\n",
+      "Episode 325, Failed! Reward -200\n",
+      "Episode 326, Failed! Reward -200\n",
+      "Episode 327, Failed! Reward -200\n",
+      "Episode 328, Failed! Reward -200\n",
+      "Episode 329, Failed! Reward -200\n",
+      "Episode 330, Failed! Reward -200\n",
+      "Episode 331, Failed! Reward -200\n",
+      "Episode 332, Failed! Reward -200\n",
+      "Episode 333, Failed! Reward -200\n",
+      "Episode 334, Failed! Reward -200\n",
+      "Episode 335, Failed! Reward -200\n",
+      "Episode 336, Failed! Reward -200\n",
+      "Episode 337, Failed! Reward -200\n",
+      "Episode 338, Failed! Reward -200\n",
+      "Episode 339, Failed! Reward -200\n",
+      "Episode 340, Failed! Reward -200\n",
+      "Episode 341, Failed! Reward -200\n",
+      "Episode 342, Failed! Reward -200\n",
+      "Episode 343, Failed! Reward -200\n",
+      "Episode 344, Failed! Reward -200\n",
+      "Episode 345, Failed! Reward -200\n",
+      "Episode 346, Failed! Reward -200\n",
+      "Episode 347, Failed! Reward -200\n",
+      "Episode 348, Failed! Reward -200\n",
+      "Episode 349, Failed! Reward -200\n",
+      "Episode 350, Failed! Reward -200\n",
+      "Episode 351, Failed! Reward -200\n",
+      "Episode 352, Failed! Reward -200\n",
+      "Episode 353, Failed! Reward -200\n",
+      "Episode 354, Failed! Reward -200\n",
+      "Episode 355, Failed! Reward -200\n",
+      "Episode 356, Failed! Reward -200\n",
+      "Episode 357, Failed! Reward -200\n",
+      "Episode 358, Failed! Reward -200\n",
+      "Episode 359, Failed! Reward -200\n",
+      "Episode 360, Failed! Reward -200\n",
+      "Episode 361, Failed! Reward -200\n",
+      "Episode 362, Failed! Reward -200\n",
+      "Episode 363, Failed! Reward -200\n",
+      "Episode 364, Failed! Reward -200\n",
+      "Episode 365, Failed! Reward -200\n",
+      "Episode 366, Failed! Reward -200\n",
+      "Episode 367, Failed! Reward -200\n",
+      "Episode 368, Better! Reward -122\n",
+      "Episode 369, Better! Reward -154\n",
+      "Episode 370, Better! Reward -168\n",
+      "Episode 371, Better! Reward -181\n",
+      "Episode 372, Better! Reward -180\n",
+      "Episode 373, Better! Reward -164\n",
+      "Episode 374, Failed! Reward -200\n",
+      "Episode 375, Failed! Reward -200\n",
+      "Episode 376, Failed! Reward -200\n",
+      "Episode 377, Failed! Reward -200\n",
+      "Episode 378, Failed! Reward -200\n",
+      "Episode 379, Better! Reward -165\n",
+      "Episode 380, Better! Reward -129\n",
+      "Episode 381, Better! Reward -188\n",
+      "Episode 382, Better! Reward -122\n",
+      "Episode 383, Better! Reward -139\n",
+      "Episode 384, Better! Reward -125\n",
+      "Episode 385, Better! Reward -131\n",
+      "Episode 386, Better! Reward -143\n",
+      "Episode 387, Better! Reward -125\n",
+      "Episode 388, Better! Reward -125\n",
+      "Episode 389, Failed! Reward -200\n",
+      "Episode 390, Failed! Reward -200\n",
+      "Episode 391, Better! Reward -131\n",
+      "Episode 392, Better! Reward -128\n",
+      "Episode 393, Better! Reward -129\n",
+      "Episode 394, Better! Reward -130\n",
+      "Episode 395, Better! Reward -125\n",
+      "Episode 396, Better! Reward -120\n",
+      "Episode 397, Failed! Reward -200\n",
+      "Episode 398, Failed! Reward -200\n",
+      "Episode 399, Failed! Reward -200\n",
+      "Episode 400, Failed! Reward -200\n",
+      "Episode 401, Failed! Reward -200\n",
+      "Episode 402, Failed! Reward -200\n",
+      "Episode 403, Failed! Reward -200\n",
+      "Episode 404, Failed! Reward -200\n",
+      "Episode 405, Failed! Reward -200\n",
+      "Episode 406, Failed! Reward -200\n",
+      "Episode 407, Failed! Reward -200\n",
+      "Episode 408, Failed! Reward -200\n",
+      "Episode 409, Failed! Reward -200\n",
+      "Episode 410, Failed! Reward -200\n",
+      "Episode 411, Failed! Reward -200\n",
+      "Episode 412, Failed! Reward -200\n",
+      "Episode 413, Failed! Reward -200\n",
+      "Episode 414, Failed! Reward -200\n",
+      "Episode 415, Failed! Reward -200\n",
+      "Episode 416, Failed! Reward -200\n",
+      "Episode 417, Failed! Reward -200\n",
+      "Episode 418, Failed! Reward -200\n",
+      "Episode 419, Failed! Reward -200\n",
+      "Episode 420, Failed! Reward -200\n",
+      "Episode 421, Failed! Reward -200\n",
+      "Episode 422, Failed! Reward -200\n",
+      "Episode 423, Failed! Reward -200\n",
+      "Episode 424, Failed! Reward -200\n",
+      "Episode 425, Failed! Reward -200\n",
+      "Episode 426, Failed! Reward -200\n",
+      "Episode 427, Failed! Reward -200\n",
+      "Episode 428, Failed! Reward -200\n",
+      "Episode 429, Failed! Reward -200\n",
+      "Episode 430, Failed! Reward -200\n",
+      "Episode 431, Failed! Reward -200\n",
+      "Episode 432, Failed! Reward -200\n",
+      "Episode 433, Failed! Reward -200\n",
+      "Episode 434, Failed! Reward -200\n",
+      "Episode 435, Failed! Reward -200\n",
+      "Episode 436, Failed! Reward -200\n",
+      "Episode 437, Failed! Reward -200\n",
+      "Episode 438, Failed! Reward -200\n",
+      "Episode 439, Failed! Reward -200\n",
+      "Episode 440, Failed! Reward -200\n",
+      "Episode 441, Failed! Reward -200\n",
+      "Episode 442, Failed! Reward -200\n",
+      "Episode 443, Failed! Reward -200\n",
+      "Episode 444, Failed! Reward -200\n",
+      "Episode 445, Failed! Reward -200\n",
+      "Episode 446, Failed! Reward -200\n",
+      "Episode 447, Failed! Reward -200\n",
+      "Episode 448, Failed! Reward -200\n",
+      "Episode 449, Failed! Reward -200\n",
+      "Episode 450, Failed! Reward -200\n",
+      "Episode 451, Failed! Reward -200\n",
+      "Episode 452, Failed! Reward -200\n",
+      "Episode 453, Failed! Reward -200\n",
+      "Episode 454, Failed! Reward -200\n",
+      "Episode 455, Failed! Reward -200\n",
+      "Episode 456, Failed! Reward -200\n",
+      "Episode 457, Failed! Reward -200\n",
+      "Episode 458, Failed! Reward -200\n",
+      "Episode 459, Failed! Reward -200\n",
+      "Episode 460, Failed! Reward -200\n",
+      "Episode 461, Failed! Reward -200\n",
+      "Episode 462, Failed! Reward -200\n",
+      "Episode 463, Failed! Reward -200\n",
+      "Episode 464, Failed! Reward -200\n",
+      "Episode 465, Failed! Reward -200\n",
+      "Episode 466, Failed! Reward -200\n",
+      "Episode 467, Failed! Reward -200\n",
+      "Episode 468, Failed! Reward -200\n",
+      "Episode 469, Failed! Reward -200\n",
+      "Episode 470, Failed! Reward -200\n",
+      "Episode 471, Failed! Reward -200\n",
+      "Episode 472, Failed! Reward -200\n",
+      "Episode 473, Failed! Reward -200\n",
+      "Episode 474, Failed! Reward -200\n",
+      "Episode 475, Failed! Reward -200\n",
+      "Episode 476, Failed! Reward -200\n",
+      "Episode 477, Failed! Reward -200\n",
+      "Episode 478, Failed! Reward -200\n",
+      "Episode 479, Failed! Reward -200\n",
+      "Episode 480, Failed! Reward -200\n",
+      "Episode 481, Failed! Reward -200\n",
+      "Episode 482, Failed! Reward -200\n",
+      "Episode 483, Failed! Reward -200\n",
+      "Episode 484, Failed! Reward -200\n",
+      "Episode 485, Failed! Reward -200\n",
+      "Episode 486, Failed! Reward -200\n",
+      "Episode 487, Failed! Reward -200\n",
+      "Episode 488, Failed! Reward -200\n",
+      "Episode 489, Failed! Reward -200\n",
+      "Episode 490, Failed! Reward -200\n",
+      "Episode 491, Failed! Reward -200\n",
+      "Episode 492, Failed! Reward -200\n",
+      "Episode 493, Failed! Reward -200\n",
+      "Episode 494, Failed! Reward -200\n",
+      "Episode 495, Failed! Reward -200\n",
+      "Episode 496, Failed! Reward -200\n",
+      "Episode 497, Failed! Reward -200\n",
+      "Episode 498, Failed! Reward -200\n",
+      "Episode 499, Failed! Reward -200\n",
+      "Episode 500, Failed! Reward -200\n"
      ]
     },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Episode 526, Failed! Reward 129\n",
-      "Episode 527, Failed! Reward 126\n",
-      "Episode 528, Failed! Reward 132\n",
-      "Episode 529, Failed! Reward 131\n",
-      "Episode 530, Failed! Reward 144\n",
-      "Episode 531, Failed! Reward 145\n",
-      "Episode 532, Failed! Reward 133\n",
-      "Episode 533, Failed! Reward 139\n",
-      "Episode 534, Failed! Reward 143\n",
-      "Episode 535, Failed! Reward 139\n",
-      "Episode 536, Failed! Reward 146\n",
-      "Episode 537, Failed! Reward 150\n",
-      "Episode 538, Failed! Reward 137\n",
-      "Episode 539, Failed! Reward 149\n",
-      "Episode 540, Failed! Reward 163\n",
-      "Episode 541, Failed! Reward 150\n",
-      "Episode 542, Failed! Reward 158\n",
-      "Episode 543, Failed! Reward 153\n",
-      "Episode 544, Failed! Reward 141\n",
-      "Episode 545, Failed! Reward 160\n",
-      "Episode 546, Failed! Reward 149\n",
-      "Episode 547, Failed! Reward 163\n",
-      "Episode 548, Failed! Reward 140\n",
-      "Episode 549, Failed! Reward 149\n",
-      "Episode 550, Failed! Reward 157\n",
-      "Episode 551, Failed! Reward 148\n",
-      "Episode 552, Failed! Reward 145\n",
-      "Episode 553, Failed! Reward 132\n",
-      "Episode 554, Failed! Reward 135\n",
-      "Episode 555, Failed! Reward 132\n",
-      "Episode 556, Failed! Reward 136\n",
-      "Episode 557, Failed! Reward 149\n",
-      "Episode 558, Failed! Reward 141\n",
-      "Episode 559, Failed! Reward 156\n",
-      "Episode 560, Failed! Reward 151\n",
-      "Episode 561, Failed! Reward 145\n",
-      "Episode 562, Failed! Reward 146\n",
-      "Episode 563, Failed! Reward 152\n",
-      "Episode 564, Failed! Reward 161\n",
-      "Episode 565, Failed! Reward 144\n",
-      "Episode 566, Failed! Reward 164\n",
-      "Episode 567, Failed! Reward 142\n",
-      "Episode 568, Failed! Reward 146\n",
-      "Episode 569, Failed! Reward 148\n",
-      "Episode 570, Failed! Reward 148\n",
-      "Episode 571, Failed! Reward 150\n",
-      "Episode 572, Failed! Reward 143\n",
-      "Episode 573, Failed! Reward 152\n",
-      "Episode 574, Failed! Reward 152\n",
-      "Episode 575, Failed! Reward 149\n",
-      "Episode 576, Failed! Reward 145\n",
-      "Episode 577, Failed! Reward 139\n",
-      "Episode 578, Failed! Reward 135\n",
-      "Episode 579, Failed! Reward 138\n",
-      "Episode 580, Failed! Reward 136\n",
-      "Episode 581, Failed! Reward 164\n",
-      "Episode 582, Failed! Reward 160\n",
-      "Episode 583, Failed! Reward 154\n",
-      "Episode 584, Failed! Reward 163\n",
-      "Episode 585, Failed! Reward 149\n",
-      "Episode 586, Failed! Reward 150\n",
-      "Episode 587, Failed! Reward 154\n",
-      "Episode 588, Failed! Reward 157\n",
-      "Episode 589, Failed! Reward 142\n",
-      "Episode 590, Failed! Reward 159\n",
-      "Episode 591, Failed! Reward 143\n",
-      "Episode 592, Failed! Reward 163\n",
-      "Episode 593, Failed! Reward 145\n",
-      "Episode 594, Failed! Reward 152\n",
-      "Episode 595, Failed! Reward 187\n",
-      "Episode 596, Failed! Reward 174\n",
-      "Episode 597, Failed! Reward 159\n",
-      "Episode 598, Failed! Reward 161\n",
-      "Episode 599, Failed! Reward 159\n",
-      "Episode 600, Failed! Reward 140\n",
-      "Episode 601, Failed! Reward 174\n",
-      "Episode 602, Failed! Reward 170\n",
-      "Episode 603, Passed! Reward 200\n",
-      "Episode 604, Passed! Reward 199\n",
-      "Episode 605, Failed! Reward 180\n",
-      "Episode 606, Failed! Reward 155\n",
-      "Episode 607, Failed! Reward 181\n",
-      "Episode 608, Failed! Reward 184\n",
-      "Episode 609, Failed! Reward 178\n",
-      "Episode 610, Failed! Reward 162\n",
-      "Episode 611, Failed! Reward 169\n",
-      "Episode 612, Failed! Reward 168\n",
-      "Episode 613, Failed! Reward 156\n",
-      "Episode 614, Failed! Reward 146\n",
-      "Episode 615, Failed! Reward 150\n",
-      "Episode 616, Failed! Reward 163\n",
-      "Episode 617, Failed! Reward 163\n",
-      "Episode 618, Failed! Reward 172\n",
-      "Episode 619, Failed! Reward 160\n",
-      "Episode 620, Failed! Reward 166\n",
-      "Episode 621, Failed! Reward 153\n",
-      "Episode 622, Failed! Reward 163\n",
-      "Episode 623, Failed! Reward 149\n",
-      "Episode 624, Failed! Reward 159\n",
-      "Episode 625, Failed! Reward 162\n",
-      "Episode 626, Failed! Reward 150\n",
-      "Episode 627, Failed! Reward 155\n",
-      "Episode 628, Failed! Reward 173\n",
-      "Episode 629, Failed! Reward 162\n",
-      "Episode 630, Failed! Reward 157\n",
-      "Episode 631, Failed! Reward 166\n",
-      "Episode 632, Failed! Reward 150\n",
-      "Episode 633, Failed! Reward 144\n",
-      "Episode 634, Failed! Reward 161\n",
-      "Episode 635, Failed! Reward 166\n",
-      "Episode 636, Failed! Reward 155\n",
-      "Episode 637, Failed! Reward 155\n",
-      "Episode 638, Failed! Reward 155\n",
-      "Episode 639, Failed! Reward 151\n",
-      "Episode 640, Failed! Reward 170\n",
-      "Episode 641, Failed! Reward 164\n",
-      "Episode 642, Failed! Reward 162\n",
-      "Episode 643, Failed! Reward 142\n",
-      "Episode 644, Failed! Reward 155\n",
-      "Episode 645, Failed! Reward 149\n",
-      "Episode 646, Failed! Reward 147\n",
-      "Episode 647, Failed! Reward 175\n",
-      "Episode 648, Failed! Reward 158\n",
-      "Episode 649, Failed! Reward 143\n",
-      "Episode 650, Failed! Reward 151\n",
-      "Episode 651, Failed! Reward 166\n",
-      "Episode 652, Failed! Reward 171\n",
-      "Episode 653, Failed! Reward 171\n",
-      "Episode 654, Failed! Reward 178\n",
-      "Episode 655, Failed! Reward 174\n",
-      "Episode 656, Failed! Reward 167\n",
-      "Episode 657, Failed! Reward 147\n",
-      "Episode 658, Failed! Reward 145\n",
-      "Episode 659, Failed! Reward 146\n",
-      "Episode 660, Failed! Reward 155\n",
-      "Episode 661, Failed! Reward 153\n",
-      "Episode 662, Failed! Reward 182\n",
-      "Episode 663, Failed! Reward 177\n",
-      "Episode 664, Failed! Reward 158\n",
-      "Episode 665, Failed! Reward 156\n",
-      "Episode 666, Failed! Reward 154\n",
-      "Episode 667, Failed! Reward 157\n",
-      "Episode 668, Failed! Reward 158\n",
-      "Episode 669, Failed! Reward 151\n",
-      "Episode 670, Failed! Reward 162\n",
-      "Episode 671, Failed! Reward 180\n",
-      "Episode 672, Failed! Reward 153\n",
-      "Episode 673, Failed! Reward 159\n",
-      "Episode 674, Failed! Reward 164\n",
-      "Episode 675, Failed! Reward 157\n",
-      "Episode 676, Failed! Reward 177\n",
-      "Episode 677, Failed! Reward 160\n",
-      "Episode 678, Failed! Reward 156\n",
-      "Episode 679, Failed! Reward 154\n",
-      "Episode 680, Failed! Reward 148\n",
-      "Episode 681, Failed! Reward 153\n",
-      "Episode 682, Failed! Reward 149\n",
-      "Episode 683, Failed! Reward 158\n",
-      "Episode 684, Failed! Reward 145\n",
-      "Episode 685, Failed! Reward 150\n",
-      "Episode 686, Failed! Reward 145\n",
-      "Episode 687, Failed! Reward 152\n",
-      "Episode 688, Failed! Reward 153\n",
-      "Episode 689, Failed! Reward 173\n",
-      "Episode 690, Failed! Reward 172\n",
-      "Episode 691, Failed! Reward 161\n",
-      "Episode 692, Failed! Reward 164\n",
-      "Episode 693, Failed! Reward 156\n",
-      "Episode 694, Failed! Reward 159\n",
-      "Episode 695, Failed! Reward 147\n",
-      "Episode 696, Failed! Reward 145\n",
-      "Episode 697, Failed! Reward 159\n",
-      "Episode 698, Failed! Reward 166\n",
-      "Episode 699, Failed! Reward 168\n",
-      "Episode 700, Failed! Reward 154\n",
-      "Episode 701, Failed! Reward 160\n",
-      "Episode 702, Failed! Reward 138\n",
-      "Episode 703, Failed! Reward 137\n",
-      "Episode 704, Failed! Reward 177\n",
-      "Episode 705, Failed! Reward 150\n",
-      "Episode 706, Failed! Reward 143\n",
-      "Episode 707, Failed! Reward 160\n",
-      "Episode 708, Passed! Reward 200\n",
-      "Episode 709, Failed! Reward 169\n",
-      "Episode 710, Failed! Reward 178\n",
-      "Episode 711, Failed! Reward 150\n",
-      "Episode 712, Failed! Reward 143\n",
-      "Episode 713, Failed! Reward 147\n",
-      "Episode 714, Failed! Reward 113\n",
-      "Episode 715, Failed! Reward 135\n",
-      "Episode 716, Failed! Reward 141\n",
-      "Episode 717, Failed! Reward 171\n",
-      "Episode 718, Passed! Reward 200\n",
-      "Episode 719, Failed! Reward 167\n",
-      "Episode 720, Failed! Reward 185\n",
-      "Episode 721, Failed! Reward 151\n",
-      "Episode 722, Failed! Reward 149\n",
-      "Episode 723, Failed! Reward 155\n",
-      "Episode 724, Failed! Reward 166\n",
-      "Episode 725, Failed! Reward 161\n",
-      "Episode 726, Failed! Reward 169\n",
-      "Episode 727, Failed! Reward 153\n",
-      "Episode 728, Failed! Reward 157\n",
-      "Episode 729, Failed! Reward 149\n",
-      "Episode 730, Failed! Reward 154\n",
-      "Episode 731, Failed! Reward 139\n",
-      "Episode 732, Failed! Reward 130\n",
-      "Episode 733, Failed! Reward 157\n",
-      "Episode 734, Failed! Reward 147\n",
-      "Episode 735, Failed! Reward 137\n",
-      "Episode 736, Failed! Reward 139\n",
-      "Episode 737, Failed! Reward 152\n",
-      "Episode 738, Failed! Reward 149\n",
-      "Episode 739, Failed! Reward 162\n",
-      "Episode 740, Failed! Reward 153\n",
-      "Episode 741, Failed! Reward 140\n",
-      "Episode 742, Failed! Reward 149\n",
-      "Episode 743, Failed! Reward 155\n",
-      "Episode 744, Failed! Reward 167\n",
-      "Episode 745, Failed! Reward 155\n",
-      "Episode 746, Failed! Reward 149\n",
-      "Episode 747, Failed! Reward 138\n",
-      "Episode 748, Failed! Reward 132\n",
-      "Episode 749, Failed! Reward 158\n",
-      "Episode 750, Failed! Reward 143\n",
-      "Episode 751, Failed! Reward 148\n",
-      "Episode 752, Failed! Reward 129\n",
-      "Episode 753, Failed! Reward 140\n",
-      "Episode 754, Failed! Reward 148\n",
-      "Episode 755, Failed! Reward 135\n",
-      "Episode 756, Failed! Reward 138\n",
-      "Episode 757, Failed! Reward 154\n",
-      "Episode 758, Failed! Reward 125\n",
-      "Episode 759, Failed! Reward 131\n",
-      "Episode 760, Failed! Reward 147\n",
-      "Episode 761, Failed! Reward 137\n",
-      "Episode 762, Failed! Reward 132\n",
-      "Episode 763, Failed! Reward 144\n",
-      "Episode 764, Failed! Reward 139\n",
-      "Episode 765, Failed! Reward 145\n",
-      "Episode 766, Failed! Reward 127\n",
-      "Episode 767, Failed! Reward 123\n",
-      "Episode 768, Failed! Reward 131\n",
-      "Episode 769, Failed! Reward 124\n",
-      "Episode 770, Failed! Reward 148\n",
-      "Episode 771, Failed! Reward 137\n",
-      "Episode 772, Failed! Reward 141\n",
-      "Episode 773, Failed! Reward 135\n",
-      "Episode 774, Failed! Reward 127\n",
-      "Episode 775, Failed! Reward 146\n",
-      "Episode 776, Failed! Reward 136\n",
-      "Episode 777, Failed! Reward 151\n",
-      "Episode 778, Failed! Reward 117\n",
-      "Episode 779, Failed! Reward 136\n",
-      "Episode 780, Failed! Reward 130\n",
-      "Episode 781, Failed! Reward 123\n",
-      "Episode 782, Failed! Reward 119\n"
+      "Episode 501, Failed! Reward -200\n",
+      "Episode 502, Failed! Reward -200\n",
+      "Episode 503, Failed! Reward -200\n",
+      "Episode 504, Failed! Reward -200\n",
+      "Episode 505, Failed! Reward -200\n",
+      "Episode 506, Better! Reward -195\n",
+      "Episode 507, Better! Reward -178\n",
+      "Episode 508, Failed! Reward -200\n",
+      "Episode 509, Better! Reward -164\n",
+      "Episode 510, Failed! Reward -200\n",
+      "Episode 511, Failed! Reward -200\n",
+      "Episode 512, Better! Reward -174\n",
+      "Episode 513, Failed! Reward -200\n",
+      "Episode 514, Failed! Reward -200\n",
+      "Episode 515, Failed! Reward -200\n",
+      "Episode 516, Failed! Reward -200\n",
+      "Episode 517, Failed! Reward -200\n",
+      "Episode 518, Failed! Reward -200\n",
+      "Episode 519, Failed! Reward -200\n",
+      "Episode 520, Failed! Reward -200\n",
+      "Episode 521, Better! Reward -175\n",
+      "Episode 522, Better! Reward -173\n",
+      "Episode 523, Failed! Reward -200\n",
+      "Episode 524, Better! Reward -173\n",
+      "Episode 525, Better! Reward -170\n",
+      "Episode 526, Failed! Reward -200\n",
+      "Episode 527, Failed! Reward -200\n",
+      "Episode 528, Failed! Reward -200\n",
+      "Episode 529, Better! Reward -162\n",
+      "Episode 530, Failed! Reward -200\n",
+      "Episode 531, Failed! Reward -200\n",
+      "Episode 532, Failed! Reward -200\n",
+      "Episode 533, Failed! Reward -200\n",
+      "Episode 534, Failed! Reward -200\n",
+      "Episode 535, Failed! Reward -200\n",
+      "Episode 536, Failed! Reward -200\n",
+      "Episode 537, Failed! Reward -200\n",
+      "Episode 538, Failed! Reward -200\n",
+      "Episode 539, Failed! Reward -200\n",
+      "Episode 540, Failed! Reward -200\n",
+      "Episode 541, Failed! Reward -200\n",
+      "Episode 542, Failed! Reward -200\n",
+      "Episode 543, Failed! Reward -200\n",
+      "Episode 544, Failed! Reward -200\n",
+      "Episode 545, Failed! Reward -200\n",
+      "Episode 546, Failed! Reward -200\n",
+      "Episode 547, Failed! Reward -200\n",
+      "Episode 548, Failed! Reward -200\n",
+      "Episode 549, Failed! Reward -200\n",
+      "Episode 550, Failed! Reward -200\n",
+      "Episode 551, Failed! Reward -200\n",
+      "Episode 552, Failed! Reward -200\n",
+      "Episode 553, Failed! Reward -200\n",
+      "Episode 554, Failed! Reward -200\n",
+      "Episode 555, Failed! Reward -200\n",
+      "Episode 556, Failed! Reward -200\n",
+      "Episode 557, Failed! Reward -200\n",
+      "Episode 558, Failed! Reward -200\n",
+      "Episode 559, Failed! Reward -200\n",
+      "Episode 560, Failed! Reward -200\n",
+      "Episode 561, Failed! Reward -200\n",
+      "Episode 562, Failed! Reward -200\n",
+      "Episode 563, Failed! Reward -200\n",
+      "Episode 564, Failed! Reward -200\n",
+      "Episode 565, Failed! Reward -200\n",
+      "Episode 566, Failed! Reward -200\n",
+      "Episode 567, Failed! Reward -200\n",
+      "Episode 568, Failed! Reward -200\n",
+      "Episode 569, Failed! Reward -200\n",
+      "Episode 570, Failed! Reward -200\n",
+      "Episode 571, Failed! Reward -200\n",
+      "Episode 572, Failed! Reward -200\n",
+      "Episode 573, Failed! Reward -200\n",
+      "Episode 574, Failed! Reward -200\n",
+      "Episode 575, Failed! Reward -200\n",
+      "Episode 576, Failed! Reward -200\n",
+      "Episode 577, Failed! Reward -200\n",
+      "Episode 578, Failed! Reward -200\n",
+      "Episode 579, Failed! Reward -200\n",
+      "Episode 580, Failed! Reward -200\n",
+      "Episode 581, Failed! Reward -200\n",
+      "Episode 582, Failed! Reward -200\n",
+      "Episode 583, Failed! Reward -200\n",
+      "Episode 584, Failed! Reward -200\n",
+      "Episode 585, Failed! Reward -200\n",
+      "Episode 586, Failed! Reward -200\n",
+      "Episode 587, Failed! Reward -200\n",
+      "Episode 588, Failed! Reward -200\n",
+      "Episode 589, Failed! Reward -200\n",
+      "Episode 590, Failed! Reward -200\n",
+      "Episode 591, Failed! Reward -200\n",
+      "Episode 592, Failed! Reward -200\n",
+      "Episode 593, Failed! Reward -200\n",
+      "Episode 594, Failed! Reward -200\n",
+      "Episode 595, Failed! Reward -200\n",
+      "Episode 596, Failed! Reward -200\n",
+      "Episode 597, Failed! Reward -200\n",
+      "Episode 598, Failed! Reward -200\n",
+      "Episode 599, Failed! Reward -200\n",
+      "Episode 600, Failed! Reward -200\n",
+      "Episode 601, Failed! Reward -200\n",
+      "Episode 602, Failed! Reward -200\n",
+      "Episode 603, Failed! Reward -200\n",
+      "Episode 604, Failed! Reward -200\n",
+      "Episode 605, Failed! Reward -200\n",
+      "Episode 606, Failed! Reward -200\n",
+      "Episode 607, Failed! Reward -200\n",
+      "Episode 608, Failed! Reward -200\n",
+      "Episode 609, Failed! Reward -200\n",
+      "Episode 610, Failed! Reward -200\n",
+      "Episode 611, Failed! Reward -200\n",
+      "Episode 612, Failed! Reward -200\n",
+      "Episode 613, Failed! Reward -200\n",
+      "Episode 614, Failed! Reward -200\n",
+      "Episode 615, Failed! Reward -200\n",
+      "Episode 616, Failed! Reward -200\n",
+      "Episode 617, Failed! Reward -200\n",
+      "Episode 618, Failed! Reward -200\n",
+      "Episode 619, Failed! Reward -200\n",
+      "Episode 620, Failed! Reward -200\n",
+      "Episode 621, Failed! Reward -200\n",
+      "Episode 622, Failed! Reward -200\n",
+      "Episode 623, Failed! Reward -200\n",
+      "Episode 624, Failed! Reward -200\n",
+      "Episode 625, Failed! Reward -200\n",
+      "Episode 626, Failed! Reward -200\n",
+      "Episode 627, Failed! Reward -200\n",
+      "Episode 628, Failed! Reward -200\n",
+      "Episode 629, Failed! Reward -200\n",
+      "Episode 630, Failed! Reward -200\n",
+      "Episode 631, Failed! Reward -200\n",
+      "Episode 632, Failed! Reward -200\n",
+      "Episode 633, Failed! Reward -200\n",
+      "Episode 634, Failed! Reward -200\n",
+      "Episode 635, Failed! Reward -200\n",
+      "Episode 636, Failed! Reward -200\n",
+      "Episode 637, Failed! Reward -200\n",
+      "Episode 638, Failed! Reward -200\n",
+      "Episode 639, Failed! Reward -200\n",
+      "Episode 640, Failed! Reward -200\n",
+      "Episode 641, Failed! Reward -200\n",
+      "Episode 642, Failed! Reward -200\n",
+      "Episode 643, Failed! Reward -200\n",
+      "Episode 644, Failed! Reward -200\n",
+      "Episode 645, Failed! Reward -200\n",
+      "Episode 646, Failed! Reward -200\n",
+      "Episode 647, Failed! Reward -200\n",
+      "Episode 648, Better! Reward -131\n",
+      "Episode 649, Failed! Reward -200\n",
+      "Episode 650, Failed! Reward -200\n",
+      "Episode 651, Failed! Reward -200\n",
+      "Episode 652, Better! Reward -121\n",
+      "Episode 653, Better! Reward -123\n",
+      "Episode 654, Better! Reward -129\n",
+      "Episode 655, Better! Reward -138\n",
+      "Episode 656, Failed! Reward -200\n",
+      "Episode 657, Better! Reward -132\n",
+      "Episode 658, Failed! Reward -200\n",
+      "Episode 659, Failed! Reward -200\n",
+      "Episode 660, Better! Reward -116\n",
+      "Episode 661, Failed! Reward -200\n",
+      "Episode 662, Failed! Reward -200\n",
+      "Episode 663, Failed! Reward -200\n",
+      "Episode 664, Failed! Reward -200\n",
+      "Episode 665, Failed! Reward -200\n",
+      "Episode 666, Failed! Reward -200\n",
+      "Episode 667, Failed! Reward -200\n",
+      "Episode 668, Failed! Reward -200\n",
+      "Episode 669, Better! Reward -117\n",
+      "Episode 670, Better! Reward -127\n",
+      "Episode 671, Failed! Reward -200\n",
+      "Episode 672, Failed! Reward -200\n",
+      "Episode 673, Failed! Reward -200\n",
+      "Episode 674, Failed! Reward -200\n",
+      "Episode 675, Failed! Reward -200\n",
+      "Episode 676, Failed! Reward -200\n",
+      "Episode 677, Failed! Reward -200\n",
+      "Episode 678, Better! Reward -127\n",
+      "Episode 679, Better! Reward -134\n",
+      "Episode 680, Failed! Reward -200\n",
+      "Episode 681, Failed! Reward -200\n",
+      "Episode 682, Failed! Reward -200\n",
+      "Episode 683, Better! Reward -130\n",
+      "Episode 684, Better! Reward -122\n",
+      "Episode 685, Better! Reward -165\n",
+      "Episode 686, Better! Reward -161\n",
+      "Episode 687, Better! Reward -129\n",
+      "Episode 688, Better! Reward -131\n",
+      "Episode 689, Better! Reward -131\n",
+      "Episode 690, Failed! Reward -200\n",
+      "Episode 691, Better! Reward -129\n",
+      "Episode 692, Failed! Reward -200\n",
+      "Episode 693, Better! Reward -124\n",
+      "Episode 694, Better! Reward -157\n",
+      "Episode 695, Better! Reward -158\n",
+      "Episode 696, Better! Reward -176\n",
+      "Episode 697, Better! Reward -156\n",
+      "Episode 698, Better! Reward -115\n",
+      "Episode 699, Better! Reward -122\n",
+      "Episode 700, Better! Reward -127\n",
+      "Episode 701, Better! Reward -115\n",
+      "Episode 702, Failed! Reward -200\n",
+      "Episode 703, Better! Reward -154\n",
+      "Episode 704, Better! Reward -124\n",
+      "Episode 705, Better! Reward -127\n",
+      "Episode 706, Better! Reward -123\n",
+      "Episode 707, Better! Reward -124\n",
+      "Episode 708, Better! Reward -129\n",
+      "Episode 709, Better! Reward -143\n",
+      "Episode 710, Better! Reward -132\n",
+      "Episode 711, Failed! Reward -200\n",
+      "Episode 712, Failed! Reward -200\n",
+      "Episode 713, Better! Reward -129\n",
+      "Episode 714, Failed! Reward -200\n",
+      "Episode 715, Failed! Reward -200\n",
+      "Episode 716, Better! Reward -123\n",
+      "Episode 717, Failed! Reward -200\n",
+      "Episode 718, Failed! Reward -200\n",
+      "Episode 719, Failed! Reward -200\n",
+      "Episode 720, Failed! Reward -200\n",
+      "Episode 721, Failed! Reward -200\n",
+      "Episode 722, Failed! Reward -200\n",
+      "Episode 723, Failed! Reward -200\n",
+      "Episode 724, Failed! Reward -200\n",
+      "Episode 725, Failed! Reward -200\n",
+      "Episode 726, Better! Reward -183\n",
+      "Episode 727, Failed! Reward -200\n",
+      "Episode 728, Failed! Reward -200\n",
+      "Episode 729, Failed! Reward -200\n",
+      "Episode 730, Failed! Reward -200\n",
+      "Episode 731, Better! Reward -174\n",
+      "Episode 732, Better! Reward -177\n",
+      "Episode 733, Failed! Reward -200\n",
+      "Episode 734, Failed! Reward -200\n",
+      "Episode 735, Failed! Reward -200\n",
+      "Episode 736, Failed! Reward -200\n",
+      "Episode 737, Failed! Reward -200\n",
+      "Episode 738, Failed! Reward -200\n",
+      "Episode 739, Failed! Reward -200\n",
+      "Episode 740, Failed! Reward -200\n",
+      "Episode 741, Failed! Reward -200\n",
+      "Episode 742, Failed! Reward -200\n",
+      "Episode 743, Failed! Reward -200\n",
+      "Episode 744, Failed! Reward -200\n",
+      "Episode 745, Failed! Reward -200\n",
+      "Episode 746, Failed! Reward -200\n",
+      "Episode 747, Failed! Reward -200\n",
+      "Episode 748, Failed! Reward -200\n",
+      "Episode 749, Failed! Reward -200\n"
      ]
     },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Episode 783, Failed! Reward 126\n",
-      "Episode 784, Failed! Reward 154\n",
-      "Episode 785, Failed! Reward 183\n",
-      "Episode 786, Failed! Reward 125\n",
-      "Episode 787, Failed! Reward 118\n",
-      "Episode 788, Failed! Reward 143\n",
-      "Episode 789, Failed! Reward 113\n",
-      "Episode 790, Failed! Reward 121\n",
-      "Episode 791, Failed! Reward 136\n",
-      "Episode 792, Failed! Reward 140\n",
-      "Episode 793, Failed! Reward 157\n",
-      "Episode 794, Failed! Reward 136\n",
-      "Episode 795, Failed! Reward 126\n",
-      "Episode 796, Failed! Reward 134\n",
-      "Episode 797, Failed! Reward 149\n",
-      "Episode 798, Failed! Reward 164\n",
-      "Episode 799, Failed! Reward 176\n",
-      "Episode 800, Failed! Reward 168\n",
-      "Episode 801, Failed! Reward 156\n",
-      "Episode 802, Failed! Reward 163\n",
-      "Episode 803, Failed! Reward 162\n",
-      "Episode 804, Failed! Reward 172\n",
-      "Episode 805, Failed! Reward 171\n",
-      "Episode 806, Failed! Reward 154\n",
-      "Episode 807, Failed! Reward 161\n",
-      "Episode 808, Failed! Reward 149\n",
-      "Episode 809, Failed! Reward 155\n",
-      "Episode 810, Failed! Reward 137\n",
-      "Episode 811, Failed! Reward 153\n",
-      "Episode 812, Failed! Reward 169\n",
-      "Episode 813, Failed! Reward 147\n",
-      "Episode 814, Failed! Reward 146\n",
-      "Episode 815, Failed! Reward 138\n",
-      "Episode 816, Failed! Reward 159\n",
-      "Episode 817, Failed! Reward 146\n",
-      "Episode 818, Failed! Reward 151\n",
-      "Episode 819, Failed! Reward 145\n",
-      "Episode 820, Failed! Reward 141\n",
-      "Episode 821, Failed! Reward 144\n",
-      "Episode 822, Failed! Reward 160\n",
-      "Episode 823, Failed! Reward 177\n",
-      "Episode 824, Failed! Reward 148\n",
-      "Episode 825, Failed! Reward 143\n",
-      "Episode 826, Failed! Reward 137\n",
-      "Episode 827, Failed! Reward 156\n",
-      "Episode 828, Failed! Reward 168\n",
-      "Episode 829, Failed! Reward 148\n",
-      "Episode 830, Failed! Reward 156\n",
-      "Episode 831, Failed! Reward 149\n",
-      "Episode 832, Failed! Reward 138\n",
-      "Episode 833, Failed! Reward 151\n",
-      "Episode 834, Failed! Reward 158\n",
-      "Episode 835, Failed! Reward 163\n",
-      "Episode 836, Failed! Reward 151\n",
-      "Episode 837, Failed! Reward 173\n",
-      "Episode 838, Failed! Reward 179\n",
-      "Episode 839, Failed! Reward 150\n",
-      "Episode 840, Failed! Reward 162\n",
-      "Episode 841, Failed! Reward 171\n",
-      "Episode 842, Failed! Reward 159\n",
-      "Episode 843, Failed! Reward 165\n",
-      "Episode 844, Failed! Reward 149\n",
-      "Episode 845, Passed! Reward 196\n",
-      "Episode 846, Failed! Reward 179\n",
-      "Episode 847, Failed! Reward 149\n",
-      "Episode 848, Failed! Reward 171\n",
-      "Episode 849, Failed! Reward 162\n",
-      "Episode 850, Failed! Reward 170\n",
-      "Episode 851, Passed! Reward 200\n",
-      "Episode 852, Failed! Reward 192\n",
-      "Episode 853, Failed! Reward 185\n",
-      "Episode 854, Passed! Reward 198\n",
-      "Episode 855, Failed! Reward 164\n",
-      "Episode 856, Failed! Reward 156\n",
-      "Episode 857, Passed! Reward 197\n",
-      "Episode 858, Passed! Reward 200\n",
-      "Episode 859, Failed! Reward 177\n",
-      "Episode 860, Passed! Reward 200\n",
-      "Episode 861, Passed! Reward 200\n",
-      "Episode 862, Passed! Reward 200\n",
-      "Episode 863, Passed! Reward 200\n",
-      "Episode 864, Passed! Reward 200\n",
-      "Episode 865, Passed! Reward 200\n",
-      "Episode 866, Passed! Reward 200\n",
-      "Episode 867, Passed! Reward 200\n",
-      "Episode 868, Passed! Reward 200\n",
-      "Episode 869, Passed! Reward 200\n",
-      "Episode 870, Passed! Reward 200\n",
-      "Episode 871, Passed! Reward 200\n",
-      "Episode 872, Passed! Reward 200\n",
-      "Episode 873, Passed! Reward 200\n",
-      "Episode 874, Passed! Reward 200\n",
-      "Episode 875, Passed! Reward 200\n",
-      "Episode 876, Passed! Reward 200\n",
-      "Episode 877, Passed! Reward 200\n",
-      "Episode 878, Passed! Reward 200\n",
-      "Episode 879, Passed! Reward 200\n",
-      "Episode 880, Passed! Reward 200\n",
-      "Episode 881, Passed! Reward 200\n",
-      "Episode 882, Passed! Reward 200\n",
-      "Episode 883, Passed! Reward 200\n",
-      "Episode 884, Passed! Reward 200\n",
-      "Episode 885, Passed! Reward 200\n",
-      "Episode 886, Passed! Reward 200\n",
-      "Episode 887, Passed! Reward 200\n",
-      "Episode 888, Passed! Reward 200\n",
-      "Episode 889, Passed! Reward 200\n",
-      "Episode 890, Passed! Reward 200\n",
-      "Episode 891, Passed! Reward 200\n",
-      "Episode 892, Passed! Reward 200\n",
-      "Episode 893, Passed! Reward 200\n",
-      "Episode 894, Passed! Reward 200\n",
-      "Episode 895, Passed! Reward 200\n",
-      "Episode 896, Passed! Reward 200\n",
-      "Episode 897, Passed! Reward 200\n",
-      "Episode 898, Passed! Reward 200\n",
-      "Episode 899, Passed! Reward 200\n",
-      "Episode 900, Passed! Reward 200\n",
-      "Episode 901, Passed! Reward 200\n",
-      "Episode 902, Passed! Reward 200\n",
-      "Episode 903, Passed! Reward 200\n",
-      "Episode 904, Passed! Reward 200\n",
-      "Episode 905, Passed! Reward 200\n",
-      "Episode 906, Passed! Reward 200\n",
-      "Episode 907, Passed! Reward 200\n",
-      "Episode 908, Passed! Reward 200\n",
-      "Episode 909, Passed! Reward 200\n",
-      "Episode 910, Passed! Reward 200\n",
-      "Episode 911, Passed! Reward 200\n",
-      "Episode 912, Passed! Reward 200\n",
-      "Episode 913, Passed! Reward 200\n",
-      "Episode 914, Passed! Reward 200\n",
-      "Episode 915, Passed! Reward 200\n",
-      "Episode 916, Passed! Reward 200\n",
-      "Episode 917, Passed! Reward 200\n",
-      "Episode 918, Passed! Reward 200\n",
-      "Episode 919, Passed! Reward 200\n",
-      "Episode 920, Passed! Reward 200\n",
-      "Episode 921, Passed! Reward 200\n",
-      "Episode 922, Passed! Reward 200\n",
-      "Episode 923, Passed! Reward 200\n",
-      "Episode 924, Passed! Reward 200\n",
-      "Episode 925, Passed! Reward 200\n",
-      "Episode 926, Passed! Reward 200\n",
-      "Episode 927, Passed! Reward 200\n",
-      "Episode 928, Passed! Reward 200\n",
-      "Episode 929, Passed! Reward 200\n",
-      "Episode 930, Passed! Reward 200\n",
-      "Episode 931, Passed! Reward 200\n",
-      "Episode 932, Passed! Reward 200\n",
-      "Episode 933, Passed! Reward 200\n",
-      "Episode 934, Passed! Reward 200\n",
-      "Episode 935, Passed! Reward 200\n",
-      "Episode 936, Passed! Reward 200\n",
-      "Episode 937, Passed! Reward 200\n",
-      "Episode 938, Passed! Reward 200\n",
-      "Episode 939, Passed! Reward 200\n",
-      "Episode 940, Passed! Reward 200\n",
-      "Episode 941, Passed! Reward 200\n",
-      "Episode 942, Passed! Reward 200\n",
-      "Episode 943, Passed! Reward 200\n",
-      "Episode 944, Passed! Reward 200\n",
-      "Episode 945, Passed! Reward 200\n",
-      "Episode 946, Passed! Reward 200\n",
-      "Episode 947, Passed! Reward 200\n",
-      "Episode 948, Passed! Reward 200\n",
-      "Episode 949, Passed! Reward 200\n",
-      "Episode 950, Passed! Reward 200\n",
-      "Episode 951, Passed! Reward 200\n",
-      "Episode 952, Passed! Reward 200\n",
-      "Episode 953, Passed! Reward 200\n",
-      "Episode 954, Passed! Reward 200\n",
-      "Episode 955, Passed! Reward 200\n",
-      "Episode 956, Passed! Reward 200\n",
-      "Episode 957, Passed! Reward 200\n",
-      "Episode 958, Passed! Reward 200\n",
-      "Episode 959, Passed! Reward 200\n",
-      "Episode 960, Passed! Reward 200\n",
-      "Episode 961, Passed! Reward 200\n",
-      "Episode 962, Passed! Reward 200\n",
-      "Episode 963, Passed! Reward 200\n",
-      "Episode 964, Passed! Reward 200\n",
-      "Episode 965, Passed! Reward 200\n",
-      "Episode 966, Passed! Reward 200\n",
-      "Episode 967, Passed! Reward 200\n",
-      "Episode 968, Passed! Reward 200\n",
-      "Episode 969, Passed! Reward 200\n",
-      "Episode 970, Passed! Reward 200\n",
-      "Episode 971, Passed! Reward 200\n",
-      "Episode 972, Passed! Reward 200\n",
-      "Episode 973, Passed! Reward 200\n",
-      "Episode 974, Passed! Reward 200\n",
-      "Episode 975, Passed! Reward 200\n",
-      "Episode 976, Passed! Reward 200\n",
-      "Episode 977, Passed! Reward 200\n",
-      "Episode 978, Passed! Reward 200\n",
-      "Episode 979, Passed! Reward 200\n",
-      "Episode 980, Passed! Reward 200\n",
-      "Episode 981, Passed! Reward 200\n",
-      "Episode 982, Passed! Reward 200\n",
-      "Episode 983, Passed! Reward 200\n",
-      "Episode 984, Passed! Reward 200\n",
-      "Episode 985, Passed! Reward 200\n",
-      "Episode 986, Passed! Reward 200\n",
-      "Episode 987, Passed! Reward 200\n",
-      "Episode 988, Passed! Reward 200\n",
-      "Episode 989, Passed! Reward 200\n",
-      "Episode 990, Passed! Reward 200\n",
-      "Episode 991, Passed! Reward 200\n",
-      "Episode 992, Passed! Reward 200\n",
-      "Episode 993, Passed! Reward 200\n",
-      "Episode 994, Passed! Reward 200\n",
-      "Episode 995, Passed! Reward 200\n",
-      "Episode 996, Passed! Reward 200\n",
-      "Episode 997, Passed! Reward 200\n",
-      "Episode 998, Passed! Reward 200\n",
-      "Episode 999, Passed! Reward 200\n"
+      "Episode 750, Failed! Reward -200\n",
+      "Episode 751, Better! Reward -178\n",
+      "Episode 752, Better! Reward -180\n",
+      "Episode 753, Better! Reward -181\n",
+      "Episode 754, Failed! Reward -200\n",
+      "Episode 755, Failed! Reward -200\n",
+      "Episode 756, Failed! Reward -200\n",
+      "Episode 757, Failed! Reward -200\n",
+      "Episode 758, Failed! Reward -200\n",
+      "Episode 759, Failed! Reward -200\n",
+      "Episode 760, Failed! Reward -200\n",
+      "Episode 761, Failed! Reward -200\n",
+      "Episode 762, Failed! Reward -200\n",
+      "Episode 763, Failed! Reward -200\n",
+      "Episode 764, Failed! Reward -200\n",
+      "Episode 765, Failed! Reward -200\n",
+      "Episode 766, Failed! Reward -200\n",
+      "Episode 767, Failed! Reward -200\n",
+      "Episode 768, Failed! Reward -200\n",
+      "Episode 769, Failed! Reward -200\n",
+      "Episode 770, Failed! Reward -200\n",
+      "Episode 771, Failed! Reward -200\n",
+      "Episode 772, Failed! Reward -200\n",
+      "Episode 773, Failed! Reward -200\n",
+      "Episode 774, Failed! Reward -200\n",
+      "Episode 775, Failed! Reward -200\n",
+      "Episode 776, Failed! Reward -200\n",
+      "Episode 777, Failed! Reward -200\n",
+      "Episode 778, Failed! Reward -200\n",
+      "Episode 779, Failed! Reward -200\n",
+      "Episode 780, Failed! Reward -200\n",
+      "Episode 781, Failed! Reward -200\n",
+      "Episode 782, Failed! Reward -200\n",
+      "Episode 783, Failed! Reward -200\n",
+      "Episode 784, Failed! Reward -200\n",
+      "Episode 785, Failed! Reward -200\n",
+      "Episode 786, Better! Reward -178\n",
+      "Episode 787, Failed! Reward -200\n",
+      "Episode 788, Failed! Reward -200\n",
+      "Episode 789, Failed! Reward -200\n",
+      "Episode 790, Better! Reward -168\n",
+      "Episode 791, Better! Reward -156\n",
+      "Episode 792, Better! Reward -114\n",
+      "Episode 793, Passed! Reward -100\n",
+      "Episode 794, Better! Reward -116\n",
+      "Episode 795, Better! Reward -154\n",
+      "Episode 796, Passed! Reward -91\n",
+      "Episode 797, Better! Reward -125\n",
+      "Episode 798, Better! Reward -165\n",
+      "Episode 799, Better! Reward -119\n",
+      "Episode 800, Better! Reward -123\n",
+      "Episode 801, Better! Reward -123\n",
+      "Episode 802, Better! Reward -118\n",
+      "Episode 803, Better! Reward -123\n",
+      "Episode 804, Better! Reward -121\n",
+      "Episode 805, Better! Reward -124\n",
+      "Episode 806, Better! Reward -126\n",
+      "Episode 807, Better! Reward -122\n",
+      "Episode 808, Better! Reward -114\n",
+      "Episode 809, Better! Reward -115\n",
+      "Episode 810, Better! Reward -123\n",
+      "Episode 811, Better! Reward -114\n",
+      "Episode 812, Better! Reward -124\n",
+      "Episode 813, Better! Reward -125\n",
+      "Episode 814, Better! Reward -131\n",
+      "Episode 815, Better! Reward -125\n",
+      "Episode 816, Better! Reward -114\n",
+      "Episode 817, Better! Reward -123\n",
+      "Episode 818, Better! Reward -125\n",
+      "Episode 819, Better! Reward -125\n",
+      "Episode 820, Better! Reward -118\n",
+      "Episode 821, Better! Reward -123\n",
+      "Episode 822, Better! Reward -125\n",
+      "Episode 823, Better! Reward -123\n",
+      "Episode 824, Better! Reward -115\n",
+      "Episode 825, Better! Reward -123\n",
+      "Episode 826, Better! Reward -160\n",
+      "Episode 827, Better! Reward -126\n",
+      "Episode 828, Better! Reward -129\n",
+      "Episode 829, Better! Reward -116\n",
+      "Episode 830, Passed! Reward -93\n",
+      "Episode 831, Better! Reward -129\n",
+      "Episode 832, Better! Reward -119\n",
+      "Episode 833, Better! Reward -123\n",
+      "Episode 834, Better! Reward -126\n",
+      "Episode 835, Better! Reward -131\n",
+      "Episode 836, Better! Reward -121\n",
+      "Episode 837, Better! Reward -129\n",
+      "Episode 838, Better! Reward -117\n",
+      "Episode 839, Better! Reward -121\n",
+      "Episode 840, Better! Reward -123\n",
+      "Episode 841, Better! Reward -128\n",
+      "Episode 842, Better! Reward -117\n",
+      "Episode 843, Better! Reward -136\n",
+      "Episode 844, Better! Reward -125\n",
+      "Episode 845, Better! Reward -128\n",
+      "Episode 846, Better! Reward -127\n",
+      "Episode 847, Better! Reward -116\n",
+      "Episode 848, Better! Reward -125\n",
+      "Episode 849, Passed! Reward -88\n",
+      "Episode 850, Failed! Reward -200\n",
+      "Episode 851, Better! Reward -184\n",
+      "Episode 852, Passed! Reward -92\n",
+      "Episode 853, Better! Reward -120\n",
+      "Episode 854, Better! Reward -118\n",
+      "Episode 855, Better! Reward -169\n",
+      "Episode 856, Better! Reward -120\n",
+      "Episode 857, Better! Reward -121\n",
+      "Episode 858, Passed! Reward -95\n",
+      "Episode 859, Passed! Reward -89\n",
+      "Episode 860, Better! Reward -162\n",
+      "Episode 861, Better! Reward -168\n",
+      "Episode 862, Failed! Reward -200\n",
+      "Episode 863, Better! Reward -162\n",
+      "Episode 864, Better! Reward -166\n",
+      "Episode 865, Failed! Reward -200\n",
+      "Episode 866, Failed! Reward -200\n",
+      "Episode 867, Failed! Reward -200\n",
+      "Episode 868, Failed! Reward -200\n",
+      "Episode 869, Failed! Reward -200\n",
+      "Episode 870, Failed! Reward -200\n",
+      "Episode 871, Better! Reward -124\n",
+      "Episode 872, Better! Reward -127\n",
+      "Episode 873, Better! Reward -122\n",
+      "Episode 874, Better! Reward -119\n",
+      "Episode 875, Better! Reward -128\n",
+      "Episode 876, Failed! Reward -200\n",
+      "Episode 877, Better! Reward -116\n",
+      "Episode 878, Better! Reward -116\n",
+      "Episode 879, Passed! Reward -87\n",
+      "Episode 880, Better! Reward -117\n",
+      "Episode 881, Better! Reward -118\n",
+      "Episode 882, Better! Reward -123\n",
+      "Episode 883, Better! Reward -122\n",
+      "Episode 884, Better! Reward -131\n",
+      "Episode 885, Better! Reward -122\n",
+      "Episode 886, Better! Reward -118\n",
+      "Episode 887, Better! Reward -120\n",
+      "Episode 888, Better! Reward -121\n",
+      "Episode 889, Better! Reward -117\n",
+      "Episode 890, Better! Reward -124\n",
+      "Episode 891, Failed! Reward -200\n",
+      "Episode 892, Better! Reward -123\n",
+      "Episode 893, Failed! Reward -200\n",
+      "Episode 894, Better! Reward -116\n",
+      "Episode 895, Better! Reward -120\n",
+      "Episode 896, Failed! Reward -200\n",
+      "Episode 897, Better! Reward -117\n",
+      "Episode 898, Passed! Reward -85\n",
+      "Episode 899, Better! Reward -118\n",
+      "Episode 900, Better! Reward -113\n",
+      "Episode 901, Better! Reward -124\n",
+      "Episode 902, Better! Reward -159\n",
+      "Episode 903, Passed! Reward -93\n",
+      "Episode 904, Better! Reward -121\n",
+      "Episode 905, Better! Reward -116\n",
+      "Episode 906, Better! Reward -116\n",
+      "Episode 907, Better! Reward -118\n",
+      "Episode 908, Better! Reward -111\n",
+      "Episode 909, Better! Reward -113\n",
+      "Episode 910, Better! Reward -118\n",
+      "Episode 911, Better! Reward -117\n",
+      "Episode 912, Better! Reward -115\n",
+      "Episode 913, Better! Reward -117\n",
+      "Episode 914, Better! Reward -116\n",
+      "Episode 915, Better! Reward -117\n",
+      "Episode 916, Better! Reward -124\n",
+      "Episode 917, Better! Reward -116\n",
+      "Episode 918, Better! Reward -117\n",
+      "Episode 919, Better! Reward -111\n",
+      "Episode 920, Better! Reward -118\n",
+      "Episode 921, Better! Reward -113\n",
+      "Episode 922, Better! Reward -114\n",
+      "Episode 923, Better! Reward -116\n",
+      "Episode 924, Better! Reward -113\n",
+      "Episode 925, Better! Reward -116\n",
+      "Episode 926, Better! Reward -115\n",
+      "Episode 927, Better! Reward -116\n",
+      "Episode 928, Better! Reward -116\n",
+      "Episode 929, Better! Reward -117\n",
+      "Episode 930, Better! Reward -115\n",
+      "Episode 931, Better! Reward -113\n",
+      "Episode 932, Better! Reward -116\n",
+      "Episode 933, Better! Reward -114\n",
+      "Episode 934, Better! Reward -113\n",
+      "Episode 935, Better! Reward -122\n",
+      "Episode 936, Better! Reward -115\n",
+      "Episode 937, Better! Reward -117\n",
+      "Episode 938, Better! Reward -127\n",
+      "Episode 939, Better! Reward -115\n",
+      "Episode 940, Better! Reward -113\n",
+      "Episode 941, Better! Reward -115\n",
+      "Episode 942, Better! Reward -111\n",
+      "Episode 943, Better! Reward -116\n",
+      "Episode 944, Better! Reward -113\n",
+      "Episode 945, Better! Reward -118\n",
+      "Episode 946, Better! Reward -116\n",
+      "Episode 947, Better! Reward -113\n",
+      "Episode 948, Better! Reward -113\n",
+      "Episode 949, Better! Reward -116\n",
+      "Episode 950, Better! Reward -118\n",
+      "Episode 951, Better! Reward -112\n",
+      "Episode 952, Better! Reward -120\n",
+      "Episode 953, Better! Reward -117\n",
+      "Episode 954, Better! Reward -117\n",
+      "Episode 955, Better! Reward -117\n",
+      "Episode 956, Better! Reward -117\n",
+      "Episode 957, Better! Reward -116\n",
+      "Episode 958, Better! Reward -117\n",
+      "Episode 959, Better! Reward -121\n",
+      "Episode 960, Better! Reward -114\n",
+      "Episode 961, Better! Reward -116\n",
+      "Episode 962, Better! Reward -116\n",
+      "Episode 963, Better! Reward -118\n",
+      "Episode 964, Better! Reward -118\n",
+      "Episode 965, Better! Reward -116\n",
+      "Episode 966, Better! Reward -115\n",
+      "Episode 967, Better! Reward -115\n",
+      "Episode 968, Better! Reward -115\n",
+      "Episode 969, Better! Reward -117\n",
+      "Episode 970, Better! Reward -115\n",
+      "Episode 971, Better! Reward -115\n",
+      "Episode 972, Better! Reward -115\n",
+      "Episode 973, Better! Reward -114\n",
+      "Episode 974, Better! Reward -124\n",
+      "Episode 975, Better! Reward -114\n",
+      "Episode 976, Better! Reward -115\n",
+      "Episode 977, Better! Reward -113\n",
+      "Episode 978, Passed! Reward -86\n",
+      "Episode 979, Better! Reward -114\n",
+      "Episode 981, Better! Reward -139\n",
+      "Episode 982, Better! Reward -120\n",
+      "Episode 983, Better! Reward -122\n",
+      "Episode 984, Better! Reward -127\n",
+      "Episode 985, Better! Reward -129\n",
+      "Episode 986, Better! Reward -197\n",
+      "Episode 987, Better! Reward -123\n",
+      "Episode 988, Better! Reward -120\n",
+      "Episode 989, Better! Reward -193\n",
+      "Episode 990, Failed! Reward -200\n",
+      "Episode 991, Better! Reward -161\n",
+      "Episode 992, Passed! Reward -107\n",
+      "Episode 993, Passed! Reward -105\n",
+      "Episode 994, Better! Reward -181\n",
+      "Episode 995, Better! Reward -111\n",
+      "Episode 996, Better! Reward -155\n",
+      "Episode 997, Better! Reward -153\n",
+      "Episode 998, Better! Reward -179\n",
+      "Episode 999, Failed! Reward -200\n"
      ]
     }
    ],
    "source": [
     "ep_list =[]\n",
-    "reward_list =[]  \n",
+    "reward_list =[] \n",
+    "index=0 \n",
+    "oh = OneHotEncoder(n_values=3)\n",
     "for ep in range(num_episodes):\n",
     "    s= env.reset()\n",
     "    s=s.reshape((1,-1))\n",
-    "    rAll =0\n",
+    "    s = s*factor\n",
+    "    total_rewards =0\n",
     "    d = False\n",
     "    j = 0\n",
     "    for j in range(200):\n",
-    "        #time.sleep(0.01)\n",
-    "        #epsilon greedy. to choose random actions initially when Q is all zeros\n",
     "        if np.random.random()< epsilon:\n",
-    "            a = np.random.randint(0,legal_actions)\n",
-    "            #epsilon = epsilon*epsilon_decay\n",
+    "            a = np.random.randint(0,len(action_list))\n",
     "        else:\n",
     "            Q = model.predict(s.reshape(-1,s.shape[0],s.shape[1]))\n",
     "            a =np.argmax(Q)\n",
     "        new_s,r,d,_ = env.step(a)\n",
     "        new_s = new_s.reshape((1,-1))\n",
-    "        rAll=rAll+r\n",
+    "        new_s = new_s*factor\n",
+    "        total_rewards=total_rewards+r\n",
     "        if show:\n",
     "            env.render()\n",
     "        if d:\n",
-    "            if rAll<195:\n",
+    "            if total_rewards<-199:\n",
     "                r =-100\n",
-    "                experience = (s,r,a,new_s)\n",
+    "                experience = (s,r,a,new_s,d)\n",
+    "                memory.append(experience)\n",
+    "                print(\"Episode %d, Failed! Reward %d\"%(ep,total_rewards))\n",
+    "            elif total_rewards<-110 and total_rewards>-199:\n",
+    "                r=10\n",
+    "                d=True\n",
+    "                experience = (s,r,a,new_s,d)\n",
     "                memory.append(experience)\n",
-    "                print(\"Episode %d, Failed! Reward %d\"%(ep,rAll))\n",
-    "                #break\n",
-    "            elif rAll>195:\n",
-    "                print(\"Episode %d, Passed! Reward %d\"%(ep,rAll))\n",
+    "                print(\"Episode %d, Better! Reward %d\"%(ep,total_rewards))\n",
+    "            elif total_rewards>=-110:\n",
+    "                r=100\n",
+    "                experience = (s,r,a,new_s,d)\n",
+    "                memory.append(experience)\n",
+    "\n",
+    "                print(\"Episode %d, Passed! Reward %d\"%(ep,total_rewards))\n",
     "            ep_list.append(ep)\n",
-    "            reward_list.append(rAll)\n",
+    "            reward_list.append(total_rewards)\n",
     "            break\n",
     "        \n",
-    "        experience = (s,r,a,new_s)\n",
+    "        experience = (s,r,a,new_s,d)\n",
     "        memory.append(experience)\n",
     "        if j==199:\n",
-    "            print(\"Reward %d after full episode\"%(rAll))\n",
+    "            print(\"Reward %d after full episode\"%(total_rewards))\n",
     "            \n",
     "        s = new_s\n",
     "    batches=random.sample(memory,batch_size)\n",
     "    states= np.array([batch[0] for batch in batches])\n",
     "    rewards= np.array([batch[1] for batch in batches])\n",
     "    actions= np.array([batch[2] for batch in batches])\n",
+    "    actions=oh.fit_transform(actions.reshape(-1,1)).toarray()\n",
+    "    actions = actions.reshape(-1,1,action_size)\n",
     "    new_states= np.array([batch[3] for batch in batches])\n",
+    "    dones= np.array([batch[4] for batch in batches])\n",
     "    Qs =model.predict(states)\n",
     "    new_Qs = model.predict(new_states)\n",
-    "    for i in range(len(rewards)):\n",
-    "        if rewards[i]==-100:\n",
-    "            Qs[i][0][actions[i]]=Qs[i][0][actions[i]]+ lr*(rewards[i]-Qs[i][0][actions[i]])\n",
-    "        else:\n",
-    "            Qs[i][0][actions[i]]= Qs[i][0][actions[i]]+ lr*(rewards[i]+gamma*np.max(new_Qs[i])-Qs[i][0][actions[i]])\n",
+    "    target_Qs=rewards.reshape(-1,1)+gamma*(np.max(new_Qs,axis=2)*(~dones.reshape(-1,1)))\n",
+    "    Qs[actions==1]=target_Qs.reshape(-1,)\n",
     "    model.fit(states,Qs,verbose=0)\n",
     "    epsilon=epsilon*epsilon_decay\n",
     "env.close()"
    ]
   },
   {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Plot of Episodes vs Reward"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Deep Q Learning"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![DQL](Reward_vs_Episode_DL_lr_0.500000_eps_1000_CartPole.jpg)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Plot for Vanilla ```Q Learning```"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![QLearning](Reward_vs_Episode_QL_lr_0.500000_eps_1000_CartPole.jpg)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The above two pictures show that Deep Q Network reached the highest points faster than traditional Q Learning Algorithm"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Further Improvement"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Text(0,0.5,'Rewards')"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEWCAYAAACnlKo3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJztnXm8HGWV97+/e282EkIS9gRCUIIKiFEui6IjKgiIiiyOoKPozMjLDIzbOAriAjOv4/q6IyPuLIILw6KgERBZZE0AISGgISQkJEgCIftyl/P+UVV9q/tW961eqru67/l+PjfpeuqpqlPbc+qc8zznkZnhOI7jOLXQ1WoBHMdxnPbFlYjjOI5TM65EHMdxnJpxJeI4juPUjCsRx3Ecp2ZciTiO4zg140rEcWpA0k8k/d9Wy1Etkj4l6QcN3udRklY0cp9O++BKxMkVkpZK2iJpo6RnwsZ6UqvlyhPhNdkeXqPo789ptjWz/zazf85aRmf04ErEySNvM7NJwBzglcB5rRJEUk+rjj0CXzazSbG/V7RaIGd04krEyS1m9gwwl0CZACBpnKSvSnpK0t8k/Y+kCeG62ySdEv5+rSST9JZw+WhJD4W/XyzpD5Kek7RG0hWSpsSOsVTSJyU9DGyS1CPplZIekLRB0s+B8bH6u0j6jaQXJD0v6Q5Jw96tUNavlpRdJ+lj4e9PSno6PMbjkt5U7TWTNCs87zMlrZS0StK/x9ZfIOny8Pd4SZeH1+EFSfdL2j1cN13S9eH5LJb0wdg+JoTW0FpJjwKHlsgwXdLVklZLelLSh6o9D6d9cCXi5BZJewHHA4tjxV8C9idQLPsBM4DPhutuA44Kf/8dsAR4fWz5tmjXwBeA6cDLgL2BC0oOfzpwAjCF4D25FrgMmAb8EjglVvffgRXArsDuwKeApHxCPwPeJUnh+U0F3gxcJeklwDnAoWa2I3AssDTxwqTjDcDscP/nSjo6oc4ZwE4E578zcBawJVx3ZXhO04FTgf+OKbXPAS8O/44N90N4Tl3Ar4E/E9ybNwEfkXRsHefi5BhXIk4euVbSBmA58CxBo0XY+H4Q+KiZPW9mG4D/Bk4Lt7uNYqXxhdjy68P1mNliM7vJzLaZ2Wrga7F6Ed8ys+VmtgU4AhgDfMPM+szsV8D9sbp9wJ7APuH6Oyw5Kd0dBMrldeHyqcDdZrYSGADGAQdIGmNmS83siQrX6OOh9RD9/bRk/YVmtsnMHgF+TKAUS+kjUB77mdmAmc03s/WS9gZeC3zSzLaa2UPAD4D3htv9PfD58B4sB74V2+ehwK5m9p9mtt3MlgDfZ+geOR2GKxEnj7wj/Bo/CngpsEtYviuwAzA/ajyB34XlAHcD+4cumTnApcDeknYBDgNuB5C0m6SrQtfReuDy2DEilsd+TweeLlEMy2K/v0JgLf1e0hJJ5yadVLj9VQw16O8GrgjXLQY+QmARPRvKN73CNfqqmU2J/Z1Rsj4u/7LwHEq5jMBdeFXo+vqypDFh3UhJx/cxI/w9PWH/EfsA0+MKjsAy273CuThtjCsRJ7eY2W3AT4AojrCGwN1yYKzx3CkMwmNmm4H5wIeBBWa2HbgL+BjwhJmtCffzBQKL4GAzmwz8A4GLq+jwsd+rgBmRGypkZkzODWb272b2IuBtwMcqxDOuBE6VtA9wOHB1bD8/M7PXEjTERuC6q5W9S2RdWVohtJouNLMDgNcAbwXeF9adJmnHkn08Hf5elbD/iOXAkyUKbkcze0sd5+LkGFciTt75BnCMpDlmNkjgGvm6pN0AJM0o8bffRhBbiOIffyxZBtgR2Ai8IGkG8B8jyHA30A98KAyyn0xg2RDK8FZJ+4VKZj2Ba2ogaUdm9iCwmsA9NNfMXgj38RJJb5Q0DthKoCwT95GSz0jaQdKBwAeAn5dWkPQGSS+X1B3K3QcMhC6qu4AvhMH3g4F/IrSagF8A50maGsat/i222/uA9WEngQmSuiUdJKko+O50Dq5EnFwTxiwuBT4TFn2SwHV0T+iKuhl4SWyT2wiUxO1llgEuBF4FrANuAP53BBm2AycD7wfWAu8q2WZ2KMdGAoXzXTP7Y4VdXgkcTRBojxgHfJHA2noG2I3ADVSOT6h4nMiakvW3EVynWwhcX79P2McewK8IFMiicJvLw3WnA7MIrJJrgM+Z2U3hugsJXFhPAr8ncIsBYGYDBNbYnHD9GgKFuVOFc3HaGPmkVI7TOUiaRdB4jzGz/tZK44wG3BJxHMdxasaViOM4jlMzuVMikuZIukfSQ5LmSTosLJekb4WjZx+W9KpWy+o4eSMcXyJ3ZTnNIndKBPgywUCpOQQjkb8clh9PEMCcDZwJXNwa8RzHcZyIPCaXM2By+Hsnhvq3nwhcGg7YukfSFEl7mtmqSjvbZZddbNasWZkJ6ziO02nMnz9/jZntOnLNfCqRjwBzw0R1XQSDoCAYLRsfJbsiLKuoRGbNmsW8efOykNNxHKcjkbRs5FoBLVEikm4m6KNeyvkECds+amZXS/p74IcEfepLRxRDcpI7JJ1J4PJi5syZSVUcx3GcBpC7cSKS1gFTzMzCEcDrzGyypO8BfzSzK8N6jwNHjeTO6u3tNbdEHMdx0iNpvpn1pqmbx8D6SoYyqr4R+Gv4+3rgfWEvrSMIlEtFBeI4juNkSx5jIh8EvqlgRrmthG4p4EbgLQSpHDYT5ANyHMdxWkjulIiZ3QkcklBuwNnNl8hxHMcpRx7dWY7jOE6b4ErEcRzHqRlXIo7jOBmwaVs/1zy4otViZE7uYiKO4zidwGevW8jVD6xg5rSJHLLP1FaLkxluiTiO42TA39ZvBWDz9uxyYX7qmke4Z8lzme0/Da5EHMdx2pSf3fsUp11yT0tlcCXiOI5TByvWbmbWuTfw20eaO/Y5L9lGXIk4juOUwcyYv2xtxToLV64H4OoHnm6GSAUG86FDXIk4juOU46d3LeWUi+/i1sefrXpbS84P2zAG3RJxHMfJN395diMAK9ZuYWvfAF/7/eNs7RsoqpOUXrwZ5ESHuBJxHMcpR9RQC/jhnU/yrT8s5qd3LW2lSAXcEnEcx8k9QUMtwfqtfQAM5KTxzokYrkQcx+lcBgaNFzZvr3n7IUtEbO8fBGBsd7pmUxk7urKOuaTFlYjjOB3LF25cxJz/vIkNoRVRKxIFJTKuJ12zmX1gPdPdp8aViOM4HcsN4diNDVtrGzUeWSJzFz4zZImUKJFWteV5iYl47izHcTqeWpvbyJr44+Ormb3bJGC4EmkVOdEhbok4jtO5RFGJkUZ3Dw4an7l2AYvDLr0R8c2e3xTEVsZ2d1clw0/+tJTfLXimqm3S4CPWHcdxMkZKF9xevHojl92zjLMun19UHm+mt5VxZ43ELY89O2y/jcBjIo7jOE1ipI/2+HiQcttFgwzHdLdqeGExbok4juPkBIuNB0kqB+gfjOrkQ4m4JeI4jpMzsh7b0UjcEnEcx8mYtEZD2fY4oTwvjXc+pHAl4jjOKCB1TGSYOyuhbkMkqp+8jBNxJeI4TseS2hIpoxqqsTqaHSrxmIjjOE6TSJuCpDRonrhVThrvwZxoEVcijuN0LFGgPK07y6meligRSe+UtFDSoKTeknXnSVos6XFJx8bKjwvLFks6t/lSO47TblTrYqo0TqRQlhNTZLTHRBYAJwO3xwslHQCcBhwIHAd8V1K3pG7gIuB44ADg9LCu4zhO3VQVWM9H252bmEhLEjCa2SJIHLRzInCVmW0DnpS0GDgsXLfYzJaE210V1n20ORI7jtNuPLNuK8ue21zVNsOUSF40RgJ5kS1vMZEZwPLY8oqwrFx5IpLOlDRP0rzVq1dnIqjjOPnmU9c8Uvg9UnNbtndWUlk+2u7Ot0Qk3QzskbDqfDO7rtxmCWVGsrIrewnN7BLgEoDe3t6cXGrHcZpJf6yVHemrPT6DYbW0SqnkxRLJTImY2dE1bLYC2Du2vBewMvxdrtxxHKcuouZ4mIc9MbCeD/IiR97cWdcDp0kaJ2lfYDZwH3A/MFvSvpLGEgTfr2+hnI7jdCDDdcjwpjqtBZC1oZCX3lktCaxLOgn4NrArcIOkh8zsWDNbKOkXBAHzfuBsMxsItzkHmAt0Az8ys4WtkN1xnPYg3tiPGBMp0yCna6db05gPDrbksMNoVe+sa4Bryqz7PPD5hPIbgRszFs1xnA5kxJhI9KN0xHoKd1bLYiI5cWjlzZ3lOI7TcEYesR7OFVLDvlrVSyon3ixXIo7jdD4jtbeRIqg0KVX5fSfXyTohY1JMxMzY3t9cP5crEcdxOpJ4GzvSV3u5ZIbJxZaiTvaWQtL+L7p1Mft/+rds2NqX7cFjuBJxHKfjGcmiKFgitey7RX6lJEvkB3c+CcDWvuZZI65EHMfpeEbqyVSIiaQJrFvycrMn1k2ygF7YHFggXU0UxpWI4zgdSdz6SGuJpNtv6batinCXP24zJXIl4jhOR1JVTKRs76wUgfUW6ZC85M5yJeI4zqinoESGZfEdXnd4F9+goNlteqWZDZup2FyJOI7T8YzkciqXgDExi29Jaf6cWc3FlYjjOB1PWnfW8O3SuLPy0zsropmj2V2JOI7TkRTFREaoW/AM1TCzYetSwVda2TQxXIk4jtOZFPXOGqGlLxdYT6Mg6hljUg95yeLrSsRxnI5npJ5MViawnlh32L7TNeZn/Og+1ocjyT/+yz9z62PPJtb75K8eLruuSI4Kh7307mWpZGoErkQcxxkF1DZOJNmdVVtg/ba/rObXfw7m0vvV/BV84Cf3J+775/OWJ64rpZLy+s6ti1NKVT+uRBzH6XjSjxMpHbHe2MD6+i399A+UHz6/LZY8ccHT65h17g38afGaMsdNfdhMcSXiOE5HUktgvZbMu9U05uu39rE9pkQeeGotA4PGbX9Zzf1Ln2fF2i2FdZ++dgEAfyjj2srLfCItmZTKcRynmaSeT6SOwYZp2Li1n22x5Ignf/cuPvi6ffn+HU8Oq/vQ8heA8nmwtvdbxfXNwi0Rx3E6knjTHjX0Nz6yin+9Yj5zFz7DV+Y+xkBogtTT06maTQfMiiwRgHuffL7iNl0ST6zeyMV/fKKg7BatWs9Zl88HYOoOY6sTuMG4JeI4TkcS/0CPGvp/veIBAG585BkAeveZxhteulshy+/wEesJEz8Nm08kvRYxsyJLBOC5jdsrbiOJf/zJ/Sx7bjPvOnRvpk0cy/HfvKOwfupEVyKO43Q4C1eu44Rv3QnAwxe8mcnjx2R+zLhrqlz8oG9gkBO+dQebtvUnrk+jH6qxRK68bzn77bZjUdnTL2wpUzugSxQsps/fsIirH1hRtL6nxf4sd2c5jlMT6zb3sbVvIFXdq+5bXvj9179tzEqkIrqKtUgiBixcuZ6lz20GhsdE1m4ePkNgqdJYt2V4na19A4nbAvzXbx4tK3M5tmwPrnOpAskDrkQcx6mJV/zn73nrt+9MVXdM91BT092kL+e4Qqg1bfqiVeuHlZUqkd8uWAVAT/fQAd/7w3sTt62F792+hOc2VXZ5tRJXIo7j1MziZwOr4toHny7rEgIY2zPU1DTL/RKPb5RzZ5UqhC6Jxc9uYO7CZ7jxkVWpjjMlDGz3dAXneN1DT3P/0rWptv3maXOGlc3Ze0rR8kBeJg4pg8dEHMepiwefWstHfv4QJ79yBl971/BGEWBs7Cu9q5bBGHVSPm5RvKK7Sxz9tdsr76tM+botfSxatZ4PX/VQarlOnDODL/72MWZO26HQS2uXSeNSbw+tH3TolojjOHWxNexttKJCgDhuiTTLnRXnpkf/xtdv+suw8qi3VkQa0cqNUP/L3zbw4aserFq2u897Ex87Zv/C8tieykL8+P2HcsHbDqj6OFnREiUi6Z2SFkoalNQbKz9G0nxJj4T/vzG27pCwfLGkb0kt+JxxHGcYUSygktulWIlkLhJQHBO57J5lfPOWvw6rUypyLc1KpFRWrdvKXyp0GpgxZQJnv+HFheX/Punlhd89sYuy04SxnPX6F/PVd75i2D5OePmevHJmsbur1SPXW2WJLABOBkrtxjXA28zs5cAZwGWxdRcDZwKzw7/jmiCn4zgjELmnKimR4sB6fh0gNz36txHrxM/y9Evu4YGnXki172vPPpL3HjGrsPzuw2cWfo/pLlZe5x7/Uk49ZK9h+7joPa8qxGDyQkvuppktMrPHE8ofNLOV4eJCYLykcZL2BCab2d0WqP1LgXc0UWSnwxgcNB54Kl3w06lMFChPq0Ranaajkdy95LnUdceP6RqmLCJ6ihRriqSPsd9Pr93CqnVbWPzshtSyNJL8fhLAKcCDZrYNmAHEO0ivCMsSkXSmpHmS5q1evTpjMZ125Pt3LOHk797F3U+kbwScZCJ3Vn8ld1ZMiZSOCm87avQejR/TXeTWixOPE5WGXF6//64V97tp+wCv/sIfRuwQkBWZKRFJN0takPB3YoptDwS+BPyfqCihWvn+FmaXmFmvmfXuumvlG+CMTh5+eh0Aqzdua7Ek7cWGrX184+a/FKUzj76iBwbLpziPN56t9uHXS63yj+nuKrLI0uzzsf86jh+e0Zu4Li9k1sXXzI6uZTtJewHXAO8zsyfC4hVA3EG4F7CydFvHSUuUv2h8mS9DJ5nv3LqY7922hOk7TSiURe1iJUvEu8EEjC2jROL6N26JjB/TDcD7XzOLyROyTxVTC7l6gyRNAW4AzjOzP0XlZrYK2CDpiLBX1vuA61okptMGfPDSeXx17lDY7cd/epJTLr6rsLytP0gjMS58SZ10jAmtjlXrthbKlBBY/+ldSzn5u38iiVaPa6iXWuT/l6OCXlldZQJC5WIlERe8/cCibsB50smt6uJ7kqQVwKuBGyTNDVedA+wHfEbSQ+HfbuG6fwF+ACwGngB+22y5nfbhpkf/VjRF6IW/fpT5y4YC6ZElMs4tkarYeVLQM2hNzA0YNar9A0Ot6+euX1jUayne8G7c1l/IubV5e3/q/Ft5o5EjyffbbRIH7DkZaD93X6t6Z11jZnuZ2Tgz293Mjg3L/6+ZTTSzObG/Z8N188zsIDN7sZmdY9XMSek4JWyNLBFXIlUxLUw7/nxCLqdKjWq8YXzrt+/k7d8Jcm4d8Nm5/N2Xb22wlOExM2ohot1G1iyM7K4bHEHhSOL9r5kV7L/NWjZ/g5xRSfT124rR0+1M5NOPN6BRs1ouJvLn5S/wxLObisrig/Ke3ZBN54asvuijRj5+TiP1oKoULyoQPoptpkM8d5YzOtnWH7iz2u2rr9UMfYUPRYKja1iud9aJFyXHRrKm9N5O32k8K9dtZc7eUwpTz9bD276TLoMxFFtpk8f38OYD96jr2K+dHSitPXcaXxSfagVuiTijiqO+citX3Lusbf3weWF7/3CFkeprO0bWHunS3UfdjEuz5Fa93wRboZr4yMMXHJuY0iSyidNclv12m8TSL55A76xpqY+bFa5EnFHF0uc2c/41C4YskRbL025EDVx8nvDoGlYbaO4byFiJlNzdSInss/MOhbJZsd+p95sgdn8DziXq5VaNcs2DM9aViDMqiXpnef+M6oga5kZYIlv7s7UGS2/t/rsH09JOGjfkxf/jf7yBfzhiJvXSiJ5aeVAIteAxEWdUknUD1ulsT4yJGH0DgyxZvanMVsVk7VIsbda/dMrBvPXg6ew1dUJRebVpWJLURV+F0fpQ3cdKNeooD4M43RJxRiXRO+12SHUkubMiBgaNL/32MY79RrocTpE1mBklN3fiuB6OO2gPxo9pTLO3U2wE+UiWSJrn7KV7BpbSa/fbpR6xmo4rEcdxqqbIEok1kX9ekb7XU/aWSHLTPba7ziwFoSY9LtbDqhExkQOn78SDnzmGUxJSwJcjB4aIKxFndOMhkeqILldSTAQom2Awia0ZWyLl7u24Oi2RpN0OjvAgpX3Opk7M11whaXAl4nQ8SaOrnfpIGicCxTP0ARz6+ZvL7uOCXy9suFxxyrXbpUkQax2UWM12WQ18zMMEr65EnI7nlkWVZqtzU6QaogBxf5lA8tiSRIKrK4xGj+cyy4Jywew0lsjnTzqown5rFqkjcSXijGq8QaiNeCA5fg2rcWdlTblYd6klEvXO+sRxLymUvXSPyWX3GymnPDw7rbdDXIk4jpOSNRu3cf/S54HigYJxV02elEi5Nr7U5RbJ3xVzDaXJqRbf/0gKJQ8KJyvyc8cdJyOS3t8dwwFnHfxuN5x3XPQnLr/nqYp18qRERmq5P3DkrKJlAe85PBh4GOmQJF1Sbq/HHbhHYb75tNvUTQ5MkVSDDSW9GFhhZtskHQUcDFxqZvVnMXOcrEl4g8tNDuSUZ8XaLYnl8bZ6bE9+rmulhnvpF08YVibB5096OZ8/6eU8siKYPrlLKtvzqrT4f957CACzzr2hJnnblbSfDVcDA5L2A34I7Av8LDOpHCdjIs9FJ7sZWkGeLJFq723cnZXk4qp1v7Vuk4ZqR9tnQdo7Pmhm/cBJwDfM7KPAntmJ5TiNI6l7Zetfvc4kV0qkDidSFJRP6kFrhf+r2X9WXXwz2W1VpL3jfZJOB84AfhOW5XPWeMdJQVcNGVOdkekZYa7wZlLtrY2PuYieiyRLZKhSLVJ1HmmVyAcI5kP/vJk9KWlf4PLsxHKcbCm4s1orRkdQFBPJkyWS8uZG9eJhssGEsqH61T81nfytkiqwbmaPAh+KLT8JfDEroRynkSS/wPn5Yu4k8nRVq22347KnsUSqcmZlNd97DpRTRSUi6REqXCszO7jhEjlOg0l6gLs8sN4w4rGBPF3OtBZDpCfi7qyCJZLzXnx5cMeOZIm8Nfz/7PD/y8L/3wNszkQix2kCeQhIdgo5aMfqIsmdFf2eOLabdVv6EutXN0dINhdppMSPzaCiA9PMlpnZMuBIM/uEmT0S/p0LHNscER2n8URdI7N6uUcredLNVbevsS+LV82cyofeNDtxLvTC/rOUJSUNmFCxbtJGwSZKem20IOk1wMRsRHKcxpL0Arsl0jhy0I4lUs9XeleX+Ngx+7PrjuOGrcvTh0ceLJG00+P+I/BjSTsRPDPrwjLHyT0Vx4m0/h10MqIRtzZxnIgV/98sWRL3m4Pnd0QlIqkL2M/MXiFpMiAzW5e9aI6THVEQNQfvYNuTh+BuEs2UayTrJDt3Vuuv/YjuLDMbBM4Jf69vhAKR9E5JCyUNSupNWD9T0kZJH4+VHSfpcUmLJZ1brwyO43Q2jWleE9KeNHT/9ZEDHZI6JnKTpI9L2lvStOivjuMuAE4Gbi+z/uvAb6MFSd3ARcDxwAHA6ZIOqOP4zigi6UXr6iq/zqmO3F7CtIMNq91tDb2zsiIP8ZlqYiIw1NUXgmv/oloOamaLIHlqR0nvAJYAm2LFhwGLzWxJWOcq4ETg0VqO74wukl6zPCSuc7Kl9c3rEFk19jnQY6lHrO+btSAAkiYCnwSOAT4eWzUDWB5bXgEcXmE/ZwJnAsycObPxgjptz1Dakxy8hW1OHhqyJFIPNqy0LjEBYzizYVXCVFO5vUhriSDpIAJX0viozMwurVD/ZmCPhFXnm9l1ZTa7EPi6mW0ssVKS7nOlkfSXAJcA9Pb2dvDtc2rF7ZDOJ+2L7w1EfaSdlOpzwFEESuRGgtjEnUBZJWJmR9cgz+HAqZK+DEwBBiVtBeYDe8fq7QWsrGH/zmgk4Yt0KItvs4XpRPJ5EavO4puyzGqIrOfzCjWGtJbIqcArgAfN7AOSdgd+0GhhzOx10W9JFwAbzew7knqA2WH24KeB04B3N/r4TmdSKf9iJ7/czSKvirhaV2XVAfYqtshDED4r0vbO2hJ29e0Px4o8S41BdQBJJ0laQZBe/gZJcyvVDyfEOgeYCywCfmFmC2s9vuO4O6vz6eB2u0AeTjGtJTJP0hTg+wSupY3AfbUe1MyuAa4Zoc4FJcs3ErjSHKcqErv4+qRUDSOvV7Ah7qzE6XGt6v3n9Ro1glSWiJn9q5m9YGb/Q9Bz6gwz+0C2ojlO4zn+oKCvh+fOyoiUF/Y9h2ffazJPHwg5EqXhpA2sXwrcAdxhZo9lK5LjNJakxmQoi69TL0WXN0etZereWRUqVgqsx7fL0Wk3nbQxkZ8AewLflvSEpKslfTg7sRwnG4YmIGqtHKOdZlz/zGYTLPxfzXwi2ZAH5ZV2sOEfJN0GHAq8ATgLOBD4ZoayOU5DiL9nw0aq5+AlbHeKLL2U2qEZGQPSNvL+QVEfad1ZtxDMH3I3gVvrUDN7NkvBHCdLhrL4uhbpVNJ+pVd0ZzUqFXweTIaMSOvOehjYDhwEHAwcJGlCZlI5TgOxYlMEKJ4K1amPWprHprizqqyfVqZaPjw6V4Wkd2d9FEDSJOADwI8JUpoMn/bLcXJGYgLGaLBhJ7/dOaYZOrzae5s4A2YFSfPx6LReirTurHOA1wGHAMuAHxG4tRynrVDhfzdFGkV+FXG2mXOrOu/cXqP6STvYcALwNWB+OHrccdqSKBbS5ZZIw6jFvZM0iK/RVD3Y0L8raiLtYMOvAGOA9wJI2jXMY+U47Ym3GB1PZnOs13CETu7AkUqJhFl8PwmcFxaNAS7PSijHaSTxnjEq+b9zX+0mktOLmL5HVAOCJ6OYtL2zTgLeTjjboJmtBHbMSijHyQofbJgP8tg7q+r9V9XFt/UyZEVaJbLdArVuUJiB0HHaloIlkoe3sM2pqYtvMwYbphasOlmSphMZ6VCd/JilVSK/kPQ9YIqkDwI3k8F8Io6TBUUDqqP/C4MNnU5lMCN3VicrhFpIO07kq5KOAdYDLwE+a2Y3ZSqZ4zSIpKCmDzZsHLU0qk1xJ1adCn64UJXkrMaK7eTAeuo51kOlcROApG5J7zGzKzKTzHEyILJACll8O/fdzjU51CGpG/qaRqxnnAwyiU3b+pk4LnUTXzMV3VmSJks6T9J3JL1ZAecAS4C/z1w6x2kwGvbDqZfaxolkIEgJjYh3VRrPkvfvj+/dvqQpxxlJTV0GrCVIvPjPwH8AY4ETzeyhjGVznIaQPLNhYW0zRelI2n0+kYi0wf6aEjBWKUsjGNvdnC+lkZTIi8zs5QCSfgCsAWaa2YbMJXOcBlH0AkddfN0UaSnNGrF+0IzGmVABAAAa8klEQVTJLHh6fc37SJyUqkZZms3UiWObcpyRemf1RT/MbAB40hWI0wl4AsbGUU1X14jmxESM17x4l5Hr1fgM5OHRqeSymzKhOUpkJEvkFZIiNS5gQrgswMxscqbSOU6dlL5kkQVSUCLNFshpGmbZKKukdjuP442aNaC2ohIxs+7miOE42TBoJeNE3J3VcOINaOq2tEnT46Zxm1Wqkpw7y8L95zuNb/pxMvWRdrCh47QlA4PJL5K7s7Ih9ZS0TdAiA2b0pBgQ1IhnYCRl1YrnrMyj33BciTgdzcCgFTVspSPWnfrJox42MwYGje4qRpUmPRKJyi5H7qxKR22WTK5EnI5mwKzMjHUBnTySuBWkbbey1uGRBZrGEoloxEyIZetWt+uG0NHuLEnvlLRQ0qCk3pJ1B0u6O1z/iKTxYfkh4fJiSd+Sf0qOiJmxaFXt3Rs7gYGBksC6Z/FtPJb4syJZX/7+UIl0ZZDfprYuvtk06JXObnAwk0MOo1WWyALgZOD2eKGkHoJ5Ss4yswOBoxjqZnwxcCYwO/w7rlnCtiuX3/sUx3/zDv60eE2rRWkZA2V6Z3VFCRjdEOlIoq/waiyRRHdWkjcr3Hc1VuxRL9ktdd1qqCRBsyyR7BOrJGBmiyDRL/1m4GEz+3NY77mw3p7AZDO7O1y+FHgH8NtmydyOPLoysEKWPreJI/cbub98J1I2sB7+7zqkfuKNaV7cWZElUk1MpFrSnuuDnzmGKTuMyUyOcjTrAylvMZH9AZM0V9IDkj4Rls8AVsTqrQjLEpF0pqR5kuatXr06Q3GdvDMwaMUzG7o7q+HUlMU3Y4dW5MZMY4m8/8hZ7Di+hze+dLi1kDhivcrznTpxbEs6crS9JSLpZmCPhFXnm9l1FeR5LXAosBm4RdJ8ghT0pZS9QmZ2CXAJQG9vr39sjmJK3VlDRO4sfzwaSV46KlRjibx0j8k8csGxqfddmJQqH6dalmaJl5kSMbOja9hsBXCbma0BkHQj8CqCOMlesXp7ASvrFtLpeAZL3FntZIls7x9EgjHdeXMYFFOcgDHdNllf/+grvLur8dduS99A4bmSWqtMKh27o3tnVWAucLCkHcIg++uBR81sFbBB0hFhr6z3AeWsGccp0D9Y2sU3Cqy3RJyq2P/Tv+V1X7q11WJkQrN6Z1UTWE8kYfOf3fsU/3XDoxg21EGjvqNkQkcPNpR0kqQVwKuBGyTNBTCztcDXgPuBh4AHzOyGcLN/IZiSdzHwBB5Ud1IQDDYcTrukPXlm/dZWizAitSRgzJooJpJVYP2yu5dllpurUTTLVduq3lnXANeUWXc5gfuqtHwecFDGojkdQPzlKTXpS91ZefdrtxupG66M/Vn94SCJnjrn1Cj92IjcV/0xd1ZQL3+UunKzIm/uLMdpKJ/630foGxg+6mooi2/rtchP71ra1mN5avnizbrRjbp2dzVYWcXjU0brLdrK40SaI0NLLBHHyZJ4mzZv2Vqe37S9sFzInZWjb8fPXb8QgKVfPKHFktRPXiy7gRoGG6ZhbHcX2/tjHyX5eYyGMVoD647TcOKvkruzGk9Nk1JlPdiwQTGRUjnHxN1jNtRBI4+PUbOebbdEnI6j9N1JakY89VrjKNdYzdp5B5Y+tzlxXeaDDaPeWd3il2e9mp0mNGbE+NieuDvLcmXRluKWiOM0CsV/qqjILZHGEr+e7+zdO7FOM/R3fywmcuisaey/+4417adU1LgSgdaPN6oUj+roLr6OkyVpAr2tfvk7i+paK9HMVPCNbeKKAuvW+MB9I3FLxHFqpNKrE73zeR4k1s4UTQBWpn2VsncCDWSUgHFsSfaA/KoQn5TKcTJBJf977qz6iV/CvFzORimR0thZcUyEXGsRd2c5To2kashy/PJ3CuXsjaa4swq5sxq737j7ysyGzjAnyjOOu7McJ0OiBi6H737bUTZPcgVFkXXvuCELs94R68WMKRkBn+defm6JOE6NlI5CT+7i2xxZauWBp9a2WoSayEunhkiKRoZErj37SHYYOzQqwhq8/0bjMRHHqZHSdyf+tRj9Lrz8OTRFtvcPcvJ372q1GKkp11aVa1+bMbYiakAbaSnM2XvKsLI8WyKjdWZDx8mUQuOS46DIhq19rRahZlK1W8reGoka0LozwVfYviiLbw4fJ4+JOE6DSDLr85SAsZSN2/pbLUJVlLuGrfxIj+IBjf5YiJ+TxZdb9BhVnpSqOTK4EnE6niR3Vp5zZ23Y2mZKpEwX34q9szL+dB9yZ9W3n5HlzKEJEuIxEcepkWExkaJ1jfeVN5on12xqtQg1k9ayy3563GyOU7Q7s1x30HB3luNkSI7j6vzblQ+2WoSqqLaLb3Ma3mxiX6XnmufeWe7OcpwaKf0aTpweN4cv/4q1mxMn0Gon0nz8qkLT3j8wyBX3LqO/zutQCKzX28JVCqzT+g4alSw/t0Qcp0ZK350k33Ahd1aOTJG3fPOOogm02oX49a13PpFfzFvB+dcs4Id3PlmXTJkF1mO/zfL5MRLhXXwdp0EkmfV5fPfXb+1nzcZtrRajYZSLO1VqeLf1DwCw8oUtdR07+kLPsosv5PM5inBLxHFqpPTVGUjQIlEDl7cuvlu2D7RahLoo7p1VnnIWwqRxwYjwjdvquw6ZBdbjubOwXHfQcCXiOA2i0suUJ3cW5DPQ32gqJWAcUiL1DbjMKndWqWt0aLxR/vDAuuPUSOmLnqQo8voBOdisN7+BFF/f5PlEJo/v4Yp/PjwsL3/xx4/tBmBTnZZIRJa9p/IQE6n0EeTjRBynQSS6s3KaxbcNdUiRS7CcO+vA6Tux247jhpWXY/P2+gZcDmY0HqjInWWt751VicEmdfRzJeJ0HMNiIom9s5ojy0g8sXpj0XInT5IVb8/LNe7R+SfFtb469/FUvdeeWL2RH9wR9O6qP7BeeQd5eY6S6OiYiKR3SlooaVBSb6x8jKSfSnpE0iJJ58XWHSfpcUmLJZ3bCrmd9iBNF9+hnEetbbRLs/W2pSVSLu1J2dGG6fYV574nn+c7ty7mE7/684jynHLxXSxcuT48VIZdfMl35oNOj4ksAE4Gbi8pfycwzsxeDhwC/B9JsyR1AxcBxwMHAKdLOqCZAjvtS6XeWa1mfUnG3mZ9PWZF5TnWVfi33NUv1/BFAfenX9g6ogwbY7nH6s+dVd/6rMlDTKRn5CqNx8wWQeKLbMBEST3ABGA7sB44DFhsZkvC7a4CTgQebZbMThtR8u7EG6ZCn52c9KoZZjW1Roy6qLatklS2cS+nRKO5zZ/fVN04miy/FawoF3z+aNazlLeYyK+ATcAq4Cngq2b2PDADWB6rtyIsc5wRSerxVAis56zVbntLpMI4kTQNeiEmUnIZImty9YZqlUi2rXyOdUjTnqXMLBFJNwN7JKw638yuK7PZYcAAMB2YCtwR7ifpXpW9QpLOBM4EmDlzZjViOx1A6QDC+MsUPUg58WYNox0D6xVexLLF5S5/udOP7mG1fv6sR6x35fVBonkxkcyUiJkdXcNm7wZ+Z2Z9wLOS/gT0Elghe8fq7QWsrHDsS4BLAHp7e9vvrXQaSrx3VukQtLw12s3qlpkVla5mvLktZyGUa/hqvU2ZTkqVh3EinoBxGE8Bb1TAROAI4DHgfmC2pH0ljQVOA65voZxOjonenXcfHlihSQ1zq1/+crSjO6soAWOKtCeVLn2586/1utRtieTaYZXM99/Xy+zdJnX2YENJJ0laAbwauEHS3HDVRcAkgt5b9wM/NrOHzawfOAeYCywCfmFmC1sgutMGRK9O1IAkNUCFLL5Nkikt7djFNy1Ds0qOHFgvvQw1K9csA+tYLBt0fm6cCJ7vZlm1reqddQ1wTUL5RoJuvknb3AjcmLFoTgcRveBJgw3z+32Zn8YoLVb0u3wXX5X5HfHG//dHlqxOntWxVuWapSVhwYQiuUMK/karO8tx6qYwBW5heWhd4Z3P4Xwi0KaWiCX/rjTHehLlFAjU/qWfdWA9j0ihJdLhgw0dJ3OSgrfDAutNkyYd7RgTSUvR7aiydS4a61PFNWp8F994Kvj8DFqNI0RXlydgdJyaGYqJlH/Bc/juA+1piRQlYIyVV5pjfcTLX9IAxpVrUgaCcuQ5t1VmFCwRVyKOUxOF+bUrNCBDgw3z1WrnTZ56GDbYcGiUTtX7ijeI1SjabGMi1vKQSNLj0iUhd2c5Tv10VdAief1CbUcdYlW6mqLAb63HqOYLWxm2cHm9VUHvLA+sOw2kHRumeojcK5Uaqvy6s9r7ZlVyZxWlgh/hG75SF9+qlEjqmmW2r3AOSct5IAqsN+tRciUyCmjvZql28pySohztGRNJptreWZUYtOTfI5Fp4DsHI9aTEHJLxGkwbf51WzXh6VZ6v5XbLr45E6hK0ohfizurVYH1vI9YT7oSwfX1wLrTQNq7WaqdUkvkxDnTh9WplHuoFbRjYL0oJhJfUXZOqpGb5tL1xalVqnFnZRhYz3j/tRLN1+KBdadhtGG7VBelaU8ADpw+mW+e9sqWyFMN7enOSha6llTwQ/ssJp7CoxpLJOv5RPIyL00RYUykWUK5EhkFtOPXbT1Epxv3h5daJYWXP2eXJm/yVEtWvbNq7uLb4BHrw7st548uBYMN3Z3lNIx2/LptBHHFkccAaBLtGBMpJ3JciRtWtFytG6iaEevxe51l54pc3KkEIQoJGF2JOI0iFw97E0nq4ltu4Fverk27W41FXXzL1KmlWY9fl6SEmuWou4tv6XI9PromIR9s6DSadm+YaqWoZ05Jb6wcvvtAe1qNVnZhCKHi53CE61/6yOali2/xwMrWu7OS4lHBOBHPneU4NZMcEymuk5ThNw+0ozurHEWzAGJF96XaxjdufQyOoEXilzDLzASDscB6nhhyZzXneK5ERgEd1C5VRVFMJPq/5KXPWxffdrRE4g9YVtfTigLrzcviW7p96fS4rSZJHUeWiMdEnIaRt4Yya5K6+JYOLszjFyS0v+uxyGNVMe1MtYH12gYbZkkerMZkd5bHRJwGk4NnvalEDXFXBXfWUN1mSJSevMmThqrTnqTw15c2jvFxIs3UIeUzEYfurOaJkpooAaPHRJyG0YbtUkOoJuFfXsjD1209VBK/Hiswfl1aaa0VzZ1iOZ2USvIuvk5jyOuAuqxJCqyX6pC8KpWceGqqovw4kZJ6UddrVHXjGz9GNV18syTfloi7s5wGEL1roy0mEtFVXofkNibSjpaIpQys13NqRSPWBytUbDDDR6zH3Vmt/1BLOm6UEcAtEadhtGG71BCKYyLJWqPdA9l5I20W3+HbWcly8fricSL5uGd5fXaCVPA+n4jTQPL6sGdF0vS4wycTymkq+Db0Z5UNrJdc9EpnVnraw5dr6+JbL5W6+A5aPt2i3sXXaTh5ayibRVGupvy964m0oQ4polLak+hjJulWlDZ4wy2T2hIwZslgHoasJ9A/aB5YdxpLTt65phH55YsHG5ZxZzVFovTkxVVTDaWpQMrWC/9PCqqXjv0ovQ7x1XkZJ5KHWxWJ8I13zSmU9Q0MgpoXO3IlMgrIw8PeCiq6s5orSmqSXI/t6o4sZ/2lsUQqubPycj3y1Dtrr6kTOHzfaUCgRJo5NXRLlIikr0h6TNLDkq6RNCW27jxJiyU9LunYWPlxYdliSee2Qu52ZbT1zkoaj1DOt93K9qhvYPinYtJHdk7azLJYmaVS66/SeVRSGqXr82KJ5C131tieoDnvG7BRERO5CTjIzA4G/gKcByDpAOA04EDgOOC7kroldQMXAccDBwCnh3WdFOS9EcqK4vkr8sfWvoFhZUkKv+NuX8LNKFUMpc9sNTGRLBv20sB6q/m3N+5Hl2D/PXakJzS9+0NLpFlKpKcpRynBzH4fW7wHODX8fSJwlZltA56UtBg4LFy32MyWAEi6Kqz7aFYyvu3bdya+5O3E39ZvBeCye5Zx4yOrWixN89gefuEnTUo1pjv4buoOX7if3r2U3zy8sqnyRSR9Uf/s3qeGlR37jdtzqQQjLv7jE4Xfdz/xXOF3vMEd19NdcC+O7+kedj4nffdPRcsr1m7mmK/dVlheu3l74ffHf/lndhjbXVaevoHGN55RAz2uZ+i42/sHC8/R+DFD3+M7jO1m8/bmtB1HvWQ3lnzhBAAmjR8DDI2kb5aSa4kSKeEfgZ+Hv2cQKJWIFWEZwPKS8sPL7VDSmcCZADNnzqxJqBfvOrHQGLUrs3efxB1/XcMRL5rWalGazqtmTuXI/XbmnDfsx5I1GzlxTvAYveeImWzY2sfRL9ud1Ru28cTqjS2V82XTJzN/6Vo2bO2jd9Y0Jo7rZtO2Acb2dLH8+c2M6+lixtQJLZUxYvqUCTz2zHrOev2LuXfJ89zx19UcNGMndp40lrWb+pg2aSxmxrotfey+43h6Z03l++/r5StzH+Mr7zyYXSeN46NH788ph8xgbHcXh82axkPLX+DAGZPZc6fxjOvppqdL7Di+hyk7jBl2/GkTx7K9f5CN2/oryrnPzjvw0PIX+ODrXtSQ8/70CS/jdbN3BeCzbz2Abf0DgZVr8N5X78MR++7MO145o1D/urOP5F+veID999iRc96wX0NkSMN/vv1A9po6gTe8dDdWb9zWtNiRsjqQpJuBPRJWnW9m14V1zgd6gZPNzCRdBNxtZpeH638I3EjgdjvWzP45LH8vcJiZ/dtIcvT29tq8efMack6O4zijAUnzzaw3Td3MLBEzO7rSeklnAG8F3mRDmmwFsHes2l5A5GsoV+44juO0iFb1zjoO+CTwdjPbHFt1PXCapHGS9gVmA/cB9wOzJe0raSxB8P36ZsvtOI7jFNOqmMh3gHHATWEPmnvM7CwzWyjpFwQB837gbDMbAJB0DjAX6AZ+ZGYLWyO64ziOE5FZTCQveEzEcRynOqqJifiIdcdxHKdmXIk4juM4NeNKxHEcx6kZVyKO4zhOzXR8YF3SamBZjZvvAqxpoDjtgJ/z6MDPufOp53z3MbNd01TseCVSD5Lmpe2h0Cn4OY8O/Jw7n2adr7uzHMdxnJpxJeI4juPUjCuRylzSagFagJ/z6MDPufNpyvl6TMRxHMepGbdEHMdxnJpxJeI4juPUjCuRBCQdJ+lxSYslndtqeRqFpL0l3SppkaSFkj4clk+TdJOkv4b/Tw3LJelb4XV4WNKrWnsGtSOpW9KDkn4TLu8r6d7wnH8eTjFAOA3Bz8NzvlfSrFbKXSuSpkj6laTHwvv96k6/z5I+Gj7XCyRdKWl8p91nST+S9KykBbGyqu+rpDPC+n8N53aqGVciJUjqBi4CjgcOAE6XdEBrpWoY/cC/m9nLgCOAs8NzOxe4xcxmA7eEyxBcg9nh35nAxc0XuWF8GFgUW/4S8PXwnNcC/xSW/xOw1sz2A74e1mtHvgn8zsxeCryC4Nw79j5LmgF8COg1s4MIpow4jc67zz8Bjispq+q+SpoGfI5givHDgM9FiqcmzMz/Yn/Aq4G5seXzgPNaLVdG53odcAzwOLBnWLYn8Hj4+3vA6bH6hXrt9EcwE+YtwBuB3wAiGMnbU3rPCeaseXX4uyesp1afQ5XnOxl4slTuTr7PwAxgOTAtvG+/AY7txPsMzAIW1HpfgdOB78XKi+pV++eWyHCihzFiRVjWUYTm+yuBe4HdzWwVQPj/bmG1TrkW3wA+AQyGyzsDL5hZf7gcP6/COYfr14X124kXAauBH4cuvB9ImkgH32czexr4KvAUsIrgvs2ns+9zRLX3taH325XIcJRQ1lH9oCVNAq4GPmJm6ytVTShrq2sh6a3As2Y2P16cUNVSrGsXeoBXAReb2SuBTQy5OJJo+3MO3TEnAvsC04GJBO6cUjrpPo9EuXNs6Lm7EhnOCmDv2PJewMoWydJwJI0hUCBXmNn/hsV/k7RnuH5P4NmwvBOuxZHA2yUtBa4icGl9A5giKZoeOn5ehXMO1+8EPN9MgRvACmCFmd0bLv+KQKl08n0+GnjSzFabWR/wv8Br6Oz7HFHtfW3o/XYlMpz7gdlhr46xBMG561ssU0OQJOCHwCIz+1ps1fVA1EPjDIJYSVT+vrCXxxHAushsbhfM7Dwz28vMZhHcyz+Y2XuAW4FTw2ql5xxdi1PD+m31hWpmzwDLJb0kLHoT8CgdfJ8J3FhHSNohfM6jc+7Y+xyj2vs6F3izpKmhBffmsKw2Wh0kyuMf8BbgL8ATwPmtlqeB5/VaArP1YeCh8O8tBL7gW4C/hv9PC+uLoKfaE8AjBD1fWn4edZz/UcBvwt8vAu4DFgO/BMaF5ePD5cXh+he1Wu4az3UOMC+819cCUzv9PgMXAo8BC4DLgHGddp+BKwliPn0EFsU/1XJfgX8Mz30x8IF6ZPK0J47jOE7NuDvLcRzHqRlXIo7jOE7NuBJxHMdxasaViOM4jlMzrkQcx3GcmnEl4jgpkDQg6aHYX8XszpLOkvS+Bhx3qaRd6t2P42SFd/F1nBRI2mhmk1pw3KUE/fvXNPvYjpMGt0Qcpw5CS+FLku4L//YLyy+Q9PHw94ckPRrO6XBVWDZN0rVh2T2SDg7Ld5b0+zBx4veI5TmS9A/hMR6S9D0Fc6R0S/pJOIfGI5I+2oLL4IxiXIk4TjomlLiz3hVbt97MDgO+Q5CXq5RzgVea2cHAWWHZhcCDYdmngEvD8s8Bd1qQOPF6YCaApJcB7wKONLM5wADwHoKR6TPM7CAzeznw4waes+OMSM/IVRzHAbaEjXcSV8b+/3rC+oeBKyRdS5CCBIIUNKcAmNkfQgtkJ+DvgJPD8hskrQ3rvwk4BLg/SA3FBIJEe78GXiTp28ANwO9rP0XHqR63RBynfqzM74gTCHIYHQLMD7PGVkrHnbQPAT81sznh30vM7AIzW0swc+EfgbOBH9R4Do5TE65EHKd+3hX7/+74CkldwN5mdivBxFhTgEnA7QTuKCQdBayxYG6XePnxBIkTIUisd6qk3cJ10yTtE/bc6jKzq4HPEKR8d5ym4e4sx0nHBEkPxZZ/Z2ZRN99xku4l+Cg7vWS7buDy0FUlgvm+X5B0AcHMgw8DmxlK5X0hcKWkB4DbCFKcY2aPSvo08PtQMfURWB5bwv1EH4TnNe6UHWdkvIuv49SBd8F1RjvuznIcx3Fqxi0Rx3Ecp2bcEnEcx3FqxpWI4ziOUzOuRBzHcZyacSXiOI7j1IwrEcdxHKdm/j9DrhNM/+wx1AAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
-    "You can try and give some trials with different architectures, parameters and hyperparameters for improvement. "
+    "plt.plot(reward_list)\n",
+    "plt.title(\"Rewards vs Episode\")\n",
+    "plt.xlabel(\"Episodes\")\n",
+    "plt.ylabel(\"Rewards\")"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {
-    "collapsed": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": []
   }
diff --git a/English/RL Tutorial 2/Experience Replay.pdf b/English/RL Tutorial 2/Experience Replay.pdf
new file mode 100644
index 0000000..e89f1b8
Binary files /dev/null and b/English/RL Tutorial 2/Experience Replay.pdf differ
diff --git a/English/RL Tutorial 2/NN.png b/English/RL Tutorial 2/NN.png
new file mode 100644
index 0000000..89cfd29
Binary files /dev/null and b/English/RL Tutorial 2/NN.png differ
diff --git a/English/RL Tutorial 2/NewDQN.png b/English/RL Tutorial 2/NewDQN.png
new file mode 100644
index 0000000..24bdf1b
Binary files /dev/null and b/English/RL Tutorial 2/NewDQN.png differ
diff --git a/English/RL Tutorial 2/NewDQN.xml b/English/RL Tutorial 2/NewDQN.xml
new file mode 100644
index 0000000..ba6b4c2
--- /dev/null
+++ b/English/RL Tutorial 2/NewDQN.xml	
@@ -0,0 +1 @@
+<mxfile modified="2019-01-24T14:16:00.367Z" host="www.draw.io" agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/9.3.1 Chrome/66.0.3359.181 Electron/3.0.6 Safari/537.36" etag="kQcbt-iKJ_vG-MMr7bHM" version="10.1.3-16" type="device"><diagram id="BS7flWhFK6w_smFJ3vTu" name="Page-1">7VvLdtowEP0alumxJUuYZV59nTanOVm0XQpbgFuDqCwK9OsrYwnbsgmOYxAhWcUaybI0c+fOjER68Hq6+sDJfPKVhTTuASdc9eBNDwDXAVj+SSXrTIJ9PxOMeRSqQbngIfpH9ZtKuohCmpQGCsZiEc3LwoDNZjQQJRnhnC3Lw0YsLn91Tsa0IngISFyVfo9CMcmkPnJy+UcajSf6y66jeqZED1aCZEJCtiyI4G0PXnPGRPY0XV3TOFWe1kv23vsdvduFcToTTV4Y3N/Bz4u7+/4Fvkk+4Ie7u0//LtQsf0m8UBtWixVrrQE5i1S2bFwtJ5GgD3MSpD1LaW8pm4hpLFuufCTJPLPAKFpR+dGrURTH1yxmfDMRHI1GIAikPBGc/aaFnhAPMcKyR62GckFXO7fpbpUnUUfZlAq+lkPUCwArfSvAbfW/LJhPiSYFy2kZUYAZb2fOdSoflFqfoGJwdiqGPjJU7NtVMTw7FSPHVLFlFGs+Lqo0lEypmoyLCRuzGYlvc+kVZ4tZmCrxxikrWWqBr3+k8ndIN38W+25W6qWstVatX1SItQoSZCGYFOVf/sLYXM2RrTVd4OP6l/thCx7QRzbuqaBD+JiKfUxatSenMRHR3/I6OreOd3YOYNI4xLYdwK2qtL0DWAAyeiZA1avfWCS/vLWTB8pEBV3DAJnjqLcMG2yX0d4s6OyQb0ZX+8hvksG8KFfADTkdduEyl5yTdWHAPHWFZLdHIWQwn2Pk9U8bLx+yFXTqdvjs3A65JxZw/BoV41ik+mEb++e6xn8WTHdcJBtXupQD5PZXead8Gqu/m1mGWpAIIqiWyqUOzZFSln1Siw1Ll117j7VN4/oBrTfu0EcecjriVFzm1K2xC8Z1QY118cFItUnNMgsv0yMM2QpikiRRUFZlU0asqqWwbVSzay17Zm6gmajiUnqKjKgruUFNkmFMdOQkw22SX78Z6/F4dDRjNUkJ9xiLriJRKE9l66dit/Q5L07Thq5N29a0HeY1qGFeYxVoF+6gDBA0aIk06ZfGTN6RodYkDToA1LqGjDUoAIN0cL8t6XiWI0T//CNEhdhR23AOLRtrd2q9zX3v3fqM+FBFTkioP6rNg3Hg0+HoMKdqbk0efNzLkQ7S4GcGawyeFK43rW+UR1IBlHdPyPqSc28M908qiPdNH24dxLGZeB6YDUAHyf0zIdjfHGLkEHzneH3LMASvDIbmObZrJqWHRqH9qgUjA4Q5MVoCIXxlILTPhYMOUHjwOngvHqwWNSaToLZoGNjFgk7SbWDBclHsN8QZtokzaB5+mEn6C8GZ1+Q3aKfPOZ5dzjGw0JZzgDkRODIY6upBszQH51iaQ7+seeuluVdXF70KU5hXwfZNYb84sHWl0Zh+d/xW4zj8a1YArY9GgXmOfeTLM6+LG40TCMYnFY0BbIkG6FpOzTq41TgFNFjlBvPaBJgpVWM0GJdl/SPfdnp11yZvaLDEDa4xkWdenR4aDedxVGQ3UnTGDa4xUaUAbI0G2cz/Ry0bnv+nH7z9Dw==</diagram></mxfile>
\ No newline at end of file
diff --git a/English/RL Tutorial 2/Q table to DQN.xml b/English/RL Tutorial 2/Q table to DQN.xml
new file mode 100644
index 0000000..5dad3e8
--- /dev/null
+++ b/English/RL Tutorial 2/Q table to DQN.xml	
@@ -0,0 +1 @@
+<mxfile modified="2019-01-24T13:59:03.688Z" host="www.draw.io" agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/9.3.1 Chrome/66.0.3359.181 Electron/3.0.6 Safari/537.36" etag="vgCu4wOsfuI5ovyf4h6e" version="10.1.3-16" type="device"><diagram id="BS7flWhFK6w_smFJ3vTu" name="Page-1">7XxZk5vI0vavmYjvuzgnAAGSLot9l1gEgjvEvu8C9Ovfwt09ttueM+0Ze3rGMYpeqKSogswntyJVv+zoauF7v03VJozKXzAkXH7ZMb9gGIpgJPy3UdYnCnk4PBGSPgufO30kmNkjernymTplYTR81nFsmnLM2s+JQVPXUTB+RvP7vpk/7xY35eeztn4SfUEwA7/8kupk4Zg+UQ8E8pEuRFmSvsyMIs9nKv+l8zNhSP2wmT8h7dhfdnTfNOPTUbXQUbkx74UvT9dxv3H21xvro3p8ywVHXdtJk6bv/0MyA0+amiY+/vM8yt0vp+cHfr7ZcX3hABwFMhs2qDnNxshs/WA7M0N5Q1o6ViVsofDQH9onCcTZEsFJqTgrS7opm/7DQLs4jrEggPRh7Jsi+uRMSN5IgoRnnu8m6sdo+c3HRH9lHkRd1FTR2K+wy/MF+PGZ38+A+5X/8yfieyaln0juheY/Ayb5deSPPIUHz2z9BhZjPx2LSZR4xeLD+7J499OxeL97zeJ3RvGLPf6UpSG0lM/Nph/TJmlqv2Q/Uqm+mepwYyKDfM5kyIV+vW70/xIvTffTc8zyfNFTa31u5dE4rs9Owp/GBpI+zqw0Tfs8xtO9bjf4v/kPn6eZ+iD6Hw+OPzsdv0+i8fcs6Zfy7KPSH7P75/fx3aWD/3QK8NqM78j3VgD0S5b+cQV4ByATfxKgz5eemwzO/NEXEJ8bqh36SgBPivN81SsZ/Hobf1wsxE+H/Nfe9f2R/5YI5h+lCuQbbfrue6gM6Ht//aRDu6nC8NsatT+8snzIq7j+2/rDg6c7+K5qR/50arfH/2YOZ/8VFpPluPGn+SD/j7wmu6l5OfGf4YMqAdgBPbbLx5PwKHn+/2GU2wvBD8asqV/I8F5vr7tC2tOcr0boP1JeCf9zbf8dALyW9yGIvi7v24HACeQ7BRjk5/ImDl/KG8W+InDyRwn88KcFDp//LQIfRn+Mvkne/zjpYoe/m3SPv28xh9Rvt8Os+rAo9aVhfD7BhP7oQ4k/NTEuyeJfMNpASv7ElKk18zM4B+DDh0U1y0kbX1gO8fkAwPnQyffD/gSvOkpmfLTt+Ji452PvbxQ0jI+b7PFVzaXOr7qobvfyOvTY8Q6noOTsDv9eyjP8G9cbPzCq6zYaNg6xsBwZHdFqD4N9OXY5CSkOByBjZj6cP3Tefmbw7+ffz7+f9/zQT0r4wVIAIG4mBBxpahEHmaGSObHk8gGPBSZdQG62kO4rXLYkD3+EdP8s5CvFZOkiDZ0lFWtqdeUDHl+V8kHlawvpq8fQcyDycqpfmhNLAVtXcwo0IqAZU0dlOksKRJApYOhiRoFSB3SFJpyTgWQOVBbkiyAMiYf6Jt2VuuJQdJuYpd+ubnthUA+YGEXb20uHpLuILFXsNOWSpUCPANsmISsNTA+c2CgQSymLWdcq+IxOjyLabkW8ijfpUT/3ZrEnI0UJEOexnR6I/XiHU0mLkaJLcqFDGdSUcGmYMIosD8yT3nekd9mnAFuNfXM0oc+lSJ2lTM+W51gS6N2IWyfberKM+D0y0mJpXNhMaWCVEuXq8Dg8o52rSjZwp3kZg3t1hnEUVSTxWerIxnZ0ATvETSaEbhBvvkbijqPMcLNQ1wkLe4LYjJ5mGCUp7CjFY21WmAQvM6nILCWWI3kaA8KuAVlpZ6lHUpzdMXxLFokrdsk1M65Xv9UoU1auO67iohsRhanu2QvR0Wd5rXQex7u9ZesgNU8SrwmVjhVA1AvUcIpijWiP5S1T9i4cisfXpDJ9HoKIMmntKhpoArhV2IdZZoJLEFDEvhEurWizYnkqjNw0DcPXFiHbJVDgQguFZKQSmzb0LcxvHSmfdCPr21YyHH+qyor3O9L3g3CCNJLm5alrWsnryK5qnMQCXTdVZNeqkCHdiplVNQBl6TpkQuys7/vbI3OmaYAuinO7nu32ZtkPg5qZobxoXGQgNXJWrN6iNj0oRE5CUG+v3G4XZ5jGMR0XnAWge3QzuAhedJ/GKTG4E6EF16h2C2XQPZ0BlEEk8Hpe0iql3+8Px3q83++DDCfl4S/Fa8VeB0EUagDMd9wUzpTI6lrAKsjCRUxyuau8g7KASR6IDkCRmPO6mo6gaIwoe5CxQaGDKIK6umb+PT9Lona9g6AfFhiscFaa38GaChzDrcygB3rKWciCFMXtERuCmcAuiMY6vZoveoeyiqbYgmMcjgftcnw8DNsLHksbMjrDoIe+ACHQaGDyzRk9324BOM3hoE3obgdzYW6c3bM/r7N+xFkmvkXRMIbw9li6CRHzsiT86DEZ3XL7Q4DEZohPhHcVzoombuERpW2xgx4UyjhTub2r73kteMUERUc5eo9ZCY5xTMg/evaAQIBTiI9lhL6HygjxCIAgUsdDhPh7D8cPupMdM2ukli0KKeLz2cf3ByOkrqEqMBkXx5CmNgcd130nPMco3VWH6HKvmgCHKkgRRFcLR5YRryoYZqD2hyeFIphHrdlO6BBVeDkHXGTnM6WK1CKgbALUVtRQOikAzY8gZ2ITgJylgU65D/18ocuCldBsBy69DuGArMGBZ3PLvfpG78CxVZHuNzMzys7C8u0pWgX7rPmY0yH6dZUJN96vbqq3p4BkN4sRSPqJGuxeTMlhOZL5OXvkWW3Q2/1TvBuVUgEoWmLL5Wpkg6Jk6AOgtJo4g7GZh1PrmyrTAIs/DQ612T+QeMgjMVLduYMZB7Nyc3zGNBABmnZ00Cg3XQGDeJKoc6vOqFQNdlmCrYXosoDmNLFIOXEUMx7P8wc168vFnYqWF7a5zvgFWKIn1Kvo7VQz20BnBtvj3m4S3axrKdcB3tnmFLTO1LLFcEVq2QmUrlQslYgpJL18gBJkjqrzlbIboEcJLgGRYafKU0qxVm2GfYgPnKJ7g8w8Crm5Q4Hmi6T1FX92z2xg764hmwhKTk8JNlvXEJG9hbiqhMes7upTXekKFzazHmt0cJgtnO8fIpXoUFrJYajyuWSTo2iJwL/bQ8OJt0q13clod3ku1ebjEe9WsbuYS2ZwR14BicBwjZkx3RJZu4fMcuxaMsqJMaEvhfZ+OckzYXeW614wqNRq2gO+ldjWp9XqsqVpVLHH67OeJYfzDSTHZjyoWumn+2ilOIqhuJRZwfVwUOXxWHCPU8sNZKfvsMqoiY6NJ2qRx/gWHjlg0FRh1mayJC6enyms36w/HH9wBkS01yk7h4t0Ww00DXJGHMCcKMSSBPD5VRmRE2m9T01o6I7d1Nfo4LMDzG4UbhQuCadFOTUzItbu70HjcMdqzKKwhJ5y1kb/zAln92BDA6nPJ803rFEbcdHpm2hLIgq01vcl1V7lYiwiTgTgsKrCmWmOzuQoyS3VGmdLE7hASU4tNU4m0V0nW2ZbdFXcy0iBEZMWhdez0Sos/3IrnPNYKqMniMRDtiVOnOQhM1BOQbzCL/TVu/sHrsJG52grRtx0aK7YwNNbFr/xkd8tmw+oD7clP5oEKzD8msfYaUrUUXTF6+UginfBAtEe9sojYryJGbpxyINY3SxwsvhlkrNeGlPAvWF4OE/KnSrDTrCjRdzrcl9ZoFLNY27xKsdh0Jw+6AZQOHC9OzsGfjKnicDDOICel3kFrHigMerKa6QbQwDOKcixGMLU260lYrd8SY+3pZAPKu7cW/kAc2LqRk6qGR1LGyajVAu2qI0OHN7p6r7y1ocyZ5SWXcgV2e4Xu57K262GGRuIgTrRFZ8BJ0Gy7TK7TzJiWaQmdUKrS9FhZWJFrTLDUloXswlR9/mZzX23DYx9HGS04uR6eg8Km6o9aUsU4a8cOtYVPWfGjnSjZjABlm+GieUoNWAeF/GUDsUecY9ZKus4uc3K1/Wq505jY9JshUUlcI2QBYnBO0g5yPLU63eaRPm7ZcSGyh5lcJpiHRt1bsSos4ndTwFb89zAd7ieXTGcbPhVU60gtc63cWrcfbGLYxSo80XvEAJOuNR6rypiIrgje/OrvHc8kUS5I2c7iA+Z4LiVcBCPYJxonskd+ThhZuLmu8MliurpSullsfk8jUd7ciK7zSbfnIfYoGF61mPwmD2Cl0IlSahzfOxzLYOqm0Oznyxnu6bt/JYuAbl/lOnmgYnwnunAgCzamrx5BtHj3u82LfQ3ix4Gx7Tlstoyd4bmKHOyTsEusdqEq3ptcx+OLVNM4e/EoLkqDhb23gk1qOUOY+dyV2hWuFqakOl24p3Sg4lbUITcAVc5rYgmDIaRlHd74HmiBZIVkhdbqod6BXKTnsgJu0d9eQgPMX/qgm0hoDwcM68/PsqGcKg7r4EHzUsPbmLtSpQv8wPwuHc0klVVZpPRzq655Bizv1UDAg2Kx8IAe7bEGy36aFRXvLNf0jIKVnaiHdW0bztSIIPyMLCTfhXUWFfKzVHUUn3eojhcpHauDkzxLje8LNV7/cPSQ6d0500hzf6MWgNK7Pa7J5fdbusS08bH3kTuh7zOd8cT6W1PsfE23KKwQ7gF+FQEKksGIh+B1jWBoPKAzpOAqRM9HznG10MJYt7AG+lINNIJ10RDu+ldxoQnQjdOuCVOhC2JxeKape0DvmKp6so6vhgeOJXrVDIVmZM6s4zWEUhgYIrasaimBxeHuTkmw8TqDMwTKwxcx9qJfrmIGRekffUINlggHrR7XrT3vHU1fI7Nec6eAsqOU3ttfE6q7WnEAiN7hBorumgqhdOt0wqndnjBO/Fab/JCyOHQL5UUTz08P8WMwIYapIJtOUYrNj555+EmzkrtfTCGaHNDAJdB35SJBW4Hsm1zJZoujSFU+ZqCyy4wlDQiNevUdqJPUUAZrDqcSyE0bhS0veCU9VYY9YxTDKJ0hokfIzashe1gZPtQDzSTBhSSwakIPkssPDC0JdsfrFBl3IQWYHThLj6np9luwPXDFQ+PCeMwE+Rpg2Vla2fzuJwjp84IE3diauHyA5ItApVNB2tbUNKsM3EU9VATg6gQ2RjkehQ3aYS5rHERhOOVipq7G5aByBo2ohtFcVJzVI9HnDUycLJbO1wsI5/PSz5n7eaNOjdizBSqACWK7JUWczx4pCt5uRmjYQ9H92aURGiZRbopMXEzz22GpgaHIG4c6XZUDm7BSZcrzO82a8HztCQgqyQa4HIdkdIqw4vVcjY8J8LUlyvoCDcaFkaN9EphbY1a1KKxoZAZ7AkGvDuTrVY7d8gF8fjuisryJTDGerRPly1chYmncqEcBbkKErF79NKlSC5IbXsLn5qCe6TKh9Lu1jptPXN1pEq+YoJPr5ft4ShokKkresjMJSZdGIxwtJfuTXPlTl6msfCpYOQlYqYjm+6kYl6CH9qkkdqa5aSi0r3KqQzZp6/ObUv7ugtt2IVpmWeqqoTKFZPEa2pbahjWqd1QZM+iKzcqVvQZhcRqOdUpv6W8Icy6uOsdJ6RpGlOe7JyOy1xdBK4utK7KNq7pngz+lHjDSXce3FKyo4vuKa5VTdFDNktVOCGKVWGdwwCA87msKcARzyRiNAjqyI0P1DS8IhNgTBPtTt6IEh6tE6GJ847QkSyIGC6iKe5+SDw7elSiydsX5XGClpa7X2JHvI2RCuhKAjkvXh8FRah8TA3c7DWODV3iIlmn8/3gpwzPp0wbpVJfGIScmRtbHPkRFzCWBRjMcoNYtZP1dJnde7blQw8ValmE2eu8OrmItojmtfcJXnXSHJEFYShnzNl+6EKUL2Nm5HvndMs8L9cSsGgQfJQlQUSUobsQWcbcWHCr5EK+hr7QUahZdJmQtrt2PpmUxKFd71yloEs21w2jKyGpbIc3iqt2DXaRwETlBMLyJqzqMTy21MI4OqO0W3xcR3ULLUCrS2kEE9WI2WB3PeFERNse63b+SXz4Dd7vOronrh4nV27egZgmoH3sopAgzrkrsZst7iKf7aIxa4uSaK6PUbp4s1zrCwWDJwTIad0ZdRp1GyqjvbZzlZvcnK1LCXaLKwqXUunkzQGzaTfPhjtezQx1OtaMePMKHU4wd0DFyTadjd4qbBnnLnHYyf0lNyvv6LGT1NWTBkMMW97cRUjKU+mXp6GTJ9SHGSR19OU7Sxax5se4R272MpQ3p1R9OL7L/W5PbG3m2s+gGYC6FzSb4jxd6DWRrmBKkJixmESPtUpRcTC0PWsAJaPzZlA12erP0uDh8jCdJK550AMM7uuzTadMiqdq224hJ0PJzvaEK7QOFD8p4p27qRD5jwVVROj4qL0iYqlo45sMKjrlYll07pzreSe5s929bs+uXZONLWd4nXY3kzoZjnJoEWv29pmIh8F0U4vVlsJ173RuIRDJ/kq0XRnRJD6JRiN32yxD1qmy3CUabkXaxCN0Vw4nvinYqckoUwnvDlGwBwfXzR6oF52nxVbAyJQasZRuOrG92LIHo0Oqvg60UiXio2tJsTInbLz7Te+US+AX1Fmt5Pac9a4quitByzAgkY1tdaBbc/IxlRHpHA1u8xtip4vOTBWd0qwk2/sVgzXVvtPIPsrWhSZkHMP8fedJtzvDerfulPIStjB3eeOSnTQkjBN7Y905p25htM0mT+cVHc0qnLizMVi7D8xmCL5mfV4HA69H+y7ZZJ7pN+g9SJg8Xa4bLIPz2B3OU7JZ3mmvBixQnWxlEB5DYBANiedxV4mNsfmF6brNdAd21WGuMhFlDfUTTKReTp21GUwT1WXLlKeO82aSPc36SSjQU1tTTdx1JTjTMzmYtAeAKbNUvfCIlPmD5mbQRZt2x3d2G9xo85IzmyspeXNf2WoDM3KlC5asG65ohVMlaGQITH8prVCuHATvrsG1uhaSxa67mm0Hc7zcHqyBasdj2GFUP7TOVqfAOfjcXqMHuBJLnbQTgK6MmlvXetAesbrebS02/7j5jJuzXG/zQwDWXfDURQFOLPQXi4uIhxClc8u4Arbg4mEVjQBaHQdmiuD0oQJ6+3l6m0U1PQwgnl4AIh8+3+n17u9XAe9eaiw/ff+H7skfVUbzlirVOgRb0TpsBaU/DFnw+bvUt9bAfMmWT56b+Mprzxfan6wGI3a/UUTxMsRTac4X1WBfKSt7NdBfXFaGvqWi8l9h/e8KpL9MWG8pAvwdYUVLNn5SkAxb7nN5w3b8sRx5a7xUI//RKua3AuOtRZ1/vDr5rwHaK1tMHP8gzl6VBBP4XwyztxS9/QCYfW+4vBcMXpkbcv9Hzc3+nX3D1yrzfjLf8IVJJ/6oIyffWVi/XVX3a9mb/v+eK+Lop1LI///12rgfVd8a+tEh/mpFHBkcolv8Y8rKUfTLiPiv/V7cd4iH/6TXJrFv8tsfWueozyADPuQs39k6v3y/9Xed+eFv5M33rxX6D3pz8nX0+YMNA/YdIvx39ubYzwCD118gQl9Hhz8aB++fPJCfW6H/Ih/N0jtZot2/lugvROB3zStw7FMM/gf5L7Lb/W8gwsZrKH0CTnx/eIVOhHxndB7eiM7jv+D88+D8DsnOnzSPOHn8uyFw/09AIPoq9SL/6LrLr5uxvAz0Fy+8YF/L4b4Vg5/gCd39xf72d3GCvydOdq8WZojXKd8/xVS94StW3wCTH7Sk+7tYeN81utdrtf9QLLxA+h+OBfJfLHwHLLxlk6u/Pxbe1UcQ2HfCAoq/MxiwPw+Gtwr1Hy+s12/h8ddvAX60sN5pifjtWvu2TOFne+P2l+Pgy7VaK1rGL7AwbsTPpP/5K5W6qaNX71+eSX6ZJfUGISjgLcCntnctWeCX4PlElYVh+Vsvez7fD+H5Xr9j9RL5lb1I8K/ABkN+GyF/boO9ry2SfuNmJPs37U2hw66Wfyu3/Sn8amNufRvaXz7bm+Kb96v4aXCBkthnwNh9iYsj+tvm5Pvj4mtLl9+Ii8ObcMHo2r8I2EzBK9+Ok18gYP+VssYfZxnesDbzsq9JXEbLs6+m/ukxFo58Locv6hre6lvxd65m2X25avJG39pHUIs/mOondD9vbAZ7E9QvxFbhu23n9qTpT4UQzxpTRvH4FUUat63eqAEqUFYn1od932DS8320hkQ/d6gvQ7yXQ32Z7KfmOk6SXw8bvz/Xt6///bqT9pNqfNyPfMf+Hw==</diagram></mxfile>
\ No newline at end of file
diff --git a/English/RL Tutorial 2/Q2DQN.png b/English/RL Tutorial 2/Q2DQN.png
new file mode 100644
index 0000000..cc112ea
Binary files /dev/null and b/English/RL Tutorial 2/Q2DQN.png differ
diff --git a/English/RL Tutorial 3/DDPG.ipynb b/English/RL Tutorial 3/DDPG.ipynb
new file mode 100644
index 0000000..50f8245
--- /dev/null
+++ b/English/RL Tutorial 3/DDPG.ipynb	
@@ -0,0 +1,714 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Reinforcement Learning Tutorial -3: DDPG"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### MD Muhaimin Rahman\n",
+    "contact: sezan92[at]gmail[dot]com"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In the last tutorial, I tried to Explain DQN. DQN solves one problem, that is it can deal with continuous state space. But it cannot output continuous action. To solve that problem, here comes DDPG! It means Deep Deterministic Policy Gradient"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- [Importing Libraries](#libraries)\n",
+    "- [Algorithm](#algorithm)\n",
+    "- [Model Definition](#model)\n",
+    "- [Replay Buffer](#buffer)\n",
+    "- [Noise Class](#noise)\n",
+    "- [Training](#training)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a id =\"libraries\"></a>\n",
+    "### Importing Libraries\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/home/sezan92/anaconda/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
+      "  from ._conv import register_converters as _register_converters\n",
+      "Using TensorFlow backend.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from __future__ import print_function,division\n",
+    "import gym\n",
+    "import keras\n",
+    "from keras import layers\n",
+    "from keras import backend as K\n",
+    "from collections import deque\n",
+    "from tqdm import tqdm\n",
+    "import random\n",
+    "import numpy as np\n",
+    "import copy\n",
+    "SEED =123\n",
+    "np.random.seed(SEED)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Important constants"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "num_episodes = 100\n",
+    "steps_per_episode=500\n",
+    "BATCH_SIZE=256\n",
+    "TAU=0.001\n",
+    "GAMMA=0.95\n",
+    "actor_lr=0.0001\n",
+    "critic_lr=0.001\n",
+    "SHOW= False"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from keras.models import Model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a id =\"algorithm\"></a>\n",
+    "### Algorithm"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The actual algorithm was developed by Timothy lilicap et al. The algorithm is an actor-critic based algorithm.. Which means, it has two networks to train- an actor network, which predicts action based on the current state. The other networ- known as Critic network- evaluates the state and action. This is the case for all actor-critic networks. The critic network is updated using Bellman Equation like DQN. The difference is the training of actor network. In DDPG , we train Actor network by trying to get the maximum value of gradient of $Q(s,a)$ for given action $a$ in a state $s$. In a normal machine learning classification and regression algorithm, our target is to get the value with minimum loss. Then we train the network by gradient descent technique using the gradient of Loss ."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\begin{equation}\n",
+    "\\theta \\gets \\theta - \\alpha \\frac{\\partial L}{\\partial \\theta}\n",
+    "\\end{equation}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here, $\\theta$ is the weight parameter of the network, and $L$ is loss"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "But in our case, we have to get the maximize the $Q$ value. So we have to set the weight parameters such that we get the maximum $Q$ value. This technique is known as Gradient Ascent, as it does the exact opposite of Gradient Descent"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\begin{equation}\n",
+    "\\theta_a \\gets \\theta_a - \\alpha (-\\frac{\\partial Q(s,a) }{\\partial \\theta_a}) \n",
+    "\\end{equation}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The above equation looks like the actual Gradient Descent equation. Only difference is , the minus sign. It makes the equation to minimize the negative value of $Q$ , which in turn maximizes $Q$ value.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "So the training is as following"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- 1) Define Actor network $actor$ and Critic Network $critic$\n",
+    "- 2) Define Target Actor and Critic Networks - $actor_{target}$ and $critic_{target}$ with exact same weights\n",
+    "- 3) Initialize Replay Buffer \n",
+    "- 4) Get the initial state , $state$\n",
+    "- 5) Get the action $a$ from , $a \\gets actor(state)$ + Noise .[Here Noise is given to make the process stochastic and not so deterministic. The paper uses ornstein uhlenbeck noise process , so we will as well]\n",
+    "- 6) Get Next state $state_{next}$ , Reward $r$ , Terminal from agent for given $state$ and $action$\n",
+    "- 7) Add the experience , $state$,$action$,$reward$,$state_{next}$,$terminal$ to replay buffer\n",
+    "- 8) Get sample minibatch from Replay buffer\n",
+    "- 9) Train Critic Network Using Bellman Equation. Like DQN\n",
+    "- 10) Train Actor Network using Gradient Ascent with gradients of $Q$ . $\\theta_a \\gets \\theta_a - \\alpha (-\\frac{\\partial Q(s,a) }{\\partial \\theta_a}) $\n",
+    "- 11) Update weights of $actor_{target}$ and $critic_{target}$ using the equation $ \\theta \\gets \\tau \\theta + (1-\\tau)\\theta_{target}$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a id =\"model\"></a>\n",
+    "### Model Definition"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "After some trials and errors, I have selected this network. The Actor Network is 3 layer MLP with 320 hidden nodes in each layer. The critic network is also a 3 layer MLP with 640 hidden nodes in each layer.Notice that the return arguments of function ```create_critic_network```."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def create_actor_network(state_shape,action_shape):\n",
+    "    in1=layers.Input(shape=state_shape,name=\"state\")\n",
+    "    l1 =layers.Dense(320,activation=\"relu\")(in1)\n",
+    "    l2 =layers.Dense(320,activation=\"relu\")(l1)\n",
+    "    l3 =layers.Dense(320,activation=\"relu\")(l2)\n",
+    "    action =layers.Dense(action_shape,activation=\"tanh\")(l3)\n",
+    "    actor= Model(in1,action)\n",
+    "    return actor\n",
+    "\n",
+    "def create_critic_network(state_shape,action_shape):\n",
+    "    in1 = layers.Input(shape=state_shape,name=\"state\")\n",
+    "    in2 = layers.Input(shape=action_shape,name=\"action\")\n",
+    "    l1 = layers.concatenate([in1,in2])\n",
+    "    l2 = layers.Dense(640,activation=\"relu\")(l1)\n",
+    "    l3 = layers.Dense(640,activation=\"relu\")(l2)\n",
+    "    l4 = layers.Dense(640,activation=\"relu\")(l3)\n",
+    "    value = layers.Dense(1)(l4)\n",
+    "    critic = Model(inputs=[in1,in2],outputs=value)\n",
+    "    return critic,in1,in2"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "I am chosing ```MountainCarContinuous-v0``` game. Mainly because my GPU is not that good to work on higher dimensional state space"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\u001b[33mWARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'>. Please provide explicit dtype.\u001b[0m\n",
+      "\u001b[33mWARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'>. Please provide explicit dtype.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "env = gym.make(\"MountainCarContinuous-v0\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "state_shape= env.observation_space.sample().shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "action_shape=env.action_space.sample().shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "actor = create_actor_network(state_shape,action_shape[0])\n",
+    "critic,state_tensor,action_tensor = create_critic_network(state_shape,action_shape)\n",
+    "target_actor=create_actor_network(state_shape,action_shape[0])\n",
+    "target_critic,_,_ = create_critic_network(state_shape,action_shape)\n",
+    "target_actor.set_weights(actor.get_weights())\n",
+    "target_critic.set_weights(critic.get_weights())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "I have chosen ```RMSProp``` optimizer, due to more stability compared to Adam . I found it after trials and errors, no theoritical background on chosing this optimizer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "actor_optimizer = keras.optimizers.RMSprop(actor_lr)\n",
+    "\n",
+    "critic_optimizer = keras.optimizers.RMSprop(critic_lr)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "WARNING:tensorflow:From /home/sezan92/anaconda/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:1290: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.\n",
+      "Instructions for updating:\n",
+      "keep_dims is deprecated, use keepdims instead\n"
+     ]
+    }
+   ],
+   "source": [
+    "critic.compile(loss=\"mse\",optimizer=critic_optimizer)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Actor training"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "I think this is the most critical part of ddpg in keras. The object ```critic``` and ```actor``` has a ```__call__``` method inside it, which will give output tensor if you give input a tensor. So to get the tensor object of ```Q``` we will use this functionality."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "CriticValues = critic([state_tensor,actor(state_tensor)])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now it is time to get the gradient value of $-\\frac{\\partial Q(s,a)}{\\theta_a}$"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "updates = actor_optimizer.get_updates(\n",
+    "    params=actor.trainable_weights,loss=-K.mean(CriticValues))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we will create a function which will train the actor network."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "actor_train = K.function(inputs=[state_tensor],outputs=[actor(state_tensor),CriticValues],\n",
+    "                   updates=updates)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a id =\"buffer\"></a>\n",
+    "### Replay Buffer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "memory = deque(maxlen=10000)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "100%|██████████| 10000/10000 [00:00<00:00, 21304.88it/s]\n"
+     ]
+    }
+   ],
+   "source": [
+    "state = env.reset()\n",
+    "state = state.reshape(-1,)\n",
+    "for _ in tqdm(range(memory.maxlen)):\n",
+    "    action = env.action_space.sample()\n",
+    "    next_state,reward,terminal,_=env.step(action)\n",
+    "    \n",
+    "    state=next_state\n",
+    "    if terminal:\n",
+    "        reward=-100\n",
+    "        state= env.reset()\n",
+    "        state = state.reshape(-1,)\n",
+    "    memory.append((state,action,reward,next_state,terminal))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([-0.49533114, -0.00344986])"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "next_state"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a id =\"noise\"></a>\n",
+    "### Noise class"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The use of Noise is to make the process Stochastic and to help the agent explore different actions. The paper used Orstein Uhlenbeck Noise class"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class OrnsteinUhlenbeckProcess(object):\n",
+    "    def __init__(self, theta, mu=0, sigma=1, x0=0, dt=1e-2, n_steps_annealing=10, size=1):\n",
+    "        self.theta = theta\n",
+    "        self.sigma = sigma\n",
+    "        self.n_steps_annealing = n_steps_annealing\n",
+    "        self.sigma_step = - self.sigma / float(self.n_steps_annealing)\n",
+    "        self.x0 = x0\n",
+    "        self.mu = mu\n",
+    "        self.dt = dt\n",
+    "        self.size = size\n",
+    "    def restart(self):\n",
+    "        self.x0=copy.copy(self.mu)\n",
+    "    def generate(self, step):\n",
+    "        #sigma = max(0, self.sigma_step * step + self.sigma)\n",
+    "        x = self.x0 + self.theta * (self.mu - self.x0) * self.dt + self.sigma * np.sqrt(self.dt) * np.random.normal(size=self.size)\n",
+    "        self.x0 = x\n",
+    "        return x"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a id =\"training\"></a>\n",
+    "### Training"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Failed! Episode 0 Total Reward -121.829750\n",
+      "Failed! Episode 1 Total Reward -118.666569\n",
+      "Failed! Episode 2 Total Reward -162.041720\n",
+      "Failed! Episode 3 Total Reward -87.619343\n",
+      "Failed! Episode 4 Total Reward -96.425173\n",
+      "Failed! Episode 5 Total Reward -143.686660\n",
+      "Failed! Episode 6 Total Reward -38.429055\n",
+      "Passed! Episode 7 Total Reward 76.095012\n",
+      "Passed! Episode 8 Total Reward 78.345259\n",
+      "Failed! Episode 9 Total Reward -0.019448\n",
+      "Passed! Episode 10 Total Reward 48.135478\n",
+      "Passed! Episode 11 Total Reward 54.750747\n",
+      "Passed! Episode 12 Total Reward 79.410100\n",
+      "Passed! ward 83.0033361Episode 13 Total Reward 83.003336\n",
+      "Passed! Episode 14 Total Reward 83.174427\n",
+      "Failed! Episode 15 Total Reward -48.217783\n",
+      "Passed! Episode 16 Total Reward 87.284775\n",
+      "Passed! Episode 17 Total Reward 32.635075\n",
+      "Passed! Episode 18 Total Reward 84.165725\n",
+      "Passed! Episode 19 Total Reward 89.819676\n",
+      "Passed! Episode 20 Total Reward 76.334761\n",
+      "Passed! Episode 21 Total Reward 58.299554\n",
+      "Passed! Episode 22 Total Reward 81.035049\n",
+      "Failed! Episode 23 Total Reward -39.691471\n",
+      "Failed! Episode 24 Total Reward -161.331777\n",
+      "Passed! Episode 25 Total Reward 86.132543\n",
+      "Failed! Episode 26 Total Reward -104.653697\n",
+      "Passed! Episode 27 Total Reward 85.415800\n",
+      "Failed! Episode 28 Total Reward -74.066867\n",
+      "Failed! Episode 29 Total Reward -29.747882\n",
+      "Passed! Episode 30 Total Reward 59.700877\n",
+      "Passed! Episode 31 Total Reward 64.003951\n",
+      "Failed! Episode 32 Total Reward -43.807664\n",
+      "Passed! Episode 33 Total Reward 81.749323\n",
+      "Passed! Episode 34 Total Reward 66.484130\n",
+      "Failed! Episode 35 Total Reward -120.525069\n",
+      "Failed! Episode 36 Total Reward -24.451582\n",
+      "Failed! Episode 37 Total Reward -207.239527\n",
+      "Failed! Episode 38 Total Reward -43.106546\n",
+      "Passed! Episode 39 Total Reward 34.880121\n",
+      "Passed! Episode 40 Total Reward 66.188821\n",
+      "Passed! Episode 41 Total Reward 29.939728\n",
+      "Failed! Episode 42 Total Reward -86.516192\n",
+      "Failed! Episode 43 Total Reward -225.143733\n",
+      "Passed! Episode 44 Total Reward 7.568837\n",
+      "Passed! Episode 45 Total Reward 84.050707\n",
+      "Failed! Episode 46 Total Reward -86.705966\n",
+      "Passed! Episode 47 Total Reward 51.108435\n",
+      "Failed! Episode 48 Total Reward -88.531423\n",
+      "Passed! Episode 49 Total Reward 79.210735\n",
+      "Failed! Episode 50 Total Reward -64.247977\n",
+      "Failed! Episode 51 Total Reward -28.334167\n",
+      "Failed! Episode 52 Total Reward -53.942703\n",
+      "Passed! Episode 53 Total Reward 56.541394\n",
+      "Failed! Episode 54 Total Reward -30.573394\n",
+      "Failed! Episode 55 Total Reward -43.900060\n",
+      "Failed! Episode 56 Total Reward -31.793103\n",
+      "Passed! Episode 57 Total Reward 49.367744\n",
+      "Failed! Episode 58 Total Reward -59.277307\n",
+      "Failed! Episode 59 Total Reward -91.342287\n",
+      "Passed! Episode 60 Total Reward 72.884442\n",
+      "Passed! Episode 61 Total Reward 81.967582\n",
+      "Failed! Episode 62 Total Reward -69.682571\n",
+      "Failed! Episode 63 Total Reward -21.655243\n",
+      "Failed! Episode 64 Total Reward -102.427871\n",
+      "Failed! Episode 65 Total Reward -18.727795\n",
+      "Failed! Episode 66 Total Reward -106.180673\n",
+      "Passed! Episode 67 Total Reward 66.997363\n",
+      "Failed! Episode 68 Total Reward -98.338890\n",
+      "Failed! Episode 69 Total Reward -17.488416\n",
+      "Failed! Episode 70 Total Reward -126.062955\n",
+      "Passed! Episode 71 Total Reward 91.446696\n",
+      "Failed! Episode 72 Total Reward -153.244886\n",
+      "Passed! Episode 73 Total Reward 32.589752\n",
+      "Failed! Episode 74 Total Reward -106.034746\n",
+      "Failed! Episode 75 Total Reward -46.197340\n",
+      "Failed! Episode 76 Total Reward -64.983000\n",
+      "Failed! Episode 77 Total Reward -50.082150\n",
+      "Failed! Episode 78 Total Reward -44.506584\n",
+      "Failed! Episode 79 Total Reward -130.709335\n",
+      "Passed! Episode 80 Total Reward 65.267968\n",
+      "Failed! Episode 81 Total Reward -32.770421\n",
+      "Passed! Episode 82 Total Reward 68.083454\n",
+      "Failed! Episode 83 Total Reward -81.669658\n",
+      "Failed! Episode 84 Total Reward -40.234104\n",
+      "Passed! Episode 85 Total Reward 89.699236\n",
+      "Passed! Episode 86 Total Reward 76.233450\n",
+      "Failed! Episode 87 Total Reward -60.663254\n",
+      "Passed! Episode 88 Total Reward 75.066797\n",
+      "Failed! Episode 89 Total Reward -46.776051\n",
+      "Failed! Episode 90 Total Reward -25.376468\n",
+      "Passed! Episode 91 Total Reward 81.885317\n",
+      "Passed! Episode 92 Total Reward 61.040792\n",
+      "Failed! Episode 93 Total Reward -59.524774\n",
+      "Passed! Episode 94 Total Reward 80.259871\n",
+      "Passed! Episode 95 Total Reward 71.395997\n",
+      "Passed! Episode 96 Total Reward 70.046112\n",
+      "Passed! Episode 97 Total Reward 87.850163\n",
+      "Passed! Episode 98 Total Reward 86.638371\n",
+      "Failed! Episode 99 Total Reward -37.902022\n"
+     ]
+    }
+   ],
+   "source": [
+    "steps_per_episodes=5000\n",
+    "ou = OrnsteinUhlenbeckProcess(theta=0.35,mu=0.8,sigma=0.4,n_steps_annealing=10)\n",
+    "max_total_reward=0\n",
+    "for episode in range(num_episodes):\n",
+    "    state= env.reset()\n",
+    "    state = state.reshape(-1,)\n",
+    "    total_reward=0\n",
+    "    ou.restart()\n",
+    "    for step in range(steps_per_episodes):\n",
+    "        action= actor.predict(state.reshape(1,-1))+ou.generate(episode)\n",
+    "        next_state,reward,done,_ = env.step(action)\n",
+    "        total_reward=total_reward+reward\n",
+    "        #random minibatch from buffer\n",
+    "        \n",
+    "        batches=random.sample(memory,BATCH_SIZE)\n",
+    "        states= np.array([batch[0].reshape((-1,)) for batch in batches])\n",
+    "        actions= np.array([batch[1] for batch in batches])\n",
+    "        actions=actions.reshape(-1,1)\n",
+    "        rewards=np.array([batch[2] for batch in batches])\n",
+    "        rewards = rewards.reshape((-1,1))\n",
+    "        new_states=np.array([batch[3].reshape((-1,)) for batch in batches])\n",
+    "        terminals=np.array([batch[4] for batch in batches])\n",
+    "        terminals = terminals.reshape((-1,1))\n",
+    "        #training\n",
+    "        \n",
+    "        target_actions = target_actor.predict(new_states)\n",
+    "        target_Qs = target_critic.predict([new_states,target_actions])\n",
+    "        \n",
+    "        new_Qs = rewards+GAMMA*target_Qs*terminals\n",
+    "        critic.fit([states,actions],new_Qs,verbose=False)\n",
+    "        _,critic_values=actor_train(inputs=[states])\n",
+    "        target_critic_weights=[TAU*weight+(1-TAU)*target_weight for weight,target_weight in zip(critic.get_weights(),target_critic.get_weights())]\n",
+    "        target_actor_weights=[TAU*weight+(1-TAU)*target_weight for weight,target_weight in zip(actor.get_weights(),target_actor.get_weights())]\n",
+    "        target_critic.set_weights(target_critic_weights)\n",
+    "        target_actor.set_weights(target_actor_weights)\n",
+    "        print(\"Total Reward %f\"%total_reward,end=\"\\r\")\n",
+    "        if SHOW:\n",
+    "            env.render()\n",
+    "        if done or step==(steps_per_episodes-1):\n",
+    "            \n",
+    "            if total_reward<0:\n",
+    "                print(\"Failed!\",end=\" \")\n",
+    "                reward=-100\n",
+    "            elif total_reward>0:\n",
+    "                print(\"Passed!\",end=\" \")\n",
+    "                reward=100\n",
+    "            memory.append((state,action,reward,next_state,done))\n",
+    "            break\n",
+    "        \n",
+    "        memory.append((state,action,reward,next_state,done))\n",
+    "        state=next_state\n",
+    "    if total_reward>max_total_reward:\n",
+    "        actor.save_weights(\"MC_DDPG_Weights/Actor_Best_weights episode %d_GAMMA_%f_TAU%f_lr_%f.h5\"%(episode,GAMMA,TAU,actor_lr))\n",
+    "        critic.save_weights(\"MC_DDPG_Weights/Critic_Best_weights episode %d_GAMMA_%f_TAU%f_lr_%f.h5\"%(episode,GAMMA,TAU,critic_lr))\n",
+    "        max_total_reward=total_reward\n",
+    "    print(\"Episode %d Total Reward %f\"%(episode,total_reward))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Video"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Please watch at 2x speed. I changed some simple mistakes after the video so the rewards are not exactly the same"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "[![](http://img.youtube.com/vi/9Fe_n-ovIaA/0.jpg)](http://www.youtube.com/watch?v=9Fe_n-ovIaA \"Keras tutorial DDPG\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 2",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.15"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/README.md b/README.md
index c1aa84c..f5c02d2 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,10 @@
 # Reinforcement Learning Tutorials using Keras
 
 ### MD Muhaimin Rahman
-<a>sezan92@gmail.com</a>
+
+<a>sezan9[at]gmail[dot]com</a>
+
+
 
 In this repository, I am trying to write tutorials on popular Reinforcement Learning Algorithms. I will design the models using Keras, as it is popular now and most importantly, it's very easy for beginners to learn. As there aren't enough Deep Learning Tutorials in Bangla, I am planning to write in both English and Bangla. As for now, I have selected the following ones for the topic of my tutorials:
 - Q Learning