Udacity-Deep-RL-Navigation-Project/Report.html at master · giakoumidis/Udacity-Deep-RL-Navigation-Project · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
<!DOCTYPE HTML>
<html>
 <head>
  <meta charset="utf-8"/>
  <title>
   Made with Remarkable!
  </title>
  <link href="http://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.1/styles/github.min.css" rel="stylesheet"/>
  <style type="text/css">
   body,table tr{background-color:#fff}table tr td,table tr th{border:1px solid #ccc;text-align:left;padding:6px 13px;margin:0}pre code,table,table tr{padding:0}hr,pre code{background:0 0}body{font:16px Helvetica,Arial,sans-serif;line-height:1.4;color:#333;word-wrap:break-word;padding:10px 15px}strong,table tr th{font-weight:700}h1{font-size:2em;margin:.67em 0;text-align:center}h2{font-size:1.75em}h3{font-size:1.5em}h4{font-size:1.25em}h1,h2,h3,h4,h5,h6{font-weight:700;position:relative;margin-top:15px;margin-bottom:15px;line-height:1.1}h1,h2{border-bottom:1px solid #eee}hr{height:0;margin:15px 0;overflow:hidden;border:0;border-bottom:1px solid #ddd}a{color:#4183C4}a.absent{color:#c00}ol,ul{padding-left:15px;margin-left:5px}ol{list-style-type:lower-roman}table tr{border-top:1px solid #ccc;margin:0}table tr:nth-child(2n){background-color:#aaa}table tr td :first-child,table tr th :first-child{margin-top:0}table tr td:last-child,table tr th :last-child{margin-bottom:0}img{max-width:100%}blockquote{padding:0 15px;border-left:4px solid #ccc}code,tt{margin:0 2px;padding:0 5px;white-space:nowrap;border:1px solid #eaeaea;background-color:#f8f8f8;border-radius:3px}pre code{margin:0;white-space:pre;border:none}.highlight pre,pre{background-color:#f8f8f8;border:1px solid #ccc;font-size:13px;line-height:19px;overflow:auto;padding:6px 10px;border-radius:3px}
  </style>
 </head>
 <body>
  <h1 id="project-report">
   Project report
  </h1>
  <h2 id="learning-algorithm">
   Learning algorithm
  </h2>
  <p>
   The learning algorithm used is vanilla Deep Q Learning as described in original paper. As an input the vector of state is used instead of an image so convolutional neural nework is replaced with deep neural network. The deep neural network has following layers:
  </p>
  <ul>
   <li>
    Fully connected layer - input: 37 (state size) output: 128
   </li>
   <li>
    Fully connected layer - input: 128 output 64
   </li>
   <li>
    Fully connected layer - input: 64 output: (action size)
   </li>
  </ul>
  <p>
   Parameters used in DQN algorithm:
  </p>
  <ul>
   <li>
    Maximum steps per episode: 1000
   </li>
   <li>
    Starting epsilion: 1.0
   </li>
   <li>
    Ending epsilion: 0.01
   </li>
   <li>
    Epsilion decay rate: 0.95
   </li>
  </ul>
  <h2 id="results">
   Results
  </h2>
  <p>
   <img alt="results" src="images/results.png"/>
  </p>
  <p>
   ```
   <br/>
   Episode 100 Average Score: 1.66
   <br/>
   Episode 200 Average Score: 8.17
   <br/>
   Episode 300 Average Score: 10.81
   <br/>
   Episode 364 Average Score: 13.00
   <br/>
   Environment solved in 264 episodes! Average Score: 13.00
  </p>
  <h3 id="trained-agent">
   Trained agent
  </h3>
  <p>
   <img alt="trained" src="images/trained.gif"/>
  </p>
  <h2 id="future-work">
   Future work
  </h2>
  <ol>
   <li>
    Extensive hyperparameter optimization
   </li>
   <li>
    Double Deep Q Networks
   </li>
   <li>
    Prioritized Experience Replay
   </li>
   <li>
    Dueling Deep Q Networks
   </li>
   <li>
    RAINBOW Paper
   </li>
   <li>
    Learning from pixels
   </li>
  </ol>
  <script src="http://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.1/highlight.min.js">
  </script>
  <script>
   hljs.initHighlightingOnLoad();
  </script>
  <script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript">
  </script>
  <script type="text/javascript">
   MathJax.Hub.Config({"showProcessingMessages" : false,"messageStyle" : "none","tex2jax": { inlineMath: [ [ "$", "$" ] ] }});
  </script>
 </body>
</html>