-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathtutorial_james.html
More file actions
308 lines (291 loc) · 17.1 KB
/
tutorial_james.html
File metadata and controls
308 lines (291 loc) · 17.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="msvalidate.01" content="8A044F6B24E3A19810D766CAE2ACA17A" />
<meta name="description" content="NeVer Tools collects knowledge and insight about formal verification of Neural Networks.">
<meta name="author" content="Navendu Pottekkat | Stefano Demarchi">
<meta name="keywords" content="VNNLIB, NeVer, NeVerTools, CoCoNet, Neural Networks Verification">
<meta name="msapplication-TileColor" content="#da532c">
<meta name="msapplication-config" content="./assets/img/favicons/never2/browserconfig.xml">
<meta name="theme-color" content="#ffffff">
<title>NeVer 2 - James</title>
<!-- Favicons -->
<link rel="apple-touch-icon" sizes="180x180" href="./assets/img/favicons/never2/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="./assets/img/favicons/never2/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="./assets/img/favicons/never2/favicon-16x16.png">
<link rel="manifest" href="./assets/img/favicons/never2/site.webmanifest">
<link rel="mask-icon" href="./assets/img/favicons/never2/safari-pinned-tab.svg" color="#5bbad5">
<link rel="shortcut icon" href="./assets/img/favicons/never2/favicon.ico">
<!-- Font Awesome icons (free version)-->
<script src="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.13.0/js/all.min.js"
crossorigin="anonymous"></script>
<!-- Core theme CSS (includes Bootstrap)-->
<link href="css/styles.css" rel="stylesheet">
<!-- Fonts CSS-->
<link rel="stylesheet" href="css/heading.css">
<link rel="stylesheet" href="css/body.css">
<!-- Custom CSS-->
<link rel="stylesheet" href="css/tutorials.css">
</head>
<body id="page-top">
<nav class="navbar navbar-expand-lg bg-secondary fixed-top" id="mainNav">
<div class="container"><a class="navbar-brand js-scroll-trigger" href="#page-top">NeVer 2 - Tutorials</a>
<button class="navbar-toggler navbar-toggler-right font-weight-bold bg-primary text-white rounded"
type="button" data-toggle="collapse" data-target="#navbarResponsive" aria-controls="navbarResponsive"
aria-expanded="false" aria-label="Toggle navigation"><i class="fas fa-bars"></i></button>
<div class="collapse navbar-collapse" id="navbarResponsive">
<ul class="navbar-nav ml-auto">
<li class="nav-item mx-0 mx-lg-1"><a class="nav-link py-3 px-0 px-lg-3 rounded js-scroll-trigger"
href="#overview">Overview</a>
</li>
<li class="nav-item mx-0 mx-lg-1"><a class="nav-link py-3 px-0 px-lg-3 rounded js-scroll-trigger"
href="#build">Build</a>
</li>
<li class="nav-item mx-0 mx-lg-1"><a class="nav-link py-3 px-0 px-lg-3 rounded js-scroll-trigger"
href="#train">Training</a>
</li>
<li class="nav-item mx-0 mx-lg-1"><a class="nav-link py-3 px-0 px-lg-3 rounded js-scroll-trigger"
href="#verify">Verification</a>
</li>
</ul>
</div>
</div>
</nav>
<header class="masthead bg-primary text-white text-center">
<div class="container d-flex align-items-center flex-column">
<!-- Masthead Heading-->
<h1 class="masthead-heading mb-0">James</h1>
<!-- Icon Divider-->
<div class="divider-custom divider-light">
<div class="divider-custom-line"></div>
<div class="divider-custom-icon"><i class="fas fa-caret-down"></i></div>
<div class="divider-custom-line"></div>
</div>
<!-- Masthead Subheading-->
<p class="masthead-subheading font-weight-light mb-0">
James is the name of a humanoid robot project developed by University of Genoa and Italian Institute of Technology (IIT).
The neural network controller for the arm of the robot consists of 8 inputs and 6 outputs: the input consists of two
vectors of four elements each, one related to the velocity and the other to the angle of the shoulder and elbow joints.
The outputs are the corresponding values of internal forces and torques in a Cartesian space.
</p>
</div>
</header>
<!-- OVERVIEW SECTION -->
<section class="masthead page-section portfolio" id="overview">
<div class="container">
<!-- Portfolio Section Heading -->
<div class="text-center">
<h2 class="page-section-heading text-secondary mb-0 d-inline-block">Overview</h2>
</div>
<!-- Icon Divider -->
<div class="divider-custom">
<div class="divider-custom-line"></div>
<div class="divider-custom-icon"><i class="fas fa-code"></i></div>
<div class="divider-custom-line"></div>
</div>
<!-- Example Section Content -->
<div class="text-center">
<div id="overview-description">
<p>
The picture below shows the formal representation of the feed-forward neural network for this example.
The architecture consists of 3 hidden layers with ReLU activation functions and a decreasing
number of neurons, namely 64, 32 and 16.
</p>
</div>
<br>
<img class="responsive" src="./assets/img/tutorials/james/formal_graph.png" alt="Convolutional diagram">
</div>
</div>
</section>
<!-- BUILD SECTION -->
<section class="page-section bg-primary text-white mb-0" id="build">
<div class="container">
<!-- Contact Section Heading-->
<div class="text-center">
<h2 class="page-section-heading d-inline-block text-white">Network building</h2>
</div>
<!-- Icon Divider-->
<div class="divider-custom divider-light">
<div class="divider-custom-line"></div>
<div class="divider-custom-icon"><i class="fas fa-hammer"></i></div>
<div class="divider-custom-line"></div>
</div>
<!-- Contact Section Content-->
<div class="text-left font-large">
<div id="build-network">
<br>
<div>
<p class="lead">
NeVer2 allows to add sequential layers directly by clicking on the corresponding buttons in the left toolbar.
The layer block will be placed in the correct position with default parameters which can be edited directly.
Note that adding a layer will block the editing of previous layers to maintain consistency. It is always possible
to remove nodes at the end of a network with the "DEL" key. This also applies to the input block, where it is
possible to change the input identifier and to set the number of inputs of the network.
</p>
<div class="text-center">
<img class="img-tutorial" src="./assets/img/tutorials/common/nodes_toolbar.png" alt="Toolbar"></img>
</div>
</div>
<br>
<div>
<p>
The first block is a Fully Connected layer consisting of 64 neurons. To do so, it is sufficient to set the
<i>out_features</i> parameter to 64 and to click on <i>Save</i>.
</p>
<div class="text-center">
<img class="img-tutorial" src="./assets/img/tutorials/james/first_layer.png" alt="Fully Connected block"></img>
</div>
</div>
<br>
<div>
<p>
To add the other layers we select the corresponding block in the left toolbar. ReLU activation functions have
no parameters and new Fully Connected layers can be added directly. In the example below it is presented the
complete network. In this example it is also visible how the last layer computes the output which is then
shown in the output block.
</p>
<div class="text-center">
<img class="img-tutorial-full" src="./assets/img/tutorials/james/james_net.png" alt="James network"></img>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- TRAIN SECTION -->
<section class="page-section" id="train">
<div class="container">
<!-- Contact Section Heading-->
<div class="text-center">
<h2 class="page-section-heading text-secondary d-inline-block mb-0">Network Training</h2>
</div>
<!-- Icon Divider-->
<div class="divider-custom">
<div class="divider-custom-line"></div>
<div class="divider-custom-icon"><i class="fas fa-brain"></i></div>
<div class="divider-custom-line"></div>
</div>
<!-- Contact Section Content-->
<div class="text-left font-large">
<div id="train-network">
<div>
Once the network is completed, it is time to train it on the dataset. In the menu bar, selecting
"Learn..." -> "Train" displays the window for the training setup. The first thing to do is to select
the dataset: we made directly available the MNIST and Fashion MNIST datasets, but it is possible to
open any dataset as a text file selecting "Custom data source...".
<div class="text-center">
<img class="img-tutorial" src="./assets/img/tutorials/james/dataset_load.png" alt="Dataset loading"></img>
</div>
In order to process correctly the dataset, it is required to provide information on the <i>Data type</i>
to expect and the <i>Delimiter</i> character, which are set by default as <i>float</i> and '<i>,</i>'.
<div class="text-center">
<img class="img-tutorial-wide" src="./assets/img/tutorials/james/training_setup.png" alt="Training parameters of the network"></img>
</div>
The learning parameters can be set as follows:<br><br>
<b><i>Optimizer</i></b>: at the moment only "Adam" (adaptive moment estimation) is supported, being
the most popular and effective gradient based optimization algorithm. Once selected, you can edit all
its related parameters.<br><br>
<b><i>Learning Rate Scheduler</i></b>: like the Optimizer, only "ReduceLROnPlateau" is supported. It adjusts the
learning rate only when a plateau in model performance is detected, e.g. no change for a given number of
training epochs.<br><br>
<b><i>Loss Function</i></b>: you can choose between "Cross Entropy" and "MSE Loss", depending on the network
structure.<br><br>
<b><i>Precision Metric</i></b>: you can choose between "Inaccuracy" and "MSE loss".<br><br>
<b><i>Epochs</i></b>: it defines the steps that the learning algorithm will perform through the training process.
<br><br>
<b><i>Validation Percentage</i></b>: (a number between 0 and 1) indicates the percentage of the dataset to use
as the validation set.<br><br>
<b><i>Training batch size</i></b> and <b><i>Validation batch size</i></b>: it defines the dimension of the
training batches.<br><br>
<b><i>Cuda</i></b>: it allows to take advantage of NVidia GPU architecture for the computation.<br><br>
<b><i>Train patience</i></b> <i>(optional)</i>: the number of epochs in which the loss may not decrease
before the training procedure is interrupted with early stopping.<br><br>
<b><i>Checkpoints root</i></b> <i>(optional)</i>: where to store the checkpoints of the training strategy, by
default the working directory of NeVer2.<br><br>
<b><i>Verbosity level</i></b> <i>(optional)</i>: how many batches between training log prints (default 1).
</div>
<br>
<div>
<p>
Once the parameters are set, clicking "Train Network" runs the training. Logs are printed in a box
below the buttons.
<div class="text-center">
<img class="img-tutorial" src="./assets/img/tutorials/james/training_loss.png" alt="Training Results"></img>
</div>
</p>
</div>
<div>
<p>
The <b>Best Loss Score</b> label indicates the smallest training loss; in this case,
with a 10 epoch long training the best loss is 0.00048, corresponding to 0.048%. Looking
at the following diagram shows how increasing the training epochs improves slightly
the training loss.
</p>
<img class="center" src="./assets/img/tutorials/james/james_loss.png" width="60%" alt="Training Loss"></img>
</div>
</div>
</div>
</div>
</section>
<!-- VERIFY SECTION -->
<section class="page-section bg-primary text-white mb-0" id="verify">
<div class="container">
<!-- Contact Section Heading-->
<div class="text-center">
<h2 class="page-section-heading d-inline-block text-white">Network Verification</h2>
</div>
<!-- Icon Divider-->
<div class="divider-custom divider-light">
<div class="divider-custom-line"></div>
<div class="divider-custom-icon"><i class="fas fa-check"></i></div>
<div class="divider-custom-line"></div>
</div>
<!-- Contact Section Content-->
<div class="text-left font-large">
<div id="verify-network">
<br>
<h1 class="titles-tutorial">In progress...</h1>
</div>
</div>
</div>
</section>
<footer class="footer text-center">
<div class="container">
<div class="row">
<!-- Footer Location-->
<div class="col-lg-4 mb-5 mb-lg-0">
<h4 class="mb-4"></h4>
<p class="pre-wrap lead mb-0"> </p>
</div>
<div class="col-lg-4 mb-5 mb-lg-0">
<h4 class="mb-4">Powered by AIMS-Lab</h4>
<a target="_blank" href="http://www.aimslab.org">
<img class="img-footer" src="assets/svg/unige.svg" alt="UniGE">
</a>
</div>
</div>
</br></br>
This tutorial was provided by Andrea Gimelli, Karim Pedemonte and Giacomo Rosato.
</div>
</footer>
<!-- Copyright Section -->
<section class="copyright py-4 text-center text-white">
<div class="container"><small class="pre-wrap"><a target="_blank" href="https://github.com/NeVerTools/NeVerTools.github.io/blob/master/LICENSE">MIT Licensed</a> | neuralverification.org 2021 | Hosted with ❤ by <a href="https://github.com/" target="_blank">GitHub</a></small></div>
</section>
<!-- Scroll to Top Button (Only visible on small and extra-small screen sizes) -->
<div class="scroll-to-top d-lg-none position-fixed"><a
class="js-scroll-trigger d-block text-center text-white rounded" href="#page-top"><i
class="fa fa-chevron-up"></i></a></div>
<!-- Bootstrap core JS -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.bundle.min.js"></script>
<!-- Third party plugin JS -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery-easing/1.4.1/jquery.easing.min.js"></script>
<!-- Core theme JS -->
<script src="js/scripts.js"></script>
<!-- GitHub buttons -->
<script async defer src="https://buttons.github.io/buttons.js"></script>
</body>
</html>