Laboratory work and experiments for Deep Learning 1 (Duboko učenje 1) at the University of Zagreb Faculty of Electrical Engineering and Computing, academic year 2025/2026.
The course studies deep learning through the representations produced by successive learned nonlinear transformations. It covers fully connected, convolutional, generative, and recurrent models, with applications in computer vision and natural language processing. The work in this repository follows the practical part of the course: models are first assembled from elementary operations and then examined through equivalent PyTorch implementations, controlled experiments, and quantitative evaluation.
| Institution | University of Zagreb, Faculty of Electrical Engineering and Computing (FER) |
| Course | Deep Learning 1 (Duboko učenje 1) |
| Academic year | 2025/2026 |
| Semester | Summer |
| Credits | 5 ECTS |
| Course coordinators | Marko Čupić and Zoran Kalafatić |
The course is organized around four broad areas: foundations of feed-forward neural networks, convolutional models, generative deep models, and recurrent sequence models. Its intended learning outcomes include the ability to:
- explain where deep learning offers an advantage over alternative machine-learning approaches;
- train fully connected, convolutional, and recurrent models in a supervised setting;
- apply deep models to image and text understanding;
- analyze and evaluate model performance;
- design deep models in a high-level programming language.
| Exercise | Focus | Main tools and data |
|---|---|---|
| Lab 1 | Fully connected models, backpropagation, regularization, and optimizer comparisons | NumPy, PyTorch, scikit-learn, MNIST, synthetic Gaussian mixtures |
| Lab 2 | Convolution and pooling from first principles, gradient checking, and CNN image classification | NumPy, Cython, PyTorch, MNIST, CIFAR-10 |
| Lab 3 | Binary sentiment classification with pooled embeddings and recurrent models | PyTorch, Stanford Sentiment Treebank, GloVe embeddings |
| Lab 4 | Metric learning with triplet loss, prototype classification, and hard example mining | PyTorch, MNIST, PCA |
The first exercise develops the basic classification pipeline on synthetic two-dimensional data and MNIST:
- data sampling, one-hot encoding, performance metrics, and decision-surface visualization;
- a two-layer fully connected network implemented directly in NumPy;
- logistic regression and configurable deep feed-forward networks implemented in PyTorch;
- experiments with network depth, activation functions, learning rates, weight decay, and optimizers;
- comparison with linear and RBF support-vector machines;
- analysis of learned MNIST weights, loss curves, and difficult examples.
The implementation is split across small scripts, while solutions.ipynb provides the exercise index.
The second exercise moves from individual layers to complete image classifiers. Its NumPy/Cython section implements convolution with im2col, max pooling, fully connected layers, ReLU and sigmoid activations, softmax cross-entropy, L2 regularization, and manual forward and backward passes. Numerical gradient checks validate each backward implementation.
The PyTorch section reproduces the MNIST network and extends the experiments to CIFAR-10. It includes:
- scheduled learning rates, momentum, and weight decay;
- cross-entropy and multiclass hinge-loss variants;
- per-class precision and recall, confusion matrices, and training curves;
- convolutional-filter visualization and inspection of the most costly misclassifications.
The third exercise treats sentiment analysis as binary sequence classification on the Stanford Sentiment Treebank. It implements dataset parsing, vocabulary construction, variable-length batching, and optional initialization from 300-dimensional GloVe vectors.
The evaluated architectures include a pooled-embedding baseline and recurrent classifiers based on vanilla RNN, GRU, and LSTM cells. Experiments cover pooling strategies, vocabulary thresholds, frozen and trainable embeddings, recurrent depth and width, bidirectionality, gradient clipping, dropout, optimizers, and Bahdanau attention. The final candidates are evaluated over five random seeds rather than a single favorable run.
The complete analysis is available in lab3_analysis.ipynb; machine-readable experiment records are stored in lab3/results.
The final exercise learns a compact convolutional embedding of MNIST digits with triplet margin loss. Training examples are formed from an anchor, a positive image from the same class, and a negative image from another class. Classification is then performed by comparing an example with the mean embedding, or prototype, of each class.
The exercise compares the learned representation with raw-pixel nearest-prototype classification, studies generalization when digit zero is excluded from training, and adds in-batch hard positive and negative mining. PCA projections illustrate how the learned feature space separates classes more clearly than the original pixel space.
The implementation and interpretation are documented in lab4_metric_embedding.ipynb.
The tables below report the saved experiment outputs. Values are percentages; Lab 3 results are means and standard deviations over five seeds.
| Model | Test accuracy | Test F1 |
|---|---|---|
| Pooled-embedding baseline | 77.00 ± 5.09 | 79.29 ± 2.86 |
| Bidirectional GRU with attention | 79.56 ± 3.86 | 81.03 ± 2.19 |
| Representation or training setup | Test accuracy |
|---|---|
| Raw-pixel identity baseline | 82.16 |
| Metric embedding, all classes | 98.46 |
| Metric embedding trained without digit zero | 94.80 |
| Metric embedding with in-batch hard mining | 97.58 |
.
├── lab1/ fully connected models and MNIST comparisons
├── lab2/task1/ convolutional layers and image classifiers
├── lab3/ sentiment models, experiment suites, and results
├── lab4/ metric-learning models, results, and figures
├── pyproject.toml project metadata and Python dependencies
└── uv.lock reproducible dependency lockfile
Source code and notebooks are versioned. Downloaded datasets, compiled Cython files, model checkpoints, submission archives, caches, and generated Lab 2 output directories are intentionally excluded.
The repository targets Python 3.12 and uses uv for dependency management.
git clone https://github.com/fran-galic/deep-learning-1.git
cd deep-learning-1
uv sync --lockedMNIST and CIFAR-10 are downloaded automatically by torchvision on first use. Lab 3 expects the course-provided SST splits and GloVe file at the following paths:
lab3/data/sst_train_raw.csv
lab3/data/sst_valid_raw.csv
lab3/data/sst_test_raw.csv
lab3/embedings/sst_glove_6b_300d.txt
The embedings directory name is retained because it is the path used by the implementation.
Lab 2 also requires a working C/C++ compiler to build its Cython extension:
cd lab2/task1
uv run python setup_cython.py build_ext --inplace
uv run python check_grads.pyThe following commands cover the main experiment entry points. Training the complete suites can take considerably longer than the gradient checks and small synthetic examples.
# Lab 1: run from its directory because the scripts use local imports
cd lab1
uv run python pt_logreg.py
uv run python pt_deep.py
uv run python mnist_shootout.py
# Lab 2: run after building the Cython extension
cd ../lab2/task1
uv run python train.py
uv run python convolutional_net_torch.py
uv run python cifar10_convnet_torch.py
uv run python cifar10_hinge_torch.py
# Labs 3 and 4: run from the repository root
cd ../..
uv run python lab3/run_suite.py --suite references
uv run python lab3/run_suite.py --suite search
uv run python lab3/run_suite.py --suite finals
uv run python lab4/run_experiments.pyTo inspect the analyses interactively, start Jupyter from the repository root with uv run jupyter lab and open the notebook for the relevant exercise.
This repository is a personal archive of completed coursework and is intended as a technical reference. Students currently taking the course should follow FER's rules on independent work, attribution, and submission.
The repository is available under the MIT License. Third-party datasets, pretrained embeddings, and any course-provided starter material remain subject to their respective terms.