MT Exercise 3: Pytorch RNN Language Models

This repo shows how to train neural language models using Pytorch example code. Thanks to Emma van den Bold, the original author of these scripts.

Requirements

• This only works on a Unix-like system, with bash.

• Python 3 must be installed on your system, i.e. the command python3 must be available

• Make sure virtualenv is installed on your system. To install, e.g.

  pip install virtualenv

Steps

Clone this repository in the desired place:

git clone https://github.com/moritz-steiner/mt-exercise-03
cd mt-exercise-03

Create a new virtualenv that uses Python 3. Please make sure to run this command outside of any virtual Python environment:

./scripts/make_virtualenv.sh

Important: Then activate the env by executing the source command that is output by the shell script above.

Download and install required software:

./scripts/install_packages.sh

Download and preprocess data:

./scripts/download_data.sh

Train a model:

./scripts/train.sh

The training process can be interrupted at any time, and the best checkpoint will always be saved.

Generate (sample) some text from a trained model with:

./scripts/generate.sh

Task 1

• Change the download_data.sh, make it directly preprocess the existing rick and morty lines;
• Enlarge the vocabulary size setting in download_data.sh, decrease ;
• Add -mps to generate.sh and train.sh, speed up the processing steps.

Task 2

• Change the main.py, make it able to take multiple dropout settings;
• Add line_chart.py at scripts, plot the data and store at los;
• Generate two sample document with text with different perplexities.