Speech to Text with DeepSpeech

We intervened this project for hyperparameter tuning and for now, only train_batch_size, dropout_rate, learning_rate and n_hidden are used. So as not to spoil the original code, we created a new optuna_hp.py on the same code and we had to change some methods and add the necessary code pieces for optuna optimization.

You can change the value ranges in hps_create_optimizer as you wish.

It is very important to choose train_batch_size and n_hidden correctly and change it in the file to avoid GPU issues.

Some changes had to be made on checkpoint_dir in order to eliminate the errors encountered with the Optuna integration. In order to resolve these, you must enter the appropriate value for your machine in the checkpoint_root_dir variable in the setup_dirs method.

Note:

In this study, the distributed Optuna - multi GPU relationship was used and the codes were updated for this purpose but suitable for single and multiple. For this non-purpose use, create_study may be modified as needed.

How to Run

Without Optuna Hyperparameter Optimization

CUDA_VISIBLE_DEVICES=0       # gpu/device id: for specific GPUs
nohup python DeepSpeech.py   # to run in the background
--scorer_path ~/name.scorer
--checkpoint_dir ~/checkpoints/
--epochs 20     
--train_batch_size 64   
--dev_batch_size 128   
--learning_rate 0.01   
--n_hidden 2056 
--dropout 0.05
--train_files ~/train.csv 
--dev_files ~/validation.csv
--test_files ~/test.csv
--export_dir ~/output_models/
--summary_dir ~/summaries/ 
--use_allow_growth true
--alphabet_config_path ~/alphabet.txt > ~/nohup_out.txt 2>&1 &

With Optuna Hyperparameter Optimization

Which model parameters?

• train_batch_size
• dropout
• learning_rate
• n_hidden

CUDA_VISIBLE_DEVICES=0      # gpu/device id: for specific GPUs
nohup python DeepSpeech.py  # to run in the background
--scorer_path ~/name.scorer
--checkpoint_dir ~/checkpoints/
--epochs 20  
--dev_batch_size 128
--train_files ~/train.csv 
--dev_files ~/validation.csv
--test_files ~/test.csv
--export_dir ~/output_models/
--summary_dir ~/summaries/ 
--use_allow_growth true
--alphabet_config_path ~/alphabet.txt > ~/nohup_out.txt 2>&1 &

It is open for improvements and please let me know what new things can be added :)