When print_every_n_steps and keep_every are set to equal, the network is not updated properly
Following the notebook example:
model = Bohamiann(print_every_n_steps=50)
model.train(x[:, None], y, num_steps=20000, num_burn_in_steps=2000, keep_every=50, lr=1e-2, verbose=True)
results in :
The reason is because the seemingly innocent print_every_n_steps does an important job in the background : by calling network_predict, implicitly the current weights is reset to the latest sampled_weights via the line : self.network_weights = weights
Upon removing the line where it prints the network performance, we can log and notice the drifting loss weights prior shown as WP here:
When
print_every_n_stepsandkeep_everyare set to equal, the network is not updated properlyFollowing the notebook example:
results in :
The reason is because the seemingly innocent
print_every_n_stepsdoes an important job in the background : by callingnetwork_predict, implicitly the current weights is reset to the latest sampled_weights via the line :self.network_weights = weightsUpon removing the line where it
printsthe network performance, we can log and notice the drifting loss weights prior shown asWPhere: