PPO Batch Size

added Bug label

changed milestone to %Official dates

assigned to @dl337788

added In Progress label

Horizon T
Number of parallel rollouts N
Minibatch size M <= T * N
train for K epochs

use torch samplers

We simply use a mini batch size of 32 per worker/environment, so 32 * N. With 4 epochs and T = 128 this will lead to 100% of the samples being used in mini batches. Some papers suggest using 3 epochs, so we will miss 25% of samples in training.

See: https://github.com/openai/baselines/blob/ea25b9e8b234e6ee1bca43083f8f3cf974143998/baselines/ppo2/ppo2.py#L97

closed via commit 21090f54

removed In Progress label