Off-Policy Policy Evaluation

For its own sake or as part of a policy iteration scheme, evaluating policies is an important matter. This post is concerned with off-policy policy evaluation, or evaluating policies with data generated by others. The ambition is to provide a smooth progression towards the Retrace estimator and some of its extensions, with a focus on each operator&rsquo;s properties and stochastic approximation…