RL's Deadly Triad Meets Optimization

\( \newcommand{\phit}{\phi_t} \newcommand{\phinext}{\phi_{t+1}} \newcommand{\enext}{\ebf_{s_{t+1}}} \newcommand{\et}{\ebf_{s_t}} \newcommand{\gradV}{\nabla_{\theta} V_{\theta}} \newcommand{\V}{V_{\theta}} \newcommand{\snext}{s_{t+1}} \newcommand{\tstar}{\theta^\star} \newcommand{\itmtx}{(\Ibf - \eta \Abf)} \newcommand{\AMC}{\Abf_{\text{MC}}} \newcommand{\Aon}{\Abf_{\text{on}}}…