Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Bounds
The Chinese University of Hong Kong · Indiana University · Department of Computer Science and Engineering, The Chinese University of Hong Kong · Cornell University
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Learning a transition model via Maximum Likelihood Estimation (MLE) followed by planning inside the learned model is perhaps the most standard and simplest Model-based Reinforcement Learning (RL) framework. In this work, we show that such a simple Model-based RL scheme, when equipped with optimistic and pessimistic planning procedures, achieves strong regret and sample complexity bounds in online and offline RL settings. Particularly, we demonstrate that under the conditions where the trajectory-wise reward is normalized between zero and one and the transition is time-homogenous, it achieves nearly horizon-free and second-order bounds.