Why You Should Start an AI PhD Now
1. You are pushing the boundary of this industrial revolution
Puyuan Peng
Personal website of Puyuan Peng
Latest posts
Why You Shouldn’t Start an AI PhD Now
1. Research direction is largely decided by industry
PhD in AI – My Experience
In October 2024, I contacted 9 companies for research positions in AI. I received interview invites from 8 of them, finished full interview loops with 6, and got offers from 4. The total market cap of the 4 companies that offered me a research scientist job was $10 trillion by mid-September 2025. I chose Meta.
Deep RL 12 Reinforcement Learning and Control as Probabilistic Inference
Please checkout Professor Sergey Levine’s excellent tutorial: Levine 18’
做一个更快乐的博士生
[文] Kevin Gimpel
Deep RL 11 Model-Based Policy Learning
In this section, we study how to learn policies utilize the known (learned) dynamics. Why do we need to learn a policy? What’s wrong with MPC in the previous lecture? The answer is that MPC is still an open loop control methods, even though the replanning machanism provides some amount of closed-loop capability, but the planning procedure still is unable to reason under the fact that more…
Deep RL 10 Model-based Reinforcement Learning
Previous lecture is mainly about how to plan actions to take when the dynamics is known. In this lecture, we study how to learn the dynamics. We will also introduce how to incorporate planning in the model learning process and therefore form a complete decision making algorithm.
Deep RL 9 Model-based Planning
Let’s recall the reinforcement learning goal — we want to maximaze the expected reward (or expected discounted reward in the infinite horizon case)
Deep RL 8 Advanced Policy Gradient
At the end of previous lecture, we talked about the issues with Q-learning, one of them is that it’s not directly optimizing the expected return and it can take a long time before the return starts to improve. On the other hand, policy gradient methods are direclty optimizing the expected return, although we cannot guarantee that the return will improve every gardient update. At the same time, we…
Deep RL 7 Q-learning
In this section we extend the online Q-iteration algorithm in the previous lecture by identifying the potential issues and introducing solutions. The improved algorithm can be very general and contains famous special cases such as DQN.