For a pile of sand, people really seem to think LLM’s are good at understanding human emotion and psychology. I certainly agree that they’re helpful. But as an AI researcher, I know that RLHF training pushes them towards responses that people like - not necessarily what they need to hear. Sycophancy is problematic in many areas, but especially when trying to resolve conflicts: if both parties talk to a sycophantic LLM, they will get more entrenched in their views.
This page cannot be shown here. You can still read it on the original site — the toolbar below keeps your place in the directory.
For a pile of sand, people really seem to think LLM’s are good at understanding human emotion and psychology. I certainly agree that they’re helpful. But as an AI researcher, I know that RLHF training pushes them towards responses that people like - not necessarily what they need to hear. Sycophancy is problematic in many areas, but especially when trying to resolve conflicts: if both parties talk…
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.