Abstract:We introduce three concepts that describe an agent's incentives: response incentives indicate which variables in the environment, such as sensitive demographic information, affect the decision under the optimal policy. Instrumental control incentives indicate whether an agent's policy is chosen to manipulate part of its environment, such as the preferences or instructions of a user. Impact incentives indicate which variables an agent will affect, intentionally or otherwise. For each concept, we establish sound and complete graphical criteria, and discuss general classes of techniques that may be used to produce incentives for safe and fair agent behaviour. Finally, we outline how these notions may be generalised to multi-decision settings. This journal-length paper extends our conference publications "Incentives for Responsiveness, Instrumental Control and Impact" and "Agent Incentives: A Causal Perspective": the material on response incentives and instrumental control incentives is updated, while the work on impact incentives and multi-decision settings is entirely new.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| ACM classes: | I.2.6; I.2.8 |
| Cite as: | arXiv:2001.07118 [cs.AI] |
| (or arXiv:2001.07118v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2001.07118 arXiv-issued DOI via DataCite |
Submission history
From: Ryan Carey [view email]
[v1]
Mon, 20 Jan 2020 14:32:07 UTC (40 KB)
[v2]
Mon, 15 Mar 2021 20:02:54 UTC (40 KB)
[v3]
Mon, 23 Jun 2025 03:26:44 UTC (84 KB)