www.lesswrong.com/posts/H8uoAmbeqjD2PG2jm/irrationality-as-a-defense-mechanism-f...
1 correction found
preferences are usually encoded as priors over observations, but ironically these are never updated.
Active inference research includes explicit methods for learning/updating an agent’s prior preferences (priors over observations). So it’s not correct that these priors are “never updated.”
Full reasoning
The post claims that in active inference, preferences (encoded as priors over observations) are “never updated.” However, the active inference literature includes preference learning approaches where an agent’s prior preferences over outcomes/observations are learned/updated from data or demonstrations.
-
Sajid et al. (2019/2021) explicitly describe scenarios where, within active inference, behaviors are learned via preference learning, including “learning the prior preferences over the observations corresponding to reward.” This directly contradicts the absolute claim that such priors are “never updated.”
-
Shin et al. (2021) propose a method for “learning a prior preference from experts” in an active inference framing—again contradicting “never updated.”
Because the post’s statement is absolute (“never”), the existence of well-documented active inference work that does update/learn prior preferences is sufficient to show the claim (as written) is incorrect.
2 sources
- Active inference: demystified and compared (Sajid, Ball, Parr, Friston)
“…agent behaviors are learned through preference learning… by … learning the prior preferences over the observations corresponding to reward.” (abstract)
- Prior Preference Learning from Experts: Designing a Reward with Active Inference (Shin, Kim, Hwang)
“…we propose a… method for learning a prior preference from experts.” (abstract)