Here is how we plan to run the month. We are still preparing, so
details will change.
Days share the same shape; the content changes. This is the
current plan.
The venue is in the Portuguese countryside, with walks right
outside and whiteboards everywhere.
To exemplify what we mean by this we will share a prototypical
discussion which happened at our first seminar, as an example
of what we want to have happen again.
A central aspect of the alignment problem is
embedded agency.
Traditional theoretical approaches towards modeling agents fail
to carry over when modeling embedded agents: for example, a
standard Bayesian agent has a prior over hypotheses, each of
which is a complete probabilistic model of the environment. The
nice theorems (convergence to the truth, low regret) assume
realizability: the true environment is one of
the hypotheses in your prior.
For an agent embedded in the real world, that assumption is
false. The world contains the agent, other agents as smart as
it, and far more detail than it can represent, so the true
environment can’t sit inside its hypothesis space. Without
realizability, the Bayesian guarantees can fail badly.
Infra-Bayesianism
is a modification of Bayesian reasoning for agents that
can’t fully model their world. It provides a mathematical
theory of what a coherent embedded agent can look like.
Coming from recent advances in theoretical neuroscience is a
completely different approach, called Active Inference. Active
Inference tries to answer the question: “can we describe
intelligent behavior as we see in the world around us from
first (physical) principles?” It is related to the theory
of Predictive Processing, which says that our brains are
prediction machines that minimize prediction error. Active
Inference basically says that all systems that maintain their
self-organization over time (approximately) minimize prediction
error, by minimizing an upper bound of it called free energy.
This is also a description of what a coherent embedded agent
can look like.
Participants were introduced to both ideas during the seminar;
some of them had a background understanding of
Infra-Bayesianism and some had a background in Active
Inference.
In the readings and workshops that we held, people inevitably
tried to make sense of these somewhat notoriously complicated
topics. Some of our fellows gave a
presentation
on the intersection.
Active Inference and Infra-Bayesianism actually don’t
share all that much in common except that they both describe
(very different) visions of what a coherent (somewhat Bayesian)
embedded agent can look like. Their approach towards normativity
is very different. Infra-Bayesianism’s main hope is to
build a mathematical language in which you can state and prove
things about agents in a world bigger than themselves, which
could be used to build some alignment protocol on top. Active
Inference was not developed for the purpose of alignment, but
rather as a general framework for thinking about intelligent
(primarily biological) systems as being approximate
Bayesians.
This kind of debate connects to a much more general debate of
the role of theoretical approaches to alignment. Theories can be
descriptive (what best explains our observations of the natural
world) vs prescriptive (aiming for knowledge like “X
algorithm works best for Y problem”). They can be
motivated almost entirely mathematically, or make various levels
of contact with other fields and empirical observations. These
discussions seem to naturally bottom out in some degree of
philosophy of science. You can see the following
post
and the following
comment,
which is essentially the spiritual successor to the discussions
had at AFFINE I.
We are still very confused about this intersection, but it is
an interesting confusion which might lead to less confusion in
the future. Being productively confused is in the spirit of
AFFINE.
The questions we care about, and why we are not sure they are
the right ones yet, are on the
Approach page.
Applications close on
22 November 2026. Dates, costs
and the FAQ are on the program page.
Applications for the January 2027 seminar are closed.
Ask to be notified about the
next one, or read more on the program page.