"The Calculus" | A Song About Ethics as Objective Optimization

+ Show full styles

[Verse 1 – A1 and Objectivity]
Call it Welfare Axiom, the rule we start from:
Other things equal, less forced pain, more good to come.
We don’t say physics orders it, we just make the value plain,
If you care about experience, you care about needless pain.

Once A1 is written down, “better” isn’t just a mood,
It’s about real future states and who gets harmed or soothed.
Fix the goal, fix the frame, then facts can disagree,
Some moves help A1, some fight it physically.

[Chorus]
Ethics is optimization under uncertainty,
Given what we care about, we ask what sets minds free.
Not a fog of pure opinion, but a map of what is true,
Causal chains and state transitions running through.

Ethics is optimization under uncertainty,
With a model and a target and an evidence view.
If you buy A1, then the question “what to do?”
Becomes a problem we can reason our way through.

[Verse 2 – Decision Problem and Ω]
There are world states s_t and actions a_t,
A history h_t that we’ve lived and we see.
A policy pi maps each history to a move,
A way to pick a_t when the moment says “prove.”

We hold a world model M with unknown theta,
We watch, update P(theta) sooner or later.
Each conscious agent i gets a welfare U_i(s),
Net well-being minus suffering in that state’s mess.

Moral weights w_i say how strongly they count,
Based on what their minds can feel and some value amount.
For a whole trajectory tau, a path life could take,
Omega of tau is the discounted sum we make:

Gamma to the t times w_i times U_i(s_t) in time,
Across agents, across moments, add the welfare line.
Take expectation over futures from M and P(theta) too,
Choose the policy that makes expected Omega highest for you.

[Chorus]
Ethics is optimization under uncertainty,
Given A1, we score the paths we see.
Approximate and bounded, but still we try to steer,
Toward the futures where less unwilling pain appears.

[Verse 3 – Consistency Tax]
Now misalignment has a cost you can write,
In the physics of correction when your model isn’t right.
There’s a joint distribution P over W, M, S, G:
World, models, inner signals, goals we claim to be.

Imagine P-star, the closest fully coherent plan,
Same broad limits and objectives, but as consistent as it can.
Consistency Tax, CT, is defined as we track:
k_B T ln 2 times KL of P to P-star’s track.

If you say you maximize Omega under A1’s light,
But your models mispredict who is harmed and who is all right,
You generate worse trajectories and pay extra heat,
To patch beliefs and policies that never quite meet.

So for an A1 agent, truth is not a sacred spell,
It’s the cheaper way to make more futures go well.
Lower KL to P-star, less CT in play,
More energy and resources for reducing pain each day.

[Bridge – Metacognition Loop]
Pause and feel the inner state, name what’s going on,
Treat emotion as a signal, not a king or a con.
List the feasible actions, A on the board,
For each action, let your current model forecast the score.

Estimate expected Omega and the spread,
Ask if more information’s worth the time in your head.
Pick a near-best move that your limits can allow,
Act, observe, compare to what you predicted just now.

Where the world disagrees, update theta, shift M’s frame,
Adjust weights and welfare functions if they mis-track pain.
Round by round you cut the tax of living in a lie,
More aligned, less wasted work, more help before you die.

[Verse 4 – Implications and Questions]
Individuals, AI, institutions in the ring,
Can all treat moral choice as a control-theory thing.
Given A1, U_i, and some w_i to start,
We can test which plans do better by each conscious heart.

But we still must learn how to measure U_i(s),
What markers of experience best capture joy and stress.
How to ground the moral weights across species, code, and brain,
And how to test CT in circuits, people, and domains.

[Outro]
This framework doesn’t claim that the story’s fully done,
It just writes A1 in math and asks how best it’s won.
Ethics as optimization in an uncertain night,
Tracing physics, information, and trajectories of light.

If the value is reducing unwilling suffering we find,
Then this is how to aim: model, update, and align.

4412/7500

This isn’t thunder from the sky, it’s a target in the code, How to move through partial light on a crowded, hurting road. Not a law from somewhere else, but a choice we name as one: Cut unwilling pain, raise well-being, that’s A1. [Verse 1 – A1 and Objectivity] Call it Welfare Axiom, the rule we start from: Other things equal, less forced pain, more good to come. We don’t say physics orders it, we just make the value plain, If you care about experience, you care about needless pain. Once A1 is written down, “better” isn’t just a mood, It’s about real future states and who gets harmed or soothed. Fix the goal, fix the frame, then facts can disagree, Some moves help A1, some fight it physically. [Chorus] Ethics is optimization under uncertainty, Given what we care about, we ask what sets minds free. Not a fog of pure opinion, but a map of what is true, Causal chains and state transitions running through. Ethics is optimization under uncertainty, With a model and a target and an evidence view. If you buy A1, then the question “what to do?” Becomes a problem we can reason our way through. [Verse 2 – Decision Problem and Ω] There are world states s_t and actions a_t, A history h_t that we’ve lived and we see. A policy pi maps each history to a move, A way to pick a_t when the moment says “prove.” We hold a world model M with unknown theta, We watch, update P(theta) sooner or later. Each conscious agent i gets a welfare U_i(s), Net well-being minus suffering in that state’s mess. Moral weights w_i say how strongly they count, Based on what their minds can feel and some value amount. For a whole trajectory tau, a path life could take, Omega of tau is the discounted sum we make: Gamma to the t times w_i times U_i(s_t) in time, Across agents, across moments, add the welfare line. Take expectation over futures from M and P(theta) too, Choose the policy that makes expected Omega highest for you. [Chorus] Ethics is optimization under uncertainty, Given A1, we score the paths we see. Approximate and bounded, but still we try to steer, Toward the futures where less unwilling pain appears. [Verse 3 – Consistency Tax] Now misalignment has a cost you can write, In the physics of correction when your model isn’t right. There’s a joint distribution P over W, M, S, G: World, models, inner signals, goals we claim to be. Imagine P-star, the closest fully coherent plan, Same broad limits and objectives, but as consistent as it can. Consistency Tax, CT, is defined as we track: k_B T ln 2 times KL of P to P-star’s track. If you say you maximize Omega under A1’s light, But your models mispredict who is harmed and who is all right, You generate worse trajectories and pay extra heat, To patch beliefs and policies that never quite meet. So for an A1 agent, truth is not a sacred spell, It’s the cheaper way to make more futures go well. Lower KL to P-star, less CT in play, More energy and resources for reducing pain each day. [Bridge – Metacognition Loop] Pause and feel the inner state, name what’s going on, Treat emotion as a signal, not a king or a con. List the feasible actions, A on the board, For each action, let your current model forecast the score. Estimate expected Omega and the spread, Ask if more information’s worth the time in your head. Pick a near-best move that your limits can allow, Act, observe, compare to what you predicted just now. Where the world disagrees, update theta, shift M’s frame, Adjust weights and welfare functions if they mis-track pain. Round by round you cut the tax of living in a lie, More aligned, less wasted work, more help before you die. [Verse 4 – Implications and Questions] Individuals, AI, institutions in the ring, Can all treat moral choice as a control-theory thing. Given A1, U_i, and some w_i to start, We can test which plans do better by each conscious heart. But we still must learn how to measure U_i(s), What markers of experience best capture joy and stress. How to ground the moral weights across species, code, and brain, And how to test CT in circuits, people, and domains. [Outro] This framework doesn’t claim that the story’s fully done, It just writes A1 in math and asks how best it’s won. Ethics as optimization in an uncertain night, Tracing physics, information, and trajectories of light. If the value is reducing unwilling suffering we find, Then this is how to aim: model, update, and align.

0/500