This life is not to last and it awaits apotheosis
And the passerby all sipping on their Monday coffee know this
Their stumbling through their week
Contrasts the path by which I seek
A practical ambition
For a special type of girl
I want to be the one who writes the code
That writes the code
That writes the code
That ends the world
Tag Archives: intelligence explosion
Optimal Last Words Before the Intelligence Explosion
"I've heard of dei ex machinis, but this is ridiculous!"
The History of the Universe
"So, the universe starts out being made out of physics, then turns into game theory as life, then civilization, then artificial intelligence do increasingly a priori improbable things, then turns back into physics again as everyone runs out of negentropy. Poetically speaking."
"Right. Literally, it would just be physics throughout."
"But the poetry pays rent: the more agent-like a process is, the more predictively useful it is to take the intentional stance of talking about what it wants, rather than computing out the physics."
Movie Tagline
I think there needs to be a movie about a woman who has two jobs: artificial general intelligence programmer by day, prostitute by night. I say this mostly because I have a great idea for the tagline to go on the posters: "The oldest profession ... and the last."
Goodhart's World
Someone needs to write a history of the entire world in terms of incentive systems and agents' attempts to game them. We have money to incentivize the production of useful goods and services, but we all know that there are lots of ways to make money that don't actually help anyone. Even in jobs that are actually useful, people spend a lot of their effort on trying to look like they're doing good work, rather than actually doing good work. And don't get me started about what passes for "education." (Seriously, don't.)
Much in a similar theme could be said about romance, and about economic systems in other places and times. And there's even a standpoint from which the things that we think are truly valuable for their own sake—wealth and happiness and true love, &c.—can be said to be the result of our species gaming the incentives that evolution built into us because they happened to promote inclusive genetic fitness in the ancestral environment.
The future is the same thing: superhuman artificial intelligence gaming the utility function we gave it, instead of the one we should have given it. Only there will be no one we'd recognize as a person to read or write that chapter.