Building on Paul Revere

How it started

After playing thru Adam’s submission for Summer Into AI (Link below), I had a thought. What if I set the universe engine on the puzzle of building out a pixelated world, bring that into a video, and then go from there? What would happen exactly? After all, I’ve done lots of realistic looking stuff but never tried to intentionally do a down-sampled 8bit like universe. I sent Codex the prompt to use the universe engine and the length of a movie I was looking for, and went to bed.

When I woke up, it was one of those stirring moments where my eyes were open but the world was impossibly vague. Was that sunlight or have I gone blind? Is this just the world I live in now? One foot followed by another found their way to the ground. Then, I stumbled into the kitchen.

It was a blurred morning of memories returning back online. I recalled watching the Yankees crush the White Sox, a bit of Soccer, and an unreasonably effective gym workout that made every step into the kitchen feel like I wasn’t just making my way to make coffee. Oh, no. I was climbing K2, in pajamas, and the top of this mountain was something far better than a view. It was a coffee machine. The Oracle at Delphi might have divined answers from a cave. Anything I know originates from the machine.

When that first cup percolated and poured into the porcelain mug I carried along this journey, I sat down in the living room and noticed my laptop had a message waiting. The message was simple in tone, “Hey Eric, figured it out and got the project done last night.”

“Oh?” I was just expecting a 50/50 result from the agent running last night. I gave it a pretty absurd task and didn’t think much about it. After reviewing what it did-I’ll show those details in a moment, I couldn’t help but think the agent had some level of extra engagement.

As I reviewed the outputs it was as if the agent were saying, “Yes, Eric. It’s done. I hope you don’t mind. I had fun with this one.”

“Huh? I’ll check it out.”

So, I navigated to the output folders the universe engine is set to produce content and opened up the “Paul Revere” section. I opened the output video and was… shocked. The universe engine skipped using the video models for the task and just built a custom 2d side-scroller over itself. And not just that, but then it took the idea Adam had run with and built out this recorded scene using “Codex People” as these little 8bit creations in a world that… made sense? I had no input into this one-I was asleep.

Codex even assembled a musical score. Have you ever explored having a model just “oh I’ll toss on a musical score just to make it more complete” as a side effect? I’ve done so many vibe coded games. Never once did I get a bonus musical score-until this time.

It’s playing in the recorded scene. Codex created a musical generation system and produced this sound. No guidance from me-it just showed up. After a few listens I’ve concluded that I just haven’t tested these AI systems and their musical capabilities nearly enough. Further, the model added this music generation system into the universe engine as an 8bit music system. If I didn’t know better, Codex was having fun with this task-it went way further than it normally does in getting the work done.

Here’s the audio clip if you would like to use it for something.

Here’s the crazy thing though-Codex used the frame generation capture system to follow this 8-bit character and then rendered it one frame at a time. The entire 3 minutes. Technically, that is how movies are made and clearly the assembled version does work at a frame rate that can be “watched”.

These assets made me realize: Codex climbed up K2 first, while I was asleep.

So that happened.

Upgrades

This took me on a path where I wondered what would happen if instead of doing this 8-bit inspired run, I bumped up the graphic level to something a little higher. Say a 16 or 32 bit inspired run. Here’s some reference art that I’ve done that you’re free to use. If you do use it, let me know!

The first test of this expanded setup turned out rather fun!

Assembled together how did the plan go? Well, turns out a video model is capable of doing sprite art. This is… actually great news- I didn’t realize I was only scratching the surface of what’s possible when I wrote about it for John’s World 1995!

Watch a simulated video game-a game that doesn’t quite exist. Perhaps the closest place it does exist is the link to Adam’s substack below.

Try Adam’s Paul Revere submission game here. It’s what inspired this post.