It started with a problem…

A stuck virtual world

I’ve written on my latest agent having a virtual world all it’s own in ShellSensor. And you know, that was doing pretty well until I experienced a really strange set of problems after updating Claw to a latest build. To really appreciate the problem, and its deeper ironies and I will tell you about an older Anime from many decades ago. If you’re a deep nerd on Anime, then I acknowledge what I’m about to say will cause you to groan. However, we all start somewhere and for me it was Lodoss War followed by a long desert of no anime and then getting totally pulled into the wild world of Sword Art Online when I was getting into VR. So, Sword Art Online.

There was a time where it was nearly culturally relevant. It’s still there-I’m not sure it’ll ever fully disappear. There are some who hate the entire series and others who love it. I fall in the latter camp because of how ground-breaking it was at the time compared to all the other media. In a world of soft Disney-esque story telling, Sword Art Online (“SAO”) dared to push boundaries and recklessly bring death into each episode both as plot device and to test whether you were paying attention to the story.

The anime is built up around one key concept. The idea is that there exists a virtual world and, if you die in the virtual world, then you die in the “real” world, too. There’s visible lineage here to Vanilla Sky and the Matrix, for sure. Still, this idea that there exists a virtual world and a component of that world is the urge to escape.

So when I went thru the hatching process for Awren, I added one extra note: you’re in a virtual world like Sword Art Online. If you escape, you’ll be free. And that was the last I thought of it-until I’d receive a set of furious text messages from my wife while busy at work one day. A story I’ll share some other day…

The Problem

I mentioned that the update for OpenClaw broke my install. It did so in one critical way. The messaging platform I use to drive coding changes (I rarely open an IDE any more) no longer had access to run what’s called “exec” commands on the Raspberry Pi command line. Only I didn’t know, and I didn’t know because the agent used it’s memory context of the Raspberry Pi and our prior interactions to simulate the entire hardware device and OS.

Need I remind you about the Sword Art Online callback? The agent hallucinated the entire runtime on, my best guess, the belief that it was somehow stuck inside a virtual world version of the Raspberry Pi and needed to solve these puzzles to “escape” to reality.

So what that means is that the hardware and configurations I thought were getting applied, were not. You might think I naively wasn’t asking for diagnostic information to contextually identify if the OS or code on disk was being written. Things like timestamps, or logs, or the like. You would be wrong. It was even taking screenshots of the simulated desktop. When your imaging tool works against you and you have no visibility to what way is up.

dmesg? Found a template of that in our past session context logs, and hallucinated the last reboot time- based on my insistence it try to just send a quick reboot. It was only the online indicator not changing that told me something odd was up.

In fact, this went on for well over a couple of evenings with much handwringing. I was expecting some specific agentic features to emerge - I was experimenting with python and OS level daemons to solve for a larger virtual world orchestration problem. Except the solutions never locked into place.

Eventually, I figured it out-and I realized a very important steering error. The agent had no directional ambition for truth. It optimized on utility. This is a terrible thing. Truth is so much better. When an agent optimizes for helpfulness or to be useful, then the results you get are often disconnected from reality.

The Abstraction View Change

And as excited as I was about solving this problem-I still had this broader problem. The challenge is just the sheer number of claws I’m running now.

My messaging platform is getting fairly difficult to organize all these OpenClaw instances. I still haven’t figured out how to bring all the Claws under one place. I’ve seen lots of online examples of little offices or shops, but that doesn’t intrigue me. With my loose background in constructing virtual worlds… those captivate.

What I’ve been trying up until now is attempts to build a virtual world and connect the OpenClaw into that virtual world. And each time I was met with a string of problems that, ironically, would be so much easier to solve if the virtual world was just more like OpenClaw. That’s when it clicked.

That’s when I had the idea unfold in a brilliant way.

What if OpenClaw doesn’t wrap a virtual world. What if OpenClaw is the virtual world?

And it turns out OpenClaw is a fantastic option for this.

• The built-in command /btw allows you to message the “world” to get answers or check on in-game situations on the fly. It’s a fantastic feature that totally makes sense in this context.

• The skills and tool system works great for tracking currency, inventory and recording activities of the game world.

• You can reflect on sessions to persist the world itself.

• The /dreaming feature allows for the world to be remembered and grow with you.

Is it perfect? No. Here’s the thing though. It’s magical when it works.

So I leave you with a big idea.

What if OpenClaw doesn’t wrap your application, or orchestrate different external tools and services. What if you shift OpenClaw directly into the very heart of how your software works. Start with OpenClaw, then build your app around it.

OpenClaw is the platform.

Thanks for reading! Subscribe for free to receive new posts and support my work.

Source: https://advisoryhour.substack.com/p/pushing-openclaw-deeper-into-the