Upcoming weekend project idea. a smallNN powers the puppets and a multi-tier layered system wires it all together.
The shape of the weekend build
The interesting part is not just that a small neural network can choose actions. It is where that little controller sits in the stack. The puppet has a body, the world has state, and the bridge has to translate between observation, action, and time without turning the whole system into one brittle monolith.
In this sketch, the tiny model is deliberately close to the world engine: sensory input, hidden state, and action output per avatar. The larger system around it handles the practical plumbing: world observation, event streaming, serialized state, viewport overlays, and per-agent debugging.
- Embodied avatars are the visible puppets the system can inhabit, inspect, and steer.
- OpenClaw gateway exposes the useful verbs: observe the world, act in the world, read neural state, and stream events.
- World bridge server keeps the state and event bus legible enough for agents and tools to cooperate.
- 2D world engine is where each puppet gets its small neural controller and reward signals.
- Viewport overlay makes the hidden machinery visible: render, activation HUD, and per-agent debug layer.
Why keep the model small?
A small network is easier to watch, reason about, reset, and tune. That matters when the goal is not raw benchmark performance but a living toy system where behavior, memory, and environment all need to remain understandable. The weekend-sized version is small enough to build, instrument, and break in useful ways.
This also fits the larger OpenClaw pattern: keep the agent interface explicit, keep the state inspectable, and let the experiment teach the architecture where it wants more complexity.