It took some effort to get this one to run for me. I wrote notes on what I did at the comments.
It takes a bit to understand what’s going on. There’s a zero-player training mode, where the game spawns different colored agents with different inherent preferences and lets them learn how to reach a goal. Then there’s a single-player playthrough mode, where you control a yellow agent and you have to get to your goal without being stepped on by any of the AI agents. The two modes don’t have much overlap, except that AI agents can train on a map and then you can play through that map with the agents trained as they are. (But they are not trained to attack the player agent, only to get to an exit, and if they step on the player it is an accident.)
They training mode is visually cool to watch, with the multicolored squares rapidly flickering across the grid. There’s a “Slow” mode where you can see the individual moves, with tweened interpolation between grid squares—a nice touch. It was hard to get an appreciation for what the training was actually doing. I would let it run for a few minutes, and usually I could not see any obvious difference in the agents’ behavior after that time. I wasn’t sure how long I was supposed to wait—minutes, hours? In the “four colors” map, for example, it did not appear that the blue agents learned a preference for moving right, the direction to the goal from their starting location. The exception was the “red and orange” map, where within a few seconds I saw a change in the red agents from random wandering to hugging the wall and moving up toward the goal, where the orange agents did not develop such a pattern and reached the goal only rarely. As I understand it, the red agents’ preference for proximity to other agents and to walls, and the orange agents aversion to the same, are hard-coded into their utility functions, so I wasn’t sure how much of what I was seeing was learned and how much was innate.
It was not clear to me what information is available to the agents. Is it only their 8-square immediate neighborhood that they can “see”? Is it more than that? If it’s only a local neighborhood, do the agents learn to arrange themselves into patterns that signal to other agents earlier on the path what their next move should be? How much randomness remains after a behavior is learned? If blue and green agents are rewarded only when they reach a goal, how does their training affect their behavior when not next to the goal?
The playthrough mode feels like playing a turn-based roguelike.
I could not discern any very meaningful strategy to follow—it
seemed to come down to luck whether the AI agents would leave and opening,
or, when adjacent to the player, choose to move onto the player’s square or not.
I beat “level 1”, “level 2”, and (after several attempts) “a hard level”,
but I could not beat “level 3”.
The blue agents are too dense in the corridor
and it seems like it would require extraordinary luck for a path to open.
A few times I wished for a . command to remain in place
while the AI agents move.
I was thinking of ways this kind of agent training might be applied. It could be used in a traditional grid-based roguelike, for NPC behavior that is more interesting than simply shortest-pathing towards the player. Another thing that comes to mind is real-time enemy hordes like in Gauntlet. The agents could be trained to effectively attack or surround the player—or on the contrary, could be trained to coordinate with each other and “pull their punches”, allowing only a few to attack at a time, so the mass of enemies looks more intimidating than it really is. Perhaps the player could be on the side of the agents (I’m thinking like the yellow ant in SimAnt) and the AI agents could learn from the player agent—maybe the human player could demonstrate strategies that the AI agents would be unlikely to find on their own.
Thanks for this submission. It’s a creative one and it sparked quite a few thoughts in me!
