Skip to main content

Indie game storeFree gamesFun gamesHorror games
Game developmentAssetsComics
SalesBundles
Jobs
TagsGame Engines
A jam submission

GAWA: Game about walking agentsView game page

A game where you can make an AI play through levels you’ve created
Submitted by IgorAnt028 — 5 hours, 39 minutes before the deadline
Add to collection

Play game

GAWA: Game about walking agents's itch.io page

Results

CriteriaRankScore*Raw Score
Creativity - how original is the idea?#201.4144.000
Enjoyment - how fun is the game?#220.7072.000
Presentation - how does it look/feel/sound?#220.7072.000
Overall#220.9432.667

Ranked from 1 rating. Score is adjusted from raw score by the median number of ratings per game in the jam.

Leave a comment

Log in with itch.io to leave a comment.

Comments

Submitted (2 edits) (+1)

It took some effort to get this one to run for me. I wrote notes on what I did at the comments.

It takes a bit to understand what’s going on. There’s a zero-player training mode, where the game spawns different colored agents with different inherent preferences and lets them learn how to reach a goal. Then there’s a single-player playthrough mode, where you control a yellow agent and you have to get to your goal without being stepped on by any of the AI agents. The two modes don’t have much overlap, except that AI agents can train on a map and then you can play through that map with the agents trained as they are. (But they are not trained to attack the player agent, only to get to an exit, and if they step on the player it is an accident.)

They training mode is visually cool to watch, with the multicolored squares rapidly flickering across the grid. There’s a “Slow” mode where you can see the individual moves, with tweened interpolation between grid squares—a nice touch. It was hard to get an appreciation for what the training was actually doing. I would let it run for a few minutes, and usually I could not see any obvious difference in the agents’ behavior after that time. I wasn’t sure how long I was supposed to wait—minutes, hours? In the “four colors” map, for example, it did not appear that the blue agents learned a preference for moving right, the direction to the goal from their starting location. The exception was the “red and orange” map, where within a few seconds I saw a change in the red agents from random wandering to hugging the wall and moving up toward the goal, where the orange agents did not develop such a pattern and reached the goal only rarely. As I understand it, the red agents’ preference for proximity to other agents and to walls, and the orange agents aversion to the same, are hard-coded into their utility functions, so I wasn’t sure how much of what I was seeing was learned and how much was innate.

It was not clear to me what information is available to the agents. Is it only their 8-square immediate neighborhood that they can “see”? Is it more than that? If it’s only a local neighborhood, do the agents learn to arrange themselves into patterns that signal to other agents earlier on the path what their next move should be? How much randomness remains after a behavior is learned? If blue and green agents are rewarded only when they reach a goal, how does their training affect their behavior when not next to the goal?

The playthrough mode feels like playing a turn-based roguelike. I could not discern any very meaningful strategy to follow—it seemed to come down to luck whether the AI agents would leave and opening, or, when adjacent to the player, choose to move onto the player’s square or not. I beat “level 1”, “level 2”, and (after several attempts) “a hard level”, but I could not beat “level 3”. The blue agents are too dense in the corridor and it seems like it would require extraordinary luck for a path to open. A few times I wished for a . command to remain in place while the AI agents move.

I was thinking of ways this kind of agent training might be applied. It could be used in a traditional grid-based roguelike, for NPC behavior that is more interesting than simply shortest-pathing towards the player. Another thing that comes to mind is real-time enemy hordes like in Gauntlet. The agents could be trained to effectively attack or surround the player—or on the contrary, could be trained to coordinate with each other and “pull their punches”, allowing only a few to attack at a time, so the mass of enemies looks more intimidating than it really is. Perhaps the player could be on the side of the agents (I’m thinking like the yellow ant in SimAnt) and the AI agents could learn from the player agent—maybe the human player could demonstrate strategies that the AI agents would be unlikely to find on their own.

Thanks for this submission. It’s a creative one and it sparked quite a few thoughts in me!

Developer(+1)

Thanks for the feedback.

I hadn’t even considered a tutorial. Now, the need for one seems quite obvious.

Originally, the game was intended to be about navigating through crowds in busy spaces. But along the way, I realized that simply creating levels and watching the agents navigate them was much more fun. You correctly identified the issue: there’s no logic to the agents' movements, which makes it uninteresting. I removed the "wait" button because it made the game too easy. I’m not sure how to make this particular mode engaging.

How quickly the agents learn the correct path is random. They might learn it within the first 30 seconds, or they might never learn it at all.

To understand what follows, it’s helpful to know how the algorithm works. When an agent receives a reward, it associates its previous actions with the total reward value it eventually earned. For instance, if an agent starts in state S, takes 3 steps, and receives a +1 reward at each step, it will store a value of approximately 3 for the actions it took. The next time it finds itself in state S, it will "know" that taking one of its actions leads to a reward of 3. I’m not sure if that’s a clear explanation, but it’s the best I could come up with (assuming I’ve understood how the algorithm works myself, of course).

This happens because their "vision" is limited to their immediate surroundings - just one cell in any direction. Consequently, they might reach some very strange conclusions about which action to take in a given cell to minimize the likelihood of future problems. You noticed this while observing the orange agents. The orange and red agents aren't hard-coded. Rather, they receive negative rewards. For example, the red agents receive +1 for reaching the finish line (as does everyone else) and -1/500 for every wall or other agent they come into contact with. The orange agents receive -1/500 for every obstacle they encounter. In theory, the red agents could choose a path without walls if they deemed it more advantageous—for instance, if the path containing walls were blocked by other agents, such as the blue ones. However, since there is no penalty for taking a longer route, they generally choose to head toward a wall. This is where the orange agents employ a clever tactic: an orange agent (let’s call it Agent A) avoids moving toward the reward, as other agents might already be there - and since its field of vision is limited to the single cell it occupies and those immediately surrounding it, it cannot know whether the path ahead is clear. Consequently, it waits until another agent (Agent B) happens to approach it. At that point, remaining stationary is no longer advantageous—as it incurs a penalty of -1/500 per turn—so the agent chooses to move elsewhere. This becomes especially apparent when randomness is disabled: the agents end up forming a grid-like pattern. Just take a look:

Furthermore, they are prepared to wait indefinitely; there is no penalty for standing still, whereas moving - which would inevitably lead to a collision with another agent - does carry a penalty.

The algorithm has a specific quirk: it lacks the concept of a "similar state." It perceives every state as entirely distinct from all others. For example, if an agent is positioned right next to the finish line and has, on previous occasions, chosen an action that successfully led it to the finish, it will unhesitatingly choose that same action again. But if another agent happens to stop nearby, the first agent no longer knows how to proceed - it has never encountered this specific situation before, even though the action that would lead it to the finish line remains perfectly viable. And now, it needs to choose the same action a sufficient number of times - but in a new situation - which will require many (possibly a great many) attempts. Yet, there are 256 such possible states for every single cell. This is the reason why their field of vision is limited to just one cell. If they could see even two cells away, the number of possible states at each point would be 16,777,216 (2^24).

All tabular reinforcement learning (RL) methods follow this pattern, but there is also "Deep RL," which utilizes neural networks. You may have heard that this approach has been used to train agents to play games like Dota 2, Doom, and others. However, such algorithms are extremely resource-intensive. Integrating them into games featuring a large number of agents would simply overwhelm the player's computer. Furthermore, there is another reason why these algorithms are only applicable in very rare cases within game development: they are entirely unpredictable. I could never have foreseen that the orange agents would come up with the strategies they did. Imagine you are a game designer who wants enemies to attack the player - for instance, by rewarding them for moving in a chain or a single-file wall (like a shield wall), while still allowing them to choose a different tactic if forming that wall would lead to certain death. At the same time, these agents are also rewarded for dealing damage to the player, healing allies, or moving as a group rather than individually. In this scenario, the agents might simply ignore the reward for moving in a chain as it’s more efficient to just rush the player and attack, as that’s faster than wasting time on formation. If you increase the reward for the chain formation, the agents might stop attacking the player altogether; they might just form a chain, break it, and form it again, or perhaps build chains of an absurd, massive size. Moreover, every time you adjust the reward structure, you have to retrain the agents from scratch, a process that could take hours. On top of that, more complex algorithms involve a multitude of parameters (such as epsilon, alpha, and gamma) that must be carefully tuned. Consequently, it is often much simpler, cheaper, and more effective to hand-code even the most complex behaviors rather than relying on reinforcement learning. The only instances where using reinforcement learning (RL) in games is justified are when a computer needs to control something that is simply impossible to hard-code (for example, entities that act like players (though I’ve never actually seen this done)) or in a specific genre of games where the core experience involves watching an agent learn (I have seen games like this).

However, the idea of using AI to replicate a player’s actions really appealed to me. Within the field of RL, there is a specific niche dedicated to algorithms designed to imitate behavior, yet I’ve never seen these applied in games, even though, from what I’ve gathered, they tend to be much more stable.

Thanks again for sharing your thoughts. I found them very interesting to read.

Submitted

Seems to be Windows only?

Developer

Sorry, the game really wasn't launching on Linux. It should be working now.

Submitted

I’m on macOS…

Developer (2 edits) (+1)

The game should run on MacOS as well! (sorry)