Games and AI Techniques (COSC2527)
Cindy in the Shop
2025 · Group Project - 3 members

✦ THE IDEA
What were we trying to build?
Cindy in the Shop was a shopping game where the goal was more complicated than simply collecting as many items as possible. Cindy had to navigate a shop, collect items with different values, and learn how to make decisions about which items were worth going after. As the difficulty increased, we introduced obstacles and planned for a guard to patrol the entrance, making the environment more challenging for the AI to navigate. Our main learning goal was to train Cindy to maximise the value of the items she collected rather than simply collecting whatever was closest.
✦ THE JOURNEY
How did we get there?
I worked as part of a three-member team to build Cindy in the Shop using Unity and Unity ML-Agents. We designed the game around a reinforcement learning problem, where Cindy had to learn how to navigate the shop and make decisions about which items to collect. The items were assigned different values, so the goal wasn't just to collect the largest number of items, but to learn behaviour that would maximise the total value collected. We also experimented with making the environment more challenging through obstacles and higher difficulty levels, including a guard that could patrol the entrance. We weren't able to implement every part of the original idea, but we were able to train the agent and observe meaningful changes in its behaviour as we adjusted the learning setup.
✦ THE CHALLENGE
Where did things get tricky?
The biggest challenge was getting Cindy to learn the behaviour we actually wanted rather than simply finding the easiest way to earn rewards. We had to think carefully about what information the agent could observe and how its reward function should encourage useful behaviour. As we experimented with the training setup, Cindy's behaviour changed in ways we didn't always expect. Some configurations produced agents that appeared to learn something, but not necessarily the strategy we had intended. Although we didn't reach the full version of the game we had originally planned, seeing the agent's behaviour change through our experiments gave us a practical understanding of how much the learning environment and reward design influence an AI agent.
✦ WHAT I TOOK AWAY
What did this project teach me?
This project was my first hands-on experience with reinforcement learning through Unity ML-Agents, and it changed the way I think about game AI. Instead of writing rules that directly told Cindy where to go and what to collect, we had to design an environment in which she could learn those behaviours herself. I became more comfortable with C#, Unity, state representation, reward design, and experimenting with different training configurations. More importantly, I learned that getting an AI agent to behave the way you expect isn't as simple as giving it a goal — the way that goal is represented and rewarded can completely change what the agent learns.