Training AI - Mansions of Madness scenario creation 001
Idea & Context.
In order to properly learn the capabilities of AI I need to try a lot of things, and not every one will be serious. One of the craziest ideas I am trying is to teach Claude Code to create scenarios for Mansions of Madness.
Mansions of Madness 2nd Edition 1 is a board game published by Fantasy Flight Games 2 that uses a companion app to guide the player through a narrative scenario. As usual, the community created an application called Valkyrie 3 that allows creating and playing user-created scenarios.
If you are into the hobby, you should try it, it’s wonderful.
My goal here is to open a Claude instance and introduce a prompt like
“Create a scenario based on Edgar Allan Poe’s ‘The Raven’, where the players need to prevent a winged monster from entering the house, the monster doesn’t attack, it only pursues the players crying ‘Nevermore’.”
The prompt will be bigger, this is just an example, and then, sit and wait. In this post series I will talk about both the process I tried to follow and the learnings achieved.
Session 001
First I want to apologize, I decided to start this series too late and I didn’t save the screenshots to properly show the process. I will do so from now on.
My first step was clear, I opened the Claude Desktop app on my Windows machine and requested Claude to read online the game rules, only for the base game, and the Valkyrie app documentation.
Once done, my plan was to create a fast iteration cycle of do and test to get early feedback, and try to understand how I was testing the iterations to teach Claude to auto-check its work.
So I asked Claude to create a sample scenario with only 2 terrain tiles and 2 monsters.
The result was horrible, the tiles were placed at random and the monsters appeared on empty spaces. The process was going to be long…
Claude explained to me that it doesn’t know the sizes of the tiles, so it cannot properly place the tiles together. I needed to enter the Valkyrie scenario edition tool to fix the tile placement and I realized something:
The app documentation doesn’t contain the sizes because the tool is visual, I can simply drag & drop the tiles together. The documentation was created for humans.
Once I fixed the tile position manually I asked Claude to check the change and learn, to apply on the next test scenario. Thanks to our manual change Claude could create the same scenario, but with the tiles properly placed.
Next step was to request Claude to create a short scenario with 5 to 7 linked tiles and one of them rotated. So Claude researched the tile sizes by taking control of my computer, opening the files and using the mouse to count the pixels of the tile images 🤦
The first session ended here with a clear learning, I need to switch to Claude Code, in the terminal, and drop the Desktop app
Session 002
The second test scenario showed the tiles as expected, but the tiles didn’t properly link the position of the doors between tiles, so our new task was to teach Claude Code to detect doors and use that information to make scenarios more complex and interesting.
On the first try Claude Code (CC from now on) placed the exploration token between tiles, but on a North, South, East & West approach. CC knew where two tiles connect but didn’t know where the doors are.
I requested CC to analyze the art of 1 tile on the scenario to try to find doors. The result was… close to reality to be honest, but not good enough. Discussing with CC, it was sure that the tile art (and doors) were placed based on a grid of 7x3.5 squares, CC even told me and added in the notes that this information had been double-checked. But my human eye told me that the grid to properly place the doors was 6x3.
This session was longer but I will finish here cause without screenshots this blog is very boring, but my learnings were really valuable.
Learnings
Documentation is done for humans, and this needs to change. The world is going full speed to change the role of the developer, and most of the code is going to be generated by AI, so the documentation needs to be created keeping in mind that a human being will probably never read it.
CC believed that the grid was 7x3.5 because “it was confirmed in a comment in the code, read in the Valkyrie code on GitHub”. As a developer I firmly consider code comments a signal of bad code, and this is a perfect example. The comment in the code was outdated information, but CC considered it a source of truth. And now I am debating between keeping my opinion and hating comments, or (like with documentation) understanding that the code is going to be generated by AI, so comments can be a great tool to help teams working in parallel with several AI providers updating the same code.
During the process, CC created several tools in the shape of Python scripts to manage and check the .ini files used by Valkyrie, and the images. The image analysis tools also created a lot of secondary images (the original images but split into smaller ones), it also created the .md files for the new skills and a lot of notes on different .md files. When I was about to commit my work to git, my working tree was completely strange to me. The learning here is to talk with CC at the beginning of each project to set where CC should store its tools, and tmp files. As well as a proper garbage removal policy.
When I asked CC to check the tiles art, CC checked the tiles art, and all my tokens were consumed in less than 10 minutes. Visual checks are really expensive and I don’t need them to teach CC to create scenarios. As long as CC had the information available, how the information achieved that state doesn’t matter. Maybe this breaks the rule of “teach CC to create scenarios”, or “let CC auto-learn”, cause my solution was to manually fill the information, but token consumption is one of the biggest issues in software companies nowadays, so it’s important to realize the impact of a bad prompt.
Conclusion
This first attempt at teaching Claude Code to build Mansions of Madness scenarios started as one of my “crazy ideas to learn AI capacities,” and two sessions in, it has already taught me a lot about how CC reasons.
The recurring pattern is clear: CC is confident, and that confidence isn’t always earned. It trusted an outdated code comment as ground truth, it “double-checked” a grid size that was wrong, and it happily burned my token budget analyzing tile art pixel by pixel when a fifteen-minute manual process would have done the job. None of this makes CC useless — quite the opposite — but it does mean my role isn’t just “prompt and wait.” It’s closer to pairing with a very fast, very literal junior developer who needs the knowledge no documentation ever wrote down, the experience.
The real lesson so far isn’t about scenarios, it’s about interfaces: humans write docs for humans, comments for humans, tools for humans — and none of that translates cleanly when the reader is an AI. Closing that gap, one tile at a time, is turning out to be the actual project.
Next up: teaching CC to auto-check the .ini files it generates, and building a proper tile/door position database so it stops guessing. Screenshots included this time, I promise.
Thanks for reading