From the university to board game testing

 Tabletop R&D is a Queen Mary University of London spin-out that developed software to test board games. The company developed automated methods to test board games after implementing the specific ruleset. In their introduction, they promise an analysis of the game and additional insights into its mechanics. They can also provide a digital twin of the game.  We spoke with Diego Perez Liebana at the UK Game Expo, one of the company's representatives.

Please tell me a few words about how you started this project?

It started as a university project. We researched games and got interested in AI to play board games. Not with generative AI, but with more traditional artificial intelligence, like Deep Blue, which played chess with Casparove or AlphaGo. We became interested in board games, developed our framework, and created a metric system for analyzing them. That’s how it started.

Now it’s a company and no longer connected to the university?

It’s a spin-out, so we are technically still owned partly by the university, but we are an independent company.

I guess the tool or framework should be customized for each new client. How big a task is it?

Certain parts, like the board, the pieces and the relationships between them, are general things in tabletop games. The specific rules, or cards, should be implemented. A simple card game could be just an afternoon. More complex games, for example, Terraforming Mars, could take maybe months. One important aspect that we don’t care much for is graphics. So that doesn’t come into play. It’s best to mention that there are things which we can't do, like TTRPG, where you can do anything, or where physics comes to play, like Jenga. These things can’t be covered with our software. Other games, where the human factor plays a huge part, like in Dixit, would be quite challenging.

When a potential client reaches you, what’s the typical question? Do they ask about specific scenarios to test? Or do they ask to test the game in general?

We have both. Normally, our default customer comes with very generic questions. But sometimes they have a specific question in mind. One of our clients asked a very specific case , where the game would be unwinnable for some of the players. It doesn’t matter what they are doing, they can’t win the game from that situation. For that game, theoretically, that could happen because of the ordering of some cards. It was a concern of the creators, but they didn’t know how often it could happen. They didn’t know how often it would happen because of the game mechanics. We run tens of thousands of games, and after that, we found that it happens 0.01% of the time. Knowing that the customer said, then it’s fine. He really wanted to know the odds of that, and with human testing, it’s not feasible to test something like this.

Can you please tell me your role in the company?

By title, I’m a director and co-founder. But the company is very small, currently we have five people and from this we have three directors, and four starting in June. Currently, we are doing it in our free time, and we do every aspect of the work, so implementation, testing, and so on.

On what hardware does it run?

Our client never has to run it, we do it. It doesn’t require many resources, whereas LLMs use many GPUs. Our program only uses CPU, so its environmental impact is very low, which is a good thing. I can even run on my own desktop computer. So it doesn’t need a cluster. If we can use a cluster, which sometimes is the case, it helps, e.g., the games can run in parallel. But we don’t need it.

Can you say a few words about pricing? From which company or project size does it make sense financially for a game developer to reach out to you?

With the price, there is a huge spread. We have to dig into the game's complexity. Does it use 200 different types of cards? Because we have to all implement. Do we have to test with one type of player, or with different players, different types of strategies? Do we have to run the game three times or a hundred times? This means a big difference in time. We can also try to optimize our players' play to be as effective as possible. This means extra effort, but basically, that means we try to train our players to be better than humans. You may have a game with multiple boards or multiple scenarios, which multiplies the number of times you need to test. We put all these things into our quote calculator, which you can even find online, and you see the amount. From there, it’s only a matter of time. Things from the cheapest thousand to hundreds of pounds, up to tens of thousands, with the maximum number of players and play accounts. And there is one possibility that didn’t happen until now, that after we implemented a game in our system, we have the logic for a potential computer implementation. You can use the mechanic for a video game.

For which project are you proudest, or which was the most interesting?

The game Turris was the most complex game. It features worker placement and hundreds of different cards, all with distinct mechanics. So it was quite complex to implement. There was a first edition of that game, and they wanted to balance it for the second edition. They were very happy with the results. We saw, for example, that if you are behind your enemy by a certain number of points, how likely are you to win the game?

Are there general takeaways from the projects?

In general, in many games, you have more resources, some of which are more important than others. And you can see from winning strategies the importance of the different resources. All these things gave a lot of insights about the game, and the creators can modify it based on them. One more thing, which helps us. With every project, our software improves because when we develop a new function for a game, we can use it in later projects as well.

Did you find a loophole in any of the games during implementation?

Almost in every game. It’s so common. (laughing) We have hundreds of different players automatically playing the game, and you play thousands of games. So with this, we can find logical problems even in edge cases. So that happens quite often. Besides that, we found loopholes in the games. Because we have to implement every element of the game, and during that, from time to time, we found game elements for which we have to ask, "How does it work in this specific case? So, that indirectly helps to fix the rules set.

For closure, did you use the software yourself or maybe for friends to find a winning strategy in a game?

Actually, that is our current method as well. Our method works by finding the best winning strategy for the game. In general, it would be during fine-tuning the playing method, but in general, we try to find different ways to play the game that are not necessarily optimal. That gives you an overview of how 90% of the players are playing the game. We try to find different ways to play the games because usually there is no single specific way to play them. But to come back to the question, the request to find unique, optimized strategies has come up very often from customers, and we can look into it.

The company’s webpage: https://www.tabletoprnd.co.uk/