One possibility would be to create an action space that consists of the complete map (including these 'inactive' hexes in my game engine), and define every possible move in that map. Now the question itself: Deep Q-learning requires a fixed-size action space, which is hard to define in BattleSheep, since the map is different in every game.

For this code, see the constructor of my Map class. 'inactive' hexes cannot be used, or could be considered 'out of bounds' during gameplay (defined HEX_OOB in my engine). During the map creation phase, the chosen hexes are set 'active' (defined HEX_FREE in my engine). In my game engine, I have solved this problem by creating large-enough map so that any kind of piece placement combination in the map creation phase will fit inside of its bounds. Since the game has two phases (map creation and the gameplay itself), the action space of the game is unknown at the start because the two players can place their pasture boards as they wish as long as it follows the rules (see the link above). I thought that a DQN should be possible to master the BattleSheep game, but I have stumbled across a problem: I have been studying the work about deep reinforcement learning networks, especially Leela chess zero (Alphazero) and AlphaGo.

The reason I created the game engine in the first place was a challenge to myself - I wanted to create an AI which would have superhuman skills in this game, since I thought that the game is pretty simple and it would be easy for a neural network to master it. I also created a SFML-based graphical interface for the game, which can be found from the same repo - here. I decided to program a game engine for this game. In this game, two to four players try to acquire as many hex tiles from a hex grid as possible. I was recently introduced to this simple board game called 'Battle Sheep'.

Battlesheep bgg