On-Device AI for Texas Hold’em: Real-Time Strategy and Predictions with Unity and Sentis

Mobile gaming relies heavily on online connectivity, but what if you found yourself by a lake with no internet connection and an itch to take on advanced poker players? In that scenario, you could test yourself against an AI poker player that requires no internet connection, yet delivers a true challenge with an added bonus: real-time predictions and strategic moves. Our project demonstrates how you can create an AI-powered Texas Hold’em game with fully offline functionality using Unity and Sentis.

Read about our exploration of the augmented reality aspect of the AI poker game game.

How to Create an AI-Powered Texas Hold’em Poker Game

Let’s explore the process of creating an AI-powered Texas Hold'em poker game — Holdem Hero — using Unity and Sentis technologies. Our primary goal is to verify the feasibility of integrating Sentis with Unity to develop a mobile app that runs AI entirely on the device, without relying on an internet connection. By leveraging the power of embedded machine learning models, we aim to create an engaging and interactive poker experience that offers players real-time predictions, hints, and the option to play against a computer agent. We will discuss the types of models typically used in similar solutions and then dive into the specifics of our application, focusing on the most interesting aspects of our development process.AI Models for Playing Poker, Chess, and Go

AI Models for Playing Poker, Chess, and Go

When building AI models for games like poker, chess, and Go, several approaches can be used, such as:

• Deep Reinforcement Learning (DRL)
• Monte Carlo Tree Search (MCTS)
• Counterfactual Regret Minimization (CFR)

Each of these techniques has been successfully applied in various AI systems for gaming. DRL, for instance, involves training an AI through trial and error, MCTS allows the AI to evaluate different game outcomes by simulating future moves, while CFR helps the AI minimize regret over time, particularly useful in games like poker that require strategic decision-making with hidden information.

These methods are particularly suitable for poker due to their ability to handle imperfect information and long-term planning in complex game environments.

Our Approach: Focusing on AI Integration with PyPoks

In our case, we prioritized AI integration over building a top-notch model from scratch, as our prototypes at Nomtek Labs have limited delivery time, typically spanning a few weeks. To streamline our development process, we decided to leverage a ready-to-use model of good quality. After exploring various options, we chose PyPoks, an open-source project that implements reinforcement learning with deep neural networks for poker environments.

PyPoks was chosen for its comprehensive implementation and well-documented codebase, which aligned well with our project timeline and requirements. However, its abstraction over PyTorch, while offering flexibility, introduces a learning curve for developers unfamiliar with its structure. This made PyPoks both powerful and slightly challenging, but ultimately a great fit for our AI-powered Texas Hold'em Poker game.

Converting PyPoks to ONNX for Sentis Integration

To integrate PyPoks models into our Unity-based mobile app using Sentis technologies, we needed to convert the models to the Open Neural Network Exchange (ONNX) format. ONNX allows for interoperability between different frameworks and platforms, which was essential for us to ensure compatibility between the PyTorch-based PyPoks model and the Unity environment.

The process of converting the PyPoks models to ONNX involved the following steps:

1. Identifying the input and output tensors of the models.
2. Tracing the PyTorch models to capture the computational graph.
3. Exporting the traced models to ONNX format.
4. Optimizing the ONNX models for mobile deployment.

Final code we used for converting the model into ONNX format looks like this:

Once the models were converted to ONNX, we were able to load them into our Unity app using the Sentis framework. Sentis provided the necessary tools to run the ONNX models efficiently on mobile devices, leveraging the power of the PyPoks AI in our poker game.

Connecting the ONNX model to Unity using Unity Sentis was a very straightforward process, which is very well described in the Sentis documentation.

AI Model Architecture and Functionality

Our Holdem Hero app uses a sophisticated AI model adapted from the PyPoks project, consisting of two interconnected neural networks:

• CardNet — A Transformer-based network that analyzes the player's cards and community cards to assess the strength of the current hand combination, almost like a player's gut feeling when they see their cards.
• Main Network — The brain of the AI, responsible for strategic decision-making. It takes the output from CardNet, financial information (previous bets, remaining chips, pot size), the player’s position, and the previous player’s action to decide the best move. This network has been trained using reinforcement learning techniques.

Together, these two networks allow the AI to assess the strength of its hand and make strategic decisions based on the current game state.

App Design and Functionality

Our Texas Hold'em Poker app, Holdem Hero, is designed with simplicity and ease of use in mind, featuring two main screens:

• Difficulty selection screen
  • Clean and minimalist interface.
  • Three difficulty levels: Beginner, Casual, and Professional.
  • Each level includes a brief description of the AI opponents' behavior.
• Gameplay screen
  • Stylish, modern table design with vibrant colors against a yellow background.
  • Two AI opponents at the top, community cards in the center, and the player’s hand at the bottom.
  • Clear display of the current pot and chip counts.

Key Features

• Unity-based development ensures smooth performance and cross-platform compatibility.
• Players compete against two AI opponents for a realistic poker experience.
• Action buttons for Raise High, Raise Low, Call, or Fold.
• AI assistant feature (magic wand icon) suggests the best move for the player.
• After each move, feedback on the player’s action is provided to help improve their strategy.

How We Coped with Difficulty System

Holdem Hero offers three distinct difficulty levels: Beginner, Casual, and Professional, all using the same AI model with clever modifications. The system adjusts the AI's decision-making process based on a "difficulty ratio," ranging from 0.0 (Beginner) to 1.0 (Professional), with Casual set at 0.5.The system works by manipulating the probability distribution of possible moves suggested by our AI model. The original model output provides probabilities for each move, which are then converted based on the difficulty level.

For example, on the Beginner level, the AI plays less aggressively and makes more mistakes, while at the Professional level, it plays a nearly optimal game.

The conversion formula is as follows:

Where:

• P_worst is the probability for the worst-playing bot
• P_moveType is the original probability from the model
• difficultyRatio is the bot's difficulty (0.0 for worst, 1.0 for best)
• P_converted is the final probability used for move selection

Here’s how the probabilities shift based on the difficulty ratio:

This smooth progression in difficulty ensures that players of all skill levels enjoy a challenging and realistic poker experience.

Feasible Integration of AI Poker Opponents with On-Device AI from Unity and Sentis

The Holdem Hero project demonstrates the feasibility of integrating AI poker opponents into a mobile game using Unity and Sentis, all while running AI entirely on-device. This not only improves player engagement by offering dynamic difficulty adjustments but also opens the door for future applications in other games and genres, expanding the potential for offline AI-driven experiences.