Poker and Perseverance
September 28, 2015
Steven A. Bogaerts, assistant professor of computer science, and students Robert T. Arrington ’15 and Clay P. Langley ’17 are hoping they have designed the winning computer player in an artificial intelligence contest of parameterized poker squares.
The contest takes place in February at the annual conference of the Association for the Advancement of Artificial Intelligence in Phoenix. They will submit a paper to present their research that began last summer. Teams participating in the contest are comprised of faculty members and students from all over the country.
The contest is to develop a computer player for the one-player game that is played on a five-by-five grid. You turn over a card and place it somewhere on the grid. Once it’s placed, you cannot move the card.
The goal is to create as many high-scoring poker hands as you can across every row and every column. “Each hand intersects with several others, so there are many decisions to make,” Bogaerts says. “Should I hope to get that perfect card for a straight flush, or should I go for the more likely, but also less rewarding, two of a kind right now?”
Parameterized poker squares becomes more difficult in that you don’t know ahead of time what the scoring system will be. “At the start of each game, the scoring system is set for that game. So in one game, a three of a kind might be the highest-scoring hand,” Bogaerts says. The next time all the hands could be worth negative points, and a pair could be worth negative 100 points. This change in scoring for each game is what makes the game “parameterized."
“Computers are very good at doing simple things very quickly, but they have no intuition; they can only do what we program them to do,” Bogaerts says.
He describes the typical fast-and-not-so-smart approach to game playing as trying to imagine what it would be like to try everything and see what works out best. “Humans do this to some extent,” Bogaerts says. “You can imagine if you were playing checkers, you would think a couple moves ahead.” In a computer, that’s easy analysis, and possible to do for many more moves ahead in the game.
A skilled human would have an advantage. He or she would know more instinctively what moves to ignore. However, in parameterized poker squares, there are so many possible moves to consider that it’s not feasible even for a computer to just try everything and see what works.
Bogaerts and his students are using a fairly new artificial intelligence game playing strategy called Monte Carlo tree search, which in recent years has garnered a lot of interest in the artificial intelligence game playing research community. Monte Carlo refers to the use of some randomness in the algorithm, like rolling the dice.
“The game space is so vast that a computer can’t explore all of it in a reasonable amount of time,” Bogaerts says. Monte Carlo tree search is a technique for systematically exploring portions of the game space that seem most promising and making a move accordingly.
Bogaerts teaches an artificial intelligence course, and he emphasizes the idea of never giving up. “I talk about how to systematically wrestle with a problem and understand it more deeply before trying to fix it,” he says. “It sounds obvious out of context, but you’d be surprise how tempting it is sometimes to try to fix a problem that you don’t fully understand.”
When he teaches a course, Bogaerts says the answers are usually known, and students are working to find those same answers. “With research, however, no one knows the answers,” he says.
“During a research project, you can no longer look to your professor for all of the answers,” Langley says. “In a class, they know what it is they're teaching you, and therefore, they can help you along the way to understanding it yourself. With research, the professor is working with the same intention to learn as his students are.”
Bogaerts explains that is part of the value of conducting research with students. “It’s showing them the importance of attacking a problem from different angles, trying to understand more about it, making more hypotheses and running experiments about what we think will work,” he says. “The scale of work that we’re doing is much more like real life than what you can do in a classroom setting.
“That kind of perseverance – comfort with the unknown and the ability to keep working on hard problems – is very valuable,” Bogaerts says.