About

In 2018 I started the Automatic Control Lab’s project to build a robotic snooker player capable of challenging a top-ranking human. It serves as a platform for numerous applied research projects in robotics, computer vision, machine learning, and artificial intelligence (AI). We called this project DeepGreen, inspired by Deep Blue, the chess computer that famously defeated Garry Kasparov back in 1996. This page describes some of the progress we made on the robot up to late 2019.

Robot setup

Vision: The robot uses information from two cameras to judge the state of the game. The first camera is mounted overhead, and has a wide-angle lens capable of seeing the entire table. The second is mounted on top of the cue, and also features a depth sensor. It is used to increase the accuracy of shots.

Artificial intelligence: Information about the table state is passed to the AI unit, which performs calculations to decide what shot to take next. This problem is difficult because unlike a chess computer, which only has a limited number of discrete decisions to make, billiard games have a continuous state and action space, and the physics of balls rolling, sliding, and colliding on the table are highly complex.

Hardware: The cueing action is based on a high-spec linear motor mounted on a Universal Robots UR3 robot arm. Information about the next shot to take is received from the AI module, and the arm performs the manoeuvres required to align the cue and hit the cue ball with the required strength.

Recent results

Camera fusion for accurate shots: This project optimally combined the outputs from two cameras to take accurate billiard shots. The overhead camera can see the whole of the table and estimate the position of all the balls, however not with a sufficiently high accuracy to take long shots. Meanwhile, the cue camera can accurately reveal the relative angles of balls in its narrow field of view, but does not have a good estimate of their distance. Resolving the two sets of measurements, which have different uncertainty characteristics, is a classical problem known as the Procrustes problem. In this project we solved this problem to take shots with an accuracy similar to that of a good amateur player.

Video: https://vimeo.com/335260829


Image and video: Thorir Mar Ingolfsson

AI and dynamic programming: This project used dynamic programming (DP) theory and Monte Carlo Tree Search (MCTS) to devise shot-taking strategies. Billiard games require two state variables to describe the location of each ball on the table, thus dynamic programming algorithms can only be applied to situations where few balls are on the table. This project first performed so-called value iteration to compute the value of shots, given adversarial actions of an opponent player.

The second part of this project performed MCTS to handle cases where the DP was not practical, i.e. where too many balls are on the table. It was shown in simulation that the tree search algorithm outperformed a naive approach where a player of the same ability failed to plan more than one shot ahead.


Image: Baldur Yngvason

Snooker table installation

Here are some pictures of the snooker table delivery and installation, which was carried out by Sascha Lippe of SnookerService.de. The table measures 3.8 x 1.8 metres, and contains 5 pieces of slate weighing 180 kg each.

Picture 1 – Picture 2 – Picture 3 – Picture 4 – Picture 5 – Picture 6