What is the problem being addressed?
Project Joseph Hoane addresses the challenge of bridging cognitive intelligence and physical intelligence in computation. While AI systems like IBM’s Deep Blue excel at making strategic decisions in chess, translating those decisions into precise physical actions, like moving a chess piece on a board, is a separate challenge. Our project focuses on giving a robotic arm the ability to understand the board, detect individual chess pieces, and move them accurately without disturbing others, effectively combining perception and action.
What is the project idea and how will it work?
The goal of Project Joseph Hoane is to bring a deceptively simple concept (playing chess) to life using robotics and computer vision. The project consists of two main components: a vision module and a robotics module.
The vision module detects both chess squares and individual pieces. Square detection allows the robot to determine whether
Vision Module GUI a piece is present in a given location, which informs its pick-and-place decisions. Piece detection identifies which type of piece is in a square, ensuring the robot moves pieces correctly according to the rules of chess.
- The robotics module programs the movement of the robotic arm in simulation and eventually on a physical robot. The arm must be able to move precisely to chess squares, pick up pieces, and place them accurately on target squares without disturbing others.
This semester marks the third and final installment of the project, with the vision module nearly complete. Our current focus is on refining the robotic arm’s movement and improving computer vision accuracy. Upcoming challenges include finalizing the simulation, building a new dataset with over 85% accuracy for detecting individual pieces, removing heuristics to improve computer vision performance, and ensuring precise physical manipulation by the robot.
