FPGA NES player

As a final project for ECE 5760, a graduate course in embedded systems that focuses on field-programmable gate arrays (FPGAs), I helped build a system to automatically play the classic NES game Super Mario Brothers.

Unlike most other AI systems for playing video games, our system is entirely implemented in hardware -- there is no CPU involved at any point. Instead, the system analyzes the output video from the NES, and with the help of the NTSC decoder modules from Terasic, tracks the location of pipes, obstacles, and enemies on the screen. That information is then used to make decisions for Mario about when to jump and for how long. The system, which uses a Cyclone II FPGA from Altera as its core, emulates the signals of an NES controller connected in the Player 1 slot of the NES.

Despite some issues working with 30-year-old hardware, we were able to create a system that could successfully complete World 1-1. (Because our system depends on analyzing the video directly, we spent more time fine-tuning the AI than working on analyzing the completely-different-looking World 1-2 and beyond.)

My team members for this project were good friends Sima Mitra and Jeremy Blum.

View project report
View schematics and code on Github
View demonstration video on YouTube