If you are reading this, there is a good chance you've at least heard of the Serpent.AI Framework. If not, welcome! Take a seat and buckle up!
In this post, I'll be going over the origins, the goals as well as a high-level overview of the different features of this rather unique Python framework.
What Serpent.AI is NOT
Before doing anything else, let's begin by addressing some common misconceptions people have about the framework.
Serpent.AI is NOT a tool that does all the machine learning for you
VERY common misconception. If anything, it does exactly the opposite: It automates and facilitates everything else so you can focus on your machine learning and AI experiments unhindered. It does provide some baselines but you are free to bring your own tools and algorithms.
Serpent.AI is NOT a tool to cheat in video games
While the framework does ship with tools to send input to video games and can be used to automate actions, it was never intended to be a tool to help cheating in games. Using Serpent.AI to cheat is probably the worst possible way to go about it: Not only is it extremely resource intensive, but you also have to infer the entire game state from frames. I can't control what people set out to do with it but planning to use it to cheat is frankly quite silly.
Like a lot of things in life, the path to Serpent.AI was not exactly straightforward. At no point in time did I ever set out to create a full-on framework to help people create game agents in Python with any game of their choosing; I like to think that no one that delivers ambitious projects ever starts out that way. It's just way too intimidating and only a matter of time before you collapse under the weight of your own zeal. In my case, it was a combination of ideas languishing on the back-burner for a few years, being lucky enough to have gained proper machine learning experience in my career, having the opportunity to take some time off and wanting to come up with unique content for my Twitch channel that ended up giving me the initial push in the right direction.
The first thing I can recall that sparked my interest in using code to turn games into machine learning experiments is seeing Seth Bling's MarI/O in action. A lot of people have seen this project on YouTube later on, but most of the training was streamed live on Twitch at time and I randomly stumbled upon it. I remember thinking: "This is cool. I want to do that!".
Fast-forward to the end of 2016, I'm bored at work and browsing Hacker News when I see something that instantly brings a smile to my face: OpenAI announces Universe. I read the entire blog post (A madman, I know) and by the end I'm already making the rounds in the office singing its praises to anyone I can find. I was already aware of OpenAI Gym but this was different: It would allow you to run your experiments on real games. At that point, I did run a few tests on some provided Flash game environments, but decided I would wait until they roll out support for some PC games. Spoiler Alert: It never happened...
The Birth of the Twitch Channel
A few months later, I officially hand in my resignation at my job. I had been toying with the idea of quitting for about 6 months at that point and seeing that nothing had really rekindled my enthusiasm, I decided it was time. I had quit jobs before but this was the first time I did so without another opportunity lined up. I figured I would get by on a few contracts and would use whatever extra time I had to start working on programming video content.
I acquired a bunch of video equipment with the plan of creating a YouTube tutorial series on Image Processing and Basic Computer Vision. The entire video setup was ready to go (RIP dining table, to this day still) but I still needed to prepare the material for the course and that was still a few days out. The eagerness of wanting to test out the equipment gave me the idea to try a short live stream on Twitch. I decided to go with something I was familiar with: Building a custom analytics server. It took me almost 2 hours to build up the courage to press that Start Streaming button but I did.
I had heard of the tales of how hard it was to get a new channel started on Twitch; How people were spending literal months stuck at 0 viewers. I was still nervous, but I wasn't exactly expecting anyone to show up. I was wrong. Turns out Creative and the Programming community work a little differently. They are smaller, niche directories so it's much easier to get eyes on your channel. All this to say that ended up with a handful of viewers, some of them even asking questions! That stream was nothing impressive (see below) but the interactivity had me hooked. By the end, I wanted to do it again. I did. I never stopped.
A random clip from the first stream ever (I think). Yes, it's bad. Very bad.
After about a dozen streams meddling with random web projects, I wasn't so much of a nervous wreck anymore and it became pretty clear that the Twitch channel was there to stay. I was having fun and even had some returning viewers. There was one thing I was missing however: A long-term project; One that could act as the backbone of the channel. I had an amount of time on my hands any employed programmer (including my past self!) would have killed for. I was free to create anything! This is where those Early Influencers I talked about earlier came in to haunt me.
I pitched the idea to my very small but loyal community at the time: I was going to start experimenting on using Python to capture frames from video games and send player input to them. My first guinea pig turned out to be Super Hexagon. It was a good candidate because it is very simple both graphically and input-wise (albeit perhaps too fast-paced).
When I say experimenting, I mean it! Here we learned the hard way to restrict inputs to the game window having focus. :D
At that point in time, most of the programming sessions were spent inside the Juptyer Notebook which is perfect to test out ideas, especially when working with visual data (e.g. game frames). I explained that my process in that case is generally to hack something together in the notebook and once satisfied, formalize it in the code base. We did a lot of that.
While not of a lot of focus was placed on machine learning, I did come up with the concepts of game/game agent plugins, game launchers (mainly Steam), crude frame capturing, frame handlers and virtual keyboard inputs. Not a single thought about Windows support at that point in time. The final prototype ended up being able to play about 15-30 seconds of the first level on its own, purely through computer vision.
A Super Hexagon game agent being somewhat autonomous and not very good at the game. It's a start!
Perhaps a pivotal moment in Serpent.AI's short history was the AIsaac undertaking. Not only did it draw a lot of attention to the channel, but it was also the first time a completely autonomous self-learning experiment was performed with the framework.
In the event that this is the first you hear of it, AIsaac was a game agent that was tasked to defeat Monstro (a boss) in The Binding of Isaac: Rebirth. We implemented our own version of Deep Q-Networks in a way that is compatible with the Serpent.AI way of doing things and ran the training 24/7 on the Twitch channel. Like any deep learning project, we had to iterate multiple times which meant starting over. It created a sense of community like nothing else though and people remember it to this day. AIsaac never did end up beating Monstro, but we got to study and understand DQNs and got to crystallize a lot of features / concepts that are still in the code base to this day!
A sample "AI run" from a version of AIsaac
As soon as it became clear that the framework was happening (i.e. Post-AIsaac), I spent some time coming up with some moderate goals for the projects that I knew I would be able to commit to. Here they are:
Serpent.AI should work on all 3 major OSes
While I'm definitely more of an Open Source / Linux type of person, there is no denying most of the PC gaming scene is still on Windows. A lot of people like to mess with me for choosing to support Windows, but I'm a fan of leaving my blinders behind when it comes to knowing your audience. It was and still is an extra headache but leaving the biggest gaming platform behind would have been shortsighted. On top of this, none of the software alternatives to Serpent.AI have gone out of their way to support Windows. Opportunity. As for Mac support... Well Linux and Windows are there so why not? The framework is full-featured on all OSes and your code is even portable from one platform to another!
Serpent.AI should work natively
If you have any experience with OpenAI Gym or Universe, this one shouldn't require an explanation! These tools, while great, require you to mess with VMs and/or Docker to get an environment working. Whilst this isn't a horrible thing per se, I've always felt like this led to extra overhead and way less overall control. One big argument seems to be that it can make installation easier, but in my opinion this benefit fails to account for how limited you end up being both graphically (modern games?) and input-wise (e.g. games that don't support virtual keys) in VMs. As such, it was very important for me that Serpent.AI have as little standing in the middle of your code and the game as possible.
Serpent.AI should allow you to use any video game
Perhaps one of my biggest gripes that lead to the creation of the Serpent.AI framework is OpenAI never adding the PC game environments to Universe. I was super excited about the announcement as I'm sure others were. I understand that licensing is hard and a bunch of the deals / partnerships that were announced on that day fell through, but this feels like a comeuppance for choosing to bundle everything including the game in those VM environments. It's no surprise publishers and developers decided not to partake in the end. Users already have accounts on platforms like Steam, Origin & GOG with individual licenses to games. Why not take advantage of this? Serpent.AI does. Your game library is your collection of environments!
Serpent.AI should be plugin-based
I like to believe that sharing helps forge and strengthen communities. We all stand to learn something from one another. Not only this but reproducibility is also at the heart of science! It always has been a no-brainer for me to have the important components in Serpent.AI (Game Agents, Game Support etc.) be plugin-based and they are!
Serpent.AI should encourage creative / different approaches
There is no question that Reinforcement Learning is all the rage at the moment. Serpent.AI alternatives have seemingly gone all-in on it. I'm of course excited to see the many advancements we'll surely see come out of that field, but also think that exclusively shaping your tools around it may be shortsighted. New interesting techniques come out all the time and just like reinforcement learning overtook the previous machine learning champion, something else will be there to take its place in the future. The circle of life. In response to this, Serpent.AI is not centered around any machine learning specialty and encourages experimentation and creativity. Bring your favorite tools, algorithms and research papers!
A few more experiments were performed on different games, namely You Must Build A Boat and Cuphead. This helped look at the existing features from a different perspective as well as ensuring Serpent.AI was working properly under Windows.
A Serpent.AI Game Agent battles Goopy Le Grande in Cuphead
The first beta version of the Serpent.AI framework was released on September 22nd 2017. It made the rounds on Reddit, Hacker News as well as various industry newsletters and podcasts. I could have never come even close to imagining the community's reception. To say it exploded would be the understatement of the year!
A plot of Serpent.AI's GitHub stars
To conclude this post, perhaps in a shameless attempt to get you interested in the capabilities of this Python framework, I'll list some features that can be found in the beta.
- Code Generation for Game Support (Steam, Executables and Web Browser)
- Code Generation for New Game Agents
- Game Launching with Before / After Callbacks
- Game Frame / Region of Interest Capture
- High FPS Full Resolution Frame Grabbing at Runtime
- Frame Transformation Pipeline in the Frame Grabber
- Easy & Trainable Game Context Classifier
- Advanced Input Controller for both Mouse and Keyboard
- Sprite Registration, Identification and Detection
- OCR Module
- DQN & DDQN implementations
- Streaming API and Analytics Client
- Visual Debugger Application to Inspect In-Memory Image Data
- And Much More...
Enjoyed the post?
Awesome! For more content, feel free to: