Project

Description

This was a school project for the course NOTS. Together with a group of four developers, our goal was to create an AI algorithm capable of solving CAPTCHAs. This meant we had to train our own model and devise strategies to overcome the very purpose for which CAPTCHAs were designed – to prevent such solutions.

While building this tool, we also had to document the entire process in the form of a research paper. The paper addressed various questions, including: What is Deep Learning? What Deep Learning techniques are necessary to tackle this problem? What data do we need to train our AI model? How much accuracy is required to determine a correct letter? How can we enhance the accuracy of our model? And finally, what are the ethical consequences of automatically solving CAPTCHAs? This 50+ pages research paper can be found on the GitHub repository below.

Eventually, we achieved the capability to automatically solve CAPTCHAs using the YOLOv5 model. To train the model, we manually labeled over 400 images using Roboflow. We then increased the dataset size using data augmentation techniques. After training the model for several days on powerful hardware provided by Google Colab and Kaggle, the AI demonstrated high accuracy in recognizing different types of CAPTCHAs. We tracked the model's performance using a tool called Weights and Biases (WandB), which provided insights on its performance compared to earlier runs. You can find the full working demo below as well.