ResNet (Residual Networks)

In December 2015, the landscape of computer vision changed permanently when researchers Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun unveiled the Residual Network (ResNet). Before this, as neural networks became "deeper"—containing more layers—they paradoxically became harder to train. This was a central hurdle in the field that had previously hampered progress. ResNet introduced a clever, elegant solution that allowed computer models to stack hundreds of layers, reaching a depth of 152 layers in its initial winning iteration, ultimately securing first place in the ImageNet 2015 competition.

Imagine you are trying to teach a student a complex task, but every time you add a new, more difficult chapter to their textbook, they forget everything they learned in the previous chapters. That was essentially the problem with early deep neural networks. As the network grew, the information required to improve the model—known as the gradient—would fade away or "vanish" before it could reach the earlier layers. ResNet acted like a highway bypass, allowing information to skip over certain layers, which kept the learning process stable regardless of how many layers were added. This allowed computers to finally "see" and understand images with human-level accuracy.

How does ResNet (Residual Networks) fit into the history of artificial intelligence?

ResNet arrived as the logical evolution of the LeNet-5 Convolutional Net and the AlexNet Convolutional Net. While those earlier architectures proved that neural networks could classify images, researchers were hitting a performance ceiling. The Vanishing Gradients Identified problem in 1991 had correctly predicted that as networks grew, the math would break down. For over two decades, the community struggled to train deep architectures. ResNet represents the moment this barrier was shattered. By enabling the training of networks with 152 layers, it surpassed the performance of the 22-layer architectures that had defined the era, fundamentally altering the trajectory of research that would eventually lead to the Vision Transformer (ViT).

What are the core technical achievements of ResNet (Residual Networks)?

The technical brilliance of ResNet lies in the implementation of "residual learning." In a standard network, each layer attempts to learn a direct mapping of the input to the output. The researchers at Microsoft Research proposed that it is easier for a layer to learn a "residual"—the difference between the input and the target—rather than the target itself. They achieved this by introducing identity "skip connections." These connections jump over one or more layers, effectively adding the original input to the output of the layer stack. This simple architectural shift meant that if a layer was not useful, the network could simply skip it, preventing the accuracy degradation that plagued previous, shallower models. Consequently, the team won the ImageNet 2015 competition with a top-5 error rate of just 3.57%, a massive leap in accuracy compared to previous years.

Why is the legacy of ResNet (Residual Networks) significant to modern computing?

ResNet is widely considered a foundational architecture because it became the industry standard backbone for nearly all computer vision tasks for years after its release. It moved the goalposts for what was computationally possible, proving that depth was a key ingredient for intelligence. This milestone paved the way for modern developments, as the underlying concept of skip connections is now a core component in the design of the The Transformer Paper architecture and almost every modern deep learning system. By solving the degradation problem, the creators of ResNet provided a blueprint for scaling artificial intelligence, essentially proving that more layers—when structured correctly—yield significantly more capable and precise software.