Algorithms · 1995
Support Vector Machines
For a decade before deep learning, it was the sharpest tool in machine learning, and it worked by bending space.
In 1995, Corinna Cortes and Vladimir Vapnik published 'Support-Vector Networks', describing a classifier that draws the boundary between two classes with the widest possible margin. Instead of any line that separates the data, it seeks the one that leaves the largest gap on either side.
Only the points nearest the boundary matter for defining it; these are the support vectors. This focus on the hardest cases, rather than the comfortable middle, gave the method strong theoretical guarantees rooted in Vapnik's earlier work on statistical learning theory.
The real magic was the kernel trick, sharpened in a 1992 paper by Boser, Guyon, and Vapnik. Data that cannot be split by a straight line in its original space often can be once mapped into a higher-dimensional space, and kernels let the machine operate in that space without ever computing the coordinates explicitly.
Through the late 1990s and 2000s, support vector machines were the default choice for serious classification, from handwriting to text to bioinformatics. They were accurate, principled, and worked well even when data was scarce.
Deep neural networks eventually overtook them on large, messy problems like images and language, where learned features beat hand-chosen kernels. But on smaller structured datasets, a support vector machine is still a formidable and often underrated baseline.
Vapnik's fingerprints are all over modern machine learning. The idea of maximizing a margin, and the theory of why that generalizes, outlived the algorithm's peak popularity.
From history to production
We turn these ideas into working systems
The same techniques, shipped into your stack with evals, observability, and measurable ROI.