Everyone talks about neural networks and LLMs. But the models quietly running your bank, your fraud alerts, and half of Kaggle are older, simpler, and far more boring. A plain-English tour of the real workhorses.
In my last post I dropped a throwaway line about “boring names, absolute workhorses” and listed a few classic machine learning models. A couple of people asked me what those actually are. Fair question. So this one is for them.
Here is the thing nobody tells you when you start out in AI. The models that get all the attention, the giant neural networks and the LLMs, are not the ones doing most of the quiet, useful work in the world. Your bank approving a loan. Your card getting flagged for fraud. A hospital deciding which test results a doctor should look at first. A lot of that runs on models that are decades old, train in seconds, and fit comfortably on a laptop. They just have unglamorous names, so nobody writes breathless threads about them.
Let me introduce the four I reach for most often. No maths. Just what each one actually does.
Start with the simplest. “Logistic regression” sounds scary, but it is basically a scorecard.
Think about how a bank used to decide a loan by hand. Good income, add some points. Existing debt, take a few off. Stable job, add a few more. At the end you total it up, and if the score crosses a line, approved.
That is all logistic regression does, except it works out the points for you. You show it old loans that got repaid and old loans that went bad, and it figures out how many points each factor is worth. A new application comes in, it adds up the score, and hands you back a probability. Yes or no, with a number attached.
It is old, it is simple, and for a lot of yes/no questions it is still hard to beat. The bonus is that you can read exactly why it said no, which matters a great deal when a regulator or a customer asks.
A decision tree is something you already understand, because you use one every time you play twenty questions.
It asks a question, splits based on the answer, asks another, splits again, and keeps going until it reaches a decision. Is income above this amount? Yes goes one way, no goes the other. On the “no” side, any missed payments in the last year? And so on down the branches until you land on a verdict.
The reason people love decision trees is that you can print one out and read it like a recipe. A police officer, a doctor, a loan officer, anyone can look at the exact path it took and understand the decision. There is no black box. The catch is that a single tree is a little naive. It can get too attached to the exact examples you trained it on. Which leads nicely into the next one.
If one tree is a bit unreliable, the fix is wonderfully simple. Grow a hundred of them, make each one slightly different, and let them vote.
This is the wisdom of the crowd. Ask one person to guess how many sweets are in a jar and they will be way off. Ask five hundred people and average their guesses, and somehow you land spookily close. A random forest does the same with trees. Each tree sees a slightly different slice of the data, so each one makes different mistakes, and when they vote together the mistakes cancel out while the real signal survives.
A random forest is the model I trust by default when I have a table of data and no time to waste. It is hard to break, it rarely embarrasses you, and it needs almost no babysitting.
Now the famous one. If you have heard of XGBoost, it is because it wins competitions. For years, if you looked at the top of a Kaggle leaderboard on any spreadsheet-style problem, you would find XGBoost or one of its close cousins sitting there.
The idea behind it is clever and very human. Instead of growing a hundred trees independently and voting, you grow them one after another, and each new tree has exactly one job: clean up the mistakes the earlier trees made.
Picture a small team writing a report. The first person does a rough draft. The second person does not start over, they just fix what the first one got wrong. The third fixes what is still off. Keep going, and the report gets sharper with every pass. Gradient boosting is that, but with trees. Each new tree puts its attention on the cases the team is still getting wrong.
It is more powerful than a random forest, and it needs a bit more care to tune, but when you want the best possible number out of a table of data, this is usually what gets you there.
Here is my honest workflow. When a new problem lands and it looks like rows and columns, a spreadsheet really, I do not start with a neural network, and I definitely do not start with an LLM. I start here.
These models train in seconds, run for free, and can tell you why they decided what they decided. In the kind of work I do, where a decision might have to stand up in front of a court or a senior officer, that last part is not a nice extra. It is the whole game.
The giant models are genuinely brilliant at messy, human things like language and images. But most real business problems are still a table of numbers with a yes or no at the end. And for that, the boring workhorses from twenty years ago quietly win, every single day.
So before anyone on your team says “let us throw a neural network at it,” it is worth asking one question first: is this just a table? Because if it is, the unglamorous answer is probably the right one.