🐍 Machine learning for market data analysis

Each week, I send out one Python tutorial to help you get started with algorithmic trading, market data analysis, and quant finance. Upgrade to a paid plan to access the Python code notebooks, Discord community, and code support.

I never got machine learning above coin-flip accuracy on tomorrow’s close, so now I use it to group stocks.

In my course, the first question I get is whether machine learning can predict tomorrow’s close. Next-day direction is the hardest version of that problem, because a single day of S&P 500 returns holds almost no repeatable pattern. Machine learning algorithms sort stocks by return and volatility much better.

So people spend months building new features while accuracy stays at a coin flip, then write off the whole field entirely.

I trained a random forest on ten years of daily S&P 500 closes to predict next-day direction. Held-out accuracy sat near 51%, and a 0.05% cost per trade made the returns negative.

I saw the same loop in myself for a year. Logistic regression became gradient boosting, indicators piled up, and I never asked what daily prices can answer.

We grouped positions by return and volatility first, then argued about which ones deserved a test.

In today’s post, you’ll group the 30 Dow Jones stocks by return and volatility with scikit-learn in Python.

Let’s go.