Stefano J Attardi’s excellent blog on using a neural network to split trendlines in his mechanical watch tracking app attracted many comments on hacker news suggesting a simpler algorithm might be more appropriate:

If I understand correctly, the reason a CNN is used here is because we want to find the splits that a human would visually agree “looks” the best? So rather than a regression it’s more like the “line simplification” problem in graphics.

Just thought this solution seems a little overkill. Surely you can pick some error metric over the splits to optimize instead?

Stefano understood intuitively the problem he wanted to solve but couldn’t write down explicit rules for doing so. He’s trying to split trend lines and in the four cases shown below the two highlighted in red boxes are considered incorrect splits:

He tried a number of classic segmented and piecewise regression algorithms first without finding a reliable solution.

To me, this looks like a hard problem to solve analytically and is a great example of an unorthodox but entirely practical application of neural networks.

But the question goes deeper than this once case. It is: if you can train a neural network to solve a problem, should you?

What is overkill, anyway?

Neural networks are easier than ever to train and deploy. Stefano’s post highlights some of the remaining gritty detail, but we’re rapidly moving towards a world in which deploying executing a neural network is as simple and normal as linking with a library and calling a function in it.

High-performance neural network processors will soon be shipping in every new phone and laptop on the planet. We are designing machine learning directly into CPU instruction set architectures. This trend is not going away - it’s only just getting started.

Stefano’s use of neural networks here, whilst currently unconventional, is absolutely a sign of things to come. If you already have the data, the benefits of a neural network are many:

• Improves over time without further developer effort. This fits the maxim of shipping something now and improving it rapidly. To get a better network you often just need more or better data. If you include some mechanism to collect data and feedback from users, this can even be automated.
• Fewer critical bugs. A neural network will sometimes predict the wrong answer, but it will never introduce a segmentation fault, security vulnerability or a memory leak.
• Predictable and scalable performance. A neural network takes the same time and memory to execute regardless of the input data, and be reducing or increasing the channels you can trivially trade performance and accuracy to match your needs.
• Faster execution and lower power consumption. This is currently questionable, but will change dramatically when every chip has a machine learning processor executing multiple teraflops per watt embedded in it.

The time is coming when hand-coding an algorithm that a neural network could have learned directly from the data will be seen as overkill - or at best, delightfully retro.

To paraphrase the tongue-in-cheek Seventy Maxims of Effective Mercenaries: one person’s overkill is another person’s “time to reload”.