What You Lose When You Discretize Your Search Space
Most optimization libraries force you to define your search space as a grid. Want to tune a learning rate between 0.001 and 1.0? You create a numpy array: np.linspace(0.001, 1.0, 100). Sounds reasonable. But you just made a decision that silently costs you performance, and you probably never thought about it.