Inferencing delivery bottlenecks away

One of the hardest parts of integrating AI in our workflows is realizing that you can, and probably should, throw away assumptions and re-evaluate trade-offs from scratch. Unit tests are insufficient on their own to guarantee the correctness of a larger system so we rely on integration / e2e to fill the gap. But the latter are also slower and resource-hungry so as the number of tests grows,…