Model Serving Architecture Patterns

Understanding latency, throughput, and the trade-offs between them