Implementing OpenAI-Compatible Tool Calling & Tool Streaming for Open-source models in vLLM

This site does not allow itself to be embedded. You can still read it on the original site — the toolbar below keeps your place in the directory.

This is a transcription of a talk I gave at vLLM's office hours after landing vLLM's first-of-its-kind tool calling implementation that allows using OpenAI-compatible tools and tool streaming with opens-source models.

Read on blacklight.sh ↗