GA
Open Source
Free Tier

Hanzo Serving

Production model inference

Deploy models to production with KServe. Auto-scaling, canary deployments, A/B testing.

DocumentationView on GitHub
Serving
Training in progress
Epoch 15/50
Loss vs Epochs
Train
Val
Loss
0.0234
Accuracy
94.2%
LR
1e-4
GPU
A100

Features

Everything you need to get started

Auto-scaling
Canary deployments
A/B testing
GPU inference
Batching
SDKs & Libraries

Official Serving SDKs

Use our official SDKs to integrate Serving into your application

Hanzo Python SDK

PyPI
pip install hanzoai
hanzoaiDocs

Hanzo TypeScript SDK

npm
npm install @hanzo/ai
@hanzo/aiDocs

Hanzo Go SDK

Go Modules
go get github.com/hanzoai/go-sdk
github.com/hanzoai/go-sdkDocs

Hanzo Rust SDK

crates.io
cargo add hanzoai
hanzoaiDocs
Community

Join the Serving Community

Get help, share ideas, and contribute to the project

Star on GitHub
Read Docs
Join Discord
Follow on X

Want to Contribute?

We welcome contributions of all kinds: bug reports, feature requests, documentation improvements, and code contributions.

Read our Contributing Guide
Built on Open Source

Powered by vLLM

32k+

Hanzo Serving is built on top of vLLM, an open-source project.High-throughput and memory-efficient inference engine for LLMs.

View on GitHubVisit vLLM

Licensed under Apache-2.0

We're grateful to the vLLM maintainers and community for their incredible work.

Ready to get started with Serving?

Deploy in minutes with Hanzo Cloud or self-host with our open-source release.

Start FreeContact Sales