ML engineers running Qwen3 models on RTX 5090
Ninfer provides the full serving stack: CLI for single requests, ninfer-serve for persistent HTTP service, and perplexity scoring. The OpenAI-compatible endpoint means existing Python clients using the openai library connect without changes. TOML configuration files manage model path, context limits, and speculative decoding settings.
Skip if:
You need to run models other than the documented Qwen3 and GLM checkpoints, or your hardware is a different GPU architecture. NInfer's model support is intentionally narrow.

