Deploy Open Models on GPUs

Choose a GPU setup and a capability to find the best open models that fit, and deploy one as an API.

Agent · 8× NVIDIA RTX 5090

Ranked by benchmark scores for Tool calling, multi-step planning and environment interaction

#	Model	Params	Precision	Max context	Score (0–10)
1	Step-3.5-Flash 2026.03	199.4B	FP8	124K	9.10
2	DeepSeek-V4-Flash-0731 2026.08	304.2B	NVFP4	242K	8.98
3	Mistral-Medium-3.5-128B 2026.07	127.7B	FP8	256K	8.90
4	MiniMax-M2 2025.12	228.7B	NVFP4	192K	8.83
5	Qwen3.5-397B-A17B 2026.02	397B	NVFP4	70K	8.81
6	Qwen-AgentWorld-35B-A3B 2026.06	34.7B	FP16	256K	8.70
7	Qwen3-Coder-Next 2026.03	80B	FP16	256K	8.70
8	Qwen3.5-122B-A10B 2026.02	122B	FP8	256K	8.47
9	gpt-oss-120b 2025.08	116.8B	FP8	128K	8.30
10	MiniMax-M2.7 2026.04	228.7B	NVFP4	200K	7.96
11	Mistral-Small-4-119B-2603 2026.07	119.4B	FP8	467K	7.80
12	Gemma 4 31B 2026.03	31B	FP16	256K	7.69
13	Step-3.7-Flash 2026.06	201.4B	FP8	117K	7.56
14	Mistral Medium 3.5 2026.04	128B	FP8	256K	7.50
15	MiniMax-M2.1 2026.02	228.7B	NVFP4	192K	7.40
16	DeepSeek-V4-Flash 2026.07	284B	NVFP4	439K	7.03
17	Leanstral 1.5 2026.06	119B	FP8	256K	7.00
18	Ling 2.6 Flash 2026.04	104B	FP16	256K	7.00
19	Nemotron 3 Super 2026.03	120B	FP8	976K	7.00
20	gpt-oss-safeguard-120b 2025.10	120.4B	FP8	128K	6.80

Deployment verified on HyperAI

Scores aggregate public benchmarks for the selected capability.
Fits usable VRAM with weights plus KV cache at 64K+ context, at the best precision.