Deploy Open Models on GPUs

Choose a GPU setup and a capability to find the best open models that fit, and deploy one as an API.

Agent · 4× NVIDIA RTX 5090

Ranked by benchmark scores for Tool calling, multi-step planning and environment interaction

#	Model	Params	Precision	Max context	Score (0–10)
1	UI-Mate-democua-27B 2026.08	27.4B	FP16	256K	10.00
2	UI-Mate-27B 2026.08	27.4B	FP16	256K	10.00
3	Qwen3.8-27B 2026.08	27.8B	FP16	256K	9.59
4	Qwen3.6-27B 2026.04	27.8B	FP16	256K	9.16
5	Mistral-Medium-3.5-128B 2026.07	127.7B	NVFP4	186K	8.90
6	Fara1.5-27B 2026.07	27.4B	FP16	256K	8.70
7	Qwen-AgentWorld-35B-A3B 2026.06	34.7B	FP16	190K	8.70
8	Qwen3-Coder-Next 2026.03	80B	FP8	256K	8.70
9	Qwen3.5-122B-A10B 2026.02	122B	NVFP4	256K	8.47
10	Qwen3.5-35B-A3B 2026.02	35B	FP16	256K	8.47
11	Qwen3.5-27B 2026.02	27B	FP16	256K	8.47
12	gpt-oss-120b 2025.08	116.8B	NVFP4	128K	8.30
13	Mistral-Small-4-119B-2603 2026.07	119.4B	NVFP4	205K	7.80
14	Gemma 4 26B-A4B 2026.03	26B	FP16	256K	7.69
15	Gemma 4 31B 2026.03	31B	FP16	220K	7.69
16	Mistral Medium 3.5 2026.04	128B	NVFP4	185K	7.50
17	Qwen3.6-35B-A3B 2026.04	35B	FP16	256K	7.49
18	GLM-4.7-Flash 2026.01	31B	FP16	198K	7.07
19	Leanstral 1.5 2026.06	119B	NVFP4	256K	7.00
20	Ling 2.6 Flash 2026.04	104B	FP8	150K	7.00

Deployment verified on HyperAI

Scores aggregate public benchmarks for the selected capability.
Fits usable VRAM with weights plus KV cache at 64K+ context, at the best precision.