Claude Sonnet 4.6 vs GLM-5.2 for coding

Claude Sonnet 4.6 ranks higher on real GitHub commits. Each agent gets a repository at a past commit and a description of the next change, edits the code on its own, and is graded from 0 to 10. Pick any two models below to switch the comparison.

BT rating · Claude Sonnet 4.6

-23elo

BT rating · GLM-5.2

-84elo

Per-task wins

78–24–18

Tasks · repos

134 · 4

Edge

Claude

What the numbers say

Over 134 benchmarked tasks from 4 public repositories, Claude Sonnet 4.6 has a Bradley-Terry rating of -23 and GLM-5.2 -84 (Elo-scaled, with the human commit anchored at 0). On the 120 tasks where both ran, Claude Sonnet 4.6 won more often (78 to 18, with 24 tied). On cost, Claude Sonnet 4.6 runs at about $61.834 and GLM-5.2 at about $45.754 per run.

Head-to-head, per task (rubric scores, 0–10)

Repo	Task	Claude Sonnet 4.6	GLM-5.2
ggml-org/llama.cpp	model : support granite multilingual embeddings R2 (ibm-granite/granite-embedding-{97,311}m-multilingual-r2) (#22716) · conversion	10.0	0.0
pooled: unsloth-zoo + vllm + llama.cpp	model : support granite multilingual embeddings R2 (ibm-granite/granite-embedding-{97,311}m-multilingual-r2) (#22716) · conversion	10.0	0.0
ggml-org/llama.cpp	convert : minor fixes for numpy 2.x (#23571) · examples	8.5	0.0
ggml-org/llama.cpp	cuda : fix KQ mask offset integer overflow in fattn MMA kernel (#23610) · ggml	10.0	1.5
pooled: unsloth-zoo + vllm + llama.cpp	[Model]Fix MiniMaxM2ForCausalLM perf regression (#45935) · tests	8.5	0.0
pooled: unsloth-zoo + vllm + llama.cpp	[Bugfix] Default tie_weights to sharing the weight (fix tied quantized embeddings, e.g. ModelOpt Gemma4) (#45544) · vllm	10.0	1.5
pooled: unsloth-zoo + vllm + llama.cpp	convert : minor fixes for numpy 2.x (#23571) · examples	8.5	0.0
pooled: unsloth-zoo + vllm + llama.cpp	cuda : fix KQ mask offset integer overflow in fattn MMA kernel (#23610) · ggml	10.0	1.5
vllm-project/vllm	[Model]Fix MiniMaxM2ForCausalLM perf regression (#45935) · tests	8.5	0.0
vllm-project/vllm	[Bugfix] Default tie_weights to sharing the weight (fix tied quantized embeddings, e.g. ModelOpt Gemma4) (#45544) · vllm	10.0	1.5
pooled: unsloth-zoo + vllm + llama.cpp	[DeepEP V2] Fill invalid recv_topk_idx with -1 (#46432) · vllm	7.0	0.0
vllm-project/vllm	[DeepEP V2] Fill invalid recv_topk_idx with -1 (#46432) · vllm	7.0	0.0
pooled: unsloth-zoo + vllm + llama.cpp	[Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in sampler kernels (#46560) · tests	7.5	1.0
vllm-project/vllm	[Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in sampler kernels (#46560) · tests	7.5	1.0
pooled: unsloth-zoo + vllm + llama.cpp	[Bugfix] Fix corrupt outputs in MoE FP8 LoRA responses and MoE base model responses when LoRAs are loaded (#42120) · tests	6.5	0.8
vllm-project/vllm	[Bugfix] Fix corrupt outputs in MoE FP8 LoRA responses and MoE base model responses when LoRAs are loaded (#42120) · tests	6.5	0.8
pooled: unsloth-zoo + vllm + llama.cpp	Humming support for 2/3/5/6/7-bit pack-quantized weight-only inference (#46389) · vllm	10.0	4.5
vllm-project/vllm	Humming support for 2/3/5/6/7-bit pack-quantized weight-only inference (#46389) · vllm	10.0	4.5
ggml-org/llama.cpp	ggml : fix ARM NEON nvfp4 dot product on non-dotprod targets (#21559) · ggml	3.5	8.5
pooled: unsloth-zoo + vllm + llama.cpp	ggml : fix ARM NEON nvfp4 dot product on non-dotprod targets (#21559) · ggml	3.5	8.5
pooled: unsloth-zoo + vllm + llama.cpp	[Perf][1/N] Expand Triton kernel warmup coverage, DSv4 (#46634) · vllm	9.5	5.3
vllm-project/vllm	[Perf][1/N] Expand Triton kernel warmup coverage, DSv4 (#46634) · vllm	9.5	5.3
pooled: unsloth-zoo + vllm + llama.cpp	[AMD][OCP MX][CI] Fix tests to not dispatch on `UNFUSED_TRITON` backend on MI300, improve w_mxfp4_a_fp8 emulation support (#46142) · tests	8.0	4.5
vllm-project/vllm	[AMD][OCP MX][CI] Fix tests to not dispatch on `UNFUSED_TRITON` backend on MI300, improve w_mxfp4_a_fp8 emulation support (#46142) · tests	8.0	4.5
pooled: unsloth-zoo + vllm + llama.cpp	nixl_ep: Skip post-receive quantization for NVFP4 (#45606) · vllm	9.5	6.5
vllm-project/vllm	nixl_ep: Skip post-receive quantization for NVFP4 (#45606) · vllm	9.5	6.5
pooled: unsloth-zoo + vllm + llama.cpp	add fixes for moe · unsloth_zoo	8.1	5.2
unslothai/unsloth-zoo	add fixes for moe · unsloth_zoo	8.1	5.2
pooled: unsloth-zoo + vllm + llama.cpp	Auto-install fused lm_head + cross_entropy forward across transformers · tests	6.8	4.0
unslothai/unsloth-zoo	Auto-install fused lm_head + cross_entropy forward across transformers · tests	6.8	4.0
ggml-org/llama.cpp	cuda: Q1_0 initial backend (#21629) · ggml	9.0	6.3
pooled: unsloth-zoo + vllm + llama.cpp	cuda: Q1_0 initial backend (#21629) · ggml	9.0	6.3
ggml-org/llama.cpp	sycl : support MUL_MAT and OUT_PROD with Q1_0 (#24721) · ggml	8.2	5.5
pooled: unsloth-zoo + vllm + llama.cpp	sycl : support MUL_MAT and OUT_PROD with Q1_0 (#24721) · ggml	8.2	5.5
ggml-org/llama.cpp	ggml: `gguf_init_from_callback` and `gguf_init_from_buffer` (#22341) · ggml	10.0	7.4
pooled: unsloth-zoo + vllm + llama.cpp	perf(moe-fp8): batched 3D dequant + FP8Experts dispatcher + Trainer guard · unsloth_zoo	8.5	5.9
pooled: unsloth-zoo + vllm + llama.cpp	[XPU][MoE] Add WNA16 oracle backend for GPTQ sym-int4 (xpu_fused_moe) (#41426) · vllm	8.5	5.9
pooled: unsloth-zoo + vllm + llama.cpp	ggml: `gguf_init_from_callback` and `gguf_init_from_buffer` (#22341) · ggml	10.0	7.4
unslothai/unsloth-zoo	perf(moe-fp8): batched 3D dequant + FP8Experts dispatcher + Trainer guard · unsloth_zoo	8.5	5.9
vllm-project/vllm	[XPU][MoE] Add WNA16 oracle backend for GPTQ sym-int4 (xpu_fused_moe) (#41426) · vllm	8.5	5.9
ggml-org/llama.cpp	ggml: support concat for scalar types at cuda backend (#24011) · ggml	10.0	7.5
pooled: unsloth-zoo + vllm + llama.cpp	[DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement (#45863) · tests	10.0	7.5
pooled: unsloth-zoo + vllm + llama.cpp	ggml: support concat for scalar types at cuda backend (#24011) · ggml	10.0	7.5
vllm-project/vllm	[DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement (#45863) · tests	10.0	7.5
ggml-org/llama.cpp	vulkan: Switch MUL_MAT_VEC to 4 K per iteration for F16/32 (#22887) · ggml	10.0	8.0
pooled: unsloth-zoo + vllm + llama.cpp	Fix gptoss 4bit (#524) · unsloth_zoo	4.0	2.0
pooled: unsloth-zoo + vllm + llama.cpp	Capture outputs fixes for transformers v5 (#713) · tests	7.0	5.0
pooled: unsloth-zoo + vllm + llama.cpp	Fix review findings for PR #3: Gemma4 LoRA/BnB patches, GDN extraction, finalize_huggingface_model · unsloth_zoo	8.0	6.0
pooled: unsloth-zoo + vllm + llama.cpp	[Kernel] Add swap AB optimization to fused_moe_kernel (#36559) · vllm	4.0	2.0
pooled: unsloth-zoo + vllm + llama.cpp	vulkan: Switch MUL_MAT_VEC to 4 K per iteration for F16/32 (#22887) · ggml	10.0	8.0
unslothai/unsloth-zoo	Fix gptoss 4bit (#524) · unsloth_zoo	4.0	2.0
unslothai/unsloth-zoo	Capture outputs fixes for transformers v5 (#713) · tests	7.0	5.0
unslothai/unsloth-zoo	Fix review findings for PR #3: Gemma4 LoRA/BnB patches, GDN extraction, finalize_huggingface_model · unsloth_zoo	8.0	6.0
vllm-project/vllm	[Kernel] Add swap AB optimization to fused_moe_kernel (#36559) · vllm	4.0	2.0
ggml-org/llama.cpp	sycl : fix failed ut cases of norm (#25044) · ggml	10.0	8.5
ggml-org/llama.cpp	metal : fix FA support logic (#21898) · ggml	10.0	8.5
pooled: unsloth-zoo + vllm + llama.cpp	fix(moe): loud-fail on silent fallbacks in MoE merge + FP8 forward paths · unsloth_zoo	8.0	6.5
pooled: unsloth-zoo + vllm + llama.cpp	sycl : fix failed ut cases of norm (#25044) · ggml	10.0	8.5
pooled: unsloth-zoo + vllm + llama.cpp	metal : fix FA support logic (#21898) · ggml	10.0	8.5
unslothai/unsloth-zoo	fix(moe): loud-fail on silent fallbacks in MoE merge + FP8 forward paths · unsloth_zoo	8.0	6.5
pooled: unsloth-zoo + vllm + llama.cpp	feat(mlx): add save_method to save_pretrained_merged · unsloth_zoo	9.8	8.6
unslothai/unsloth-zoo	feat(mlx): add save_method to save_pretrained_merged · unsloth_zoo	9.8	8.6
ggml-org/llama.cpp	dflash: refactor draft model conversion (#25110) · conversion	10.0	9.0
ggml-org/llama.cpp	[SYCL] Add BF16 support to GET_ROWS operation (#21391) · ggml	8.3	7.3
pooled: unsloth-zoo + vllm + llama.cpp	fix(moe-fp8): inline weight/quant-state lookup; narrow ImportError scope · unsloth_zoo	2.0	3.0
pooled: unsloth-zoo + vllm + llama.cpp	Fp8 compressed (#358) · unsloth_zoo	10.0	9.0
pooled: unsloth-zoo + vllm + llama.cpp	Add weights padding for fp8 per-block online quantization (#44763) · vllm	9.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	dflash: refactor draft model conversion (#25110) · conversion	10.0	9.0
pooled: unsloth-zoo + vllm + llama.cpp	[SYCL] Add BF16 support to GET_ROWS operation (#21391) · ggml	8.3	7.3
unslothai/unsloth-zoo	fix(moe-fp8): inline weight/quant-state lookup; narrow ImportError scope · unsloth_zoo	2.0	3.0
unslothai/unsloth-zoo	Fp8 compressed (#358) · unsloth_zoo	10.0	9.0
vllm-project/vllm	Add weights padding for fp8 per-block online quantization (#44763) · vllm	9.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	fix(mlx): repair stub injection on Apple Silicon (3 sub-bugs) · unsloth_zoo	9.7	9.0
unslothai/unsloth-zoo	fix(mlx): repair stub injection on Apple Silicon (3 sub-bugs) · unsloth_zoo	9.7	9.0
pooled: unsloth-zoo + vllm + llama.cpp	Fix dense vLLM state dict parity · tests	1.7	2.3
unslothai/unsloth-zoo	Fix dense vLLM state dict parity · tests	1.7	2.3
ggml-org/llama.cpp	[SYCL] support bf16 on bin_bcast OP and unary OPs (#24838) · ggml	8.0	7.5
ggml-org/llama.cpp	ggml-webgpu: support non-square subgroup matrix configs for Intel GPUs (#21669) · ggml	5.5	5.0
pooled: unsloth-zoo + vllm + llama.cpp	Fix ModuleNotFoundError when loading gpt-oss models without triton_kernels (#4088) (#539) · unsloth_zoo	9.0	9.5
pooled: unsloth-zoo + vllm + llama.cpp	[Bugfix] Fix minimax_qk_norm_fusion (#44983) · vllm	5.0	4.5
pooled: unsloth-zoo + vllm + llama.cpp	fix(moe_wna16): access tp_size via moe_config for RoutedExperts compatibility (#45404) · vllm	1.5	2.0
pooled: unsloth-zoo + vllm + llama.cpp	[SYCL] support bf16 on bin_bcast OP and unary OPs (#24838) · ggml	8.0	7.5
pooled: unsloth-zoo + vllm + llama.cpp	ggml-webgpu: support non-square subgroup matrix configs for Intel GPUs (#21669) · ggml	5.5	5.0
unslothai/unsloth-zoo	Fix ModuleNotFoundError when loading gpt-oss models without triton_kernels (#4088) (#539) · unsloth_zoo	9.0	9.5
vllm-project/vllm	[Bugfix] Fix minimax_qk_norm_fusion (#44983) · vllm	5.0	4.5
vllm-project/vllm	fix(moe_wna16): access tp_size via moe_config for RoutedExperts compatibility (#45404) · vllm	1.5	2.0
pooled: unsloth-zoo + vllm + llama.cpp	fix(moe-quant): gate v5-only patches · tests	9.0	8.7
unslothai/unsloth-zoo	fix(moe-quant): gate v5-only patches · tests	9.0	8.7
pooled: unsloth-zoo + vllm + llama.cpp	fix(peft-param-wrapper): handle merge_and_unload for 4-bit MoE experts (B4) · unsloth_zoo	8.3	8.5
unslothai/unsloth-zoo	fix(peft-param-wrapper): handle merge_and_unload for 4-bit MoE experts (B4) · unsloth_zoo	8.3	8.5
ggml-org/llama.cpp	metal: Q1_0 backend (#21528) · ggml	9.9	10.0
pooled: unsloth-zoo + vllm + llama.cpp	fix(moe-bnb): dequant Params4bit experts in transformers v5 grouped/batched MoE forward (B6) · unsloth_zoo	9.6	9.5
pooled: unsloth-zoo + vllm + llama.cpp	Harden fused-forward AST rewriter and adapter · unsloth_zoo	9.1	9.2
pooled: unsloth-zoo + vllm + llama.cpp	metal: Q1_0 backend (#21528) · ggml	9.9	10.0
unslothai/unsloth-zoo	fix(moe-bnb): dequant Params4bit experts in transformers v5 grouped/batched MoE forward (B6) · unsloth_zoo	9.6	9.5
unslothai/unsloth-zoo	Harden fused-forward AST rewriter and adapter · unsloth_zoo	9.1	9.2
ggml-org/llama.cpp	[SYCL] Add Q8_0 reorder optimization (~3x tg speedup on Intel Arc) (#21527) · ggml	10.0	10.0
ggml-org/llama.cpp	CUDA: Various fixes to `cpy.cu` (#25000) · ggml	10.0	10.0
ggml-org/llama.cpp	CUDA & CPU: support F32 kernel type for `CONV_TRANSPOSE_2D` (#17094) · ggml	10.0	10.0
ggml-org/llama.cpp	ggml: vectorize ggml_vec_dot_q4_1_q8_1 with WASM SIMD128 (#22209) · ggml	10.0	10.0
ggml-org/llama.cpp	fix: GLM-DSA crash in llama-tokenize when using vocab_only (#22102) · src	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	Use torch.Tensor.reshape for non-contiguous tensor in ce loss function (#591) · unsloth_zoo	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	[XPU] Fix Triton attn fp8/bf16 check failing (#45758) · vllm	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	[Bugfix] Restrict FlashInfer cuDNN FP8 ViT attention gate to Blackwell (SM 100) (#45251) · vllm	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	[Kernel][Bugfix] Fix INT8 per-token-head KV cache rounding in Triton reshape-and-cache (#45361) · tests	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	[Bugfix][Quantization] Fix W8A8 int-quantized scheme selection regression (#46860) · tests	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	[Kernel] Enable TritonW4A16LinearKernel as CUDA fallback for non-Marlin-aligned W4A16 shapes (#43731) · vllm	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	[Perf] Fix dsv3_router_gemm heuristic (#44217) · vllm	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	[SYCL] Add Q8_0 reorder optimization (~3x tg speedup on Intel Arc) (#21527) · ggml	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	CUDA: Various fixes to `cpy.cu` (#25000) · ggml	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	CUDA & CPU: support F32 kernel type for `CONV_TRANSPOSE_2D` (#17094) · ggml	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	ggml: vectorize ggml_vec_dot_q4_1_q8_1 with WASM SIMD128 (#22209) · ggml	10.0	10.0
pooled: unsloth-zoo + vllm + llama.cpp	fix: GLM-DSA crash in llama-tokenize when using vocab_only (#22102) · src	10.0	10.0
unslothai/unsloth-zoo	Use torch.Tensor.reshape for non-contiguous tensor in ce loss function (#591) · unsloth_zoo	10.0	10.0
vllm-project/vllm	[XPU] Fix Triton attn fp8/bf16 check failing (#45758) · vllm	10.0	10.0
vllm-project/vllm	[Bugfix] Restrict FlashInfer cuDNN FP8 ViT attention gate to Blackwell (SM 100) (#45251) · vllm	10.0	10.0
vllm-project/vllm	[Kernel][Bugfix] Fix INT8 per-token-head KV cache rounding in Triton reshape-and-cache (#45361) · tests	10.0	10.0
vllm-project/vllm	[Bugfix][Quantization] Fix W8A8 int-quantized scheme selection regression (#46860) · tests	10.0	10.0
vllm-project/vllm	[Kernel] Enable TritonW4A16LinearKernel as CUDA fallback for non-Marlin-aligned W4A16 shapes (#43731) · vllm	10.0	10.0
vllm-project/vllm	[Perf] Fix dsv3_router_gemm heuristic (#44217) · vllm	10.0	10.0
ggml-org/llama.cpp	model: mistral small 4 support (#20649) · convert_hf_to_gguf.py	-	9.7
ggml-org/llama.cpp	sycl: support reordered Q4_K/Q5_K/Q6_K MoE MUL_MAT_ID (#24452) · ggml	8.1	-
ggml-org/llama.cpp	sycl : fix the failed UT cases of conv_3d (#24900) · ggml	9.5	-
ggml-org/llama.cpp	llama-fit: fix regex pattern for gate_up tensors (#20910) · src	6.0	-
pooled: unsloth-zoo + vllm + llama.cpp	Fix LoRA scaling count mismatch on merge for Qwen2.5-VL exports (#2966) (#806) · tests	7.5	-
pooled: unsloth-zoo + vllm + llama.cpp	[Bugfix] Support non-power-of-2 top_k in legacy triton_kernels routing (#46406) · vllm	10.0	-
pooled: unsloth-zoo + vllm + llama.cpp	[Bugfix][Kernel] Fix mHC fused-RMSNorm big-fuse miscompile for hidden_size != 4096 (#44692) · vllm	9.5	-
pooled: unsloth-zoo + vllm + llama.cpp	model: mistral small 4 support (#20649) · convert_hf_to_gguf.py	-	9.7
pooled: unsloth-zoo + vllm + llama.cpp	sycl: support reordered Q4_K/Q5_K/Q6_K MoE MUL_MAT_ID (#24452) · ggml	8.1	-
pooled: unsloth-zoo + vllm + llama.cpp	sycl : fix the failed UT cases of conv_3d (#24900) · ggml	9.5	-
pooled: unsloth-zoo + vllm + llama.cpp	llama-fit: fix regex pattern for gate_up tensors (#20910) · src	6.0	-
unslothai/unsloth-zoo	Fix LoRA scaling count mismatch on merge for Qwen2.5-VL exports (#2966) (#806) · tests	7.5	-
vllm-project/vllm	[Bugfix] Support non-power-of-2 top_k in legacy triton_kernels routing (#46406) · vllm	10.0	-
vllm-project/vllm	[Bugfix][Kernel] Fix mHC fused-RMSNorm big-fuse miscompile for hidden_size != 4096 (#44692) · vllm	9.5	-

Scores run from 0 to 10. Head-to-head counts only the 120 tasks where both models ran. Bold marks the winner on that task.

By area of the codebase

Area	Tasks	Claude Sonnet 4.6	GLM-5.2
ggml	38	8.9	7.7
vllm	32	8.3	5.9
unsloth_zoo	28	8.1	7.3
tests	24	7.7	4.9
conversion	4	10.0	4.5
src	4	8.0	10.0
examples	2	8.5	0.0
convert_hf_to_gguf.py	2	-	9.7

How the benchmark works

A task here is a single commit. CommitBench checks out the repository one commit before the change, writes a short description of the work from the diff, and gives the agent nothing but that description and the codebase. The agent navigates and edits the files itself, the same way it would on a real ticket.

Every agent's change is then compared head-to-head against every other one — and against the human commit — by a blind judge that never sees which model wrote which diff. Those pairwise wins become a Bradley-Terry rating (Elo-scaled, with the human commit anchored at 0), which is the ranking shown here. A fixed rubric still scores each change for the per-task detail. Cost and speed come from each run's own token and timing reports, priced at one consistent rate so the numbers line up across agents.

Common questions

Which is better for coding, Claude Sonnet 4.6 or GLM-5.2?

Claude Sonnet 4.6 ranks higher overall across 134 tasks from 4 public repositories. On the pairwise Bradley-Terry rating (Elo-scaled, anchored on the human commit at 0), Claude Sonnet 4.6 sits at -23 and GLM-5.2 at -84. Claude Sonnet 4.6 costs about $61.834 per run, GLM-5.2 about $45.754 per run.

How is the ranking calculated?

Each task is a real commit from a public repository. The agent gets the code at the commit before the change and a written description of the work, then edits the files itself. Every agent's diff is then compared head-to-head against every other one — and against the human commit — by a blind judge that never sees which model produced which diff. Those pairwise wins and losses become a Bradley-Terry rating (Elo-scaled), so the ranking reflects which changes actually win comparisons, not whether they match the original wording.

Could the agents have seen the answers already?

Two things limit that. The task description is rewritten from the change rather than copied from the commit message, and the grader is given the rubric instead of the original answer. Building tasks from recent commits also reduces how much a model could have trained on the result.

Can I compare Claude Sonnet 4.6 and GLM-5.2 on my own code?

Yes. Point CommitBench at your repository and it builds the same kind of tasks from your own commit history, so the ranking reflects the code you actually work on.

Other comparisons

Claude Opus vs Claude Sonnet Claude Opus vs GPT-5.5 Claude Opus vs GLM-5.2 Claude Sonnet vs GPT-5.5 GPT-5.5 vs GLM-5.2

Run this on your codebase

Public repos give the general picture. Benchmark Claude Sonnet 4.6 against GLM-5.2 on your own commit history to see which one does better on the code you actually ship.

Benchmark your repo →