Comparing LLMs for Condensing a 757-Page Physics Bowl Problem Set

A hands-on, entirely subjective benchmark — four LLMs, one batch LaTeX condensing task, and a lot of waiting for DeepSeek.

My existing Physics Bowl classified problem set ran to 757 pages. I wanted a shorter version that could also be converted into Anki-style flashcards. The approach: loop through every problem, feed the question and detailed solution to an LLM, and ask it to output a condensed version. All models were called via the OpenRouter API. Key parameters (costs and latency fluctuate):

Model	Input $/M tok	Output $/M tok	Latency s
o1-mini	1.1	4.4	0.72
DeepSeek-R1	0.8	2.4	0.73
Claude Sonnet	3	15	1.38
Gemini Flash 2.0	0.1	0.4	0.59

DeepSeek-R1's official pricing is $0.14 in / $0.55 out, but the official API was so overloaded (19.71 s latency) that I used an accelerated variant instead.

Results: page count

Starting from 757 pages, post-condensation counts were:

Model	Pages after	Pages saved
o1-mini	488	269
DeepSeek-R1	507	250
Claude Sonnet	459	298
Gemini Flash 2.0	477	280

Side-by-side output comparison

Example: a relativity problem at the harder end of the set. Original question and full solution first:

o1-mini — most concise, occasionally drops intermediate steps:

DeepSeek-R1 — good mathematical rigour, full information:

Claude Sonnet — detailed, well-labeled steps, closest to my own solution style:

Gemini Flash 2.0 — balanced length; sometimes heavy on equations, light on prose:

LaTeX quality issues

Surprisingly, Claude Sonnet — normally the top model on OpenRouter for coding — produced the most LaTeX errors in this task. I ended up writing a separate script to have ChatGPT fix Claude's compilation errors. GPT-o1-mini and Gemini were both solid, with roughly five minor errors each across the entire problem set.

DeepSeek had two distinct problems. First, it leaked its chain-of-thought into the output even when instructed not to — and the thinking text was not in LaTeX format, requiring extensive cleanup. Second, for the first half of the loop DeepSeek simply echoed the input verbatim rather than condensing it, producing a 636-page first draft. I had to rerun the entire DeepSeek pass to get a usable result.

Gemini occasionally went to the opposite extreme — outputting only equations with no prose explanation. Technically correct, but hard to follow.

Objective timing data (10-problem test)

Model	Avg time (s)	Avg length	Min (s)	Max (s)
GPT-o1-mini	7.10	1,788	6.02	9.14
DeepSeek	55.89	1,641	24.81	95.95
Claude	7.99	1,017	5.01	11.52
Gemini	4.53	1,441	3.83	5.63

DeepSeek's chain-of-thought reasoning is great in a chat interface but brutal for batch API calls. While other models finished in the time it takes to eat lunch, DeepSeek needed the time it takes to sleep. Gemini-2.0 is underrated — fast, multimodal, and largely correct, even if its explanations need polish.