DeepSeek's V4 Flash aces the leaderboards but stumbles on real work, then raises its prices

Key points

• DeepSeek V4 Flash completed only 53.8% of 240 complex multi-step tasks in independent testing published in August 2025.
• DeepSeek raised API prices by as much as 1,100% after V4 Flash became the most-used model on the OpenRouter platform by weekly usage volume.
• V4 Flash now costs 22 cents per million input tokens off-peak, up from previous pricing, with peak rates double that.
• Despite the increases, DeepSeek remains cheaper than rivals from OpenAI, Anthropic, Google and others across all pricing tiers.
• Enterprise adoption is still early, with most serious use limited to low-risk, batch-style workloads rather than live customer-facing systems.

DeepSeek V4 Flash arrived in public beta on 31 July 2025 and immediately broke records. Within days it sat at the top of OpenRouter's weekly usage chart, a platform where developers route requests to dozens of AI models. Researchers called it a "total monster." The price was part of the appeal: a fraction of what OpenAI or Anthropic charge for comparable intelligence.

Then two things happened at once.

How did it actually perform on real tasks?

Not as cleanly as the leaderboards implied. Testing firm Composio put V4 Flash through 240 runs across eight different "agent harnesses", the software frameworks that let an AI model take real actions inside tools like Gmail, GitHub, Slack and Google Sheets, rather than simply answer questions. Only 129 of those 240 runs passed. That is 53.8%.

Of the 30 workflows tested, only six were completed successfully by every harness. The same model, on the same tasks, produced very different results depending on how it was set up.

That gap matters. It tells us that raw model intelligence is only part of the story. The scaffolding around the model, including how it retries failed steps, how it caches (saves and reuses) earlier work, and which tools it can reach, shapes the outcome just as much.

Meta software engineer Naman Ahuja saw this dynamic in his own experiments. Working independently of his employer, he built a home-automation agent using V4 Flash to coordinate a thermostat, a Ring security system, and door locks whenever he left home. His finding was blunt: "Once a model can take actions, reliability matters as much as intelligence." The system needed structured outputs, confirmation that each action actually worked, and clear rules about what the model was allowed to do.

What is the price increase, and who pays more?

DeepSeek raised prices significantly. The table below shows the new API rates.

A "token" is roughly three-quarters of a word. Sending a million tokens is roughly equivalent to processing 750,000 words of text. The increases range from 51% to over 1,100% depending on the model and time of day. DeepSeek says 17 of every 24 hours stay at the cheaper off-peak rate, a deliberate nudge to shift batch jobs to overnight windows.

Analyst Sanchit vir Gogia of Greyhound Research framed it clearly: "This is not a simple price rise. It is a pricing architecture that makes the timing of inference an economic variable." Work that can wait, such as summarising documents overnight or generating test data, can stay cheap. Live customer interactions cannot.

Tech analyst Carmi Levy called it a risky move for a company still building credibility, but added that DeepSeek remains cheaper than OpenAI, Anthropic, Google, and others even at the new rates. The business case still works; it just needs tighter maths.

What does this mean if you are thinking about using it?

For most businesses, DeepSeek is best suited right now to contained, repetitive tasks with clear success metrics, where a wrong answer is easy to catch and correct. Think document classification, code review queues, or overnight data processing runs.

Enterprise use of AI models for live, high-stakes workflows requires something harder to demonstrate: consistent reliability, data privacy guarantees, and audit trails. DeepSeek has not yet fully established those credentials at scale, first reported by VentureBeat.

Expect cautious pilots rather than full rollouts. IT teams will test in sandboxed environments before bringing budget requests to senior leadership.

The one doable takeaway: if you are weighing DeepSeek for a project, schedule your heavy processing jobs during off-peak hours and start with a workflow where a 46% failure rate is recoverable. Build in retries and a fallback model from day one. The model is genuinely capable; the surrounding setup is what decides whether that capability shows up reliably.