DeepSeekV3 vs Claude-Sonnet vs o1-Mini vs Gemini-ept-1206: AI Coding Assistants Tested in Real-World Scenarios

In today's world of AI-assisted programming, choosing the right AI assistant has become increasingly important. As a long-time user of AI coding assistants, I recently conducted an interesting experiment comparing four mainstream AI coding assistants in a real project. This experiment not only gave me deeper insights into each model but also revealed some surprising results.

Experiment Background: A Real Development Need

During the Christmas holiday, I started developing a smarter home assistant project, aiming to create something better than Google Home and Alexa. One of the key features was implementing an AI memory system - for instance, when a user says "I don't like eggs, remember that," the system would avoid recommending recipes with eggs in the future.

To implement this feature, I needed to develop an Azure Functions project as a proxy, handling data interactions with Azure Table Storage, and integrate it into an existing Blazor WASM application. This seemingly simple requirement actually involved multiple aspects including project creation, cloud deployment, and existing project feature expansion, making it perfect for testing AI coding assistants.

Claude-Sonnet: The Reliable Veteran

Claude-Sonnet performed like a seasoned senior engineer. Throughout the development process, it demonstrated exceptional code quality control, automatically detecting and fixing issues in the code, and even intelligently pre-filling tool URLs after deployment. However, this "veteran's" services don't come cheap. In the basic API version, it hit the limit after just $0.2, forcing a switch to OpenRouter. More surprisingly, the cost through OpenRouter soared to $2.1, with some performance degradation.

DeepSeekV3: The Dark Horse

DeepSeekV3's performance was truly impressive. I tested it through both OpenRouter and the official API, with strikingly different results. Via OpenRouter, it seemed somewhat clumsy, with code duplication and limited functionality. However, when using the official API, it was like a different model altogether - code quality nearly matching Claude's, smooth operation, and unique solution approaches. Most impressive was its price advantage, completing the entire task for just $0.02. In the deployment phase, while it chose a more traditional manual zip deployment method, it showed some surprising capabilities, like autonomously finding resources and constructing storage connection strings.

Gemini-ept-1206: Growing Pains of a Promising Newcomer

Gemini feels like a promising but inexperienced newcomer. It showed the strongest interaction among all models, proactively asking about runtime versions and other details. It excelled in deployment configuration, anticipating environment variable setup. However, it also showed some "growing pains": slow processing speed, often taking 20 minutes to complete tasks; token limit constraints, frequently requiring multiple sessions; and most frustratingly, even after 24 hours, its cost statistics remained opaque, making it impossible to accurately assess usage costs.

o1-Mini: Unfulfilled Promises

o1-Mini's performance was rather disappointing. It started well, with smooth project setup and acceptable initial code quality. But things went downhill from there: slow response times, frequent incorrect assumptions (like creating resource groups in wrong geographical locations), and inefficient problem-solving. After spending $2.2, it even suggested downgrading the .NET version to solve issues, forcing me to terminate the test early.

Practical Insights and Recommendations

Through this experiment, I've drawn some practical conclusions. For individual developers and small projects, DeepSeekV3 is undoubtedly the best choice, perfectly balancing code quality and cost. For those with sufficient budget, Claude-Sonnet remains a reliable choice for enterprise-level development. Gemini suits scenarios requiring detailed interactive guidance, while o1-Mini might find its niche in specific algorithm optimization problems.

It's worth noting that using these models through OpenRouter often affects their performance, so it's recommended to use official APIs when possible. Additionally, we must recognize that the AI coding assistant field is rapidly evolving, with all models continuously improving their capabilities. The competitive landscape could change significantly in the future. Choosing the right AI assistant should be based on specific project requirements, budget constraints, and development scenarios, rather than blindly following any particular option.