Quick takeaways
- Compare models by workflow, not by leaderboard rank.
- Long context, reasoning, speed, and cost all matter depending on the job.
- Re-test model fit when the task changes or the vendor releases a new version.
A developer's introduction to large language models
Andrej Karpathy covers how large language models work, what they excel at, and how to think about choosing one.
How to compare models
Use a real task, the same input, and a simple rubric. For example, compare models on a customer research summary, code review note, or SEO outline. Look at correctness, clarity, structure, and cleanup required.
- Pick a task your team does weekly.
- Prepare the same input and prompt for each model.
- Score outputs on correctness, clarity, structure, and tone.
- Measure cost and speed if you plan to run the task at scale.
- Document the winner and why, then re-test quarterly.
Selection criteria
Price tier matrix
API pricing varies by model strength and provider. This table gives a rough sense of where each family sits. For exact rates, see the AI model API pricing page.
| Tier | Typical price / 1M tokens | Good for | Examples |
|---|---|---|---|
| Budget | ~$0.05 - $0.25 | High-volume, low-risk tasks | Qwen, DeepSeek-V3, Gemini Flash |
| Mid-range | ~$0.50 - $3.00 | Most daily work | GPT-4o, Claude Sonnet, Gemini Pro |
| Premium | ~$10.00+ | Hard reasoning, agents, coding | Claude Opus, OpenAI o-series |
Use case matrix
General assistant model
Use for drafting, editing, and structured thinking.
Code-aware model
Use for repo changes, debugging, and code explanation.
Source-heavy model
Use for summaries, citations, and decision support.
Multimodal model
Use for image, voice, or mixed-format tasks.
Best for specific jobs
Best AI model for research
Models that handle citations, source synthesis, and deep-dive analysis.
Read guideBest AI model for long context
Models that can process long documents, books, and large codebases.
Read guideBest local LLM
Self-hosted options for privacy, control, and cost at scale.
Read guideBest AI model for coding
Top picks for repo work, debugging, and learning to code.
Read guideDecision tree
Use this tree to narrow your choice:
- Is the task about writing, editing, or reasoning? Start with a general assistant like GPT-4o or Claude Sonnet.
- Is the task about code? Use a code-aware model or coding agent like Claude Code, Cursor, or Codex.
- Does the task need long documents or many files? Prioritize context window and retrieval support.
- Does the task need images, audio, or video? Use a multimodal model like Gemini or GPT-4o.
- Is the task high-volume and low-risk? Consider a smaller, cheaper model or API tier.
Popular families
Explore the current model family pages
OpenAI GPT family
General-purpose models for writing, reasoning, voice, images, and coding.
Open familyClaude family
Writing, analysis, coding, and agentic workflows.
Open familyGemini family
Multimodal, Google-native, and voice-ready workflows.
Open familyGrok family
Live search, reasoning, multi-agent, and media generation.
Open familyDeepSeek family
Reasoning, coding, and open-model experimentation.
Open familyLlama family
Open models for self-hosting and customization.
Open familyMistral family
Document intelligence, coding, and platform control.
Open familyQwen family
Multilingual, multimodal, safety, and translation workflows.
Open familyWithout AI vs. with AI
| Task | Without AI | With AI |
|---|---|---|
| Model choice | Teams pick models based on leaderboard hype or brand recognition. | Models are chosen by use case, context length, speed, cost, and review needs. |
| Latency | Latency is guessed from marketing claims and benchmark tables. | Latency is measured on real inputs at expected throughput and concurrency. |
| Cost | Cost is ignored until the first big bill arrives. | Per-token and per-request economics are modeled before scaling the workflow. |
| Context length | Long documents are truncated or chunked without a strategy. | The right context window and chunking approach are selected for the task. |
| Quality | Quality is judged from public benchmarks alone. | Teams score outputs on the same real task using a simple rubric and re-test quarterly. |
FAQ
Which model is best for beginners?
The best beginner model is usually the one already integrated into your workflow and easy to review.
Should teams use multiple models?
Yes, if different workflows benefit from different strengths. Keep the stack small enough to govern.
How often should we re-test model fit?
Quarterly, or whenever a major version is released or your workflow changes significantly.
Do benchmarks matter?
Benchmarks give a rough signal, but your real task and review process matter more.
Should we self-host an open model?
Consider self-hosting when data residency, customization, or cost at scale is critical. It adds infrastructure and maintenance overhead.
How do we avoid model lock-in?
Use portable prompts, avoid proprietary features for core workflows, and test alternatives periodically.
Is the most powerful model always the best choice?
No. The best model balances capability, cost, speed, and the amount of review your use case can tolerate.
How do I compare models fairly?
Use the same prompt, input, and rubric across candidates; score correctness, structure, tone, cost, and latency.
Should I use one model or many?
Use one general model for simple work and route specialized tasks to models that excel at coding, reasoning, or long context.
How often should I re-evaluate model choice?
Re-test when the task changes, usage scales, or a vendor releases a new version that claims better performance.