As we increase our use of AI coding assistants (Claude, Cursor, etc.), it’s worth understanding how context works—both to get better results and to use resources efficiently.
What is Context?
Context is everything the AI “sees” when you ask a question:
- The files you’ve mentioned or opened
- Your conversation history
- System instructions and documentation
- Your current question
- The AI’s response
All of this gets sent to the model for every single message you send.
What are Tokens?
Tokens are how AI models measure text. Roughly:
- 1 token ≈ 4 characters or ¾ of an English word
- A typical sentence is 15-25 tokens
- A question like “How do I add a unique constraint?” is about 10 tokens
File sizes in tokens (approximate):
| File Size | Tokens | Example |
|---|---|---|
| 1 KB | ~250 | Small config file |
| 5 KB | ~1,250 | Typical source file |
| 20 KB | ~5,000 | Large module |
| 100 KB | ~25,000 | Very large file |
When you @ mention a file, the entire file gets added to the context… every time you send a message.
Why This Matters
1. Cost is calculated per token
Every token in your context costs money—both input (what you send) and output (what the AI generates). A conversation with 50K tokens of context costs significantly more than one with 5K tokens.
Example:
- Simple question with minimal context: ~2K tokens → low cost
- Same question with 5 large files mentioned: ~50K tokens → 25x the cost
2. Bigger context ≠ better results
This is counterintuitive, but more context can actually hurt response quality:
- The AI may get distracted by irrelevant information
- Important details can get “lost” in a sea of text
- The model has to decide what’s relevant, and it doesn’t always choose well
- Response time increases with context size
A focused question with precisely relevant context will outperform a vague question with everything attached.
Context must be treated as a finite resource with diminishing marginal returns.
As its context length increases, a model’s ability to capture these pairwise relationships gets stretched thin, creating a natural tension between context size and attention focus.
Recommendations
- Start fresh conversations for unrelated topics. Each new question in the same chat carries forward all previous context. If you’re switching tasks, start a new conversation.
- Be selective about which files you @ mention. Only include files directly relevant to your question. If you need context from a large file, consider copying just the relevant function or section instead.
- Ask about specific sections. Instead of “review this file,” try “review the validation logic in the handleSubmit function” or copy the text you want to put on the table. This focuses the AI’s attention and often produces better results.
- Use separate chats for different tasks. Working on both a new design for a webpage and a job description? Use two conversations. This keeps each context focused and prevents cross-contamination.
- Summarize and restart long conversations. If a conversation has grown long, consider asking the AI to summarize the key decisions, then start fresh with that summary.
Quick Self-Check
Before sending a message, ask yourself:
- Do I need all these files in context, or just one or two?
- Is this related to my previous questions, or should I start fresh?
- Can I be more specific about what I need?
Being mindful of context helps us use AI tools more effectively while being responsible with shared resources.
Leave a Reply