An empty chat box is not always freedom.
Sometimes it is a note from the product team:
We connected the model. You handle the rest.
The user must explain the task, collect the relevant information, summarize the history, write the instruction, specify the output format, and try again when the model misunderstands.
We call this “natural interaction with AI.” If older software had required users to write SQL before every action, the marketing department might have chosen a less romantic phrase.
Chat is a powerful interface. It is excellent for expressing uncertain ideas, asking questions, and exploring a problem collaboratively.
But chat does not need to be the entire interface of every AI product.
In many cases, the real value is not in the sentence the user types. It is in everything the system already understands around that sentence.
The real interface for AI is context.
A prompt is not enough
Early generative AI products were obsessed with prompts.
Write better instructions. Assign a role. Add examples. Tell the model not to invent facts. If it still invents facts, add three exclamation marks so it understands that this time you are serious.
For a one-step task, prompt quality matters. Once a system uses tools, performs several actions, or carries state over time, the problem becomes larger than the instruction itself.
The model may also need:
- the user’s current state;
- the object being edited;
- earlier decisions;
- available tools;
- permission boundaries;
- externally retrieved information;
- previous tool results;
- completion conditions.
Anthropic describes this shift as the move from prompt engineering to context engineering. Context includes not only the system prompt but the full collection of instructions, tools, data, and history available to the model at a given moment.
It is also finite. Adding information does not automatically add useful intelligence.
The objective is not to tell the model everything.
It is to provide the useful information at the moment it becomes necessary.
The user should not be a freight company
Poorly designed AI products turn users into context couriers.
They copy the document.
They summarize the previous conversation.
They explain which customer is involved.
They describe the file format.
They restate company rules.
They provide an example of the desired result.
The next day, they do it again because the new conversation offers “a clean start.” AI products have taught us that a clean start can involve an impressive amount of manual labor.
A well-designed system obtains context from the environment where the work is already happening.
Which document is open?
Which text is selected?
Which record is active?
What role does the user have?
Are they allowed to perform the action?
What was decided in a similar case?
Which information is current, and which was true last year?
If the user must type these facts every time, the chat interface is not flexible. It is lazy.
This is the difference between a model wrapper and a product. A wrapper offers access to intelligence. A product takes responsibility for the conditions that make that intelligence useful.
Model memory is not an information system
Large models contain an enormous amount of knowledge. That does not make the knowledge easy to update, attribute, or align with the current facts of a particular organization.
The original Retrieval-Augmented Generation research showed that giving a model access to relevant external documents could produce more specific and factual language on knowledge-intensive tasks. External knowledge can also be updated and traced in ways that information embedded in model parameters cannot.
The ReAct research demonstrated that models can improve some tasks by alternating reasoning with actions that gather information from external environments, rather than relying only on internal text generation.
The product implication is simple:
Do not expect the model to know everything.
Design where it should look.
This often leads to a new mistake: “If context is useful, let’s put all the data into the context.”
No.
That is merely the AI version of emptying the entire kitchen onto one plate.
Retrieval is not valuable because it makes the prompt longer. It is valuable because it lets the system choose evidence that is relevant, current, and attributable. A database dump is not context engineering. It is surrender with a token budget.
A context window is not a storage unit
Long context windows are impressive. A developer can provide hundreds of pages of documents, meeting notes, and conversation history in one request.
That does not mean the model will use every part equally well.
The Lost in the Middle study found that performance could decline significantly when relevant information appeared in the middle of a long context. Models often used information near the beginning or end more successfully while missing critical details buried between them.
Anthropic similarly frames context as a limited attention budget and recommends finding the smallest set of high-signal tokens that maximizes the chance of the desired result.
Context engineering is not data accumulation.
It is selection, ordering, summarization, and just-in-time retrieval.
A good system:
- separates permanent rules from temporary state;
- removes stale information;
- retrieves only records relevant to the task;
- preserves sources and dates;
- keeps critical constraints visible;
- loads detail through tools when needed;
- asks questions instead of inventing certainty.
More context is not automatically better context.
Sometimes it simply gives the model a larger selection of things to misunderstand.
Chat will remain, but it should not be alone
Chat is not going away, nor should it.
Some tasks are genuinely conversational. When users do not yet understand what they need, dialogue can help clarify the problem.
The mistake is converting every interaction into a conversation.
An action that appears when text is selected is an AI interface.
A form prepared from existing records is an AI interface.
A short explanation attached to an anomaly is an AI interface.
A comparison screen showing proposed changes before approval is an AI interface.
A search system that quietly identifies the right files is an AI interface.
None of these require the user to begin with, “Hello, please act as a highly experienced professional.” It would be healthy if interacting with software required less amateur theatre.
The best AI interactions may not look like AI at all. They may look like a normal product that already knows what page is open, what object is selected, and what the user is allowed to do. The model sits behind the workflow rather than demanding a starring role in it.
The best context is invisible, not secret
Automatically collected context introduces another question: what does the system know about the user?
The distance between a helpful context-aware product and a disturbing one can be a single missing explanation.
Users should be able to understand:
- which information is being used;
- where it came from;
- how current it is;
- what action the system intends to take;
- what will be stored;
- how incorrect context can be changed.
Context can be effortless without making control invisible.
The system should spare users from repeatedly transporting information, but it should not quietly write an unauthorized biography about them.
This is particularly important when context crosses boundaries. A message from one workspace may not belong in another. A private document may be relevant to the user but not appropriate for the current task. More access does not automatically create a better answer. Sometimes it simply creates a more comprehensive privacy incident.
The real interface is everything we do not need to explain again
A good AI product does not require users to become prompt specialists.
It knows which object is being edited.
It retrieves the relevant records.
It respects the user’s permissions.
It distinguishes current information from old information.
It asks for approval before consequential actions.
It makes uncertainty visible.
Chat may be one part of that experience. It should not be the entire product.
An empty chat box demonstrates the flexibility of the model.
A good context system demonstrates that the product team understood the work.
The model may provide the intelligence.
Context determines whether that intelligence becomes useful.
The real interface for AI is everything the user does not need to explain again.
Related reading
Sources and further reading
- Anthropic, Effective Context Engineering for AI Agents.
- Patrick Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020.
- Shunyu Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models.
- Nelson F. Liu et al., Lost in the Middle: How Language Models Use Long Contexts.