Applied AI / LLM engineer
Ships features on top of foundation models: retrieval, agents, evals.
What they do
An applied AI or LLM engineer builds product features on top of foundation models. They design the retrieval, prompting, tool use and agent logic, then build the evaluations and guardrails that keep those features dependable in production. They usually work in the product codebase, close to users, rather than training models.
Core skills
- Retrieval-augmented generation: chunking, embeddings, vector and hybrid search, re-ranking
- Prompt and context design, structured outputs and tool or function calling
- Agent and workflow orchestration, with clear limits on what the model may do
- Evaluation: test sets, model-graded and human review, regression checks on every change
- Guardrails, safety and handling of sensitive data
- Production engineering: latency, cost per request, caching, fallbacks between model providers
- Strong general software engineering, usually in Python or TypeScript
Signals we look for
- An LLM feature they shipped to real users, and what changed after launch
- How they measured quality, beyond "it looked right in testing"
- A failure they found in production and how they fixed or contained it
- Trade-offs they made between quality, latency and cost
Often confused with: Machine learning engineer. An LLM engineer builds with models; an ML engineer trains and serves them.
