You do not need a research lab. You need a team that can ship. Most companies — even very large ones — would be better off with three senior engineers who have shipped a bad LLM product and learned why, than with ten freshly-minted PhDs and one VP of AI.
The shape of the team
Our default first hire is not a researcher. It is a senior full-stack engineer who has shipped at least one LLM-powered feature in production and seen it break. That person sets the engineering culture: they will say no to demos, they will write evals before they write prompts, they will refuse to ship without observability. You build the team around them.
The five roles, in order of hiring
- Senior engineer #1. Full-stack, has shipped LLM features, has scars. Will set the bar.
- Senior engineer #2. Same profile. Two seniors form a team; one senior is a single point of failure.
- Evaluation lead. Could be ML-leaning or QA-leaning, but obsessed with measurement. Builds the eval harness.
- Designer. An interaction designer, not a visual designer. Owns the human side of the agent.
- Product. Last, not first. A product manager without a working prototype to react to is not effective in this domain.
Who you don't need (yet)
You do not need a Chief AI Officer. You do not need a research scientist. You do not need a fine-tuning specialist (you'll fine-tune later, if at all, and you can hire a contractor for it). You do not need an MLOps engineer (your senior engineers can wear this hat for the first year). Hire these later, when scale forces you to.
Where to find the people
Three sources, in order. First: senior engineers at companies who shipped an LLM thing in 2023–2024 and learned the hard way. Their LinkedIn says "interested in AI engineering." These are the most valuable people in the market and they are findable. Second: ex-startup founders whose companies didn't make it. They have shipped under constraint. Third: junior engineers who have built, on their own time, a side project that demonstrates taste. Hire one for every two seniors.
How to interview
- Have them debug a broken eval. Better than any take-home. Reveals how they think about correctness.
- Have them critique a bad prompt. Reveals taste, opinions, and depth.
- Ask: when have you killed an LLM feature? The best engineers have killed at least one.
- Skip Leetcode. Whether they can solve graph problems on a whiteboard tells you nothing about whether they'll build a good agent.
Compensation reality
Senior AI engineers in 2026 are paid 30–40% above general senior engineers in the same market. This is real and won't normalize for at least 18 months. Budget for it. The alternative — saving 30% on salary and shipping six months later — costs more.
The org structure question
Centralized AI team, embedded AI engineers, or a hybrid? Our experience: start centralized so the team can develop a shared culture and tools. Embed individuals into product teams once they have something to give. The pure embedded model — every product team hires their own AI engineer — produces eight slightly different RAG implementations within a year. Don't do it.