How do you build an AI team at a startup?
Short answer
Build an AI team at a startup by hiring for the product, not the model. Start with one or two engineers who can ship an AI feature end to end, involve a domain expert from the first week, and add specialists in data, machine learning or infrastructure only when a measured bottleneck appears. Most early AI products need software and data engineering more than research, so machine learning specialists usually join after the first data and evaluation results.
By Dr. Dulan Dias, co-founder and CTO of Tellme AI
Key facts
- Chapter 16 of From Model to Product names seven roles in an AI product team: machine learning engineer, data engineer, backend engineer, infrastructure or DevOps engineer, product manager, domain expert and QA or testing engineer, and notes that in small teams one person may perform several of them.
- The 2015 NeurIPS paper Hidden Technical Debt in Machine Learning Systems, by D. Sculley and colleagues, shows that only a small fraction of a real-world machine learning system is the machine learning code; the surrounding infrastructure is vast and complex.
- Rule #1 of Google's Rules of Machine Learning, written by Martin Zinkevich, is: Don't be afraid to launch a product without machine learning.
- In DSIT's AI Labour Market Survey 2025, published on 28 January 2026, 35% of the 119 organisations surveyed had AI vacancies they found hard to fill, mainly because candidates lacked work experience (31%) or technical skills (30%).
- The same survey found that AI professionals and AI leaders made up 44% and 42% of hard-to-fill AI vacancies, against 14% for AI workers whose main job is not AI.
- DSIT's AI sector study 2025, published on 29 October 2026, estimated that 130,400 people worked in AI-related roles in the UK in 2025, up 51% on 2024, and reported competition for experienced AI talent as a prominent concern.
Definitions
- AI engineer
- An engineer who builds products on top of pre-trained models, through APIs or open weights: prompts, retrieval, integration, evaluation and the backend around them.
- Machine learning engineer
- An engineer who trains, evaluates and serves models from data, and owns model quality, size and inference speed.
- Data engineer
- An engineer who builds the pipelines and storage that deliver consistent, versioned data to training, evaluation and inference.
- Domain expert
- Someone who knows the real-world rules and edge cases of the field the product serves, such as a clinician, lawyer or underwriter, and can say what a correct answer is.
What roles does an AI product team need?
Chapter 16 of From Model to Product, Working in Teams, describes seven roles in an AI product team, each with its own primary concern. A startup will not hire seven people at once, but every responsibility still has to be covered by someone, and an uncovered one shows up later as a failure in production.
The third column is the practical question for a team of two or three.
| Role | Primary concern | Who covers it before you hire |
|---|---|---|
| Machine learning engineer | Model accuracy and performance | The AI engineer, using provider models and an evaluation set |
| Data engineer | Data quality and pipelines | The backend engineer |
| Backend or software engineer | APIs and system logic | The first engineer, often the technical co-founder |
| Infrastructure or DevOps engineer | Deployment, scaling, monitoring | The backend engineer, on managed cloud services |
| Product manager | Requirements and user needs | The CEO or another founder |
| Domain expert | Real-world correctness | An adviser, design partner or early customer |
| QA or testing engineer | Validation and reliability | Everyone, through an evaluation set run on every change |
Who should you hire first, and in what order?
This order suits a product built on foundation models. If the product's value is a model trained on your own data, move the machine learning hire earlier.
- An engineer who can ship an AI feature end to end: model calls, retrieval, backend, deployment and logging. Often the technical co-founder, and the hire that decides how fast you learn.
- Time from a domain expert, from the first week. They write the first evaluation cases and say what a wrong answer costs. Part time is fine; absent is not.
- A second engineer weighted towards backend and data: pipelines, data quality, the evaluation harness and the logs that make quality measurable.
- Infrastructure ownership, once deployment, cost and reliability work start crowding out product work. Until then, managed services and one named owner among the engineers are enough.
- A machine learning specialist, once you have data and a measured quality gap that prompting and retrieval cannot close.
- A product manager and dedicated testing ownership, once founders can no longer hold the priorities and release pace makes regressions expensive.
Generalists or specialists?
Hire generalists first and specialists for a bottleneck you can measure. Early on the work changes weekly, and an engineer who can move between the model call, the database and the deployment pipeline is worth more than a specialist waiting for their part.
Dr. Dulan Dias built Tellme AI's production MVP end to end as its founding engineer within 4 to 6 months: the Next.js frontend, the FastAPI and MongoDB backend, the admin portal, CI/CD and the AWS infrastructure. That is the profile worth hiring first: someone who can carry a product to production.
These signals justify a specialist.
| Signal | Specialist to add |
|---|---|
| Quality has plateaued on the evaluation set after prompt and retrieval work | Machine learning engineer |
| Data arrives late, inconsistent, or different in training and production | Data engineer |
| Deploys feel risky, and cost or uptime incidents keep recurring | Infrastructure or platform engineer |
| Engineers argue about what a correct output is | More domain expert time, then a product manager |
| Regressions reach customers after model or prompt changes | Dedicated testing and evaluation ownership |
When should you hire ML specialists rather than AI engineers?
Hire AI engineers first when the product is built on foundation models through APIs or open weights, because most of the work is integration, retrieval, evaluation and cost control. Hire a machine learning engineer early when the product's value is a model trained on your own data, such as an image classifier, a forecast or a risk score, or when models must run on devices or at volumes a hosted API cannot serve.
Two well-known sources argue for patience. Rule #1 of Google's Rules of Machine Learning is not to be afraid to launch a product without machine learning, and the 2015 NeurIPS paper Hidden Technical Debt in Machine Learning Systems shows that model code is a small fraction of a real system. The difference between the roles is covered in what is AI engineering, and the model choice in RAG vs fine-tuning.
How do you work with domain experts?
Domain experts make an AI product correct rather than merely plausible. Give them work with a clear output, not a standing invitation to meetings.
- Writing and reviewing the evaluation set: real inputs, the right answer and why.
- Reviewing a sample of production failures every week and labelling the cause.
- Owning the rules the product must follow, such as regulations, policies or thresholds, kept outside the model where they can be changed and audited.
- Defining where the product should say it does not know and hand over to a person.
- Spotting the edge cases engineers would never think to test.
Hiring AI engineers in the UK
Experience is the scarce thing. In DSIT's AI Labour Market Survey 2025, 35% of organisations had hard-to-fill AI vacancies, and the most common reason was a lack of work experience (31%), just ahead of a lack of technical skills (30%). AI professionals and AI leaders made up 86% of the hard-to-fill roles between them. DSIT's AI sector study 2025, published in October 2026, adds that competition for experienced AI talent and visa barriers remain prominent concerns.
For a startup, that argues for hiring strong software engineers and growing their AI skills, testing candidates on a realistic task such as improving an evaluation score, and looking beyond the usual routes: in the same survey, apprenticeships rose from 3% of AI hires in 2020 to 19% in 2025. Treat the survey as a signal rather than a census, since 119 organisations responded.
How do you keep a small AI team working as one?
The hardest problems in a small AI team are usually about shared understanding, not algorithms. Chapter 16 of the book gives the example of the word model, which means a trained network to a researcher, a service endpoint to a backend engineer and a feature to a product manager.
Agree definitions for model, accuracy, latency and confidence; write down assumptions such as image resolution or data freshness; share prototypes early; and keep exploratory notebooks apart from production code. Give everyone a stake in the whole system: the machine learning engineer cares how the model is deployed, and the backend engineer cares how it behaves.
Frequently asked questions
How many people does a startup need to build an AI product?
A startup can build a first AI product with very few people when it builds on foundation models: one strong engineer who can ship end to end, plus founder and domain expert time, can reach a production MVP. Products that train their own models, run on devices or serve regulated customers need more, because data engineering, infrastructure and testing become full-time work sooner.
Should the first AI hire be a data scientist?
For most startups the first AI hire should not be a data scientist. A data scientist analyses data and builds models for insight, while an early product needs someone who can ship and run software. Hire an engineer who can build an AI feature end to end first, and add data science once you have enough data and a question that analysis can answer.
Can you outsource AI development instead of building a team?
You can outsource a first build, but not the judgement that keeps an AI product working. Evaluation, data rights, model choice and cost per user need someone inside the company who will still be there in a year. If you use an agency, keep the code, cloud accounts and evaluation set in the company's name, and hire the first in-house engineer before the agency hands over.
How do you interview AI engineers?
Interview AI engineers on realistic work rather than puzzles. Give a small task with a real model: improve a prompt or retrieval step against a provided evaluation set, explain the trade-offs, and say what they would monitor in production. Ask about a model failure they debugged. Strong candidates raise data, evaluation and cost unprompted, not only which model they would choose.
Who should own evaluation in a small AI team?
One named engineer should own the evaluation harness, with the domain expert owning what counts as correct. Everyone else contributes cases, especially from production failures. Without a single owner the evaluation set goes stale, and model or prompt changes start reaching customers untested. As the team grows, this becomes a dedicated testing or quality role.
When should an AI startup hire a product manager?
An AI startup should hire a product manager when founders can no longer keep priorities clear across customers, engineers and domain experts, often once the product has paying users and more than one team. Until then a founder usually plays the role. A good AI product manager turns vague goals such as helpful answers into measurable targets the engineers can test.
How can a startup compete with big tech for AI talent?
A startup competes for AI talent with scope, ownership and speed rather than salary. Offer real responsibility for a product in production, equity that could matter and fewer layers between an idea and its users. Widen the pool by hiring strong software engineers who want to learn AI, and by considering apprenticeships, which made up 19% of AI hires in 2025 in DSIT's labour market survey.
Sources
About the author
Dr. Dulan Dias is co-founder and Chief Technology Officer of Tellme AI in Belfast; founder of EchonLabs, grown from 2017 to more than 60 people before its acquisition in 2026; holder of a PhD in AI and data science from Queen’s University Belfast; and the author of From Model to Product.