Most founders who've hired an AI vendor regret how they picked them. Not because the work was necessarily wrong — sometimes it was fine — but because they evaluated the vendor the same way they'd evaluate a web design agency. Different game.
The tells that matter aren't on the website.
Why most AI vendor searches go wrong
Every agency pivoted to AI in 2023. Design shops, offshore dev factories, management consulting arms — they all added "AI" to their service pages and hired one person who'd done a Coursera course.
The result: a market where credentials are meaningless and portfolios are written to obscure what actually happened.
"We built an AI assistant that reduced support tickets by 40%" is not a portfolio. It tells you nothing about technical depth, production stability, or whether they understand the difference between a chatbot and an agent. A real AI development company can tell you specifically which embedding model they'd use for your use case and why, what their production observability setup looks like, where they've failed and what they learned from it.
Generic answers to specific questions are your answer.
The portfolio questions worth asking
Most case studies are marketing. You need to get past them.
Ask for one past project where the scope changed significantly after build started. How did they handle it? What was the real impact on timeline and cost? Ask for a project where something failed in production — not a bug, but a fundamental architectural choice that turned out to be wrong. What did they do? Ask if they've had a client kill a project mid-build. Any shop that's done enough work has. How they talk about it tells you more than any success story.
If they can't give you specifics on any of those, they either haven't shipped enough or they're hiding the real outcomes.
What the billing model tells you about risk
There are three delivery models in AI development, and the risk profile differs significantly between them.
Time-and-materials. You pay by the hour or sprint. Scope creep is their revenue. This works if you have an internal technical lead who can audit progress. Without one, costs compound quickly.
Fixed-price. Fixed scope, fixed price, fixed timeline. Good shops front-load scoping heavily because they're on the hook if something goes wrong. Bad shops use fixed price as a hook and add change orders throughout.
Outcome-based. Payment is tied to delivery of agreed acceptance criteria, with a contractual refund if they miss. This is rare. When a company genuinely offers it — not as a pledge but as a written guarantee — it usually means they scope carefully and have done this enough times to know what they can commit to.
That's the model behind ClearShip, our fixed-price AI development offering. Not because it's clever marketing — it forces us to scope carefully and not take projects we can't guarantee.
The five questions worth asking before you sign
What do you build on, and why?
A vendor with a specific answer has thought about your use case. "We use whatever's best for you" usually means they don't have strong enough opinions to choose — which means they haven't done enough of this to form any.
What does your evaluation setup look like?
LLM outputs are probabilistic. Every production AI system needs a way to measure whether it's working — automatically, not just through user complaints. If they've never built an eval pipeline, don't hire them for production work.
Who exactly will build my project?
Agency bait-and-switch is common: senior engineers sell, junior engineers build. Find out who writes code on your project. Ask to meet them before you sign.
What happens when a model provider changes their API?
OpenAI has broken production code with API changes multiple times. Anthropic has deprecated models mid-project. A vendor who's shipped enough has a plan for this. One who hasn't will shrug.
What's your change order policy?
This is where fixed-price deals get messy. Get the policy in writing: what triggers a change order, who decides, what it costs. The policy matters less than whether they can explain it clearly without hesitation.
Red flags that are easy to miss
If the proposal came back in 48 hours, it was written before they understood your problem. Scoping a real AI project takes time. Good shops ask a lot of questions and take a week.
Watch how they explain their architecture. Strong AI engineers can describe what they're building to a non-technical founder in plain language. Buzzword soup — "we use a multi-modal RAG pipeline with agentic orchestration" with no further explanation — usually means they're hoping you won't ask follow-ups.
Check how varied their portfolio is. Building chatbots is not the same skill as building AI agents or RAG systems. If every case study is a chatbot, that's the one thing they know how to do. Make sure it matches what you need.
Watch for anyone pushing fine-tuning before you've even explored RAG. Fine-tuning is expensive, slow, and usually unnecessary — but it sounds impressive, which makes it a common oversell. A vendor recommending fine-tuning for a knowledge retrieval problem either doesn't understand the difference or is optimizing for a bigger invoice. The actual difference between RAG and fine-tuning is simpler than most vendors make it sound.
Finally: ask them about a project that went wrong. Every shop has one. If they can't name something they got wrong in the last year, they're not self-aware, not honest, or both.
How to compare two proposals
When you have two or three proposals in hand, compare on these:
| Criteria | What to look for |
|---|---|
| Scope definition | Specific acceptance criteria, not vague deliverables |
| Timeline | Broken into phases? What's the first delivery milestone? |
| Team | Named people, not "our engineering team" |
| Evaluation plan | How will you both measure whether it's working? |
| Change order policy | Clear, in writing, with cost estimates |
| Payment terms | Milestone-based beats 50/50 up front/on delivery |
Price is the least diagnostic signal. The range for similar work is wide — a $15K proposal and a $90K proposal for the same scope are both common. The question isn't which is cheaper. It's which is more likely to actually ship.
If you're not sure what you need to build yet
Hiring a development company before you've scoped the problem is expensive. You'll get a build that matches what they know how to sell, not necessarily what your business needs.
The AI Profit Leak Audit is a 30-page assessment ($497, delivered in 7 days) that maps your operations, identifies the highest-ROI AI opportunities, and gives you a prioritized build roadmap with a vendor shortlist and build-vs-buy calls. Most clients find it changes what they thought they needed.
If you already have a use case defined and want to compare your build options, AI development services covers what to expect from each engagement type.
Frequently asked questions
How much does an AI development company charge?+
A focused boutique shop building a production AI agent or RAG system typically runs $25,000–$80,000 for a full build. Prototypes run $8,000–$25,000. Large agencies charge $200,000+ for similar scope because of overhead. Offshore teams are cheaper upfront but rarely have the LLM depth needed for production-grade systems.
How long does AI development take?+
A first production agent or integration with a focused team is typically 3–8 weeks. More complex multi-agent systems or full products run 8–16 weeks. Be skeptical of vendors promising production-ready work in under two weeks — they're describing a prototype or they haven't scoped your project.
Should I hire a large AI agency or a boutique firm?+
For a $1M–$15M business building a first AI product, a specialized boutique shop almost always delivers better value. Large agencies have overhead, slower cycles, and tend to put senior talent on the pitch and junior talent on the build. Boutique shops are faster and more specialized, but they do less hand-holding — you'll need someone internally who can define requirements clearly.
What's the difference between an AI development company and an AI consulting firm?+
Consulting covers strategy, architecture decisions, and vendor evaluation. Development is building the thing. Some companies do both. Make sure the company you hire can do both, or pair a consulting engagement with a separate build partner.
Do I need an internal technical team to work with an AI development company?+
Not necessarily. What you need is someone who can define business requirements: what the system needs to do, what good output looks like, what failure looks like. The technical translation is the vendor's job. Some shops — including ours — are set up to work with fully non-technical teams.