How to Evaluate a Fractional CTO

The title problem

Fractional CTO is not a regulated title. There’s no certification, no licensing board, no standard body that validates who can call themselves one. Anyone with a laptop and a LinkedIn profile can claim it.

This creates a real problem for founders who need technical leadership but don’t have the technical background to evaluate candidates. The person sounds confident. They use the right buzzwords. They’ve done this before. Probably. Maybe.

I wrote this because the pattern of bad engagements is consistent enough to document. Here are the questions that separate someone who ships from someone who collects retainers.

The 8 questions

1. “Have you done the CTO role full-time, in a real company, with real users?”

Good answer: “Yes. I was CTO at [company] for [years]. We had [size] team and [metric] users. I hired, I architected, I shipped, I handled incidents at 3 AM.”

Bad answer: “I’ve been consulting for years. I’ve worked with many startups.”

A fractional CTO who has never been a full-time CTO hasn’t made the hard decisions. They’ve advised. They haven’t owned. Ownership and advice are different muscles.

2. “Can you show me a production system you built?”

Not a diagram. Not a slide. A running system, ideally one you can access or see screenshots of.

Good answer: “Yes, here’s the system. Here’s what it does. Here’s what broke after launch and how we fixed it.”

Bad answer: “I can’t share client details due to NDAs.” Or they show you a diagram, not a system.

If they can’t show you anything real and running, they haven’t shipped anything real and running.

3. “Tell me about something that broke and how you handled it.”

This is the most revealing question. Everyone has war stories. The quality of the story tells you about their judgment.

Good answer: A specific incident with specific details. “We had a database migration that corrupted customer records. We restored from backup, identified the root cause (missing foreign key constraint), added tests, and implemented a rollback plan for all future migrations. The incident took 4 hours to resolve, and the same mistake never happened again.”

Bad answer: “Nothing major broke.” Or a vague story with no specifics. Or a story where they blame someone else.

If nothing ever broke, they never shipped anything worth shipping.

4. “Can I talk to a non-technical founder you’ve worked with?”

Technical founders can evaluate technical people. Non-technical founders need someone who can translate. That’s a different skill.

Good answer: “Yes, here’s a founder who was in your position - they knew they needed technical help but couldn’t evaluate it themselves. They’ll tell you how I communicated.”

Bad answer: “All my clients are technical.” Or “I can’t share references.”

If they’ve only worked with technical people, they haven’t developed the translation skill that makes fractional CTOs valuable to non-technical founders.

5. “What’s your handover process?”

Fractional CTOs should plan to leave from day one. If you can’t run without them, they haven’t done their job.

Good answer: “Documentation from day one. Your team shadows key decisions. By month 3, they’re making decisions themselves. By month 6, I’m available for questions, not operations.”

Bad answer: “We can discuss a retainer if you need ongoing support.” Or “Your team isn’t ready yet.”

If they don’t have a handover plan, their business model is dependency.

6. “What’s your actual availability?”

Fractional means different things to different people. 5 hours/week? 20 hours/week? Fixed schedule or as needed? Clarify before you commit.

Good answer: “I’m available [specific days/hours]. I respond to urgent issues within [timeframe]. Here’s what I can and can’t commit to in that time.”

Bad answer: “We’ll figure it out as we go.” Or “I’m very flexible” without specifics.

If the availability isn’t scoped, it will expand in ways that frustrate both of you.

7. “How do you stay current?”

Technology changes fast. A fractional CTO who learned their craft 10 years ago and hasn’t updated is dangerous.

Good answer: “I ship - I’m building systems myself. I also read, attend conferences, experiment with new tools. Here’s what I’ve learned recently and how it changed my approach.”

Bad answer: “I read Hacker News.” Or “I follow the trends.”

The best way to stay current is to build. If they’re not building, they’re not current.

8. “How will I know you’ve succeeded?”

Without success criteria, you can’t evaluate whether the engagement is working. Define them upfront.

Good answer: “By month 3, your team will have [specific capability]. By month 6, [specific metric]. We’ll review quarterly against these criteria.”

Bad answer: “We’ll see how it goes.” Or “You’ll feel it.”

If they can’t define success, they can’t be held accountable for failure.

Red flags

• Can’t show production code - they’ve advised, not built.
• Owns a proprietary platform - “We use our framework” = lock-in.
• No handover plan - dependency is the business model.
• Can’t explain things simply - they don’t understand it well enough or they’re hiding something.
• Blames previous clients - you’ll be the next one they blame.
• Says “your team isn’t ready” without a plan - that’s a stall, not an assessment.
• Only talks about technology, not outcomes - they’re a technician, not a leader.

Green flags

• Has shipped production systems they can show - real experience, not theory.
• Has turned down work - standards over revenue.
• Talks about handover in the first conversation - they plan to leave you capable.
• Explains complex things simply - they communicate, not obscure.
• Asks about your business before your tech stack - they know the problem comes before the solution.
• Provides references from non-technical founders - they can translate.

The test

Here’s a practical test. Ask the candidate to spend 30 minutes (free, no commitment) diagnosing one specific technical problem you’re facing. Not solving it - diagnosing it.

A real fractional CTO will identify the root cause, ask good questions about context, and tell you what they’d need to investigate further. A pretender will give you generic advice that sounds insightful but doesn’t apply to your specific situation.

The diagnosis is the product. If they can’t diagnose, they can’t deliver.

FAQ

How many candidates should I evaluate? 3-5. More is procrastination. After 5, you’ll see the same patterns and your judgment will degrade.

Should I ask for a free trial? No. Ask for a paid diagnostic session (€500-€1000). Free trials attract pretenders. Paid diagnostics attract professionals.

What if I can’t evaluate their technical answers? Bring someone who can, even if it’s a friend or former colleague who’s technical. Or look for the non-technical signals: clarity, honesty, specificity, and whether they ask you good questions about your business.