How to vet an AI automation freelancer.
A polished demo tells you little. What you want to know is whether this person can keep a real system running after the first week. These are the checks we use ourselves.
6 min read. Updated
1. Ask to see something live
Ask for a screen share of a system running for a real client, including its run history. A demo built last week shows someone can follow a tutorial. Months of run history shows the system survived real use.
2. Ask how it fails
This is the most useful question you can ask. Someone with real experience will quickly tell you about a time an API changed, a file arrived broken or a queue backed up, and how they found out. If every story is a success story, be careful.
3. Check how they treat your keys and data
A careful builder will:
- Work in accounts that you own
- Store keys in a secrets manager or credentials store, not in the workflow itself
- Ask which personal data is involved before choosing tools
- Give each connection only the access it needs
4. Set a short, fair test
Give them a small task on made-up data, and keep it to about 90 minutes. Include one case that should not be processed automatically, and see whether they catch it. Then ask for one change on a call. AI tools are fine, as long as they can explain the result.
Never use a real client task as an unpaid test. It is unfair, and good people will walk away.
5. Run a mock first meeting
The best builders ask about your problem before they name a tool. Describe a vague need and see what they do. Do they find out the volume, the exceptions and who owns the process? Then ask for a one-page scope.
6. Call the references yourself
Find the client company's contact details on their own website rather than using a number the freelancer gives you. Ask two things: would you hire them again, and who holds the accounts and keys today?
Red flags
Walk away if you see any of these:
- They want to host everything in their own accounts
- They cannot show a system that has run in production
- They promise a percentage saving before seeing your process
- They will not write down what is and is not included
Questions
Should I pay for a test?
For a long test, yes. A short test on made-up data is common and reasonable. Keep it brief, and let them keep what they build.
What if I am not technical?
Live work, references and the mock meeting do not need technical knowledge. For the interview and test, ask someone technical to help, or use a marketplace that does this checking for you.