Your first AI project: a clear outcome and data ready for the task
Choose a measurable AI task, fix the specific data it needs and test the complete workflow, including review effort, errors and permitted actions.
A useful first AI project needs a specific job to improve, information suitable for that job and a way to judge the result. Those decisions give a pilot a purpose before anyone chooses a model or builds a connection.
You do not need to clean every system before testing any use of AI. You do need to understand and address the data, access and quality issues that the chosen task depends on.
Choose work that can be evaluated
Consider a service team preparing replies. Staff read the request, find records, draft an answer and check it before sending. AI might help with the draft, provided the information is appropriate and uncertain points remain visible.
Define what a satisfactory answer contains. It might need the correct customer context, an answer supported by current policy and an explicit indication when a source is missing. Include realistic exceptions in the test set, not just short, complete enquiries.
Keep the initial authority narrow. A drafting pilot may read approved material and prepare text while having no permission to send messages or change a customer record. The reviewer needs relevant knowledge and time to check the output properly.
Check the data for that particular task
| Proposed use | Main data question | A sensible first boundary |
|---|---|---|
| Drafting from approved material | Is the material current and suitable to use? | Review before anything is sent |
| Answering policy questions | Who owns the source and its version? | Show sources and escalate missing evidence |
| Forecasting demand | Is the history consistent and representative? | Compare against an agreed baseline |
| Updating operational records | Are identity, permissions and validation reliable? | Require explicit approval and a recovery route |
These are design considerations, not automatic risk ratings. Even a simple summary can expose information to the wrong person or remove an important qualification.
For each source, name an owner and confirm the permitted purpose. Test completeness, dates, duplicate records and conflicting definitions. Fix the gaps that would make this use unreliable, while keeping wider data improvements in a separate, prioritised backlog.
Measure the complete workflow
Observe the work before introducing the tool. Include time spent finding information, checking details and correcting errors. A faster first draft has limited value if the reviewer spends longer repairing it.
Choose a primary measure that matters to the process owner, such as turnaround time or handling effort. Keep a quality measure alongside it. Use comparable work for the baseline and pilot, including difficult cases that occur in ordinary use.
Record errors and the effort required to correct them. Check whether the system identifies missing information or produces an answer that appears complete despite weak evidence. Include requests outside the agreed scope.
Agree access, review and fallback before use
Confirm which environment is suitable for the information involved. The OAIC recommends against entering personal information, particularly sensitive information, into publicly available generative AI tools. Assess the actual tool, purpose and handling arrangements before using real records.
Name the person who can pause the pilot. Decide how staff return to the normal process and how an incorrect change would be contained or reversed. Human review needs to be an effective operating step, not a reassuring label.
Decide what would justify continuing
Set a review point that allows enough representative work to be evaluated. Compare time, quality, review effort, support demands and operating cost. Ask whether the team could sustain the improvement in ordinary use.
The outcome may be a wider rollout, a narrower task or a decision to stop. The value of the pilot is evidence for that decision. A broad data programme or an impressive demonstration is not a substitute for showing that one useful task works better.