3 min read
Use business records to test an AI task
Keep learning examples, test cases, and feedback separate.
Decide what you need to learn or check
Training teaches a model from examples. Evaluation checks an AI system against a task. Feedback records what a person accepted or changed.
Each use may need different fields and permissions. State the use before choosing records.
Write a task with a checkable result
Describe the starting request, relevant details, allowed actions, and a successful result. Include other acceptable answers where needed.
For a delivery task, the checks might require an approved route and a confirmed date. A plausible but invented date should fail.
Keep the answer out of the input
Keep the expected answer separate from what the AI receives. Showing the answer can make the test misleading.
Keep training and test records separate too. Copies of the same task should not appear in both.
Document how a reviewer decides
Write down the checks and what each result means. Explain how to handle missing details or disagreements.
An employee’s correction can help, but still needs review. Check the reason and whether other answers could work.
Record the limits of the result
Keep the tested system version, cases, checks, and limits with the results. Note any tasks that were left out.
A result on one test set does not prove performance on every task. Changes to the model or work may need new tests.
See what your data could earn.
Describe your records to Korra and get a starting range. Your files stay in your systems.
What’s my data worth?