Responsible AI guide
How Accurate Is AI Detection?
False positives, text length, editing and model changes all affect reliability.
Accuracy is not one universal number
A detector’s reported performance depends on the test data, languages, model versions, text length and definition of success. A result obtained on long, unedited English essays may not apply to short answers, translated writing, technical material or text revised by a person.
Marketing claims should therefore be read with the test conditions. Ask whether the evaluation was independent, whether false positives were reported and whether the tested material resembles your use case.
False positives and false negatives have different costs
A false positive labels human writing as AI-like. In education or employment, that error can unfairly damage a person. A false negative misses AI-assisted writing. The acceptable balance depends on the decision, but consequential use should generally place strong weight on avoiding unsupported accusations.
Changing the threshold can reduce one error while increasing the other. A high score is not immune from this trade-off.
- Short passages provide fewer stable signals.
- Editing and paraphrasing can change the classification.
- Formulaic genres may resemble model-generated prose.
- Second-language writing may be represented unevenly in training data.
- New language models can reduce the relevance of older evaluations.
Evaluate accuracy for your own context
Build a small, ethically collected test set with known origins and representative writing. Include the languages, genres and lengths that matter to your organisation. Run the detector without revealing the expected answer to the reviewer, then examine both overall performance and the specific cases it gets wrong.
Repeat the test after major product updates. A one-time evaluation should not become a permanent policy.
Use uncertainty as information
The practical value of detection is not perfect classification. It is the ability to identify patterns that deserve closer examination while making uncertainty visible. A responsible output should help the reviewer ask better questions.
For any consequential decision, combine the result with drafts, revision history, source use and a fair conversation with the writer.
Terbit editorial analysis
Accuracy claims need a defined dataset, language mix, text length, decision threshold and error cost. A single percentage without those details cannot tell a school or employer whether the tool is suitable.
False positives deserve special weight where consequences are serious. A cautious system that declines short or ambiguous samples can be more responsible than one that always returns a score.
How this guide was reviewed
- Checked against current primary or official guidance listed below.
- Separated provider claims from Terbit Zero AI analysis and practical recommendations.
- Used a transparent scenario to show the reasoning process; examples are illustrative, not benchmark results.
Simple accuracy worksheet
Build a labelled set of 20 human-written and 20 AI-assisted passages that match the real language and document type.
- Blind the reviewer to the labels and record every output.
- Calculate false positives and false negatives separately.
- Repeat after material tool updates; do not generalise beyond the tested sample.
Comparison framework
| Measure | Formula | Interpretation |
|---|---|---|
| False-positive rate | Human passages wrongly flagged ÷ human passages | Risk of accusing human work |
| False-negative rate | AI-assisted passages missed ÷ AI-assisted passages | Risk of missing the target signal |
| Repeat consistency | Unchanged outputs ÷ repeated runs | Stability for identical input |
| Coverage | Valid results ÷ all submitted passages | Whether the tool handles the real document mix |
Practical checklist
- Check the evaluation conditions behind accuracy claims.
- Measure false positives separately.
- Test representative text and languages.
- Never treat a detector as the sole decision-maker.
Sources and further reading
- Turnitin — Using the AI Writing ReportChecked 16 October 2026
- UNESCO — Guidance for generative AI in education and researchChecked 16 October 2026
- NIST — AI Risk Management FrameworkChecked 16 October 2026