How to validate a diagnosis recorded in electronic health records

Table 1

Test measures

	Reference standard		Outcome	Test measures
	Positive	Negative	Outcome	Test measures
Diagnostic algorithm
Positive	True positives correctly identified (A)	False positives (B)	Total identified as positive	PPV=A/(A+B)
Negative	False negatives (C)	True negatives correctly identified (D)	Total identified as negative	NPV=D/(C+D)
Outcome	True positives	True negatives
Test measures	Sensitivity=A/(A+C)	Specificity=D/(B+D)

Table 2

Test measures that it is possible to calculate or estimate by each validation technique

Technique	PPV	NPV	Sensitivity	Specificity
Manual validation of physical records	Can be calculated	Can be calculated	Can be calculated	Can be calculated
Questionnaires for healthcare practitioners or patients	Can be calculated	Can be calculated
Validation of machine-learning algorithms	Can be calculated	Can be calculated	Can be calculated	Can be calculated
Comparison to an external database (complete overlap)	Can be calculated	Can be calculated	Can be calculated	Can be calculated
Comparison of rates in a comparable population			Estimate only	Estimate only
Internal validation using additional parameters	Estimate only	Estimate only
Internal validation using free text in the database	Estimate only	Estimate only
Sensitivity analyses using restrictive algorithms