The ROC curve is a visualization tool for classification. ROC curves visualize true positive and false positive rates which also can be taken out of a confusion matrix. The steeper the curve (towards the upper left corner) the better the classification. Other performance measures are , specificity and predictive accuracy.
Example data from jrocfit.org (J. Eng) visualized in
EXCEL (T. Kind)
I personally don't like ROC curves, but don't let this strong opinion get between you and the ROC
curve.
(This statement is subject to change).
Summary statistics:
Number of Cases: 50
Number Correct: 42
Accuracy: 84.0%
Sensitivity:
88.0%
Specificity: 80.0%
Pos Cases Missed: 3
Neg Cases Missed: 5
(A rating of 3 or greater is considered positive.)
Fitted ROC Area: 0.905
Empiric ROC Area: 0.892
JROCFIT:
Maximum likelihood estimation of a binormal ROC curve from categorical rating data.
Java translation by John Eng, M.D.
The Russell H. Morgan Department of Radiology and Radiological Science
Johns Hopkins University,
Baltimore, Maryland, USA
Version 1.0.2, March 2004
Original Fortran program ROCFIT by Charles Metz and colleagues
Department of Radiology, University
of Chicago
January 1994
------------------------------------------------------------
DATA CHARACTERISTICS:
- Data collected in 5 categories with category 5 representing strongest evidence of positivity (e.g., that abnormality is present).
- Number of actually negative cases = 25
- Number of actually positive cases = 25
RESPONSE DATA:
| Category | 1 | 2 | 3 | 4 | 5 |
| Actually negative cases | 11 | 9 | 3 | 1 | 1 |
| Actually positive cases | 1 | 2 | 3 | 9 | 10 |
OBSERVED OPERATING POINTS:
| FPF: | 0.0000 | 0.0400 | 0.0800 | 0.2000 | 0.5600 | 1.0000 |
| TPF: | 0.0000 | 0.4000 | 0.7600 | 0.8800 | 0.9600 | 1.0000 |
INITIAL VALUES OF PARAMETERS:
A = 1.7621
B = 0.9538
Z(K): -0.1507 0.8415 1.4053 1.7511
LOGL = -68.6817
FINAL VALUES OF PARAMETERS:
Procedure converges after 6 iterations.
A = 1.8683
B = 1.0118
Z(K): -0.1202 0.7566 1.2524 2.0602
LOGL = -65.0240
VARIANCE-COVARIANCE MATRIX:
| A | 0.2517 | 0.1303 | 0.0546 | 0.0592 | 0.0391 | -0.0306 |
| B | 0.1303 | 0.1384 | 0.0205 | -0.0005 | -0.0375 | -0.1332 |
| Z(1) | 0.0546 | 0.0205 | 0.0626 | 0.0363 | 0.0274 | 0.0127 |
| Z(2) | 0.0592 | -0.0005 | 0.0363 | 0.0694 | 0.0608 | 0.0580 |
| Z(3) | 0.0391 | -0.0375 | 0.0274 | 0.0608 | 0.0951 | 0.1126 |
| Z(4) | -0.0306 | -0.1332 | 0.0127 | 0.0580 | 0.1126 | 0.2485 |
CORRELATION MATRIX:
| A | 1.0000 | 0.6983 | 0.4350 | 0.4480 | 0.2526 | -0.1225 |
| B | 0.6983 | 1.0000 | 0.2207 | -0.0049 | -0.3270 | -0.7182 |
| Z(1) | 0.4350 | 0.2207 | 1.0000 | 0.5507 | 0.3551 | 0.1018 |
| Z(2) | 0.4480 | -0.0049 | 0.5507 | 1.0000 | 0.7492 | 0.4414 |
| Z(3) | 0.2526 | -0.3270 | 0.3551 | 0.7492 | 1.0000 | 0.7325 |
| Z(4) | -0.1225 | -0.7182 | 0.1018 | 0.4414 | 0.7325 | 1.0000 |
SUMMARY OF ROC CURVE:
Area = 0.9055
Std. Dev. (Area) = 0.0425
ESTIMATED BINORMAL ROC CURVE WITH ASYMMETRIC 95% CONFIDENCE INTERVAL:
| FPF | TPF | 95% Conf. Interv. |
| 0.005 | 0.2301 | (0.0169, 0.7407) |
| 0.010 | 0.3135 | (0.0430, 0.7718) |
| 0.020 | 0.4168 | (0.0996, 0.8061) |
| 0.030 | 0.4860 | (0.1545, 0.8282) |
| 0.040 | 0.5384 | (0.2056, 0.8449) |
| 0.050 | 0.5807 | (0.2523, 0.8587) |
| 0.060 | 0.6159 | (0.2949, 0.8705) |
| 0.070 | 0.6461 | (0.3337, 0.8808) |
| 0.080 | 0.6723 | (0.3690, 0.8901) |
| 0.090 | 0.6955 | (0.4012, 0.8985) |
| 0.100 | 0.7161 | (0.4306, 0.9062) |
| 0.110 | 0.7347 | (0.4575, 0.9132) |
| 0.120 | 0.7515 | (0.4821, 0.9198) |
| 0.130 | 0.7668 | (0.5047, 0.9258) |
| 0.140 | 0.7809 | (0.5255, 0.9314) |
| 0.150 | 0.7938 | (0.5447, 0.9366) |
| 0.200 | 0.8454 | (0.6214, 0.9577) |
| 0.250 | 0.8822 | (0.6757, 0.9723) |
| 0.300 | 0.9096 | (0.7160, 0.9824) |
| 0.400 | 0.9466 | (0.7727, 0.9934) |
| 0.500 | 0.9691 | (0.8119, 0.9978) |
| 0.600 | 0.9832 | (0.8424, 0.9994) |
| 0.700 | 0.9918 | (0.8684, 0.9999) |
| 0.800 | 0.9967 | (0.8927, 1.0000) |
| 0.900 | 0.9992 | (0.9189, 1.0000) |
| 0.950 | 0.9998 | (0.9357, 1.0000) |
ESTIMATES OF EXPECTED OPERATING POINTS ON FITTED ROC CURVE:
| Expected Operating Point | 95% C.I. of Lower Bound | 95% C.I. of Upper Bound |
| (FPF, TPF) | (FPF, TPF) | (FPF, TPF) |
| (0.0197, 0.4144) | (0.0012, 0.1141) | (0.1394, 0.7800 |
| (0.1052, 0.7261) | (0.0317, 0.4959) | (0.2585, 0.8874 |
| (0.2246, 0.8649) | (0.1015, 0.7192) | (0.4050, 0.9479 |
| (0.5478, 0.9767) | (0.3557, 0.9324) | (0.7292, 0.9935 |
WARNINGS AND ERROR MESSAGES:
Chi-square goodness of fit not calculated because
some expected cell frequencies are less than 5.
Chi-square goodness of fit not calculated because
some expected cell frequencies are less than 5.
Links:
- ROC Analysis - Web-based Calculator for ROC Curves
- The magnificent ROC - an interactive tutorial by Lara Hopley and Jo van Schalkwyk.
- ROCCET - a webservice from Jianguo Xia at the Wishart group in Canada.