Method Overview
Precision-recall curves are a standard way to analyze the quality of a classifier, as are ROC curves. The difference between ROC and Precision-recall (PR) is that PR is mostly concerned with what is happening at the top of the ranking, while ROC looks at the whole ranking. The precision-recall method in ErmineJ is similar in intent to the method popularized in “GSEA“, but uses standard precision-recall (GSEA originally used Kolmogorov-Smirnov statistics, but was changed to use a modified K-S statistic that makes it more precision-recall-like).
Like the ROC method, the PRC method uses only the ranks of the gene scores. That is, all it cares about is the ordering of items obtained by your gene scores (e.g., t-test or fold-change), but doesn’t use the information about the relative values of the scores.
When to use PRC scoring
PR-scoring is rather like the ORA method in that it is concerned with what is going on at the “top” of your list, but unlike ORA it does not use a threhold. Thus PR-scoring would be appropriate whenever you consider the ORA method appropriate. The PR method requires a resampling step, so it is slower to run than ORA.
Walkthrough
The precision-recall method workflow is the same as the Gene set resampling method. The difference is in how the scores are computed. Whereas for GSR the score is the mean (or other summary statistic) of the gene scores in the set, PR uses the average precision, which is closely related to the area under the precision-recall curve.