Re: [R] Logistic regression model + precision/recall

From: Frank E Harrell Jr <f.harrell_at_vanderbilt.edu>
Date: Wed 24 Jan 2007 - 14:13:34 GMT

nitin jindal wrote:
> Hi,
>
> I am using logistic regression model named lrm(Design)
>
> Rite now I was using Area Under Curve (AUC) for testing my model. But, now I
> have to calculate precision/recall of the model on test cases.
> For lrm, precision and recal would be simply defined with the help of 2
> terms below:
> True Positive (TP) - Number of test cases where class 1 is given probability

>> = 0.5.

> False Negative (FP) - Number of test cases where class 0 is given
> probability >= 0.5.

Why 0.5?

>
> Precision = TP / (TP + FP)
> Recall = TP / ( Number of Positive Samples in test data)

Those are improper scoring rules that can be tricked. If the outcome is rare (say 0.02 incidence) you could just predict that no one will have the outcome and be correct 0.98 of the time. I suggest validating the model for discrimination (e.g., AUC) and calibration.

Frank

>
> Any help is appreciated.
>
> I an write a long code with for loops and all, but is there any inbuild
> function or just few commands that would do the task.
>
> regards,
> Nitin
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Thu Jan 25 16:37:32 2007

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Thu 25 Jan 2007 - 07:30:30 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.