[R] entropy and conditional entropy for continous variables

From: Rajarshi Guha <rxg218_at_psu.edu>
Date: Thu 12 May 2005 - 06:27:13 EST

  this is not a R question per se, but since I'm on the lookout for an R solution I thought this was the best place:

I would like to calculate the entropy for a variable and the conditional entropy between two variables, H(X|Y) for variables X & Y

I have coded the case for the categorical case but I'm having problems understanding how to do it for the continous case.

>From what I understand, for continous variables the entropy would be

\integral_{A} f(x) log( f(x) ) dx

where f(x) is the density of X and A is the support.

I have tried obtaining the density of X by using the density estimation methods of the KernSmooth package and then doing a numerical integration - however I am confused about how to interpret the support in this context.

So the question is, are there any R functions to determing the entropy and conditional entropy for continous variables? (Searching CRAN did not turn up anything significant)


Rajarshi Guha <rxg218_at_psu.edu> <http://jijo.cjb.net> GPG Fingerprint: 0CCA 8EE2 2EEB 25E2 AB04 06F7 1BB9 E634 9B87 56EE

All theoretical chemistry is really physics; and all theoretical chemists
know it.
-- Richard P. Feynman

R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Thu May 12 06:34:49 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:31:41 EST