[Rd] Implementing a single-precision class with raw

From: Colin A. Smith <colin_at_colinsmith.org>
Date: Sat 20 Aug 2005 - 06:38:42 GMT


A package that I develop (xcms) sometimes needs to read and process vectors several hundreds of megabytes in size. (They only represent parts of a large data sets which can approach nearly 100GB.) Unfortunately, R sometimes hits the 2GB memory limit of Win32. To help cut the memory footprint in half, I'm implementing a "float" class as a subclass of "raw". Because almost all the computation on the large vectors is done in C code, having a somewhat limited single-precision data type is okay.

I've run into a limitation with the .C() function where it does not handle raw vectors, which it will do in 2.2.0. In the meantime, I'm using the .Call() function to access the raw vectors. However, there don't seem to be any macros for handling raw vectors in Rdefines.h. I've made a guess at what those macros would be and was wondering whether my guesses were correct and/or might be included in 2.2.0:

#define NEW_RAW(n) allocVector(RAWSXP,n)
#define RAW_POINTER(x) (RAW(x))
#define AS_RAW(x) coerceVector(x,RAWSXP)

I'm not sure whether coerceVector(x,RAWSXP) will actually work. Also, there isn't an Rf_isRaw() function, which would be useful for an IS_RAW(x) macro.

Another issue with the "float" class is that it will run into endian issues if it ever gets saved to disk and moved cross-platform. I don't really anticipate that happening but it might be nice to incorporate serialization hooks if possible. Are there any facilities in R for doing that?

Thanks for any feedback or suggestions.

-Colin

http://abagyan.scripps.edu/~csmith/float.R http://abagyan.scripps.edu/~csmith/float.c 



R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Sat Aug 20 16:44:43 2005

This archive was generated by hypermail 2.1.8 : Mon 20 Feb 2006 - 03:21:16 GMT