Re: [Rd] Implementing a single-precision class with raw (now integer)

From: Colin A. Smith <>
Date: Thu 25 Aug 2005 - 01:25:39 GMT

On Aug 22, 2005, at 3:38 , Prof Brian Ripley wrote:

>> Unfortunately, R sometimes hits the 2GB memory limit of Win32.
> The rw-FAQ explains why that is _not_ the limit!

Sorry to be uninformed, I see from the Microsoft documentation that the limit can be increased to 3GB when you change the boot parameters of Windows and a flag in the R binary.

> Why via "raw"?
> I believe the intention is that this sort of thing be done via
> external references, but as float and int are the same size on all
> current platforms, I would have considered R integers for storage.
> Then for example subsetting would work and you had a 4x larger size
> limit on 64-bit platforms. (You would also have got automatic handling
> of endianness.)

I was concerned that on some platforms (particularly 64 bit ones) an integer may actually be 64 bits. However, if all current platforms have standardized on float and int being the same size, then using the integer data type would be more elegant and solve any endian issues. (Is that really the case?) Another motivation for using raw was so other R functions (particularly internal ones) wouldn't assume the data was integer and produce erroneous results. I'd rather have user-visible error thrown.

To have the best of both worlds, I'm now using an S4 class with a single slot (f) that stores the float data as an integer vector. It uses the extensible conversion mechanism created by Duncan Lang to allow the .C() function to handle float objects. (The custom converter also allows proper checking for single-precision NA values, which don't get caught if passed as integers.)

At some point, I would like to release this for others to use. Before I get too far into it, I would greatly appreciate it if anyone had any ideas, unforeseen caveats, or other suggestions about my current direction. My implementation is available here:

>> I've run into a limitation with the .C() function where it does not
>> handle raw vectors, which it will do in 2.2.0.
> That is just not true!

I should have been a bit more explicit and said that R 2.1.x doesn't have RAWSXP handlers in either RObjToCPtr() or CPtrToRObj(), so all the C code gets is a SEXP. R-devel does and passes raw vectors as unsigned char *. Of course that doesn't matter now that I'm using an extensible converter.


-Colin mailing list Received on Thu Aug 25 11:30:33 2005

This archive was generated by hypermail 2.1.8 : Mon 20 Feb 2006 - 03:21:18 GMT