From: Vadim Kutsyy <vadim_at_kutsyy.com>

Date: Fri, 01 Aug 2008 10:22:43 -0700

R-devel_at_r-project.org mailing list

https://stat.ethz.ch/mailman/listinfo/r-devel Received on Fri 01 Aug 2008 - 17:32:15 GMT

Date: Fri, 01 Aug 2008 10:22:43 -0700

Martin Maechler wrote:

*> [[Topic diverted from R-help]]
**>
*

> Well, fortunately, reasonable compilers have indeed kept

*> 'long' == 'long int' to mean 32-bit integers
**> ((less reasonable compiler writers have not, AFAIK: which leads
**> of course to code that no longer compiles correctly when
**> originally it did))
**> But of course you are right that 64-bit integers
**> (typically == 'long long', and really == 'int64') are very
**> natural on 64-bit architectures.
**> But see below.
**>
*

well in 64bit Ubunty, /usr/include/limits.h defines:

/* Minimum and maximum values a `signed long int' can hold. */

# if __WORDSIZE == 64 # define LONG_MAX 9223372036854775807L # else # define LONG_MAX 2147483647L # endif # define LONG_MIN (-LONG_MAX - 1L)

and using simple code to test

(http://home.att.net/~jackklein/c/inttypes.html#int) my desktop, which
is standard Intel computer, does show.

Signed long min: -9223372036854775808 max: 9223372036854775807

> If you have too large a numeric matrix, it would be larger than

*> 2^31 * 8 bytes ~= 2^34 / 2^20 ~= 16'000 Megabytes.
**> If that is is 10% only for you, you'd have around 160 GB of
**> RAM. That's quite a impressive.
**>
*

> cat /proc/meminfo | grep MemTotal

MemTotal: 145169248 kB

We have "smaller" SGI NUMAflex to play with, where the memory can increased to 512Gb ("larger" version doesn't have this "limitation"). But with even commodity hardware you can easily get 128Gb for reasonable price (i.e. Dell PowerEdge R900)

> Note that R objects are (pointers to) C structs that are

*> "well-defined" platform independently, and I'd say that this
**> should remain so.
**>
**>
*

I forgot that R stores two dimensional array in a single dimensional C
array. Now I understand why there is a limitation on total number of
elements. But this is a big limitations.

> One of the last times this topic came up (within R-core),

*> we found that for all the matrix/vector operations,
**> we really would need versions of BLAS / LAPACK that would also
**> work with these "big" matrices, ie. such a BLAS/Lapack would
**> also have to internally use "longer int" for indexing.
**> At that point in time, we had decied we would at least wait to
**> hear about the development of such BLAS/LAPACK libraries
*

BLAS supports two dimensional metrics definition, so if we would store
matrix as two dimensional object, we would be fine. But than all R code
as well as all packages would have to be modified.

R-devel_at_r-project.org mailing list

https://stat.ethz.ch/mailman/listinfo/r-devel Received on Fri 01 Aug 2008 - 17:32:15 GMT

Archive maintained by Robert King, hosted by
the discipline of
statistics at the
University of Newcastle,
Australia.

Archive generated by hypermail 2.2.0, at Fri 01 Aug 2008 - 22:35:57 GMT.

*
Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel.
Please read the posting
guide before posting to the list.
*