[Rd] Allocated Memory Warnings in Vista 32 bit with 4 GB (PR#12557)

From: <drmark_at_marktab.com>
Date: Sun, 17 Aug 2008 19:15:17 +0200 (CEST)


Hello

I am running Windows Vista 32 with 4 GB (installed, though Windows of cours= e only recognizes 3326 MB, as reported by Windows "My Computer")

I am running R 2.7.1

I was trying to read in a comma delimited single column CSV file, assign th=
at file to a variable ("data") and then extract a sample (assigned to "part=
ial").  I was getting memory allocation errors, and from the log below, I s=
tarted to see a pattern which indicates that the "error" is related to how =
R views memory allocation, and perhaps how R is using or reading memory on =
my specific system.  The error, I would guess, is probably related to how R=
 is reporting the memory available.

My source files have the same number of rows (or cases) as the name of the = file. Thus, data10000 has 10,000 observations, and data300000 has 300,000 = observations.

Here is my history which shows the problem, and inconsistency in memory all= ocation reporting:

> data=3Dread.csv("data10000.csv",header=3DFALSE)
> partial=3Dsample(data,5000,T)
> data=3Dread.csv("data100000.csv",header=3DFALSE)
  Error in file(file, "r") : cannot open the connection   In addition: Warning message:
  In file(file, "r") :
    cannot open file 'data100000.csv': No such file or directory
> data=3Dread.csv("data300000.csv",header=3DFALSE)
> partial=3Dsample(data,5000,T)

  Error: cannot allocate vector of size 2.3 Mb   In addition: Warning messages:

  1. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  2. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  3. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  4. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
    > partial=3Dsample(data,5,T)
    Error: cannot allocate vector of size 2.3 Mb In addition: Warning messages:
  5. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  6. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  7. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  8. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
    > partial=3Dsample(data,1,T)
    Warning messages:
  9. In attributes(.Data) <- c(attributes(.Data), attrib) : Reached total allocation of 1535Mb: see help(memory.size)
  10. In attributes(.Data) <- c(attributes(.Data), attrib) : Reached total allocation of 1535Mb: see help(memory.size)
    > partial=3Dsample(data,1,T)
    > partial=3Dsample(data,5000,T)
    Error: cannot allocate vector of size 2.3 Mb In addition: Warning messages:
  11. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  12. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  13. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  14. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
    > partial[1:5,]
    [1] 0.3204279 1.6583593 -0.3456585 1.2951363 -1.1096974
    > rm(partial)
    > partial=3Dsample(data,5000,T)
    Error: cannot allocate vector of size 2.3 Mb In addition: Warning messages:
  15. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  16. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  17. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  18. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  19. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  20. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
    > partial[1:5,]
    Error: object "partial" not found
    > partial=3Dsample(data,1,T)
    Error: cannot allocate vector of size 2.3 Mb In addition: Warning messages:
  21. In attributes(.Data) <- c(attributes(.Data), attrib) : Reached total allocation of 1535Mb: see help(memory.size)
  22. In attributes(.Data) <- c(attributes(.Data), attrib) : Reached total allocation of 1535Mb: see help(memory.size)
  23. In attributes(.Data) <- c(attributes(.Data), attrib) : Reached total allocation of 1535Mb: see help(memory.size)
  24. In attributes(.Data) <- c(attributes(.Data), attrib) : Reached total allocation of 1535Mb: see help(memory.size)
    > partial=3Dsample(data,1,T)
    Warning messages:
  25. In attributes(.Data) <- c(attributes(.Data), attrib) : Reached total allocation of 1535Mb: see help(memory.size)
  26. In attributes(.Data) <- c(attributes(.Data), attrib) : Reached total allocation of 1535Mb: see help(memory.size)
    > partial=3Dsample(data,1,T)
    > partial=3Dsample(data,5000,T)
    Error: cannot allocate vector of size 2.3 Mb In addition: Warning messages:
  27. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  28. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  29. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size)
  30. In names(y) <- make.unique(cols) : Reached total allocation of 1535Mb: see help(memory.size) >=20
I have been able to reproduce the situation several times.  For example, wh=
at is curious is that a command like partial=3Dsample(data,1,T) may or may =
not produce a warning message.  In the history, obviously 2.3MB is well bel=
ow the allocated 1535MB.  I had been getting this error on my original larg=
er source data (1,000,000 observations), but since I am a new R user, it wa=
s not clear to me what these errors mean.
If you need the original source files, or have additional questions, please=
 let me know.  I am a certified .NET software developer, and although I do =
not know everything about Windows development, I believe my abilities are a=
bove average.

Mark Tabladillo=20
Atlanta, GA
marktab_at_email.phoenix.edu
Alternate: x2008_at_marktab.com

        [[alternative HTML version deleted]]



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Mon 18 Aug 2008 - 09:55:59 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Mon 18 Aug 2008 - 10:37:21 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive