Re: [Rd] parse_Rd and/or lazyload problem

From: Duncan Murdoch <murdoch_at_stats.uwo.ca>
Date: Sat, 31 Oct 2009 10:27:32 -0400

On 30/10/2009 10:58 PM, Mark.Bravington_at_csiro.au wrote: > The errors are generated inside lazyLoadDBfetch; I don't call object.size, so unfortunately that's not the issue (unless o.s. is called somewhere inside the .Primitive for lazyLoadDBfetch).

Does this happen in R-patched? I've seen similar errors in 2.10.0, but not in a current build. (Still haven't got to your code, this was in mine. I'm reluctant to spend time on code that is messing with internals, because you might be using things in a way not intended by the author. Now, if you can show me some code that demonstrates the problem without using internals directly, I'll follow up.)

Duncan Murdoch

> 

>>> I'm encountering problems when making lazy-loadable databases of the output from 'parse_Rd'. The lazy-
>>> load database is of seemingly limitless size when I try to reload it... Admittedly I am using functions
>>> that I'm not really supposed to use, which is why this isn't a bug report, but there does seem to be
>>> something strange going on; my code is very similar to code that lives inside
>>> 'tools:::.install_package_Rd_objects'. The problems occur with just-released R2.10.0 on Windows.
>> object.size() has problems when working on Rd objects, because it counts
>> every environment separately, even though they may all be references to
>> the same one. I haven't looked at your code, but that could be a problem.
>>
>>
> 
> Mark
> 
> ________________________________________
> From: Duncan Murdoch [murdoch_at_stats.uwo.ca]
> Sent: 31 October 2009 08:59
> To: Bravington, Mark (CMIS, Hobart)
> Cc: r-devel_at_r-project.org
> Subject: Re: [Rd] parse_Rd and/or lazyload problem
> 
> On 30/10/2009 12:50 AM, Mark.Bravington_at_csiro.au wrote:

>> I'm encountering problems when making lazy-loadable databases of the output from 'parse_Rd'. The lazy-load database is of seemingly limitless size when I try to reload it... Admittedly I am using functions that I'm not really supposed to use, which is why this isn't a bug report, but there does seem to be something strange going on; my code is very similar to code that lives inside 'tools:::.install_package_Rd_objects'. The problems occur with just-released R2.10.0 on Windows.
> 
> object.size() has problems when working on Rd objects, because it counts
> every environment separately, even though they may all be references to
> the same one. I haven't looked at your code, but that could be a problem.
> 
> Duncan Murdoch
> 

>> The examples below use files which can be found at ftp://ftp.csiro.au/MarkBravington, but you'll obviously need to modify the paths. The file "scrunge.Rd" is just "Rdiff.Rd" from the 'tools' package. The file "fakepack.7z" should unzip to create a fake package with a DESCRIPTION file and a "man" directory that contains two Rd files.
>>
>> Transcript of first example:
>> eglist <- list( scrunge=tools:::prepare_Rd( 'd:/temp/scrunge.Rd',
>> defines=.Platform$OS.type, stages='install', warningCalls=FALSE))
>> tools:::makeLazyLoadDB( eglist, 'd:/temp/ll', compress=FALSE)
>> tools:::fetchRdDB( 'd:/temp/ll')
>> # Error: cannot allocate vector of size 1.4 Gb
>>
>> The second example triggers an error with 'tools:::.install_package_Rd_objects' itself. It doesn't happen the first time, but frequently does after the Rd files have been changed. Transcript:
>>
>> # Make sure d:/temp/help/ is empty, then
>> test> tools:::.install_package_Rd_objects( 'd:/temp/fakepack', 'd:/temp')
>> test> tools:::fetchRdDB( 'd:/temp/help/temp')
>> # All good. Next, I *removed* one of the two Rd files in "d:/temp/fakepack/man", ...
>> # ...deleted "d:/temp/help/temp*", and tried again
>> test> tools:::.install_package_Rd_objects( 'd:/temp/fakepack', 'd:/temp')
>> test> tools:::fetchRdDB( 'd:/temp/help/temp')
>> Warning: Reached total allocation of 1535Mb: see help(memory.size)
>> Warning: Reached total allocation of 1535Mb: see help(memory.size)
>> Warning: Reached total allocation of 1535Mb: see help(memory.size)
>> Warning: Reached total allocation of 1535Mb: see help(memory.size)
>> Error: cannot allocate vector of size 1.9 Gb
>> # Or on other occasions I get
>> Error: internal error -3 in R_decompress1
>>
>> Mark Bravington
>> CSIRO CMIS
>> Hobart
>> Australia
>>
>> --please do not edit the information below--
>> Version:
>> platform = i386-pc-mingw32
>> arch = i386
>> os = mingw32
>> system = i386, mingw32
>> status =
>> major = 2
>> minor = 10.0
>> year = 2009
>> month = 10
>> day = 26
>> svn rev = 50208
>> language = R
>> version.string = R version 2.10.0 (2009-10-26)
>> Windows XP (build 2600) Service Pack 2
>> Locale:
>> LC_COLLATE=English_Australia.1252;LC_CTYPE=English_Australia.1252;LC_MONETARY=English_Australia.1252;LC_NUMERIC=C;LC_TIME=English_Australia.1252
>> Search Path:
>> .GlobalEnv, ROOT, package:grDevices, package:ad, package:chstuff, package:handy2, package:tweedie, package:statmod, package:handy, package:debug, package:mvbutils, mvb.session.info, package:tools, package:tcltk, package:stats, package:graphics, package:utils, package:methods, Autoloads, package:base
>> ______________________________________________
>> R-devel_at_r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel


R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Sat 31 Oct 2009 - 15:49:45 GMT

This archive was generated by hypermail 2.2.0 : Sun 01 Nov 2009 - 04:10:18 GMT