Re: [Rd] save.image compression_level argument

From: Andreas Eckner <andreas_at_eckner.com>
Date: Wed, 13 Jul 2011 12:35:31 -0400

> On Mon, 11 Jul 2011, andreas@eckner.com wrote:
>
>> Hi,
>>
>> in "save.image", it would be nice if there was a "compression_level"
>> argument that is passed along to "save".
>>
>> Or is there a reason for disabling the "compression_level" option for
>> saving workspaces, but enabling it for manually saving individual
>> objects?
>
> Why not just call save() yourself? save.image() is a rarely used
> (directly) convenience wrapper.

I believe a decent number of R users indeed use save.image() due to its convenience.

> And why do you want to change the compression level? There is very little
advantage in using it for 'gzip' compression, and really the only significant use is reduce it to save time for xz compression.
>
> If you are volunteering others to add a feature, it behooves you to
explain why it would be useful to you and might be to others. Nothing is 'disabled': this is a new feature request ....

For large workspaces on the order of several hundred MB (I'm working on astronomical datasets), 'gzip' with the default compression_level=6 can take several minutes. Using compression_level=1 is about three times faster, while the file size only increases by ~10% (results may of course vary across application and system, but the numbers are probably representative). When saving work in progress (as opposed to sharing it externally), the time it takes to save workspace is probably the prime concern for most users.

I have attached a proposed modified definition of save.image() at the end of the message.

Cheers,
Andreas

>>
>> Thanks,
>> Andreas
>>
>> ______________________________________________
>> R-devel_at_r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> --
> Brian D. Ripley, ripley_at_stats.ox.ac.uk
> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
> University of Oxford, Tel: +44 1865 272861 (self)
> 1 South Parks Road, +44 1865 272866 (PA)
> Oxford OX1 3TG, UK Fax: +44 1865 272595

save.image <- function (file = ".RData", version = NULL, ascii = FALSE,

                        compress = !ascii, safe = TRUE, compression_level)
{

    if (! is.character(file) || file == "")

        stop("'file' must be non-empty string")

    opts <- getOption("save.image.defaults")     if(is.null(opts)) opts <- getOption("save.defaults")

    if (missing(safe) && ! is.null(opts$safe))

        safe <- opts$safe
    if (missing(ascii) && ! is.null(opts$ascii))

        ascii <- opts$ascii
    if (missing(compress) && ! is.null(opts$compress))

        compress <- opts$compress
    if (missing(version)) version <- opts$version

    if (safe) {

        ## find a temporary file name in the same directory so we can
        ## rename it to the final output file on success
        outfile <- paste(file, "Tmp", sep = "")
        i <- 0
        while (file.exists(outfile)) {
            i <- i + 1
            outfile <- paste(file, "Tmp", i, sep = "")
        }

    }
    else outfile <- file

    on.exit(file.remove(outfile))
    if (! missing(compression_level))

        save(list = ls(envir = .GlobalEnv, all.names = TRUE), file = outfile,

            version = version, ascii = ascii, compress = compress,
            envir = .GlobalEnv, precheck = FALSE, compression_level =
compression_level)

    else

        save(list = ls(envir = .GlobalEnv, all.names = TRUE), file = outfile,

            version = version, ascii = ascii, compress = compress,
            envir = .GlobalEnv, precheck = FALSE)
    if (safe)
        if (! file.rename(outfile, file)) {
            on.exit()
            stop("image could not be renamed and is left in ", outfile)
        }

    on.exit()
}

R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Thu 14 Jul 2011 - 11:43:06 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 14 Jul 2011 - 15:30:09 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive