Re: [Rd] CRAN mirror size mushrooming; consider archiving some?

From: Uwe Ligges <ligges_at_statistik.tu-dortmund.de>
Date: Tue, 26 Jul 2011 20:07:38 +0200

On 25.07.2011 19:47, Paul Johnson wrote:
> Hi, everybody
>
> I'm setting up a new CRAN mirror and filled up the disk space the
> server allotted me. I asked for more, then filled that up. Now the
> system administrators want me to buy an $800 fiber channel card and a
> storage device. I'm going to do that, but it does make want to
> suggest to you that this is a problem.

Why? Just for the mirror? That's nonsense. A 6 year old outdated desktop machine (say upgraded to 2GB RAM) with a 1T harddisc for 50$ should be fine for your first tries. The bottleneck will probably be your network connection rather than the storage.

> CRAN now is about 68GB, and about 3/4 of that is in the bin folder,
> where one finds copies of compiled packages for macosx and windows.
> If the administrators of CRAN would move the packages for R before,
> say, 2.12, to long term storage, then mirror management would be a bit
> more, well, manageable.
>
> Moving the R for windows packages for, say, R 2.0 through 2.10 would
> save some space, and possibly establish a useful precedent for the
> long term.

That is right, but then users of R < 2.11.0 could no longer use install.packages() and friends. If we want to move stuff around in future, we may want to implement that in R first. We thought about removing old binaries before, but then disk space increased roughly as exponentially as repository space in the past and we decided to stay with it as is.

> Here's the bin/windows folder. Note it is expanding exponentially (or nearly so)

And you see that quite a lot of efforts were made during the last release cycles to reduce the amount of used memory (e.g. using better compression).

Best wishes,
Uwe

> $ du --max-depth=1 | sort
> 1012644 ./2.6
> 103504 ./1.7
> 122200 ./1.8
> 1239876 ./2.7
> 1487024 ./2.8
> 15220 ./ATLAS
> 167668 ./1.9
> 17921604 .
> 1866196 ./2.9
> 204392 ./2.0
> 2207708 ./2.10
> 2340120 ./2.13
> 2356272 ./2.12
> 2403176 ./2.11
> 298620 ./2.1
> 364292 ./2.2
> 438044 ./2.3
> 595920 ./2.4
> 698064 ./2.5
>



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Tue 26 Jul 2011 - 18:15:11 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Tue 26 Jul 2011 - 22:20:14 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive