RE: [R] parsing speed

From: Liaw, Andy <andy_liaw_at_merck.com>
Date: Tue 17 May 2005 - 23:43:17 EST


> From: Barry Rowlingson
>
> Jan T. Kim wrote:
>
> > Generally, I fully agree -- modular coding is good, not only in R.
> > However, with regard to execution time, modularisation that involves
> > passing of large amounts of data (100 x 1000 data frames etc.) can
> > cause problems.
>
> I've just tried a few simple examples of throwing biggish
> (3000x3000)
> matrices around and haven't encountered any pathological
> behaviour yet.
> I tried modifying the matrices within the functions, tried
> looping a few
> thousand times to estimate the matrix passing overhead, and in most
> cases the modular version run pretty much as fast as - or
> occasionally
> faster than - the inline version. There was some variability
> in CPU time
> taken, probably due to garbage collection.
>
> Does anyone have a simple example where passing large data
> sets causes
> a huge increase in CPU time? I think R is pretty smart with its
> parameter passing these days - anyone who thinks its still like Splus
> version 2.3 should update their brains to the 21st Century.

I think one example of this is using the formula interface to fit models on large data sets, especially those with tons of variables. Some model fitting functions have the default interface f(x, y, ...), along with a formula method f(formula, data, ...). If x has lots of variables (say over 1000), using the formula interface can take several times longer than calling the "raw" interface directly.

Andy  

> Baz
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
>
>
>



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Tue May 17 23:54:54 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:31:48 EST