[R] Fw: Permutations with replacement

From: Jesse Albert Canchola <jesse.canchola.b_at_bayer.com>
Date: Tue 22 Aug 2006 - 03:51:39 EST


My apologies, I forgot to CC: to the list on my previous communication with Daniel.

Jesse

Jesse Albert Canchola/EMVL/DIAG/US/BAYER 08/21/2006 09:36 AM

To
"Daniel Nordlund" <res90sx5@verizon.net> cc

Subject
RE: [R] Permutations with replacement

Thanks, Daniel. I need to enumerate all possibilities of 8^8 and take a random sample of 10,000 from there. Then I will use the sampled possibilities to do a combination of data frames/files then do some math on those files and develop the probability distribution from resulting sampling statistics (I couldn't get the available bootstrap packages to do what I want). BTW, the preferred solution (however inelegant) is reprinted below. I did have a memory problem with the hypercube for 8^8 so I did an 8^7 hypercube that worked and concatenated a 1-8 to the 8^7 matrix that resulted in 8 large matrices which I attempted rbind together to create the 8^8 but ran into more memory problems (on matrix number 7 of 8- it's a Windows problem - I used the --max-mem-size=2G - to no avail). The final solution was to take a random sample of 10,000/8=1250 from each of the files (doing four of the 8 files at a time, and for which I permuted the rows to make it more random), removed the heavy-laden files then rbind 'ed the smaller sampled files together to make the 10,000.

Here is the final final code:

# IDEA: We cannot simply do a permutation of the 8 classes/file id's
since this will not allow/simulate repeats of numbers
# as in a bootstrap (e.g., for 3 items - 1,2,3 - we would also want
the possibility 1,1,1 or 2,2,2 or 3,3,3 etc.
# The 8!=40,320 permutations with replacement would become an
8^8=16,777,216 so we would want to take a random
# sample of 10,000 from the posibilities
library(combinat)
# THIS WILL NOT WORK DUE TO THE LIMITATIONS OF WINDOWS MEMORY (PROBLEMATIC
AS IN THE FAQs) SO WE WILL
# USE A WORKAROUND
# WORKAROUND: 1) Construct the 8 to the 7 power hypercube. 2) For these
data, create eight additional data sets that include
# the last position to finish the construction of an 8^8
hypercube
#x <- rep(8,8) # for partitions of 8 units into classes {1,2,3,4,5,6,7,8}
#hcube8 <- hcube(x, scale=1, transl=0)
#hcube8

#step 1: 8^7 = 2,097,152

x1 <- rep(8,7)
x1
hcube87 <- hcube(x1, scale=1, transl=0)
#this will generate 2,097,152 results from 1-8 but for only 7 positions

#step 2: column bind each file with 1-8 in the 8th position
#x1a <- cbind(hcube87,1)
#x2a <- cbind(hcube87,2)
#x3a <- cbind(hcube87,3)
#x4a <- cbind(hcube87,4)
#x5a <- cbind(hcube87,5)
#x6a <- cbind(hcube87,6)
#x7a <- cbind(hcube87,7)
#x8a <- cbind(hcube87,8)

#turns out this method also chokes with the memory limitations
# Step 2 will be modified as follows:
# Step 2a: as before; Step2b: Sample 1/8 from each piece and after every 4
processes, delete the objects to allow for the rest.

#Step 2a

x1a <- cbind(hcube87,1)
x1a <- x1a[sample(1:2097152),] #randomly permute the rows for more randomness
x2a <- cbind(hcube87,2)
x2a <- x2a[sample(1:2097152),] #randomly permute the rows for more randomness
x3a <- cbind(hcube87,3)
x3a <- x3a[sample(1:2097152),] #randomly permute the rows for more randomness
x4a <- cbind(hcube87,4)
x4a <- x4a[sample(1:2097152),] #randomly permute the rows for more randomness

x1b <- x1a[sample(1:1250,replace=FALSE),]
x2b <- x2a[sample(1:1250,replace=FALSE),]
x3b <- x3a[sample(1:1250,replace=FALSE),]
x4b <- x4a[sample(1:1250,replace=FALSE),]

rm(x1a,x2a,x3a,x4a) #remove the big files

x5a <- cbind(hcube87,5)
x5a <- x5a[sample(1:2097152),] #randomly permute the rows for more randomness
x6a <- cbind(hcube87,6)
x6a <- x6a[sample(1:2097152),] #randomly permute the rows for more randomness
x7a <- cbind(hcube87,7)
x7a <- x7a[sample(1:2097152),] #randomly permute the rows for more randomness
x8a <- cbind(hcube87,8)
x8a <- x8a[sample(1:2097152),] #randomly permute the rows for more randomness

x5b <- x5a[sample(1:1250,replace=FALSE),]
x6b <- x6a[sample(1:1250,replace=FALSE),]
x7b <- x7a[sample(1:1250,replace=FALSE),]
x8b <- x8a[sample(1:1250,replace=FALSE),]

rm(x5a,x6a,x7a,x8a) #remove the big files

#Step 3: combine all the randomly sampled files
m <- rbind(x1b,x2b,x3b,x4b,x5b,x6b,x7b,x8b)

# NOTE: each number in the matrix represents a file "name" from 1-8.
# the first pointer should be numeric and then subsequent as character
# since the first time you assign a number to a character in a matrix
# the rest of the numbers in the matrix are coerced to character
m[m==1]='a'; m[m=='2']='b'; m[m=='3']='c' ; m[m=='4']='d'; m[m=='5']='e' ; m[m=='6']='f'; m[m=='7']='g' ; m[m=='8']='h' m
########### end R code ############

Thanks, David. That worked fabulously!

Here is the R code for the hypercube test example:

########## begin R code ############
library(combinat)
x <- rep(3,3) # for partitions of 3 units into the three classes {1,2,3}

hcube(x, scale=1, transl=0)
########### end R code ############

For the larger one I want (i.e., 8^8), I will take a random sample of 10,000 from the 16,777,216 possibilities.

Regards,
Jesse Canchola

<davidr@rhotrading.com>
Sent by: r-help-bounces@stat.math.ethz.ch 08/18/2006 01:33 PM

To
"Jesse Albert Canchola" <jesse.canchola.b@bayer.com>, "r-help" <r-help@stat.math.ethz.ch>
cc

Subject
Re: [R] Permutations with replacement

If you also want 1,1,1 and so on, the number of these is n^n, (n choices for each of n slots.)
In that case, you could use hcube from combinat.

David L. Reiner
Rho Trading Securities, LLC
Chicago IL 60605

"Daniel Nordlund" <res90sx5@verizon.net> 08/18/2006 05:16 PM

To
"'Jesse Albert Canchola'" <jesse.canchola.b@bayer.com>, "'r-help'" <r-help@stat.math.ethz.ch>
cc

Subject
RE: [R] Permutations with replacement

> -----Original Message-----
> From: r-help-bounces@stat.math.ethz.ch [mailto:r-help-bounces@stat.math.ethz.ch]

> On Behalf Of Jesse Albert Canchola
> Sent: Friday, August 18, 2006 1:02 PM
> To: r-help
> Subject: [R] Permutations with replacement
> 
> Is there a simple function or process that will create permutations with
> replacement?
> 
> I know that using the combinat package
> 
> ###### begin R code ######
> > library(combinat)
> > m <- t(array(unlist(permn(3)), dim = c(3, 6)))
> 
> # we can get the permutations, for example 3!=6
> # gives us
> 
> > m
>      [,1] [,2] [,3]
> [1,]    1    2    3
> [2,]    1    3    2
> [3,]    3    1    2
> [4,]    3    2    1
> [5,]    2    3    1
> [6,]    2    1    3
> ###### end R code ##########
> 
> I'd like to include the "with replacement possibilities" such as
> 
> 1,1,3
> 1,1,2
> 2,3,3
> 

Isn't what you want just sampling with replacement?

  x <- c(1,2,3)
  sample(x,3,replace=TRUE)

Hope this is helpful,

Dan

Dan Nordlund
Bothell, WA USA


The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you.

For alternate languages please go to http://bayerdisclaimer.bayerweb.com



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Tue Aug 22 03:59:07 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Tue 22 Aug 2006 - 04:23:28 EST.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.