Re: [Rd] concurrent requests (Rook, but I think the question is more general)

From: Simon Urbanek <simon.urbanek_at_r-project.org>
Date: Wed, 24 Oct 2012 15:23:15 -0400

On Oct 24, 2012, at 3:09 PM, Richard D. Morey wrote:

> On 24/10/12 8:53 PM, Simon Urbanek wrote:
>> On Oct 24, 2012, at 2:13 PM, Richard D. Morey wrote:
>> 

>>> This question involves Rook, but I think the answer will be general enough that it pays to post here. At any rate, I don't know enough to know whether this is a Rook only issue or a general R issue.
>>>
>>> Here's what I'd like to do (and indeed, have code that should do this):
>>>
>>> 1. Start R, Rook
>>> 2. Start an analysis via a HTTP request to Rook. This analysis uses .Call() to some compiled C code, if that matters. The C code calls a callback function to update a variable with its progress.
>>> 3. While the analysis is happening, use Rook to obtain current status with an HTTP request
>>>
>> You can't. R doesn't support threading so it's simply not possible to have an asynchronous eval. The R HTTP server works by simply enqueuing an eval to run while R is idle, it can't do that if R is busy. (Note that the HTTP server was *only* designed for the internal help).
>> 
>> What you can do is have your C code start another thread that reports the progress when asked e.g. on a socket, but that thread is not allowed to call any R API so you want that progress to be entirely in your C code.
> How can I start a new thread? By running R again from the command line, or is there a better way?
> 

No, you have to use the system thread API like pthreads, NSThread etc. If you have to ask about this, you probably don't want to go there ;) - threads can be quite dangerous if you are not familiar with them.

Another poor man's solution is to simply have your C code write out a file with the progress. Along the same lines you could use a shared object to store the progress (e.g. via bigmemory) ...

>> Note that if your C code is robust enough, it can call R_CheckUserInterrupt() to allow external events to happen, but a) your C code must in that case be prepared for early termination (clean up memory etc.) and b) I don't remember if the httpd is allowed to run during interrupt check on all platforms - you may want to check that first.
> I do use R_CheckUserInterrupt(), but if I understand what you're saying then given that it doesn't work, httpd must not run during the interrupt check. At least, on OSX, which is what I'm testing on.
> 

Yes, if your code calls R_CheckUserInterrupt() and httpd doesn't respond at that point then it may not be allowed to run. On OS X you can try your luck with R_ProcessEvents() as well.

Cheers,
Simon

>> Cheers,
>> Simon
>> 
>> 
>> 
>> 

>>> The problem is that once the analysis starts, Rook does not respond to requests. All of the status requests to Rook pile up, and then are answered when the analysis (step 2) is done. Here is some example code to demonstrate what the issue:
>>>
>>> ##########
>>>
>>> library(Rook)
>>> s <- Rhttpd$new()
>>> s$add(
>>> name="pingpong",
>>> app=Rook::URLMap$new(
>>> '/ping' = function(env){
>>> req <- Rook::Request$new(env)
>>> res <- Rook::Response$new()
>>> res$write('This is ping.')
>>> Sys.sleep(20)
>>> res$finish()
>>> },
>>> '/pong' = function(env){
>>> req <- Rook::Request$new(env)
>>> res <- Rook::Response$new()
>>> res$write("This is pong.")
>>> res$finish()
>>> },
>>> '/?' = function(env){
>>> req <- Rook::Request$new(env)
>>> res <- Rook::Response$new()
>>> res$redirect(req$to_url('/pong'))
>>> res$finish()
>>> }
>>> )
>>> )
>>>
>>> s$start(quiet=TRUE)
>>> s$browse('pingpong')
>>>
>>> #############################
>>>
>>> If you request /ping, R runs Sys.sleep() for 20 seconds. This is where my .Call() statement would be. While the .Call() (Sys.sleep()) function is doing its thing, I need to get Rook to respond on /pong (which would simply respond with the progress), but if you run this code, request /ping, then immediately request /pong, you'll see that the /pong request will not be answered until the Sys.sleep() is done.
>>>
>>> Of course, for a progress report to be useful, the requests have to be answered immediately. Is this a Rook issue, or an R issue? Or am I asking something unreasonable?
>>>
>>> ______________________________________________
>>> R-devel_at_r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>>
>
>

R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Wed 24 Oct 2012 - 19:26:11 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Wed 24 Oct 2012 - 20:10:50 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive