Re: [R] RCurl and postForm()

From: Duncan Temple Lang <duncan_at_wald.ucdavis.edu>
Date: Fri, 29 Apr 2011 15:11:12 -0700

Hi Ryan

 postForm() is using a different style (or specifically Content-Type) of submitting the form than the curl -d command. Switching the style = 'POST' uses the same type, but at a quick guess, the parameter name 'a' is causing confusion and the result is the empty JSON array - "[]".

A quick workaround is to use curlPerform() directly rather than postForm()

 r = dynCurlReader()
 curlPerform(postfields = 'Archbishop Huxley', url = 'http://www.datasciencetoolkit.org/text2people', verbose = TRUE,

              post = 1L, writefunction = r$update)  r$value()

This yields

[1]
"[{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":0,\"end_index\":17,\"matched_string\":\"Archbishop Huxley\"}]"

and you can use fromJSON() to transform it into data in R.

  D.

On 4/29/11 12:14 PM, Elmore, Ryan wrote:
> Hi everybody,
>
> I think that I am missing something fundamental in how strings are passed from a postForm() call in R to the curl or libcurl functions underneath. For example, I can do the following using curl from the command line:
>
> $ curl -d "Archbishop Huxley" "http://www.datasciencetoolkit.org/text2people"
> [{"gender":"u","first_name":"","title":"archbishop","surnames":"Huxley","start_index":0,"end_index":17,"matched_string":"Archbishop Huxley"}]
>
> Trying the same thing, or what I *think* is the same thing (obvious not) in R (Mac OS 10.6.7, R 2.13.0) produces:
>

>> library(RCurl)

> Loading required package: bitops
>> api <- "http://www.datasciencetoolkit.org/text2people"
>> postForm(api, a="Archbishop Huxley")

> [1] "[{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":44,\"end_index\":61,\"matched_string\":\"Archbishop Huxley\"},{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":88,\"end_index\":105,\"matched_string\":\"Archbishop Huxley\"}]"
> attr(,"Content-Type")
> charset
> "text/html" "utf-8"
>
> I can match the result given on the DSTK API's website by using system(), but doesn't seem like the R-like way of doing something.
>
>> system("curl -d 'Archbishop Huxley' 'http://www.datasciencetoolkit.org/text2people'")

> 158 141 141 141 0[{"gender":"u","first_name":"","title":"archbishop","surnames":"Huxley","start_index":0,"end_index":17,"matched_string":"Archbishop Huxley"}] 17 599 72 --:--:-- --:--:-- --:--:-- 670
>
> If you want to see some additional information related to this question, I posted on StackOverflow a few days ago:
> http://stackoverflow.com/questions/5797688/post-request-using-rcurl
>
> I am working on this R wrapper for the data science toolkit as a way of illustrating how to make an R package for the Denver RUG and ran into this problem. Any help to this problem will be greatly appreciated by the Denver RUG!
>
> Cheers,
> Ryan
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Fri 29 Apr 2011 - 22:18:13 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 29 Apr 2011 - 23:30:36 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive