Re: [Rd] problem gsub in the locale of CP932 and SJIS (PR#9751)

From: <ripley_at_stats.ox.ac.uk>
Date: Mon, 25 Jun 2007 11:24:45 +0200 (CEST)


Thanks for this.

I don't think the patch is quite right. As I understand it, mbstate_t should be initialized at the start of the string, not before each character, and that is what is done in the rest of R.

Also, do you have an example I can use to test the patch, please?

R 2.5.0 is now in code freeze and I don't think this is vital for that.

On Sun, 24 Jun 2007, nakama_at_ki.rim.or.jp wrote:

> Full_Name: Ei-ji Nakama
> Version: R-2.5.0
> OS: any
> Submission from: (NULL) (219.117.236.5)
>
>
> problem by operation of gsub in the locale of CP932 and SJIS.
> The inconvenient character code which used 0x5c after the first byte.
>
> --- R-2.5.0.orig/src/main/character.c 2007-04-03 11:05:05.000000000 +0900
> +++ R-2.5.0/src/main/character.c 2007-06-24 22:31:06.000000000 +0900
> @@ -986,6 +986,17 @@
> char *p = repl;
> n = strlen(repl) - (regmatch[0].rm_eo - regmatch[0].rm_so);
> while (*p) {
> +#ifdef SUPPORT_MBCS
> + if(mbcslocale){
> + int clen;
> + mbstate_t mb_st;
> + mbs_init(&mb_st);
> + if((clen = Mbrtowc(NULL, p, MB_CUR_MAX, &mb_st)) > 1){
> + p+=clen;
> + continue;
> + }
> + }
> +#endif
> if (*p == '\\') {
> if ('1' <= p[1] && p[1] <= '9') {
> k = p[1] - '0';
> @@ -1014,6 +1025,18 @@
> int i, k;
> char *p = repl, *t = target;
> while (*p) {
> +#ifdef SUPPORT_MBCS
> + if(mbcslocale){
> + int clen;
> + mbstate_t mb_st;
> + mbs_init(&mb_st);
> + if((clen = Mbrtowc(NULL, p, MB_CUR_MAX, &mb_st)) > 1){
> + for ( i=0; i<clen; i++)
> + *t++ = *p++;
> + continue;
> + }
> + }
> +#endif
> if (*p == '\\') {
> if ('1' <= p[1] && p[1] <= '9') {
> k = p[1] - '0';
>
> ______________________________________________
> R-devel_at_r-project.org mailing list
>
https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Brian D. Ripley,                  ripley_at_stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Received on Mon 25 Jun 2007 - 09:29:43 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Mon 25 Jun 2007 - 09:35:33 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.