Re: [R] Proportion test in three-chices experiment

From: Rafael Laboissiere <laboissiere_at_cbs.mpg.de>
Date: Mon 18 Jul 2005 - 04:12:08 EST

> I suspect that there are more direct ways to do this test, but it
> is unclear to me just what the issue is. For example, if there
> are many subjects and very few stimuli for each, you might want
> to get some sort of measure of ability for each subject (many
> possibilities here, then test the measure across subjects with a
> t test. The measure must be chosen so that you can specify a
> null hypothesis. It must be directional.
>
> If you have a few subjects and many trials per subject, then you
> could do a significance test for each subject.
> You want a directional test, because you have a specific
> hypothesis, namely, that the correct answer will occur more often
> than predicted from the marginal frequencies in the 3x3 table.
> (I assume it is a 3x3 table with stimuli as rows and responses ad
> columns, and you want to show that the diagonal cells are higher
> than predicted.) One possibility is kappa, which is in the vcd
> package, and also in psy and concord, in somewhat different
> forms.

Thanks for your reply, Jonathan. Thanks also to Spencer, who suggested using the BTm function. I realize that my description of both the experiment and the involved issue was not clear. Let me try again:

My subjects do a recognition task where I present stimuli belonging to three different classes (let us say A, B, and C). There are many of them. Subjects are asked to recognize each stimulus as belonging to one of the three classes (forced-choice design). This is done under two different conditions (say conditions 1 and 2). I end up with matrices of counts like this (in R notation):

# under condition 1
c1 <- t (matrix (c (c1AA, c1AB, c1AC,

                    c1BA, c1BB, c1BC, 
		    c1CA, c1CB, c1CC), nc = 3))

# under condition 2
c2 <- t (matrix (c (c2AA, c2AB, c2AC,
                    c2BA, c2BB, c2BC, 
		    c2CA, c2CB, c2CC), nc = 3))

where "cijk" is the number of times the subject gave answer k when presented with a stimulus of class j, under condition i.

The issue is to test whether subjects perform better (in the sense of a higher recognition score) in condition 1 compared with condition 2. My first idea was to test the global recognition rate, which could be computed as:

# under condition 1
r1 <- sum (diag (c1)) / sum (c1)
# under condition 2
r2 <- sum (diag (c2)) / sum (c2)

The null hypothesis is that r1 is not different from r2. I guess that I could test it with the chisq.test function, like this:

p1 <- sum (diag (c1))
q1 <- sum (c1) - p1
p2 <- sum (diag (c2))
q2 <- sum (c2) - p2

chisq.test (matrix (c(p1, q1, p2, q2), nc = 2))

What do you think?

I also thought about testing the triples like [c1AA, c1AB, c1AC] against [c2AA, c2AB, c2AC], hence my original question.

-- 
Rafael

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Received on Mon Jul 18 04:20:02 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:33:46 EST