I have a data set of 85 targets, each of which has been judged on 31 items by either 2 or 3 judges. About half of the targets have been judged by a random set of 2 of the 3 judges, and about half have been judged by all 3. I would ideally like to eventually average the ratings from as many judges as there are for each target, but I'm running into a question of which intraclass correlation coefficient to use. How can I use SPSS to compute inter-rater reliability with a variable number of judges?
Thank you for your help!