To do: Measuring category correlation of accuracy scores

Suppose we know, for each question, for each branch, for each forecaster, the forecaster’s marginal accuracy score on that branch.  Also, each question, across all branches, has multiple category assignments.  Suppose we have 10 scored questions, each with categories overlapping other questions.

How do you measure the correlation between accuracy scores and categories?

Advertisements

4 thoughts on “To do: Measuring category correlation of accuracy scores

  1. You’ve posed it as accuracy scores and categories for a given forecaster – vector could be of categories, perhaps use mean accuracy for all questions assigned a particular category. Though, seems to me it would be better to use each question and not deal with categories at all. See if correlation actually exists for questions across a category, perhaps? I’m not certain that it will.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s