Retrospective Brier and Accuracy scores

For closed questions, for my forecasts, knowing the final outcome, plot my running Brier and Accuracy scores from start to finish of the question.

I’m trying to come up with a measure of foresight.  A low initial Brier line would say that I guessed right in the beginning.  Basically how well do I guess up front.  I’m also thinking about weighting the Brier score so that later observations  less important than initial observations in the quality measure. My initial lines tend to be on top floating above the crowd.

As a matter of practice going forward I’m going to stay closer to the crowd and express my personal view as a spread on the crowd, I’m not going to go 0/1 which was my initial instinct.  However there may be some players in the population who are successful with a 0/1 strategy.

So:

avg_prediction_qid_2_uid_102_branch_a-majorityavg_prediction_qid_2_uid_102_branch_a-pluralityavg_prediction_qid_2_uid_102_branch_not-a-pluralityavg_prediction_qid_10_uid_102_branch_yesavg_prediction_qid_11_uid_102_branch_noavg_prediction_qid_11_uid_102_branch_yes-and-the-resolution-will-become-lawavg_prediction_qid_11_uid_102_branch_yes-but-the-resolution-will-be-vetoed-by-the-president-and-the-veto-will-standavg_prediction_qid_16_uid_102_branch_yesavg_prediction_qid_20_uid_102_branch_daniel-scioliavg_prediction_qid_20_uid_102_branch_mauricio-macriavg_prediction_qid_20_uid_102_branch_sergio-massaavg_prediction_qid_20_uid_102_branch_someone-elseavg_prediction_qid_26_uid_102_branch_a-team-from-the-american-leagueavg_prediction_qid_26_uid_102_branch_a-team-from-the-national-leagueavg_prediction_qid_27_uid_102_branch_yesavg_prediction_qid_28_uid_102_branch_yesavg_prediction_qid_37_uid_102_branch_yesavg_prediction_qid_41_uid_102_branch_conservativeavg_prediction_qid_41_uid_102_branch_liberalavg_prediction_qid_41_uid_102_branch_new-democratic-partyavg_prediction_qid_41_uid_102_branch_otheravg_prediction_qid_42_uid_102_branch_yesavg_prediction_qid_43_uid_102_branch_adam-bainavg_prediction_qid_43_uid_102_branch_anthony-notoavg_prediction_qid_43_uid_102_branch_jack-dorseyavg_prediction_qid_43_uid_102_branch_no-one-will-be-named-ceo-before-the-end-of-the-yearavg_prediction_qid_43_uid_102_branch_someone-elseavg_prediction_qid_82_uid_102_branch_yes

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s