Conditional probability: adapting forecasts on questions with resolved bins on GJOpen

GJOpen currently leaves resolved bins open on questions such as WTI Jan 1 Close with 1 out of 3 closed bins and Syrian Refugees in EU with 2 out of 4 closed bins.

To get the correct current forecast of a user who may have put credibility into closed bins, it is necessary to assume the bin is closed and apply a simple conditional probability calculation to adjust the forecasts on the other bins, so that the total probability of remaining bins adds up to 100.

Questions that started with a single bin should not be adjusted because the total probability of such questions can lie between 0 and 100.

This leads to the following Python function to adapt a forecast F={bin1:f1,…,binN:fN} for an N-bin question into an M<N-bin question in which the N-M bins C={binA,binB,…} have been closed:

import numpy as np
def adapt_forecast(F, C):
 if len(F)==1:
  return F
 else:
  open=[x for x in F if x not in C]
  bet = np.sum([F[x] for x in open])
  scale = 100.0/bet
  return {x:0 if x in C else scale*F[x] for x in F}

For example, here is a prediction from user doublehanded who made just one forecast on this question (which is typical, most users in a question will make just a single prediction):

  1. (de facto closed). 0% Less than 560,000
  2. (de facto closed). 55% Between 560,000 and 710,000, inclusive
  3. 45% More than 710,000 but less than 1 million
  4. 0% 1 million or more

We should then read this as follows:

  1. doublehanded gets marginal Brier scores of 2 for bin 1 and 0.605 for bin 2.
  2. The adapted forecast of doublehanded is now
    • 100% More than 710,000 but less than 1 million
    • 0% 1 million or more

which is the result of adapt_forecast({1:0,2:55,3:45,4:0}, [1,2])

Advertisements

One thought on “Conditional probability: adapting forecasts on questions with resolved bins on GJOpen

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s