Machine forecasting tools

1.  SENSE-EU:  This one goes on top, because it was the only one to get #BREXIT right.

Showcases online conversation summarisation technology developed within the SENSEI European Project. Every day, more than 300 000 posts across multilingual media sources on the topic of the UK EU Referendum are captured and automatically analysed by SENSEI technology. A team of human and machine data curators generate this analysis and teach the algorithms to continue improving. You can find out more about the multilingual media sources we monitor and about our algorithms.

This has been commercialized as: https://websays.com/en/about/ by Hugo Zaragoza: https://unorganizedmachines.wordpress.com/hugo-zaragoza/ and https://www.linkedin.com/in/hugozaragoza.

2. Bing.  Bing?

brexit – Bing

  • Bing Predicts: Vote on the UK’s EU referendum
    55.44% Remain, 44.56% Leave

Bing uses web and social data to make predictions. See how

Last updated on 13 June 2016

3. Kensho, where the claim is that “A lot of people who make over $350,000 are about to get replaced by software.”

4. Instadat, where the claim is that

Our evolving deep neural net and variable engineering feedback loop capitalizes on Artificial Intelligence planning systems to generate automated forecasts with unprecedented adaptability and accuracy.

5. Geofeedia, where the claim is that

Our patented, cloud-based, location-based intelligence platform lets you predict, analyze, and act on real-time social media content by location from anywhere in the world—with a single click.

6. These physics guys analyzing the New online ecology of adversarial aggregates: ISIS and beyond.

7. Inline research assistant Helix.

8. Graph databases as shown in Panama Papers exploit. “How the ICIJ Used Neo4j to Unravel the Panama Papers”.

 

Social network intelligence software which blurs the line between analyst and operative

The CIA has traditionally separated intelligence analysts and operatives (collectors) into two separate workforces with something of a wall between them.  A big wall, two separate silos joined at the top of the house.  This may effectively be changing somewhat as the CIA breaks itself into smaller groups.  Obviously there is a difference between having 1000 analysts in one room and 1000 operatives in another, and 20 halls divided in the middle with each hall have 50 analysts on one side and 50 operatives on the other.  It will inevitably be the cases that there is more communication across both sides of a smaller hall than between two very large big halls.

That being said, the original division was there because operators are not traditionally known to be good at analysis, and analysts have been known to make terrible operatives.

Now we have a lot of concerns about cyberspace, and a couple of software offerings to work in the social cyber-subspace, which blur the line between analyst and operative by allowing analysts to inject themselves via fake personas into the social space that they are analysing.  The assumption here is that the line-blurring is somewhat more risk-free when you are working behind a computer terminal than when you are jumping out of a plane.  It is not always clear that the risk is less.

So, with those caveats (don’t try this at home?), here are some nifty off-the-shelf systems you can buy to spy on and pseudonymously interact people in social networks:

  • HIWIRE™ Intelligence Desk™: “A comprehensive array of tools needed throughout the web intelligence production lifecycle, from social media mapping and analysis, to cultivation and operation of virtual avatars, all the way through data collection, processing and report generation.”
  • Geofeedia: “Our patented, cloud-based, location-based intelligence platform lets you predict, analyze, and act on real-time social media content by location from anywhere in the world—with a single click.”

I haven’t got a price list for these, but I’m pretty sure they’re not on sale at Target.

What voting schemes are more effective at eliciting correct forecasts from small teams?

Consider the following voting schemes:

  • Plurality
  • Cumulative voting
  • Approval voting
  • Plurality with elimination
  • Borda Count
  • Successive elimination

Now suppose

  • We have a team of 10 people
  • We have a question with 3 outcomes A, B or C, and the correct outcome is B
  • Team members can express likelihood of outcomes by order for example A>B>C.
  • The 10 team members express their preferences as follows:
    • 2 for B>A> C
    • 2 for B>C>A
    • 1 for A>B>C
    • 2 for A>C>B
    • 2 for C>B>A
    • 1 for C>A>B
  • We randomly select a leader among those 10 people (this is important for certain voting schemes)
  • The team has to vote on a single outcome, A, B or C

Which voting schemes give the best outcome on average, where we perform repeated experiments to randomly select the team leader after performing a single assignment of forecasts to team members (it doesn’t matter what order, say 1-3 have A, 4-8 have B and 9-10 have C)?

This is a numerical thought experiment which I will do in Python.  The hard part is coding the 6 voting rules and then the simulation is easy.  TBA!

Crowd forecasting sites

I’m thinking I’m winding down my interest in this topic, but a few notes for old time’s sake.  There are some variables:

  • Format:
    • Market format: People bet real or fake money and are ranked by their account balance
    • Survey format: People make forecasts, and are ranked by their Brier Score
  • Compensation:
    • Reputational: Leaderboards, badges, titles
    • Prize-based: The market compensates the forecaster with case prizes
    • Pay for time, not accuracy: With cash, or coupons and similar barter compensation
    • Market-based: Zero-sum game where one forecaster’s loss is another’s gain.  Within this, whether it is
      • Fake money
      • Fiat currency (a/k/a real money)
      • Cryptocurrency (Bitcoin, Litecoin and similar)
  • Socialness
    • Unsocial.  Forecasters don’t communicate
    • Team-oriented.  Forecasters communicate only with their team
    • Social.  Forecasters communicate with all other forecasters
  • Openness
    • Result open/closed. Forecasters can see the aggregate forecast
    • Competitive open/closed/medium.  Forecasters can see some or all of the leaderboards and scores of other forecasters
    • Analysis open/closed/medium.  Forecasters can see some or all of other forecaster’s forecasts
  • Agenda
    • Agenda-driven.  Site exists to promote discussion framed around a view
    • Analysis-driven.  Site exists to predict the outcome of a well-defined scenario accurately
    • Market research.  Site exists to gather forecaster’s opinions regarding a future event without actually expecting an accurate forecast of the outcome of that event.
  • Domain.  What kind of events does the site focus on.
  • Commercial/non-profit, and for commercial ventures, early stage or profitable, and software vendor to research or analysis vendor to clients.
  • Public or private site.  Open to the public or instantiated for particular clients.

With these distinctions in mind, here are some sites I’ve come in contact with and their categorization, in no particular order:

Inkling Markets.  Market format, fake money market-based and reputational, social, open results, medium competitive, analysis closed, analysis driven, geopolitical and financial domains.   Commercial software vendor, early stage, acquired.  Public site.

Early Warning Project Expert Opinion Pool. Actually I think the platform for this is Inkling Markets under the hood.  I’m not in it so I haven’t looked.

GJOpen.  Survey format, reputational compensation, results open, medium competitive, analysis open, mixed agenda, geopolitical, financial, sports and commerce domains.  Public and private sites.  Commercial early stage. Platform from CultivateLabs, formerly Inkling Markets.

AlphaCast.  A blend of InklingMarkets and GJOpen, mixed by CultivateLabs.  Just starting up.  Market format, reputational compensation, results open, medium competitive, analysis open, mixed agenda, geopolitical, financial, sports and commerce domains.  Public and private sites.  Commercial early stage.

PredictIt.  Market format, US dollar, social, results open, competitive closed, analysis closed, analysis driven, geopolitical domain, non-profit, public site.

Survey Monkey.   Survey format, no compensation, unsocial, results closed, market research, any domain.  Why do I include it?  Because depending on what you are pushing a forecasting site to do, it does the same thing.  Anybody can use it to make a custom survey.  Public and private site.  Commercial profitable.

SwagBucks and similar.  There are a bunch of these.  Get paid to take a survey.  Survey format, pay for time compensation, unsocial, results closed, market research. Public site.  Commercial profitable.

Foresight Engine.  Survey format, reputational compensation, results open, medium competitive, analysis open, agenda-driven, geopolitical, financial, science and commerce domains.  Public and private sites.  Nonprofit.

Unu.  Survey format, reputational compensation, results open, medium competitive, analysis open, mixed agenda, geopolitical, financial, sports and commerce domains.  Public site.  Commercial early stage.

Huunu.  No relation to Unu.  Market format, prize-based, social, analysis-driven, medium competitive, results open, analysis closed, client-defined domains.  Private sites.  Commercial early stage software vendor.

Hypermind and Almanis.  Market format, prize-based, social, analysis-driven, medium competitive, results open, analysis closed, geopolitical and financial domains.  Public and private sites.  Commercial early stage software vendor.

Ladbrokes and similar.  Bookie sites.  Market format, market based, fiat currency, analysis-driven, unsocial, results open, analysis closed, sports, geopolitical and financial domains.  Public site, commercial profitable.

Predictious and similar.  Bitcoin prediction markets.  Market format, market based, Bitcoin, analysis-driven, unsocial, results open, analysis closed, sports, geopolitical and financial domains.  Public site, commercial profitable.

Zocalo.  Market format open-source prediction market software.  On you how you apply it!