Spring reading: The Sparrow (*spoilers*)


Mary Doria Russell’s The Sparrow, like Michel Faber’s The Book of Strange New Things, is a novel which attempts to give a realistic vision of first contact with a self-aware, intelligent alien species.  This is also a novel by an author new to writing science fiction. In Faber’s case, because he comes from what is called the “literary community” (no Wikipedia definition available), and in Russell’s case, this being a first novel by an anthropologist who had only written academic articles previously.

Russell’s aliens are, in a way, much less alien than Faber’s aliens.  Both have arms and legs, are bipedal, have a language, live in houses, and have some technology.  Faber’s aliens are otherwise as different from people as squid and hamsters.  Russell’s aliens are modelled after kangaroos and tigers.

In both cases, Russell and Faber downplay and lower the drama and strangeness of first contact.  They both use the colonial analogy, like Marco Polo first meeting the Chinese or Spaniards first encountering Aztecs.  In this analogy, the presence of others is not so astonishing and is not the focus of awareness.  Rather simply the inability to recognize social cues and differences in status hierarchy.  For example imagine Meriwether Lewis running into the Kim Jong Un after walking over a hill.  The social expectations and assumptions will be quite different, and one party may behave with a level of haughtiness and indifference, despite the novelty and strangeness of the encounter, that catches the other party quite unaware.

That, in essence, is the plot of The Sparrow.  Russell’s aliens are just not as alien as Faber’s because she makes a lot of assumptions that are Earth-normative:  The atmostphere and gravity of the alien planet are not discussed and one assumes identical to Earth’s.  There are two similar species.  One turns out to be domesticated herbivore prey.  The other turn out to be carnivore predators, who look similar as a result of evolutionarily adaptive aggressive mimicry.  The herbivores are like big cuddly kittens and have very dextrous hands and are very social and warm.  The carnivores have larger teeth and three-fingered, sharp claws, and are very hierarchical and cold.  They have different kinds of intelligence, but both species are intelligent and capable of change.

The novel adopts challenge to religious faith as a theme but somewhat tiresomely overplays it.   Both the humans being social at rest in the exploration group and the humands giving each other a hard time in the Jesuit context are somewhat heavily and stereotypically written.   The construction of the dual species, and the ecological imbalance accidentally introduced by the visiting human party are cleverly designed, as one would expect of a good scientist exploring a scenario in their domain.

I bought the sequel, Children of God, which will arrive in a few weeks.  The New York Times didn’t like it.  Russell plays out the scenario a little more with a return visit.  I’m looking forward to it!  My primary takeaways from this book:

  • First contact with aliens could play out just like first contact in the human context, for example when Christian missionaries came to Japan in the 1500’s.
  • We have to be very careful about unintended ecological impact of human ideas on alien society.  The predator/prey society depicted in Russell’s book had strict population controls and no risk of prey insurrection.   The human concept of gardening interfered with population controls and the human concept of strength in numbers and retaliation upset the political stability of the dual society.
  • The aliens might not like us, find us that interesting, and may look down on us, even if they are technologically inferior, so we have to be very careful about making assumptions about social hierarchies, status, and level of empathy.  (The essence of diplomacy, I suppose.
  • Maybe we can make asteroids into self-fueling space ships.  (Her one cool and relatively unexplored technical idea.)

Finally, note the novel is written in 1998 and she has a relatively uneven scorecard as a futurologist.  She got a few things right, like tablet computing, but mostly her timeline is way too ambitious, considering it’s 2017 as I write the following conditions were supposed to hold in 2016:

  • Students do not yet become indentured servants to pay for college scholarships.
  • Japan is not the dominant economic, military and political power.
  • Asteroids are not yet so thoroughly routinely mined that you can go to a broker for a used one equipped with engines that has just the right shape.
  • Jesuits don’t commission space explorations.

Collaboration at NCTC and in prediction markets and forecasting tournaments

I am reading Bridget Nolan’s 2013 UPenn thesis on workplace collaboration at the NCTC, and thinking about how that compares to workflow and collaboration in play-money prediction markets such as Almanis and Hypermind and AlphaCast, real-money markets such as PredictIt, and accuracy-score forecasting tournaments such as GJOpen.

The workflow at NCTC is described as follows:

  • Tasking. Analysts receive Taskings
  • Research. Analysts write, as quickly as possible, a, usually short, analysis related to the tasking. This could be either a forecast or an interpretation of a past event, but most likely a forecast.
  • Coordination. Analyst coordinates by sharing the analysis with all other analysts with a stake in the topic.  Analysts are indexed by region and functional area and home agency, so multiple analysts could have a stake in a topic.
    • Analysts must converge on a commonly acceptable text with coordinating analysts.
    • Analysts can game the coordination phase by
      • Limiting the review period to short or 0 (“Flash”) time periods
      • By inventing an exclusive “compartment” that coordinating analysts don’t have access to, and stealing the analysis by placing it in the compartment
  • Review. Once coordination converges, the analysis must then be approved and re-edited by all the hierarchical superiors of each participating analyst.  The claim is that there are 14 layers of management, which seems unlikely, but you never know. The Government pay grades go from GS-1 (the lowest) to GS-15, so maybe analysts come in at GS-1 and they have people at every pay grade.  GS-1 is $18,343 per year however, which implies that there are a lot of analysts subsisting well below the poverty level for the DC Metro Area, which also seems unlikely, but, again, you never know.
  • Publishing.  The analysis is delivered to the original client of the tasking.
  • Compensation.
    • Client Feedback. Some analysts are notified if the client likes or reads the product.
    • Performance Review.  Analysts are compensated by the number of pieces they are involved in that get published.
    • Work Time Away.  Time spent on foreign field visits, training and interagency meetings is not considered for performance and is effectively a form of compensation for writing analysis pieces.

For each published piece, this process can take from 10 minutes to several years, and it is a major source of stress and dissatisfaction for analysts.

Now let’s consider the case of prediction markets and forecasting tournaments.  First of all, there are two major areas of tradecraft which are out of control of the analyst, but which determine the quality of the overall process:

  • Question formation.  How to formulate a “tasking” which has an unambiguous answer, has a reasonable forward time period, covers all the possible outcomes, and is not overly specific to the extent that the actual answer to the question becomes disconnected from the tasking client’s original intent.
  • Question resolution.  How to decide when the conditions of the question have been met, and score the question accurately with unimpeachable sources, so that all analysts agree that the question has been closed and scored fairly and correctly.

Let’s assume that all the markets I mentioned up front are equally competent at question formation and resolution.  Then what distinguishes them are mainly

  • Scoring and compensation model.
    • GJOpen uses Brier Score.  Compensation is reputational: you can say you won a challenge.
    • Almanis and Hypermind use play money and the site pays cash to analysts.  This is called “creating a market for expertise”.
    • AlphaCast uses play money but also reports Brier Score.  It is a demo site for aficionados, some of whom have accumulated extremely large play money scores in a fairly small crowd.   GJOpen uses AlphaCast software underneath, so the pitch here is just to sell the software as an OEM to other vendors.
    • PredictIt uses real cash, supplied by the analysts, in a zero sum market.  Each question is a futures contract.  The site takes an 18% rake.
    • All sites have leaderboards for bins of related questions and overall cash or quality of forecasts.  Almanis has leaderboards for commenting and question posing (analysts can self-task).
  • Effect of compensation on forecasting style.  All sites penalize analysts for participating in questions they aren’t good at.  Accuracy sites however all the analyst to forecast in as many questions as they want.  Cash sites limit the analyst to forecasting in questions they have a remaining cash balance for.  Analysts can also lose all credibility by putting all of their cash on a single question with the wrong forecast.  It is impossible to lose all credibility in an accuracy-based site. Nevertheless, other analysts can see your general credibility by looking at your overall accuracy score.  (But of more relevance is looking at credibility by topic.) Play money compensation sites with thin participation can quickly become dominated by a few obsessive players accumulating very large balances.
  • Socialness.  The amount to which analysts on a site share information is more dependent on the collaboration features provided than on the scoring model. Reddit-like tree-formatted dialogues, notifications when others have responded, ability to notify/call out particular analysts, and ability to see other analysts scores, forecasts, comments and personal profiles, all have a strong impact on how much sharing happens.
  • Teaming.  In a big site with good socialness you will find that analysts form into cliques naturally and tend to have patterns of association over time.
  • Information sources.  Most analysts in public sites just use Google and are limited to what Google Search digs up.  Not much source analysis is done.  The quality of the market is dependent on the prior expertise of the participants.  Most participants simply regurgitate the most recent news as a forecast, thus acting as a Mechnical Turk news digesting machine.
  • Tools.  Public sites do not provide any kind of question domain-specific modelling tools or advanced tools for filtering and visualizing news or publicly available data.
  • Workflow.  Questions get published.  Analysts make predictions.  Questions get closed.

OK, why am I lining up NCTC workflow and forecasting workflow?  Well, the question is, what if you unplugged the Task/Coordinate/Edit/Publish model and replaced it with the Question Posing/Forecasting/Scoring model, would you get a better result?   This is the question being asked by IARPA ACE, CREATE and HFC competitions.  However it’s not clear how much of a gap there is between those competitions and current workflow at NCTC and similar places.  That is, I don’t know if there is any traction or application in the work IARPA is doing that has been translated into the actual analyst workplace.  A few observations are relevant though:

  • Seemingly large crowds in public sites boil down to a much smaller number of fanatics.  Say you have 20,000 registered users.  150 of those will forecast a lot of questions.  50 of those will be consistently accurate, and it’s not clear whether that 50 are accurate just based on survivor bias.
  • Prediction markets are sometimes terrible, especially on binary election questions. Think Scottish Referendum, Brexit, Trump.  These questions were all called wrong.
  • The particularities of intelligence analysis are not reproduced in other occupations. Forecasting markets as a work paradigm are not a toy for intelligence analysis, they are a real solution. They are a not a real solution for other occupations such as banking.
  • Analysts at NCTC are selected maybe 20% on accuracy and 80% on other factors such as
    • Commitment to training to be a professional analyst
    • Willingness and ability to pass a security clearance
    • Commitment to the occupation of being a professional analyst
  • The pre-existing workplace for analysts, with its particularities, will not go away.

The last point is most important: Analysts are a static, small population.  They can’t easily leave their jobs, and their jobs are relatively stable.  They are qualified by many other factors than accuracy.  They are a pre-existing population.  There has been much talk of “Superforecasters” a/k/a unicorns in the prediction market arena.  To adopt forecasting market methodology in intelligence analysis in a pre-existing workplace, we need to think in other terms rather than the search for these unicorns.  We have to ask: Will adopting this technology and implementing a different workflow and compensation model improve collaboration in this workplace with these people?  I think it will, at least in the sense that what Nolan describes is clearly not working well, and in the sense that the prediction market model provides a much more objective standard for scoring both the analysts and the quality of question posing and resolution.

Spring Amazon Pilots: Oasis versus it’s source novel The Book of Strange New Things (*spoilers*)


I watched the pilot for Oasis, which I liked, and immediately bought the book it’s based on, called The Book of Strange New Things.

Critics are mixed on the book.  It is literary fiction?  Is it science fiction?  Is it good fictiongood literary science fiction, bad science fiction by an arriviste?  I.e. reviewers get hung up on what category it is.  By analogy, I went to a Japanese noodle place in Charlotte that was fantastic.  “These are the best Japanese noodles I’ve ever had”, I thought.  Noodles with pecan smoked pork belly.   Spicy.  Fantastic.  Unique.  Delicious.


The catch: The chef is Italian, and learned to cook in fancy restaurants in Arizona.


So are these Japanese noodles or not?  It’s kind of like that. People get irritated when category rules are not abided by.  This same thing is happening with a new Scarlett Johansson movie, based on a Japanese manga.  Shouldn’t the heroine be Japanese?


Aren’t they appropriating?  Should straight actors be allowed to play gay men?  Should gay men be allowed to play straight men?  Should women be allowed to play men?  Should men be allowed to play women?  Like appropriating is a really nasty word.  Like we don’t appropriate all the time.  Boundaries must be respected!  So boring. Of course, I got it when Keanu Reeves pretended to be a samurai


(or was it Tom Cruise?),


but I lose patience when people get huffy about comic book adaptions.  Because face it, people, manga are comic books, written in a kind of boring, Pikachu, stereotypical style.


Get over it.

So anyway, this book, it’s like that.  Take a great science fiction genre (priest visits alien culture), and let a really good literary fiction author cook one up.  It’s good.  It’s not quite like anything a died-in-the-wool science fiction author would come up with.  It’s fresh.  So there.  I liked it.

What is really great about this book is that it does some things with space exploration and aliens which are really quite novel:

  • Space travel is depicted as painful, debilitating, messy, harsh and brief, like flying from Charlotte to Baltimore on American Airlines, only worse.  The spaceship is not pretty, dramatic or interesting, the controls are dull and simple, and it is not well decorated in any way.  It’s just dull.
  • The new world has some mystery elements but it is most of all like being stuck in a truck stop in some abandoned Southwestern town in the middle of the desert with only the most basic amenities.  It’s the Jim Jarmusch of new worlds.  There are some novel elements to the climate, the ground cover, the air, the water.  It’s different. But it’s not really exciting.  It’s a habitable place, far away, a lifeline from a dying Earth.  But it’s not great.
  • The aliens are by and large simple, pleasant and somewhat primitive villagers in some peaceful Amazonian culture untouched by modern technology.  They get by just fine.  They like aspirin, which they trade for food.  They are ugly and different, and also similar in some ways.  They can pick up our language.  We can’t pick up theirs.  They are well characterized, and different.  You rarely get to see such a depth of characterization in the science fiction genre, just as you rarely get to see the addition of really delicious pecan smoked pork belly to an otherwise standard Japanese dish that, in Japanese hands, would be rendered competently but unremarkably, and leave no particular memory.  We are used to Star Wars bar-level characterization of aliens, different on the surface.  This author goes deep on the differences, yet comes up with some believably while not jumping straight to scary. They are different, but in the way some species of squid is different from a hamster. Not like us, but not so different that there is no basis for communication.  And there is something about us that they like, and they appropriate it.  This literary description of alien appropriation of human culture is fascinating and unique. Some readers found it boring.  I do not.

The book and the TV series pilot diverge quite a lot.  Really a lot.  The TV series, in it’s hour long introduction, doesn’t really get around to the aliens until the end, and it makes them scary.  It makes a big deal out of disappearances which the book doesn’t really make a central plot point, even halfway through, where I am now.  The TV show centers a lot more on the characters at the human base.  The book centers on the alien colony, the existence of which is not mentioned at all in the pilot.

I hope the pilot gets chosen for development.  I kind of hope they follow the book.  If they don’t, they have their work cut out for them, and it will be quite a different entity from the book.  I’m not against this.  The TV Series “The Expanse” is quite different tactically from the book series it is based on.   Tactically.  But it still follows the major plotline running through the books.  It will be interesting to see whether the Amazon TV Series for Oasis will follow the same strategy, or whether it will go strategically as well as tatically different.  We already know it is tactically different.  In the TV show, the wife is dying, and is not a central focus.  In the book, the wife is not at all dying, while the world may be, and she is kind of a central, parallel narrative focus, although one of the themes of the book is how the priest detaches himself from the marriage as he gets more involved with the alien culture and the people on the human base.

Final note, in the book he invents an alphabet for the aliens or native Oasans that are central to the book (not clear how the TV show will go on this, it could throw them out completely, as depicted in the book, which may be an unhappy experience for the author but typical for Hollywood).  Writing the book he used Thai characters.  The publisher then had someone design a type font not related to any current language, so as not to offend.  Faber only actually gives us a phonic correspondence for the “s”, “t” and “ch” sounds, in cases where the Oasans are trying to pronounce English words and fail.  On occasion he makes up longer words in Oasan, unpronounced and in so doing introduces another 10 or so letters that we don’t have sounds for.  I had hoped there would be more clues and I was prepared to reproduce the full alphabet here along with English phoneticisation of the untranslated Oasan words, but there just aren’t enough examples and when they are trying to speak English, they only stumbled on “s”, “t” and “ch”.  So that’s disappointing. But apparently this was not the most fun part of writing the book for him, he didn’t really care about it that much, it was just a gimmick to convey a little more otherness.  He also gives some rules for the Oasan grammar, in particular:

  • A few thousand words
  • No cases
  • No distinction between singular and plural
  • No genders
  • Three tenses: Gone, here, and expected to come

Some things I wonder about while watching 9-year-olds play Little League Baseball

Do they steal bases in the Major Leagues?

9 year olds are not good at catching balls and tossing balls.  There is a lot of room for base stealing as the tiny catcher, laden with gear, scrambles to pick up and forward a ball that slipped through his grasp.  So: Do they steal bases in the Major Leagues?  The answer?  Yes they can, but not so much.  The players are just too good for there to be much opportunity.  They are a long way from being 9-year-olds, and their precision is just too great.

Does anybody try to bat at bad pitches?

The most common play in the game I watched was a walk.  This is the first year in their Little League progression that the kids do the pitching.  When they are younger, a machine pitches, which is dull, but the ability is just not there yet in a kid younger than 9.  Of course, with no training, it is barely there when they are 9, and they tend to throw a lot of pitches that are too high, too low, aimed at the batter, or way away from the batter.

The batter has some constraints too, I think.  He must stay in the batter’s box.  The pitch, not to count as a ball, must be in the strike zone.  So the question is, are there any well known baseball players who have trained to hit balls which are pitched outside of the strike zone, and do so with regularity?  I’ve just never heard of it, and now I need to know.  Just finding the right question to ask is hard: “Hitting a pitch outside of the strike zone”.  Luckily, asking that exact question gets you to the right place.  Here’s the answer:

  • Batters do it all the time, because pitchers are trained to make pitches which are just outside of the strike zone, but otherwise plausible.  It’s legal to swing at one, and you’ll get a strike if you miss.
  • Batters often connect with these bad pitches, but generally do not get good solid hits off of them.  The weak hits they get lower their overall performance.
  • Because bad pitches are hard to make into good hits, there are no professional players that spend time trying to learn how to make good hits out of bad pitches. Which is kind of sad, because it looks like a good arbitrage, because it’s the kind of thing that nobody would expect you to spend time on.

Some memories of Cambridge, MA when I was 10 or 11 in the early 1970s

A couple of times when I was 10 or 11 years old, I hitchhiked from Ipswich, MA to Cambridge, MA to hang around Harvard Square.  Things were different in 1971, and I was in charge of my oldest brother and sister, while my mother spent the summer in San Miguel de Allende, an artist’s colony in Mexico that was not too scary at that time, and is probably not too scary now, indeed quite upscale, not quite the Mexico depicted in TV series like Breaking Bad, which also exists.  Some things I remember:

Truc in Harvard Square.


It was a place to buy blue kaleidescope prism glasses.


Dannon’s frozen yogurt, soft serve in a cone or in the form of Danny’s yogurt bars.  Not the Oikos Greek stuff they sell now, it was more like TCBY’s Italian tart flavor, but disappeared from the market for some reason.  It was great.  Back then, it was really modern.

Elsie Burger behind Harvard Square made the best burger.  I’ve tried to reproduce the sauce but I can’t, a cross between Russian Dressing and mustard relish.

I saw the movie Joe at the Orson Wells Cinema near Central Square when I was 10, with my older sister.  This is really not a movie to take a 10-year-old to.  There are many things I shelter my children from now.


Steve’s Ice Cream with mix-ins was a big new thing around 1977. Mix-ins were modern.  They’d take a scope of high-fat, stretchy ice cream, put a dent in it, sprinkle in some chopped up Reese’s Pieces or M&Ms, and mash it together then stick it on a cone.  Super modern concept at the time.  Cold Stone Creamery stores do that now.  I think Steve’s, like Ben&Jerry’s which I think came after it, expanded into pints you could buy at a grocery store, but I don’t see those in North Carolina at all, whereas Ben&Jerry’s is pretty much national now.


Rock and roll concerts in Cambridge Commons.  These felt like happenings, at least to an 11 year old, much more eventful events than they would seem to me now.  Honestly, I think they really were, if you read the link, intended to be modern and challenging, much more by design than I realized at the time.


Time dilation and cellular automata representation of physical space


One of the plot points of the movie Interstellar  centers around the idea that travelling through a wormhole to get someplace distant and then returning would result in the same time dilation as if the astronauts travelled at near light speed in normal space to and from the same object.  That is, if you travelled through a worm hole and came back, what you returned to would be much older than what you left.

It would be fun to take a cellular automata model of physical space which was correct for relativity, and fold it over and pinch it to connect two distant points in the cellular space, and then see if the predicted time dilation effect was reproduced in the model.

I Googled “cellular automata relativity” and it seemed the only two papers written on this topic are by two Canadian electrical engineers, Tom Ostoma and Mike Trushyck, in 1999.   They never published again.  I read through these papers and they talk about cell propagation and the number of rules per cell, with the cells being of volume related to Planck constant and the universal clock time for the whole automata, operating outside of time within the automata, such that the speed of light, viewed as velocity of propagation of information between cells, was reproduced.

Subsequently I found the 1994 MIT PhD thesis by Mark Andrew Smith on Cellular Automata Methods in Mathematical Physics which plows similar ground. However, as with Ostoma and Trushyck, publishing on this topic appears to be a career-killing move, as I can find no subsequent trace of this author.   Smith bylined briefly in 2001 as a member of the Harvard Department of Molecular and Cellular Biology, and after that I can truly find no trace of him on the web.  So, a seven year gap between publications, and then nothing.  One fears the worst, especially as he mentions in his thesis that

My first years at MIT were ones of great academic isolation, and I would have dropped out long ago if it were not for the fellowship of friends that I got to know at the Thursday night coffee hour in Ashdown House.

Or are the missing cellularautomatophysicists here, one wonders:


The idea of physical space quantized into cellular automated seemed satisfying until I looked through the window.  My eyes could see a lot of visual information coming to me from a distance.  That’s not counting a lot of non-visual information such as radio waves and magnetism and so on.   Presumably everything I am seeing at a distance is propagated to me and summarized on a cell by cell level by a complex of information representing all of this information that is arriving to the cell from all of these sources at the same time.  It doesn’t seem like it can be summarized in a single bit.  So, how many bits are needed to represent the information being received by a single cell?  If one posits an “operating system” or collection of common rules that is replicated in every cell, what are the implications for the size of that rule set to manage and store the amount of information being received from each neighboring cell?

Dear readers, do you know of any other credible published work on cellular automata models for relativity which are concrete enough to be programmed as simulations?  What are your thoughts on the bits-per-cell problem and how we perceive, at a single point, light arriving from a distance from many sources?

Spring binge watch: Patriot


I watched this dreary, arty, self-conscious and beautifully photographed series on Amazon over the past month or two.  It is 8 hour-long episodes but seems longer.  I fell asleep several times.   Variety harshed it completely.  Critics label it a rip-off of the Coen Brothers shtick. I’m not sure I see that.  I see that it’s kind of there but I didn’t like the Coen Brothers output so much that it takes away from the originality of this show.  The show is original.  It’s just a one-joke story with some gratuitous violence thrown in on occasion, some largely undeveloped characters other than the central few, a Maltese Falcon MacGuffin that wanders through the plot, and a conclusion that begs sequel.

The show is about an all-American midwestern assassin with a Dad with Oliver North ambitions.  Father and son are psychopaths, Dad a cheerful one and son more sombre.  There is also a kid brother with a conscience.  The opening credits show a pair of rowdy boys doing various Americana-level boy things like jumping off swings, riding sheep, flinging rocks with slingshots, riding motorcycles, smoking and firing a rifle.  There is no backstory for the transition from adolescence to assassin, but some kind of military career is assumed.

I wanted to like this more and there were some surprising and nice unfoldings of characters, but by and large the artistic range was thin.  This film is blues and jazz, and I’d recommend it as a detour for viewers who like pretty jazzy things and don’t mind the lack of plot.