Sunday, April 19, 2015

The Hand We've Been Dealt

(Yes, the title references both The Walking Dead and Miles Naismith Vorkosigan.)

One of the recurring themes from the on-going Hugo kerfuffle is a sense of unfairness.  Among the charges:

a) There was a pre-existing bias against conservatives & libertarians (1) among the small (2) subsection of SFF Fandom (3) who nominate and vote for the Hugos.  The most vocal people expressing this pov are called by those who oppose them “SJW” – social justice warriors.  (It is not meant as a compliment.)

b) That there was a countering bias against women, non-Caucasians, and non-heterosexuals in terms of characters, authors, and fans.(Sometimes this is expressed in terms of matching the general issues of American society, other times it is described as unique to SFF fandom/ SFF creators.) The most vocal people expressing this pov are called by those who oppose them “fascist racist sexists homophobes”. (4)(Also not a compliment.) (They have employed the term "racist" against a Caucasian man in a twenty-year old marriage with an African-American woman, to which I can only say damn, that's dedication to the Cause.)

c) There was an active on-going cabal of influential people who habitually manipulated some if not all of the nominations in order to steer the finalist lists to include selected works and people. (5)

d) That Sad Puppies/Rabid Puppies (SP/RP) (6) unfairly stacked the deck against all other parts of fandom in order to get a selected group of writers on the finalist list.

Noses are so far bent out of joint that it’s a wonder any of us can see straight.

Among many people who consider themselves defending the Hugos and SFF / SFF Fandom against the sorts of people and thoughts exemplified by the SP/RP, there has been an oft-repeated sense that The Hugos Were Fine Why Did You Have To Break Them?  As evidence for how The Hugos Were Fine, quotes like this one from Rcade are common:

 What makes me bitter is the strategy of bloc voting, because it made it impossible for nominations I made as an individual in good faith to appear on the ballot. Out of 80 slots on the ballot, my nominations appear 0 times. That’s never happened before. Normally I see around 2-6.

This is given as part of the justification for charges such as that put forth by such otherwise temperate and polite people as Connie Willis that the SP/RP were outright “cheating” and “ballot stuffing.” (7)

The purpose of this post is to demonstrate that such an assumption is inaccurate and – instead of being proof of cheating – is instead evidence in support of an insular common opinion amongst the historical voters for the Hugos.


As we’re talking about cheating, let’s talk cards.

Consider a deck of common playing cards. Take out the jokers and the extra cards with the name of the card manufacturer on them, and you are left with 52 cards.  Four suites – hearts, clubs, spades, diamonds (9) – of 13 cards – ace, 2-10, jack, queen, king. For the purposes of this example, face cards are ace, jack, queen, king. (And that makes four, oh best beloved.)

So.  We have 52 cards, and we want to know, what are the best two cards?

Differentiation depends on, well, differences. If there are no differences, then what appear to be varying levels of support are no more than random chance.

If all the cards are indistinguishable from each other in value of Bestness, and if we ask a large enough group (say, a bazillion gazillion) (8) of people, we would end up with 1326 different unique combinations of 2 cards from that group.  That number – 1326 – is calculated using a mathematical formula called the factorial – generally written like so: factorial of (n) = n!  The factorial of a number is equal to that number times all the whole numbers smaller than it.  Thus:

5! = 5*4*3*2*1 = 120
4! = 4*3*2*1 = 24
3! = 3*2*1 = 6

And so on. 

In our example we talk about “sets of two” – this is the smaller grouping drawn from a larger grouping.  The size of this set is k, so that k =2 if we mean, sets of two, or k = 3 if we mean, sets of three.  For every value of n and k, we can determine how many unique sets of size k are in that group numbering n, using the factorial formula.

The formula for determining the number of unique sets is:

K! (N-k)!

So for a group of size n, choosing smaller groups of equal size k, we can calculate how many unique groups of size k there are in a group of size n.

If we also want to know how many groups of size k we find that include any one item, we imagine we have a group of size (n-1), pick our groups of size k from them, and then subtract.  The remainder is the number of unique small groups that were made up of ONLY the items not included in the second , smaller group.

(Wikipedia also has an explanation of the math, in case I have confused anyone.)

In our set of 52 cards, there 1326 unique sets of two cards.  If everyone’s opinion of the “Bestness” of cards was equal, we would find that each of these 1326 sets would have an equal representation in our poll, and that there would not be any one pair – either the 2 of clubs and the 10 of spades, or the ace of diamonds and the queen of hearts, nor any other pair – would be determined to be best by a greater number of people than any others. (Such is the power of large sample groups, to which all stats nerds burn incense daily.)

If I, as High Queen of the Universe, were to anoint two cards of my choosing from the deck of 52 and declare them to be The Best, no matter what two cards I picked, 92.4% of the people expressing an opinion on the cards would be unhappy, for neither of their cards would match the two I had picked.  The other 7.6% would be moderately pleased, as one of their cards would match one of mine, and 1/326 th of the people would be very pleased, as my choice would exactly match theirs.

But wait, one says – this is a stupid example, because everyone knows that not all cards are alike!  Face cards are clearly More Best than the rest, and so any example that ignored this difference is clearly useless.

Fine.  Let’s run the numbers for ‘two picked from 12 face cards’ – and we come up with 66 unique sets. Everyone of the bazillion gazillion sorts themselves along those lines – again, giving equal weight to any of the face cards – into 66 groups. I as High Queen of the Universe again pick The Best – and this time, there are 31.8% of the people who are moderately pleased, 68.2% who think I clearly suck as a universal monarch, and 1/66 of the people who think my opinions (at least in cards) are perfect.

However, for the people who didn’t share the opinion that face cards are CLEARLY More Best, my disapproval rate is much higher: only 1.58% of the people who were selecting from the whole deck had EITHER of their two cards match EITHER of mine.

With me so far? Good.

If you do the math out, you see that if one is picking sets from larger decks, the numbers get crazy large crazy fast.  More sets, larger decks, and the number of people who think it is clearly time to pick another universal hereditary ruler start to equal EVERYONE.

But what the heck does this have to do with picking Hugos?

Firstly, consider that instead of a deck of 52, we have a deck of “all the novels published that year.”  And we have everyone vote on what they think the best five are….wait.

No, we already decided that there are cards which are clearly better than others.  Face cards, in our deck.  And for the Hugos we have…oh, every one of the novels nominated during the nomination round.  There.  We’ve narrowed the pool of “best SFF novel” from the tens of thousands published that year to…around 400 (it was 230 novels in 2005, and last year at LonCon it was 648.  We’ll use 400 because I’m High Queen of the Universe.)  At any rate, tens of thousands down to 400 is sorta like 52 to 12, except that it’s several orders of magnitude in difference, and so it’s not really the same. At All, because 52 to 12 doesn’t even come close to approximating the degree in change from tens of thousands to 400.

And as it turns out, my version of MS EXCEL crashes when I go over 170 for my n.  So we can’t even use that. Let’s use 160.  (See: High Queen of the Universe.)

If we pick sets of 5 cards from a deck of 52, that there are 2.598960 MILLION different combinations of sets of 5.  For [our 'face cards set' (slight edit)] 160, it’s 98,446,083,840.  Yes, that’s 98 BILLON. And change.  When the High Queen of the Universe comes down and anoints The Five Best Cards, out of those 160, 14.85% of the people see that at least ONE of their cards matches at least ONE of The Five Best.  (Remember, in our last example, we were talking sets of two.  Now we have sets of 5.  That changes the math.)

(Also?  “One out of five” is a lower standard of happy than “one out of two” – or at least I think so.  See: High Queen of the Universe)

And remember, we’re just talking the people who picked face cards.  The people who were picking from the larger set of the whole deck/all the books published that year, they’re much less happy.  (And I can not do that math because, again, when n > 170, Excel = miserable.)

So.  That’s how it is when we look at picking the five best novels from the 160 face card/clearly best novels that year.  15% of the people have gotten at least one of their novels selected.  The rest are unhappy, and collecting pitchforks.

But it gets worse.

What if instead of picking from all the face card novels, I only picked from diamond suite novels?  If instead of picking from 160, what if I had narrowed my selections down to only those which were the ace, jack, queen and king of diamonds, so now I (as High Queen of the Universe) was selecting from 40 novels, while everyone else was selecting from all the face cards (160 novels) or (even worse) all the novels selected (tens of thousands.)

In that case, there are 658,008 sets of 5, from the 40 diamond face cards. (Note the change from the 2.5 million sets of 5 from 52 cards.  Numbers don’t change geometrically here.) Now, 50% of those whose tastes also ran to just diamond face cards have at least one of five selections equal to one (or more) of mine.  Of those still picking from all the face cards, it’s less than one in a hundred.  In fact, it’s a lot less – it’s 4 in ten thousand.

For those picking from the wider pool of all the deck of published cards?  Doesn’t even register.

And remember that I’m talking about out of 160 novels.  It’s been a very long time since we had only 160 novels that someone thought was Hugo worthy.

So when a fan says Up until now, I generally agreed with the Hugo nominations…it means, I think, that their tastes agree with the tastes of the Queen of the Universe.  Or the average of the Hugo nomination voters, who – at less than 1K – are numerically indistinguishable from a single Queen of the Universe, when looked at on that scale.

When SP/RP say, Up until now, most of what I liked never made it to the Hugos…well, it *might* mean that they had a fancy for cards numbering 2-10.  But it could also mean that they liked face cards of suites other than diamonds.

If we were to imagine SFF as a deck of cards(note: Examples not chosen with any intent in mind) – with Literary SFF as diamonds, and MilSF&Space Opera as clubs, and Humor as hearts, and, oh, Movies&TV&Tieins as spades…well, it would easy enough to see that even if one really liked the most excellent work in clubs AND hearts, if the High Queen of the Universe (or the Hugo voters) were only picking from the 40 items in diamond face cards…well, you’d be SOOL(link).  And the High Queen of the Universe would be aghast at suggestions of bias, because She was selecting evenly from the 40 items in diamond face cards- and what could be wrong with that?

Likewise, if one were to imagine a revolt by people who liked just spades, who all gathered together to sacrifice fluffy kittens and blend puppies so that a pleasing aroma rose unto the sky, and the High Queen deigned to select from the spades face cards instead of diamonds…that would look very much like a betrayal to those who liked diamonds. (We are ignoring those who have objections to animal sacrifice of any sort, because they are obviously in league with the Elder Ones.)

To sum up, because it is too much to explain: the SFF field is huge.  The number of Hugo voters is small.  We need to fix this.


(1) These two things are not the same.

(2) No matter how one slices it, WSFS members, WorldCon attendees, and Hugo voters are a very very small fraction of the total number of people who read, watch, write, draw, or play science fiction and fantasy.  Annual nominating membership was under 1,000 people for decades.   It’s only in the last five years that it has hit 2 thousand. Attending membership was under 10,000.  In comparison, Dragon*Con - held the same weekend - was 40,000 in 2010, and is projected to exceed 60,000 in 2015. 

(3) SFF Fandom: that portion of the global human populations who read, watch, write, draw, or play science fiction and fantasy.  At the very minimum, we’re talking 100,000 people – assuming we limit the number to those who can read or speak English.  This group HEAVILY overlaps with, but does not equal, those people who are SFF creators – writers, artists, directors, editors, etc.  (In much the same way, SFWA membership heavily overlaps with, but does not equal, “people who have published something in SFF in the last five years.”

(4) The author of this particular article is Kameron Hurely, two-time winner of the Hugo award, short listed by Chaos Horizon last November as an strong contender  for a Hugo this year for her novel The Mirror Empire.  Yet somehow neither the author nor the editor saw fit to mention this conflict of interest.  I suppose in a world where The Rolling Stone exists this is to be considered of no great note.

(5) While the actions and words of a couple of editors associated with Tor had done a great deal to avoid disproving this perspective, it is my opinion that the fault lies most with a narrow pov on the part of Hugo voters, each of whom is voting their individual preferences, with perhaps some minor influence by those who are attempting to push specific works or authors. More specifically – we can’t get rid of people’s individual preferences and likes, but we can avoid choosing from people who only like one sort of things.

(6) Sad Puppies here. Rabid Puppies here.  These are two different groups who share some overlapping goals.  In combining them, I am unfortunately continuing the disastrously inaccurate lumping together of goals, membership, motivation, and nominated works that has characterized the “trufan” response to the whole mess.  For the purpose of this discussion, I think the shorthand is accurate enough to continue, although I may come to regret saying that.

(6.1) I am not Vox Day, either.  If you have a question about, or an issue with, something VD has said, go take it up with him.  If you have a question about, or an issue with, something I have said, I am willing to discuss that.  For the purposes of this post, the only opinion that VD and I share which is relevant is that the current Hugo process is broken.

(7) Urging other fans – who then purchase their own memberships to WorldCon, and then vote their own ballot - to support particular authors or works has been widely acknowledged as “within the rules.” There are those who disagree and/or who hold that having recommended 5 works on a five opening ballot constitutes undue influence.  Complicating this judgement is Vox Day’s verbage regarding the Rabid Puppy slate: “Those who trust my judgement will vote the slate exactly as it appears.”  Be that as it may, the range of votes even across Puppy dominated categories does not support the charge of lock-step voting. (Obligatory link to herding cats video.)

(8)In the most practical terms, one needs 40 “normal average” individuals to achieve a measurable range of values for any test (like blood pressure or lung volume) and that sample sizes of 100 individuals per data point is sufficient to get a good random distribution, but there are different schools of thought on this.

(9) Assuming traditional French suites, not the German.

Comments and critique of all sorts welcomed!  Please leave a note or drop an email - excel spreadsheets available on demand.

No comments: