Hugo Voting: Let’s Look at the Record (Again)

by George Flynn

[originally published in Apa:WSFS 1995 #4, July 1995; updated in January 1997]

Back in 1988, I wrote an article on Hugo voting statistics for The Mad 3 Party (#30, November 1988). That was long enough ago that it may be worthwhile to do it again. So here’s the new version, with lots more boring numbers. The 1988 version began as follows:

I am often amazed by the kinds of misinformation that people believe about Hugo voting figures. Just an example: One widely distributed report on this year’s nominations said, "There were 122 ballots with best fanzine nominations (29%, a high number)." The 122 was in fact the number of fanzines nominated, not the number of people nominating; but where did the idea come from that 29% would be "a high number"? (As we shall see, 45% would be typical.) This idea that hardly anyone votes on the fan Hugos is perhaps the most widespread of the misconceptions I alluded to.

As Mike Glyer pointed out in a recent File 770, it’s just ten years since full Hugo voting counts began to be routinely released. (Mike and I sponsored the rule that now requires this publication.) The numbers come out every year, but not many people wade through them to see what they all mean. After ten years, though, we should have enough data to do some useful analysis. The numbers that follow mostly come from the annual tabulations in the newszines, with some exceptions that I’ll note.

(The exceptions are from official reports of Hugo administrators, some of them by me.) Well, there’s probably just as much misinformation around today, but we do have an additional eight years of data, including some details not available for earlier years. The rest of this article is organized mostly like the original one; however, I’ve come up with many new ways to tweak the numbers. And we’ll see whether the conclusions turn out the same . . .

Table 1 — Total Vote Counts (1971–96)

(* = overseas Worldcon)

How many people vote on the Hugos? The left side of Table 1 gives the numbers ofnominating and final ballots each year since 1971, along with the numbers of site-selection ballots for comparison. (In this and all the tables, the "year" is that in which the voting takes place, not the year of the works being voted on.) Also included is the number of categories on the final Hugo ballot (including Campbell and Gandalf awards). Note that in the early ’70s the size of the Worldcon was growing rapidly. But since the mid-’70s the "normal" Hugo nominating-ballot count has been fairly stable at around 500. The final-ballot count has fluctuated more, but the "normal" count seems to have declined from around 1200 in the late ’70s to perhaps 900 today. Still, there seems to be a relatively stable population of Hugo voters — more stable, at least, than the Worldcon membership itself.

The most obvious exceptions to this stability can be accounted for by geographical factors: Australia (1975 and 1985) and Holland (1990) had about half the usual vote. (For Winnipeg in 1994, the final vote was similarly low; but since ConFrancisco also distributed nomination ballots, the nominating vote in 1994 was the second-highest ever.) As for other fluctuations, the high final vote in 1980 may derive from the special effort Noreascon II made to get ballots to last-minute joiners; did something similar occur in 1976? Conversely, some relatively low vote counts (e.g., the 1988 and 1991 nomination counts) have been attributed to late distribution of ballots.

In 1996, there were 111 nominating ballots and 605 final ballots for the 1946 "Retro-Hugos"; 558 of the latter voters cast ballots for both 1946 and 1996 Hugos. (Since the voting pattern is likely to be quite different, the Retro-Hugos will not be considered further in this article.)

What fraction of the Worldcon membership votes? Now consider the right side of Table 1. The "Advance Memb." column gives the best numbers I have been able to find for the final pre-convention Worldcon membership; some of these numbers are obviously more accurate than others. And the "Final Hugo Vote Ratio" column gives the number of final Hugo ballots divided by the "Advance Memb." As you can see, this ratio has dropped from around 40% to around 15%. Depressing, yes? One problem with these numbers is that the Worldcon membership tends to increase rapidly at the last minute, and many of these people join too late to vote. So the voting percentage among people who actually get a chance to vote is higher than tabulated here, but probably not by much. (For what it’s worth, in 1980 I estimated that 17% of the eligible-at-the-time members nominated, while 37% cast final ballots. But these figures were probably higher than average, and for most years there are no good data available for the membership-at-the-time.)

Explanation of category tables: Many of the tables that follow give assorted statistics for individual Hugo categories. For these I generally have data only from 1978 through 1996. Since (as described above) some years had votes much lower than usual, I’ve divided the results into "Low" and "Other": "Low" includes Aussiecon II (1985) and ConFiction (1990) for both nominating and final ballots, plus ConAdian (1994) for the final ballot; "Other" covers everything else. Rather than giving the numbers for every year, for both "Low" and "Other" I simply list the range (high and low bounds) and the average. With regard to Hugo categories, note that the Non-Fiction Book category first appeared on the Hugo ballot in 1980; the Original Artwork category was tested in 1990, and appeared regularly from 1992 to 1996 (when it was abolished). What had been the "Fanzine" category in 1978–83 was divided in 1984 into "Semiprozine" and "Fanzine," so separate tabulations for the two periods are given for these categories.

[I defined the"Low" and "Other" sets in the 1994 version of this article. The 1995 final counts were generally somewhat lower than the previous range of "Other" years, but not by enough to justify a complete reorganization.]

The average overall votes (from Table 1) for the subsets defined here are as follows:

Nominating: "Low": range 222–291, average 256

Nominating: "Other": range 352–660, average 515

Final: "Low": range 443–491, average 473

Final: "Other": range 744–1788, average 1137

 

Table 2 — Minimum Nominating Vote (1978–96)

How many nominations does it take to get on the Hugo ballot? Table 2 gives the minimum nominating vote by category, arranged as described in the preceding section. That is to say, the lowest-ranking nominee to actually make the final ballot for Best Novel, say, has gotten as few as 25 nominations (1978 and 1990) and as many as 96 (1983), with an "Other" average of 49. It is depressing to see how few nominations it takes to get on the ballot, with the "Other" average under 25 in six categories. (Interestingly, the low end of the "Other" range in 9 of 11 possible categories is from 1978, although the total nominating vote wasn’t particularly low that year. This is probably because since 1982 there has been a rule requiring nominees to receive at least 5% of the number of nominating ballots in their category in order to appear on the final ballot. We’ll have more to say about this later.)

The variation between categories — and years — is largely a function of the number of likely nominees. For example, there are only a few well-known professional editors, so the vote in that category tends to be concentrated and the minimum-vote-to-nominate consistently high. Similarly, Dramatic Presentation usually has a high minimum, because in most years there are only a few good SF movies (the low of 8 votes in 1978, when Star Wars blew everything else away, may have been a case of too few plausible choices). In the fiction categories, the vote typically gets more and more scattered as you go to shorter lengths: short stories get almost as few nominations as fanzines. The highest minimum ever was 119 for Dramatic Presentation in 1983; while Original Artwork had the lowest minima for both "Low" and "Other" years. (By the way, the Novella average would be 41 if the category shifts hadn’t been made in 1994; the other figures would be unchanged.)

Table 3 — Maximum Nominating Vote (1978–96)

How about the high end of the nominating range? Table 2 gave the figures for the lowest-ranking nominees in each category; in Table 3 are the corresponding figures for the highest-ranking nominees. The ranges here are somewhat wider, since the maximum vote is affected more by single blockbuster nominees. As you might expect, the Dramatic Presentation nominee with the most votes nearly always beats out anything in the other categories (the exceptions were 1979, when Varley’s "The Persistence of Vision" outscored Superman; and 1995, when both Pro Editor and Semiprozine had higher-scoring nominees). The single nominee with the most votes (338) is still Star Wars in 1978 — and even that received less than 2/3 of the total nominating ballots. As you might surmise from that datum, the votes cast in individual categories are always appreciably lower than the overall total; the next few tables will give details.

The lowest maximum vote was 11 for Original Artwork in 1995, when the range of nominations was 8–11. Generally, however, the narrowest ranges appear in the Novelette and Short Story categories: the closest things to ties were 43–49 for Novelette in 1983, and 26–30 for Short Story (seven nominees) in 1992, again illustrating the typical flat distribution of votes for short fiction. (By the way, the "Other" maximum and minimum averages in most categories are several votes lower than those I calculated in 1978: further evidence of the decline in Hugo votes. [Three categories had their lowest maxima in 1995, and one in 1996.] Without the category shifts in 1994, the Novelette average would have dropped further to 50.)

Table 4 — Nominating Ballots by Category (1980–96 Partial)

How many people make nominations in each category? Unfortunately, these numbers haven’t been published every year; the data I have at hand are only for 1980, 1983, 1985–87, 1989–94, and 1996. (For these years the "Other" range of overall nominating votes is still 352–660, but the average is 524.) Table 4 gives the number of nominating ballots cast by category, for those years only. For the "Low" years, the lower end of the range in every case was in 1985. For the "Other" years available, Chicon V (1991) had the lowest vote in 11 of 13 possible categories; ConAdian (1994) was highest in 6 of 14 categories. (Nevertheless, the average "Other" vote has declined significantly since 1988 — by 41 votes in the average category, ranging from 13 for Semiprozine to 78 for Dramatic.) Print chauvinists will be gratified that (for these years at least) the Novel category always gets the most nominating ballots, the highest being 606 in 1983; however, Dramatic Presentation usually takes the lead on the final ballot. At the other extreme, Original Artwork drew the fewest nominating ballots every year it was on the ballot.

Table 5 — Nominating Percentages by Category (1980–96 Partial)

The raw numbers are interesting, but it may be more useful to consider the numbers of nominating ballots in each category as percentages of the total nominating vote (see Table 1); these percentages are given in Table 5. As one might expect, in this table there is no significant difference between the "Low" and the "Other" years, so I have added a column giving the overall averages. Curiously, Noreascon II (1980) had the highest percentages in 8 of 12 possible categories; I have no idea why (though I ran the Hugos that year). The lowest percentages are scattered over time, but I note that four categories each had their lowest percentages in 1987 and 1996. The "Other" averages for most of the professional categories have decreased by 2–3% since 1988. For what it’s worth, here’s a table of the percentage nominating in the average category over time:

There seems to be a slight downward trend, though not as strong as the trend we will see below for the final-ballot percentages. It seems that Hugo voters (and fans in general?) are tending to have more specialized interests. (Also, we have tended to add relatively low-vote categories.)

While several categories fairly consistently fall below 50%, in these years only Original Artwork went below 25% (in 1992 and 1996); the lowest in other categories is 26% for Fan Artist in 1986. In particular, the numbers for Fanzine demonstrate the point I made in the 1988 introduction: nearly as many people nominate fanzines as nominate in the short-fiction categories. (As we’ll see below, even the low-vote categories get somewhat higher voting percentages on the final ballots.)

Table 6 — Number of Items Nominated (1980–96 Partial)

(*: "Other" numbers for these categories do not include 1980)

How many different items receive nominations? Here’s something I didn’t include in 1988. We have data on this question from 1980–85, 1988–91, 1994, and 1996. (As you may have gathered, any two Worldcons seldom report quite the same sets of data.) These numbers appear in Table 6, in the usual format. (In 1980 I lumped the three short-fiction categories together on the first pass; the best I can say is that 468 items were nominated in one or more of these categories.) The "Low" years still have smaller numbers, but not by as much as in Tables 2–4. Curiously, 1996 had the lowest "Other" numbers in seven categories. There certainly are a remarkable number of things that someone considers worthy of a Hugo nomination. Nevertheless, the relative paucity of candidates in the Novella and Semiprozine categories is particularly striking.

Table 7 — Total Number of Nominating Votes Cast, and Nominations per Ballot (1982–96 Partial)

How completely do people fill out the nomination ballots? For 1982–83, 1989, 1992–94, and 1996 (all "Other" years) I have the total numbers of votes cast by category on all the nomination ballots. Since for all of these years except 1982 we also know the number of ballots cast in each category, we can divide and obtain the average number of nominations per ballot in each category. As you can see, I have separated 1992 from the other years, since in almost every case the 1992 numbers are significantly lower than all the others. (I suspect an error in the 1992 report; for example, analysis of my 1989 data shows that the averages would typically be lowered about 0.7 if the "Total Votes Cast" actually omitted nominees with fewer than 5 votes.) In any case, it is clear that the average voter makes only about half as many nominations as are allowed. (This is not necessarily a bad thing: "Bullet-voting" for one or two nominees is certainly a rational strategy if one’s paramount aim is to get those particular nominees onto the ballot. It’s on the final ballot that failing to vote for all the nominees wastes one’s voting power.) Excluding 1992, 1996 had the lowest numbers of votes in eight categories, and the lowest ratios in four.

By the way, while I have not made a full calculation, it appears that the fraction of all votes cast for those nominees that actually make it to the final ballot varies from around 20% for Novelette and Short Story up to around 70% for Semiprozine. (The reasons are left as an exercise for the reader.) While at the other extreme, a substantial number of people pick nominees that receive only a couple of votes; just as an illustration, here’s a breakdown of the novel nominations in 1989 (for a total of 283 items):

What would be the effect of changing the minimum vote to get on the final ballot? At present the only minimum is the "5% rule" (in Sec. 2.6), which requires that nominees receive votes on at least 5% of the nomination ballots cast in the category (or be among the top three vote-getters). This rule went into effect in 1982, but it has actually been invoked only in the Original Artwork category in 1994 and 1995. (However, it would have applied to the Campbell Award in 1989, to Short Story in 1994, and to Music in 1995; in each case other factors supervened . . . And in 1991 it was initially applied erroneously in eight categories.) Two motions to change this rule were defeated in 1994: one would have reduced the 5% criterion to 2% (under which nothing would ever be eliminated), the other would have replaced it with a 10-vote minimum. Other criteria have also been suggested, so it seems worthwhile to survey the data.

Here is a tabulation of the number of nominees in some recent years that would have been made ineligible by several possible criteria, in the absence of an at-least-three-nominees clause. (For 1994, I am giving the numbers as if the short-fiction category shifts had not been made, since that’s more likely to correspond to future practice.)

1996

³ 10%	Novel 2, Novelette 4 (of 6), Short Story 3, Orig. Artwork 2, Fan Writer 2, Campbell 2
³ 10 votes	Orig. Artwork 2, Campbell 1
³ 20 votes	Novelette 3 (of 6), Short Story 2, N-F Book 2, Orig. Artwork 3, Fanzine 2 (of 6), Fan Writer 3, Campbell 3
³ 25 votes	Novelette 4 (of 6), Short Story 4, N-F Book 3, Prof. Artist 2, Orig. Artwork 5, Fanzine 2 (of 6), Fan Writer 4, Fan Artist 2, Campbell 3

1995

³ 5%	Orig. Artwork 2, Music 3 (I can’t calculate any other percentages for 1995, since the numbers of nomination ballots per category weren’t released.)
³ 10 votes	Orig. Artwork 3, Music 4
³ 20 votes	Short Story 2 (of 6), N-F Book 3, Orig. Artwork 5, Fan Writer 2, Campbell 4, Music 4
³ 25 votes	Novelette 2 (of 6), Short Story 3 (of 6), N-F Book 3, Orig. Artwork 5, Fan Writer 3, Fan Artist 2, Campbell 5, Music 5

1994

³ 5%	Short Story 3 (of 6), Orig. Artwork 2
³ 10%	Novel 2, Novelette 2, Short Story 4 (of 6), N-F Book 1, Dramatic 1, Orig. Artwork 4, Fan Writer 1, Campbell 2
³ 10 votes	Orig. Artwork 1
³ 15 votes	Orig. Artwork 3
³ 20 votes	Short Story 3 (of 6), Orig. Artwork 4
³ 25 votes	Short Story 3 (of 6), N-F Book 2, Orig. Artwork 4, Fan Writer 1, Fan Artist 3 (of 6), Campbell 2

1993

³ 10%	Novelette 1, N-F Book 3 (of 6), Fan Writer 3, Campbell 3 (of 6)
³ 10 votes	Orig. Artwork 2
³ 15 votes	N-F Book 3 (of 6), Orig. Artwork 4, Fan Writer 3, Campbell 3 (of 6)
³ 20 votes	Novelette 1, N-F Book 4 (of 6), Orig. Artwork 4, Fan Writer 4, Fan Artist 2 (of 6), Campbell 4 (of 6)
³ 25 votes	Novelette 3, N-F Book 5 (of 6), Orig. Artwork 4, Semiprozine 1, Fanzine 2, Fan Writer 5, Fan Artist 2 (of 6), Campbell 4 (of 6)

(Previous versions of this article included data back to 1990, but this table was getting too damn long. See also Table 8 below for more information about low-vote nominees.)

One thing to bear in mind is that sometimes nominees with only a few nominating votes go on to win. The most striking case was in 1991, when Julia Ecklar got only 9 nominations for the Campbell Award, but on the final ballot led with 197 first-place votes, and won. And there are ten other cases since 1989 of nominees receiving over ten times as many first-place final votes as they had had nominations. (But nine of those were in Original Artwork, and only two of them won.) Since the electorate for the final ballot is so much larger (partly because many people haven’t had time to survey the field by the nomination deadline), it is perhaps unwise to limit their choices on the basis of the nominators’ apathy.

Table 8 — Numbers of Nominees Matching Various Criteria (1989–96 Partial)

(*: not including 1989 or 1995)

For how many nominees should nomination numbers be published? This is of course the subject of a Constitutional amendment we ratified in 1995. It requires publishing the numbers for all nominees down to 15th place or 5% of the ballots cast in the category, whichever is less. Might this be too low (so as to threaten the privacy of individual voters), as David Bratman suggested? For reference, Table 8 gives a summary of the data David and I supplied to Apa:WSFS in 1994, plus some data for 1995 (numbers but not percentages) and 1996. Draw your own conclusions. (But note that this table is available only because the 1989 and 1993–94 administrators did release the numbers down to 5% and 15th place. For 1990 — a "Low" year — the 15th-place numbers were generally not reported, but the 10th-place ones, in the same order as above, were: 18, 21, 11, 9, 8, 12, 16, 14, 3, <9, 11, <8, 8, 6.) For Original Artwork, the ³5% figures for the six available years (including 1995) were 9, 8, 12, 3, 3, 15, with an average of 9, and the two consecutive 3’s appeared ominous; also, in 1995 there were only 5 nominations for 5th place, and no data for 10th or 15th. (1996 was somewhat better, but not a whole lot, and the category has now been eliminated.)

That’s about all the conclusions I can derive from the nominating-ballot data. (Note that the last four sections were all added since the 1988 version.) Let’s move on to the final ballots.

How many people vote in each category on the final ballot? Here we once again have numbers for every year since 1978. Table 9 gives the final-ballot vote by category (corresponding to Table 4 for the nominating ballots), and Table 10 the final-ballot percentages (corresponding to Table 5 for the nominating ballots).

Table 9 — Final Ballots by Category (1978–96)

For the absolute numbers in Table 9, the "Low" years clearly form a tight cluster in every category. Beyond being significantly higher, the numbers for the "Other" years are not so obviously a single population, because they are distorted by the long-term downward trend that we have already noted (cf. Table 1). This is the biggest change since 1988, when I found "no clear-cut evidence of any trend over time." Some details that make this trend especially obvious: 1980 had the highest vote counts in 10 of the 12 categories that then existed (1984 was higher for Fan Writer and Fan Artist, as well as the then-new Semiprozine). Conversely, 1995 had the lowest "Other" counts in every category except Fanzine and Fan Writer (for which 1994 was lowest); before that, 1993 was lowest in every category; and before that, 1989 was lowest in 10 categories, 1991 in the other 3. Indeed, no individual-year count since 1988 has been anywhere near as high as the corresponding category average I calculated then. The category averages have themselves fallen by an average of 152 votes since 1988 (including 17 from 1995 to 1996). (By the way, this same trend makes the 1984–96 Fanzine average appear abnormally low relative to the other fan categories, which include the higher 1978–83 data: for 1984–96, the "Other" average is 557 for Fan Writer and 561 for Fan Artist.)

Given all this evidence of a downward trend, this time around I have added a "Recent Other" tabulation, covering the last five "Other" years. The drop from earlier years is striking, but the downward trend is noticeable even within this period. Let us look at the percentages in Table 10 to see if we can eliminate the effect of the decline in the overall vote.

Table 10 — Final-Ballot Percentages by Category (1978–96)

As in Table 5, there is no significant difference between the "Low" and the "Other" years. However, the surprising thing is that the decline over time is still present! The highest percentages were in the period 1978–82 for every category that then existed (9 of 11 possible in 1978–79), and the lowest percentages for every current category were in 1991–96 (7 of 14 categories in 1996, and 4 more in 1995). And as one might expect, the "Other" averages have typically declined by several percent since 1988. To display this trend more clearly, here are the year-by-year average percentages of all the categories — that is, the percentage of final ballots that include votes in the average category (which is equivalent to the percentage of categories in which the average voter votes):

Given the downward trend, the averages for the entire period are of questionable significance; so I have included in Table 10 an additional two columns summarizing the data for the last six years only, to give some idea of what the percentages are today.

For yet another way of seeing the change over time in individual categories, here is a comparison of the 1978–88 averages (based on my 1988 article; numbers in parentheses are for shorter periods) with the 1991–96 averages: