were experimented with in an effort to combine the source
lists into one in a way that acknowledges and reflects their evaluations of the
works included. The best means seems to be one cumulating the rankings of the
sources. The version of this method originally adopted had the problematic feature
of exaggerating the scores of works appearing in multiple sources, to the extent that
it distorted the rankings. The version now in use does not have this problem. The
method used to reconcile the sources is as follows:
- First, to ensure that equivalent
ratings in each source were given equal weight, the rankings in each
source list were converted into points on a 100-point scale (with the
last 50 on the longest source dipping into negative numbers, since there
are 150 works in that source). A first-place rank was given 100 points,
a second place rank 99, and so on -- down to 1 point for a 100th-place
rank on the three shorter lists, and down to -49 points for the 150th-place
rank on the longest list.
- Second, the points for every
work on each list were totaled to determine its cumulative popularity
(or weight) in the sources.
- Finally, the works were re-ranked
according to total points received, those with the most ranking
highest and those with the least ranking lowest. In cases where
two or more works ended up with the same number of points, the
works(s) appearing in the most sources are ranked first. Works that
both share the same number of points and appear in the same number of
sources are ranked alphabetically by author and title.
The highest number of points
a work could receive according to this system is 400, for appearing
in first in all four lists. No one work actually did - the most points
actually awarded was 385. The lowest number of points a work
could receive is -49, for appearing last in the longest list alone.
As the last item in the longest list also appeared in one of the
others, the fewest points actually awarded were -48.
To provide the user of the composite list
with some idea of the basis for each work's ranking, I have shown for each work
the number of source lists it appears in, along with the points or weight
it was awarded. Together with the original rankings in the sources, also provided,
this will let the user check the compiler's math, should he or she so desire. As I
stake no claim to infallibility, reports on any errors discovered will be welcome.
The column key will further explain the information
provided on each work.
It might be asked why such a complex system
was used. Originally I utilized a simple 150-point scale, which
remained in place on this site for over six years. Its flaws were
highlighted by the observations of John Young. John demonstrated that
rating 100-title lists on the same scale as a 150-title list disadvantaged
the former. In fact, it awarded so many extra points based on the number of lists a work appeared
on that its points from its original rankings in those same lists were often
irrelevant -- a serious distorion. His favored solution was to drop the Library Journal list's titles
101-150 and then re-rate the four lists on a 100-point scale. While this notion
has merit, it would in my opinion have meant falsifying the comparison between the four lists, for two
reasons: (1) dropping the last 50 of the 150-title list misrepresents that list;
(2) since some of these fifty titles are duplicated in the other lists, it denies their
rankings in the longest list their due effect on their rankings in the shorter lists,
rendering their final rankings inaccurate.
I have, however, made John's solution the basis
of an alternative ranking.
The present system is essentially a modification
of John's 100-point scale, retaining the last 50 titles of the longest list while
representing their rankings in negative numbers, since in relation to the three lists
with only 100 entries all these titles would rate at less than 1. This removes the
distortion inherant in my original system. In fact, the practical effect is now that
a title on any of the other lists duplicated in the final third of the Library Journal
list has its score dragged down, not augmented, by that duplication. Unique
titles in this portion of the Library Journal list, as in the previous system, rank
below all titles in the other lists.
If you are interested in seeing the effect
John's unmodified proposal has on the ratings, see the
Alternative Rankings. His removal of the last fifty
entries in the longest list from consideration it eliminates their drag-down effect
on some of the ratings, but that same removal essentially falsifies the comparison.
Also in the
Alternative Rankings you will also find my original
alternative ranking, in which the bias created by exaggerating a work's rating in
accord with the number of sources it appears in is taken out. Its defect is that in
treating works rated among the "best" by all sources the same as those only one source
elevates to that distinction it favors eccentic choices over consensus ones, and a
work appearing in one source only can "beat out" one appearing in all the rest.