photo.not
Phil Greenspun's Photo.net
provides users with a place to talk about photos, a place to show
their uploaded photos, and a ranking system for photos and
photographers. Many thousands of photos have been rated, with
over two million ratings. A page is available at
http://www.photo.net/photodb/top-rated-photographers that
attempts to index the work not just of the photos, but the
photographers, based on user ratings.
Individual image ratings have two indices, labeled "Aesthetics"
and "Originality," each on a scale of 1-10. These indices are
combined across multiple photos to create the photographer's
overall "score."
The method used is described thus:
We currently sort by the sum of the average aesthetic and
originality ratings of each photographer's photos. We only
rank those photographers who have at least 3 photos, each of
which has been rated by more than 10 members and the sum of
the average ratings is greater than 7. ( We are now computing this every hour to
reduce server load )
So theoretically the top score possible is a perfect 20 -- ten
and ten for both ratings, at least ten times, of at least three
photos. In practice, the leaders tend to hover around 15.5.
Photographers are as status-conscious as anyone (perhaps more
so), and quite a few of them spend a lot of time and effort
worrying about their rating. Currently there are about 1900
photographers with ratings high enough to appear on the list
page. While many decry the listing as pointless and
anti-artistic, the amount of noise, effort (and outright cheating)
expended at the rankings indicates its important status value to
the photo.net community. The need for hour-by-hour recalculation of the stats
is ample evidence of interest!
There are numerous problems with any rating system when applied
to subjective material like photographs. For Photo.Net's rating system,
three obvious problems readily spring to mind,
without even discussing the composite scores.
- The incredibly ambiguous nature of the rating indices. A page has
been prepared describing these "standards:" it can be found at
http://www.photo.net/gallery/photocritique/standards/#Aesthetics
and after reading it really feel no more informed than when I
began (when I supply ratings, I've come to regard photo.net's
"aesthetics" as an Aristotelian synonym for "professional polish" and
"originality" as a sort of "cleverness/emotion" knob).
- Ratings are usually done en masse from the front
photo.net page's various "Rate this Photo" or "rate yesterday's
photos" links. The photos are usually shown around 200 pixels
wide. Such a size obviously favors images that read well at such
a size -- large group portraits or other sorts of images that
require fine detail will simply turn to mush.
- Ratings go to the predictable mid-road crowd-pleasers -- cute animals,
craggy mountain peaks, exotic arabs silhouetted against the
sunset, sexy babes in varying states of discrete undress. No
Joel-Peter Witkin here! While users sometimes opine that "the
ratings should not be a popularity contest," they clearly are
-- with popular, safe, themes dominating the ratings. There's
nothing wrong with that, it's part of the democratic nature
of photo.net -- but a viewer hoping to be visually challenged
will find little to surprise their eye among the top
member-rated choices.
The scheme used to rate photographers, layered atop the image
ratings, also has some obvious weaknesses (used in the
past to manipulate the standings):
- The criteria are a bit vague in terms of the specific
selections used for the rating. It appears that the best
strategy for gaining a high ranking is to keep your three
highest-rated photos -- and only those -- in your photo.net
portfolio. A photographer with three "14's" will rank higher
than a photographer with three "15's" and a "10." Shifts dur
to portfolio mainenance often occur -- it seem to be a common
practice to kill low-ranking images (photographers with
large, terrific portfolios, or with long histories on photo.net,
are thus at some disadvantage).
- The rankings appear to be summed across photos, rather than
individual rankings. Thus a photo with exactly ten ratings
can be very influenced by even a single 10-of-10 "ringer"
rating, and that higher photo-total will be factored into the
member rankings -- while superior photos, that may have received
hundreds of more-honest ratings, will hold steady for long
periods of time without those capricious variations.
Many people have expressed dissatisfaction with the scoring. One
user,
Robert
Brown, has recently suggested the following alternative scoring
scheme:
(AVERAGE SCORE x 10) + 1 point for each photo with more than 10 ratings.
Using this scheme, photographers with large bodies of work, or
who have been present on photo.net for a long time (and have thus
accumulated many ratings) are at an advantage. But one needs ask:
are many good photos really better than a smaller number of
truly great ones? This scheme would simply replace one limited
rating system with another.
The truth is, there are many possible rating systems, and they're
all wrong. So why not give the users
a choice of which prism they choose to view through?
Photo.net is, at its computer core, just a big database. How odd
that there's only one indexing scheme! It seems obvious that the
same process that collates the results for the
existing list could also build alternatives at the same time.
If alternative rating systems were available, viewers would have
more opportunities for browsing, photographers might have less to
fight about in the boards, and less "unfairness" to complain at
Phil about (then again, some people live for that sort of thing).
A number of possible rating schemes can spring readily to mind,
in addition to the original one and the "Brown Method" described
above. In fact, any info in the database could be used to build
an index of one sort or another:
- Rankings based on "Originality" and "Aesthetics" alone, or
with weightings other than 50/50.
- Rankings based on total ratings for all images -- so that
four "15's" beat three "15's"
- Rankings that factor-in the total ratings for images -- so
that 100 ratings on picture "A" and 10 ratings on picture "B"
are calculated as 110 ratings, not two.
- Rankings of the number of comments -- most contentious and
controversial
- Rankings on images within specific age ranges for the images
and/or the user accounts (or even based on recent ratings
only)
- Rankings based on a higher (or lower) number of image ratings
- Rankings based on additional criteria such as digicams or
black and white (easy to determine based on the required image
data) -- even by f/stop, focal length, or (ewww) brand of equipment
- Rankings of photo.net subscribers or non-subscribers
- History view -- see how rankings have varied over time
(maybe limited to the top 20, or (for subscribers) to
yourself...)
- Rankings based on historical precedents -- to account for the
reported inflationary upward drift of "average" scores
- Rankings based on community diversity -- to identify the differences
between broad appeal and mutual-admiration logrolling
- Raw ratings counts overall
Pick your poisons!
photo.not is brought to you by
Kevin Bjorke &
botzilla.com