Everyone has an opinion on the Web
February 2007
PDF versions are also available (280KB): formatted for A4 or
US Letter size.
1. Introduction
The development of the Web has made it easy to express an opinion on any subject. Essays, articles, white papers, op-ed pieces and blogs (diaries) are all variations on a common theme that long predates the hyper-text transfer protocol, yet publication to a global audience is now child's play. What's not so obvious, at least to some users, is that the mechanism for passing on recommendations between readers is also developing rapidly. Reviews of all kinds, such as those of books, music, plays and restaurants, continue to be an important part of traditional media, but how does this work on the Web and does it work in the user's interest as well as it could? This paper highlights existing and developing systems that facilitate the exchange of recommendations between individuals, looks at what this means for user protection, and considers what remains to be done. The potential is significant, exciting and well within our grasp.
2. Existing Rating And Recommender Systems
The most pervasive recommender system on the Web is the Google Page Rank algorithm. In 1998 Stanford University students Larry Page and Sergey Brin devised an automatic way of essentially classifying the Web according to what was popular [Google]. The key metric they used the number of hyperlinks that pointed to a particular Web page. In its simplest form, if website A links to website B then it's a good bet that the author of website A thinks the content on website B is worth seeing, that is, B is recommended by A. If more people link to B than A, then website B should have a higher page rank than website A, leading to greater prominence is search results.
There is a lot more to the Google search engine than this, but it remains true that one of the best ways to raise your search engine visibility is to get other people to link to you.
There are other established methods of passing on recommendations. Star rating systems are commonplace and well understood by users of sites such as eBay and Amazon. Like the Page Rank algorithm, these star ratings are based on vox populi. Everyone who enters into a transaction on eBay is asked to rate their experience (repeatedly if you don't comply) and anyone can post a book review on Amazon. Interestingly, readers of reviews posted by Amazon users are then asked whether they found the review helpful - in other words, readers are asked to rate the rating with the most helpful being promoted.
This is not a trivial point. With systems set up to gather recommendations from anyone and everyone, the question arises very quickly: can you trust the rating? The artificial boosting of a 5 star rating system, a practice known as shilling, requires significant effort, however, it is possible [Shill]. Boosting page rank is an established commercial business, known as Search Engine Optimisation. Some SEO tips come down to good practice, such as adhering fully to technical standards, separating content and style etc. but there is a constant tension between the SEO companies and the search engines as one side tries to boost the prominence of their clients while the other side tries to see through the smoke to find what people are actually looking for. Want to know exactly how Google decides which site comes top of their results for a given search term? So does everyone else - and they're not telling.
There are other established ways in which recommendations are passed on. Popularised largely by del.icio.us and with notable similar services (digg.com for example), the sharing of tags and bookmarks (what Microsoft Internet Explorer calls 'Favorites') allows users to pass their recommendations on directly to their contacts and for users to see a generalized information about pages based on how many people have bookmarked and tagged it. These are available as lists and automated feeds of various kinds such as what's most popular at the moment [DEL] (see Figure 1) and what has been most recently tagged. The sharing of recommendations directly between people known to each other is important. If 9 out of 10 people in the general population say that a film is terrific but three of your close friends tell you it's a turkey - you're probably going to be inclined to leave it in the poultry barn.
Figure 1. A sample "most popular" page on del.icio.us
This is an overwhelmingly positive aspect of social networks. They bring like-minded people together who can then share ideas, images, stories, music and movies. The creative potential is enormous.
Research into shared ratings shows that if your opinion is inline with the majority, then generalized user opinions are as valuable as those of your peers. If you agree with the many film critics who put Star Wars and Citizen Kane at or near the top of the greatest movies ever made, there are plenty of ways to find the next film you'll enjoy. However, if your taste differs from the norm, you'll benefit far more from sharing the opinions of those with whom you have specific tastes in common [Golbeck].
Having being pioneered on the sites already mentioned, the central notion of user-generated ratings and comments is finding new expression all the time. RatePoint.com and jyte.com being two recently launched examples.
2.1. So is the job done now?
With a growing number of such systems are in place, what is there left to do? Quite a lot, starting with showing users of all ages, but especially children, how to recognize the rating information available to them, to understand its usefulness and limitations, and to encourage them to share bookmarks with parents, teachers and friends.
Users have very blasé attitudes to assessing the quality of online content. According to work done in the European SAFT project, 50% of the Norwegian children surveyed think that all or most of the content on the net is true and can be trusted. The researcher also reports anecdotally of children being taught to look for things like copyright notices on websites on the basis that "only the good guys will want to protect their copyright;" and that, not surprisingly, a teenager's primary concern when assessing quality is "does it look good?" Research conducted in 2005 by Carleton University in Ottawa showed that users typically decide on the quality and trustworthiness of a website within 0.2 seconds . No wonder phishing scams work.
But what does constitute a 'good website'? The question is meaningless without a set of criteria against which such a judgement can be made. A set of recipes for birthday cakes is clearly not a 'good website' on which to check the likely re-sale value of your car. Likewise, finding what is popular (as Google's Page Rank algorithm, tags and shared bookmarks are very good at helping you do) does not mean you've found the best resources available for your needs - it just means you're looking at what everyone else is looking at. The list of most popular bookmarks on del.icio.us at the time when the screenshot was taken, 25 February 2007, hardly looks like a list of quality material (Figure 1).
There are, however, any number of organisations and individuals who are in a position to publish their informed opinions - opinions that will stand up to challenge and scrutiny, acting as a seed around which a cloud of user-generated comments and ratings can form. In other words, vox populi can be given a welcome boost by the opinions of experts and professionals. This is where Description Resources, DRs, also known as Content Labels, can play a pivotal role.
3. Description Resources
As Description Resource is: "A resource that contains a description, a definition of the scope of the description and assertions about both the circumstances of its own creation and the entity that created it" [cLabel]. Breaking down the jargon a little, we can recognize the essential elements:
- A Description Resource is a resource in its own right. It therefore has its own URI and exists independently of the content it describes.
- The scope of the resource is defined. That is, the resource, or more usually, the group of resources, that are described. Typically this will be "all the pages on example.com" or "everything in the news section of example.com."
- When was the description published, and by whom.
- The description itself.
Points 2 - 4 are intuitive. The critical importance of point 1, deliberately put at the head of the list, however, may not be so immediately obvious. It is this: by having a URI of its own, the DR is itself open to comment and challenge by others in their own recommendations, blogs etc. A DR produced by a recognized body, such as a film classification board, game rating system, or trustmark, can act as the seed for a user-generated cloud of comment by virtue of its identity on the Web.
3.1 Provenance of the Description
If a DR exists independently of the resource it describes, is it not possible for anyone to claim anything, perhaps using other people's descriptors? Well, yes. It's perfectly possible for anyone to post data on the Web that says, for example, that the British Board of Film Classification gave the 1997 film 'Titanic' an 18 classification. But that's perfectly possible now, indeed, since this document is published on the Web, that data exists in cyberspace, even though it is untrue (BBFC gave it a 12).
The way to avoid people abusing proprietary descriptions in this way is for the classification body concerned to publish the data themselves in an interoperable format. That way, if a machine comes across data on the Web that declares that a given movie has a given classification, it is easy to authenticate that claim directly with the classification board through an automated process.
The same is true for game ratings, trustmarks and content descriptions in general. If the labelling authority exposes its data in a suitable manner, any machine wishing to make use of that data will always be able to authenticate the data wherever it was found. In this way, the proprietary data and the brand behind it are protected. If, however, the rating exists in a form that is not machine processable, it is much more open to being abused.
A W3C Working Group is now developing a Protocol for Web Description Resources (POWDER) that will build on previous work carried out in the QUATRO project and in the W3C Content Label Incubator Activity [WCL-XG]. The group is chartered through until March 2008 by when a set of Recommendations will be available through which DRs can be published and authenticated either individually or as part of an online repository.
4. The Importance of Controlled Vocabularies
The photo sharing service Flickr was a key driver in making the practice of tagging popular. Users now routinely post their content to websites, be it a blog entry, image or video, and ascribe one or more tags to it. The practice of tagging is now very widespread - we've already seen how it's used in del.icio.us for example. Tags serve a self-selecting group of like-minded people who use similar terms to describe similar things. How would you tag this image?
 |
Some obvious ones might be:
Less obvious ones might be:
And, you'd really have to be familiar with this particular sculpture of Einstein and its location to tag it as:
|
Other users who use the same descriptive terms as you, that is, those who have a similar active vocabulary and are of the same general mindset, would no doubt find your tags useful. Other people's tags may be quite different and so would not be helpful to you in locating this image.
Professional opinion cannot be open to such variation, rather, to be useful and authoritative, it must be couched in terms that are clearly defined and unaffected by the ephemera of linguistic fashion. In other words, professional opinion, expressed in Description Resources designed to be processed by machines and understood as authoritative by humans, should be given using controlled vocabularies. Film classifications again provide a good example of this.
In Germany, films are classified according to a set of age categories: All ages, 6+, 12+, 16+ and 18+. Even a Harry Potter film would never be given the classification '9¾'. The ICRA vocabulary is another example. It lists a series of content types that are present or not present in the described resources. The vocabulary is clearly defined in its human and machine-readable formats. Updates to the vocabulary are infrequent and fully documented.
Adhering to a controlled vocabulary does not mean that everyone has to use the same terms. For example, Broadcaster A might describe a programme as containing "occasional scenes of mild nudity" whereas broadcaster B might describe the same programme as containing "infrequent scenes of partial nudity." As long as each broadcaster adheres strictly to their own published vocabulary, users and machines can extract meaningful information. A user might post a comment that says 'when broadcaster A says "occasional" you can bet it's in every scene whereas when broadcaster B says "infrequent" you know if you blink you'll miss it.' More seriously, search engines, personalisation software and filters can make a comparison between what the two broadcasters say about their programmes without either feeling constrained by their competitor's terminology.
5. Some Examples
The following two examples summarize the threads developed so far.
5.1 Movie Rating Example
Suppose two users wish to make comments about the film Titanic as shown diagrammatically in Figure 2.
Figure 2 Two user tags for 'Titanic'
There is an immediate problem. Try doing a search on any film classification board's database, or the Internet Movie Database, for 'Titanic'. You'll see multiple movies and trailers with 'Titanic' in the title. The British Board of Film Classification's database, for example, lists 50 such works, two of which are full feature films of that name: one classified 'A' on 10th April 1953 and the other classified '12' on 14 November 1997. IMDB gives 73 full or partial hits.
In other words, as far as a machine - say an aggregator, portal or search engine is concerned - there's no way of knowing what these two users are talking about. From a data processing point of view, there are no semantics in their comments.
Much greater specificity is required in order to make a comment about 'Titanic.' Fortunately there is a well established way of providing a unique identifier for a given resource - called a Unique Resource Identifier (URI) of which the best known example is a URL (like http://www.example.com). Here's the IMDB's URI for the 1997 film Titanic directed by James Cameron and starring Kate Winslet and Leonardo DeCaprio:
http://www.imdb.com/title/tt0120338/.
The URI used doesn't have to be in the IMDB space. The movie's ISAN number would be equally suitable as it too is an example of a URI. Furthermore, ISAN is a non-profit organisation cf. IMDB which is wholly owned subsidiary of Amazon.com.
Figure 3 shows the same basic set up as before but with the crucial difference that the comments are being made about a resource that is uniquely identified.
Figure 3. Two user tags for the 1997 film 'Titanic' directed by James Cameron starring Kate Winslet and Leonardo DeCaprio <http://www.imdb.com/title/tt0120338/>
So much for the user-generated tag side. What about the professional opinion of the film classification boards? It's critical that it is clear which classification board is classifying which movie, what the classification is etc. This means using a controlled vocabulary that is itself referenced by URIs as shown in Figure 4.
Figure 4. A machine-processable, professional classification of the same movie
The user-generated tags and the professional classification exist wholly independently of each other, as do the classifications of other movie ratings. It is the usage of Unique Resource Identifiers that makes it possible for machines to harvest the data, wherever it comes from, and present it to end users in a variety of ways.
As noted in section 3.1, this does not facilitate the malicious use of the relevant classification body's data. On the contrary, it protects that data.
5.2 The ICRA Example
Self-labelling allows providers to make declarations about their own content. Using the ICRA system, declarations can be made about whether defined types of content, such as sex, nudity and violence, are or are not present. Parents can use that data, either directly or through tools that do it for them, to decide whether a particular item, is or is not suitable for their children. Under the ICRAchecked scheme, labels and the content they describe are compared and, if the comparison is favourable, the label is added to an online database that can be interrogated. In this way, an ICRA label provides an authoritative content description covering a particular area of interest. It is initiated by the content provider and backed up by an independent organisation, using terms that are stable and well defined.
Such a description can be used in different ways. For example, it can be converted into an age-based rating and displayed in search results [SEARCH]; it can be used by content aggregators to compile lists of sites suitable for particular audiences (such as children); it can be displayed to end users through browser extensions [ViQ]; used directly or indirectly in filtering software [PLUS] and so on.
But an ICRA description of a website only covers one aspect. Trustmarks can cover other areas, such as privacy, e-commerce, medical or scientific accuracy, accessibility and suitability for display in mobile devices. If this information is made available to end users, and the various systems mentioned earlier, such as del.icio.us and RatePoint, add the dimension of popularity, then users will be in a much better position to judge online content.
This will require the use of tools that provide simple visual cues - icons in the browser chrome as ViQ, the QUATRO project browser extension offers for example - with the more detailed information available if the user asks for it. Remember that, on average, content providers have got a fifth of a second to prove their content to be 'good.' Furthermore, back end systems will need to recognize and authenticate the available information before forwarding content to users. To aid and promote the development of such tools, Description Resources and the output of recommender systems should be available in common formats. This does not preclude the development of proprietary systems that may offer a richer feature set, tailored to a particular market, but it does promote the general approach and help to create a media literate audience that demands assurance that what they're looking at is trustworthy and of good quality. That's surely something from which everyone can benefit.
6. Summary
There is a growing number of services that allow users to share recommendations and opinions of online content. This is to be welcomed and users of all ages should be encouraged to make use of these systems to help guide others, especially their children, as they navigate the Web. However, such systems rely on user-generated tags and bookmarks which tend to reflect popularity and the current Zeitgeist rather than inherent quality.
There is therefore a need for professional opinion to be expressed about online resources. Such opinions should be expressed in well-defined terms with information available about whose opinion it is, when it was expressed and under what circumstances. The interests of the bodies expressing those opinions, and those of the users, are both best served if those opinions are open to authentication by automated means.
There is some way to go to make this a reality, however, a lot of work has already been done or is well underway. With expert opinion and vox populi working together, the remaining challenge will be to improve media literacy among end users.
Phil Archer
26th February 2007
Links and References
- http://infolab.stanford.edu/~backrub/google.html
- Shill
- See, for example, Shilling Recommender Systems for Fun and Profit
- DEL
- http://del.icio.us/popular/
- Golbeck
- See extensive work in this area by Jennifer Golbeck et al at the University of Maryland. For example Generating Predictive Movie Recommendations from Trust in Social Networks
- SAFT
- The original report seems to have disappeared from the Web but there are snippets in the project's final report
- Carleton University
- Reported at http://news.bbc.co.uk/2/hi/technology/4616700.stm
- cLabel
- See glossary in W3C Content Label Incubator Activity
- POWDER
- http://www.w3.org/2007/powder/
- QUATRO project
- http://www.quatro-project.org/
- WCL-XG
- See W3C Content Label Incubator Activity
- Questacon
- http://www.questacon.edu.au
- IMDB
- http://www.imdb.com/
- British Board of Film Classification
- http://www.bbfc.co.uk/search/index.php
- ISAN
- http://www.isan.org
- SEARCH
- http://checked.icra.org/search/
- ViQ
- See the QUATRO project's VIQ browser extension
- PLUS
- See ICRA's free ICRAplus filter
