He Typed, She Typed
How valid are the comments that people make about your agency, its services — or even you — online?
The Organization for the Advancement of Structured Information Standards hopes to find out.
A new OASIS technical group met for the first time in May to begin work on specifications to define tools that can aggregate large data sets of often conflicting opinions about people, services or organizations. The goal: represent that data in a fair and meaningful way.
“This will make systems less susceptible to data manipulation and help make trustworthiness scores more stable, constructive and contextually relevant,” says James Bryce Clark, director of standards development for OASIS.
The Open Reputation Management Systems Technical Committee wants to develop a standard way of giving relevance to a reputation score within a given context. It doesn’t think it can go so far — at least, not yet — as to be able to craft an algorithm for computing reputation scores.
The ACSI Connection
For the government, the use of such a standard reputation system could help it combat a declining perception about e-government by the public. For the third straight quarter, the American Customer Satisfaction Index (ACSI) E-Government Satisfaction Index inched downward. Although a handful of websites continue to rank high in customer satisfaction ratings, overall reviews of the government’s online activities aren’t keeping pace with the private sector.
“The current quarter’s aggregate score of 72.4 is half a point lower than last quarter (72.9), a full point lower than first quarter 2007 (73.4), and a point and a half lower than its all-time high (74 in June 2006), making it the lowest e-gov score reported in three years,” the 2008 first-quarter report notes. By comparison, the aggregate score for private retail, brokerage and travel sites was 81.6 out of 100.
Interestingly, federal sites that provide transactional services saw a bump in customer satisfaction. Overall, for such government sites, ACSI reports a 1.5 percent increase in satisfaction. “The significant score increase for this group of sites is an indication that citizens are eager for government sites to evolve from information sources to conduits for conducting business with the government,” the report notes.
Susceptible to Tampering
Many reputation systems for online services currently exist — such as those used by Amazon, eBay and Epinions — but they are all fairly rudimentary and susceptible to manipulation, according to a study by a team of researchers at Queensland University of Technology in Brisbane, Australia, and the College of Information Technology in Malaysia.
The study, "A Survey of Trust and Reputation Systems for Online Service Provision," points to seven chief issues that OASIS will have to overcome in creating a standard reputation system approach:
- low incentives for providing ratings;
- bias toward positive ratings;
- unfair ratings;
- change of identities that let poorly rated people, services or organizations disguise their identity;
- quality variations over time;
- discrimination that singles out individual users or rare blips in service;
- ballot-box stuffing.
According to the OASIS committee, a chief solution to these hurdles is to create a standard framework for all reputation data captured online. The committee says such a framework will let users “acquire raw reputation data and calculate their own reputation values, either using their personal experience or with the help of data aggregators.”
How It Might Work
This diagram, created by committee member Daniela Bourges-Waldegg of IBM, illustrates in principle how such a system would work:
The use of reputation tools to create a community online around federal sites as well as create a degree of trust among users could play to the government’s favor, based on the ACSI findings: “Any incoming administration will certainly be looking for ways to cut costs, and encouraging citizens to interact with the government online rather than through more costly channels is certainly an effective way to do so.”
