WikiTrust

WikiTrust (Alternative names for this tool)
Keyword(s)	reputation, quality, reliability, histories
Operating system(s)	Unknown [+]
Language(s)	English
Programming language(s)	Unknown [+]
Author(s)	Luca de Alfaro, Bo Adler, Ian Pye
License(s)	New BSD License, GPL
Website	http://sites.google.com/site/ucscwikitrust/home, https://addons.mozilla.org/en-US/firefox/addon/wikitrust/, https://github.com/collaborativetrust/WikiTrust
Related material
Related tool(s)	Unknown [+]
Related dataset(s)	Unknown [+]
Search
Google Scholar
Export and share
BibTeX, CSV, RDF, JSON
Browse properties · List of tools

WikiTrust is an open-source, on-line reputation system for Wikipedia authors and content.

Publications

Title	Author(s)	Keyword(s)	Published in	Language	DateThis property is a special property in this wiki.	Abstract
Towards Content-driven Reputation for Collaborative Code Repositories	Andrew G. West Insup Lee	WikiTrust Wiki Code repository SVN Reputation Trust management Content persistence Code quality	WikiSym	English	August 2012	As evidenced by SourceForge and GitHub, code repositories now integrate Web 2.0 functionality that enables global participation with minimal barriers-to-entry. To prevent detrimental contributions enabled by crowdsourcing, reputation is one proposed solution. Fortunately this is an issue that has been addressed in analogous version control systems such as the *wiki* for natural language content. The WikiTrust algorithm ("content-driven reputation"), while developed and evaluated in wiki environments operates under a possibly shared collaborative assumption: actions that "survive" subsequent edits are reflective of good authorship. In this paper we examine WikiTrust's ability to measure author quality in collaborative code development. We first define a mapping from repositories to wiki environments and use it to evaluate a production SVN repository with 92,000 updates. Analysis is particularly attentive to reputation loss events and attempts to establish ground truth using commit comments and bug tracking. A proof-of-concept evaluation suggests the technique is promising (about two-thirds of reputation loss is justified) with false positives identifying areas for future refinement. Equally as important, these false positives exemplify differences in content evolution and the cooperative process between wikis and code repositories.
Multilingual Vandalism Detection using Language-Independent & Ex Post Facto Evidence	Andrew G. West Insup Lee		PAN-CLEF	English	September 2011	There is much literature on Wikipedia vandalism detection. However, this writing addresses two facets given little treatment to date. First, prior efforts emphasize zero-delay detection, classifying edits the moment they are made. If classification can be delayed (e.g., compiling offline distributions), it is possible to leverage ex post facto evidence. This work describes/evaluates several features of this type, which we find to be overwhelmingly strong vandalism indicators. Second, English Wikipedia has been the primary test-bed for research. Yet, Wikipedia has 200+ language editions and use of localized features impairs portability. This work implements an extensive set of language-independent indicators and evaluates them using three corpora (German, English, Spanish). The work then extends to include language-specific signals. Quantifying their performance benefit, we find that such features can moderately increase classifier accuracy, but significant effort and language fluency are required to capture this utility. Aside from these novel aspects, this effort also broadly addresses the task, implementing 65 total features. Evaluation produces 0.840 PR-AUC on thezero-delay task and 0.906 PR-AUC with ex post facto evidence (averaging languages). Performance matches the state-of-the-art (English), sets novel baselines (German, Spanish), and is validated by a first-place finish over the 2011 PAN-CLEF test set.
Evaluating WikiTrust: A trust support tool for Wikipedia	Teun Lucassen Jan Maarten Schraagen		First Monday	English	May 2011	Because of the open character of Wikipedia readers should always be aware of the possibility of false information. WikiTrust aims at helping readers to judge the trustworthiness of articles by coloring the background of less trustworthy words in a shade of orange. In this study we look into the effects of such coloring on reading behavior and trust evaluation by means of an eye–tracking experiment. The results show that readers had more difficulties reading the articles with coloring than without coloring. Trust in heavily colored articles was lower. The main concern is that the participants in our experiment rated usefulness of WikiTrust low.
Wikipedia Vandalism Detection: Combining Natural Language, Metadata, and Reputation Features	B. Thomas Adler Luca de Alfaro Santiago M. Mola Velasco Paolo Rosso Andrew G. West	Wikipedia Wiki Collaboration Vandalism Machine learning Metadata Natural Language Processing Reputation	Lecture Notes in Computer Science	English	February 2011	Wikipedia is an online encyclopedia which anyone can edit. While most edits are constructive, about 7% are acts of vandalism. Such behavior is characterized by modifications made in bad faith; introducing spam and other inappropriate content. In this work, we present the results of an effort to integrate three of the leading approaches to Wikipedia vandalism detection: a spatio-temporal analysis of metadata (STiki), a reputation-based system (WikiTrust), and natural language processing features. The performance of the resulting joint system improves the state-of-the-art from all previous methods and establishes a new baseline for Wikipedia vandalism detection. We examine in detail the contribution of the three approaches, both for the task of discovering fresh vandalism, and for the task of locating vandalism in the complete set of Wikipedia revisions.
Assigning Trust to Wikipedia Content	B. Thomas Adler Krishnendu Chatterjee Luca de Alfaro Marco Faella Ian Pye Vishwanath Raman		WikiSym	English	2008	The Wikipedia is a collaborative encyclopedia: anyone can contribute to its articles simply by clicking on an "edit" button. The open nature of the Wikipedia has been key to its success, but has also created a challenge: how can readers develop an informed opinion on its reliability? We propose a system that computes quantitative values of trust for the text in Wikipedia articles; these trust values provide an indication of text reliability. The system uses as input the revision history of each article, as well as information about the reputation of the contributing authors, as provided by a reputation system. The trust of a word in an article is computed on the basis of the reputation of the original author of the word, as well as the reputation of all authors who edited text near the word. The algorithm computes word trust values that vary smoothly across the text; the trust values can be visualized using varying text-background colors. The algorithm ensures that all changes to an article's text are reflected in the trust values, preventing surreptitious content changes. We have implemented the proposed system, and we have used it to compute and display the trust of the text of thousands of articles of the English Wikipedia. To validate our trust-computation algorithms, we show that text labeled as low-trust has a significantly higher probability of being edited in the future than text labeled as high-trust.
Measuring Author Contributions to the Wikipedia	B. Thomas Adler Luca de Alfaro Ian Pye Vishwanath Raman		WikiSym	English	2008
A content-driven reputation system for the Wikipedia	B. Thomas Adler Luca de Alfaro			English	2007	We present a content-driven reputation system for Wikipedia authors. In our system, authors gain reputation when the edits they perform to Wikipedia articles are preserved by subsequent authors, and they lose reputation when their edits are rolled back or undone in short order. Thus, author reputation is computed solely on the basis of content evolution; user-to-user comments or ratings are not used. The author reputation we compute could be used to flag new contributions from low-reputation authors, or it could be used to allow only authors with high reputation to contribute to controversialor critical pages. A reputation system for the Wikipedia could also provide an incentive for high-quality contributions. We have implemented the proposed system, and we have used it to analyze the entire Italian and French Wikipedias, consisting of a total of 691,551 pages and 5,587,523 revisions. Our results show that our notion of reputation has good predictive value: changes performed by low-reputation authors have a significantly larger than average probability of having poor quality, as judged by human observers, and of being later undone, as measured by our algorithms.