John Riedl

From WikiPapers
Jump to: navigation, search

John Riedl is an author.

Publications

Only those publications related to wikis are shown here.
Title Keyword(s) Published in Language DateThis property is a special property in this wiki. Abstract R C
Tell me more: An actionable quality model for wikipedia Classication
Flaw detection
Information quality
Machine learning
Modelling
Wikipedia
Proceedings of the 9th International Symposium on Open Collaboration, WikiSym + OpenSym 2013 English 2013 In this paper we address the problem of developing actionable quality models for Wikipedia, models whose features directly suggest strategies for improving the quality of a given article. We rst survey the literature in order to understand the notion of article quality in the context of Wikipedia and existing approaches to automatically assess article quality. We then develop classication models with varying combinations of more or less actionable features, and nd that a model that only contains clearly actionable features delivers solid performance. Lastly we discuss the implications of these results in terms of how they can help improve the quality of articles across Wikipedia. Categories and Subject Descriptors H.5 [Information Interfaces and Presentation]: Group and Organization InterfacesCollaborative computing, Computer-supported cooperative work, Web-based interac- Tion. Copyright 2010 ACM. 0 0
The Rise and Decline of an Open Collaboration System: How Wikipedia's Reaction to Popularity Is Causing Its Decline Governance
Peer production
Quality control
Retention
Wikipedia
American Behavioral Scientist English 2013 Open collaboration systems, such as Wikipedia, need to maintain a pool of volunteer contributors to remain relevant. Wikipedia was created through a tremendous number of contributions by millions of contributors. However, recent research has shown that the number of active contributors in Wikipedia has been declining steadily for years and suggests that a sharp decline in the retention of newcomers is the cause. This article presents data that show how several changes the Wikipedia community made to manage quality and consistency in the face of a massive growth in participation have ironically crippled the very growth they were designed to manage. Specifically, the restrictiveness of the encyclopedia's primary quality control mechanism and the algorithmic tools used to reject contributions are implicated as key causes of decreased newcomer retention. Furthermore, the community's formal mechanisms for norm articulation are shown to have calcified against changes-especially changes proposed by newer editors. 0 0
In Search of the Ur-Wikipedia: Universality, Similarity, and Translation in the Wikipedia Inter-Language Link Network Wikipedia
Tobler's Law
First Law of Geography
WikiSym English August 2012 Wikipedia has become one of the primary encyclopaedic information repositories on the World Wide Web. It started in 2001 with a single edition in the English language and has since expanded to more than 20 million articles in 283 languages. Criss-crossing between the Wikipedias is an interlanguage link network, connecting the articles of one edition of Wikipedia to another. We describe characteristics of articles covered by nearly all Wikipedias and those covered by only a single language edition, we use the network to understand how we can judge the similarity between Wikipedias based on concept coverage, and we investigate the flow of translation between a selection of the larger Wikipedias. Our findings indicate that the relationships between Wikipedia editions follow Tobler's first law of geography: similarity decreases with increasing distance. The number of articles in a Wikipedia edition is found to be the strongest predictor of similarity, while language similarity also appears to have an influence. The English Wikipedia edition is by far the primary source of translations. We discuss the impact of these results for Wikipedia as well as user-generated content communities in general. 0 0
Bots and cyborgs: Wikipedia's immune system Robot
Cyborgs
Social computing
Wikipedia
Computer English 2012 Bots and cyborgs are more than tools to better manage content quality on Wikipediathrough their interaction with humans, they're fundamentally changing its culture. 0 0
In search of the ur-Wikipedia: Universality, similarity, and translation in the Wikipedia inter-language link network First law of geography
Tobler's law
Wikipedia
WikiSym 2012 English 2012 Wikipedia has become one of the primary encyclopaedic information repositories on the World Wide Web. It started in 2001 with a single edition in the English language and has since expanded to more than 20 million articles in 283 languages. Criss-crossing between the Wikipedias is an inter-language link network, connecting the articles of one edition of Wikipedia to another. We describe characteristics of articles covered by nearly all Wikipedias and those covered by only a single language edition, we use the network to understand how we can judge the similarity between Wikipedias based on concept coverage, and we investigate the flow of translation between a selection of the larger Wikipedias. Our findings indicate that the relationships between Wikipedia editions follow Tobler's first law of geography: similarity decreases with increasing distance. The number of articles in a Wikipedia edition is found to be the strongest predictor of similarity, while language similarity also appears to have an influence. The English Wikipedia edition is by far the primary source of translations. We discuss the impact of these results for Wikipedia as well as user-generated content communities in general. 0 0
Searching for the goldilocks zone: Trade-offs in managing online volunteer groups Online volunteer group
Productivity
Trade-off
Wikipedia
Withdrawal
English 2012 Dedicated and productive members who actively contribute to community efforts are crucial to the success of online volunteer groups such as Wikipedia. What predicts member productivity? Do productive members stay longer? How does involvement in multiple projects affect member contribution to the community? In this paper, we analyze data from 648 WikiProjects to address these questions. Our results reveal two critical trade-offs in managing online volunteer groups. First, factors that increase member productivity, measured by the number of edits on Wikipedia articles, also increase likelihood of withdrawal from contributing, perhaps due to feelings of mission accomplished or burnout. Second, individual membership in multiple projects has mixed effects. It decreases the amount of work editors contribute to both the individual projects and Wikipedia as a whole. It increases withdrawal for each individual project yet reduces withdrawal from Wikipedia. We discuss how our findings expand existing theories to fit the online context and inform the design of new tools to improve online volunteer work. 0 0
Some of all human knowledge: Gender and participation in peer production Gender
Open source
Peer production
Wikipedia
English 2012 The promise of peer production includes resources produced by volunteers and released freely for the world to use. Wikipedia and Open Source Software are famous examples of peer-produced projects. Anyone is free to participate, but not everybody does. Wikipedia aims to collect the "sum of all human knowledge", but only about 13% of editors on the site are female [3]. In Open Source Software, the percentage of female contributors has been estimated near 1% [4]. If women are not well represented among authors of the most widely accessed reference source on the planet, are important voices muted? Could these projects be even more impactful with more female participation? This panel includes experts in gender theory and open collaboration, activists, and representatives from peer-produced projects to discuss recent findings and trends in this complex and often contentious research space. 0 0
Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work WikiWork
Wikipedia
Experience
Motivation
Productivity
Quality
Revert
WikiSym English 2011 Reverts are important to maintaining the quality of Wikipedia. They fix mistakes, repair vandalism, and help enforce policy. However, reverts can also be damaging, especially to the aspiring editor whose work they destroy. In this research we analyze 400,000 Wikipedia revisions to understand the effect that reverts had on editors. We seek to understand the extent to which they demotivate users, reducing the workforce of contributors, versus the extent to which they help users improve as encyclopedia editors. Overall we find that reverts are powerfully demotivating, but that their net influence is that more quality work is done in Wikipedia as a result of reverts than is lost by chasing editors away. However, we identify key conditions – most specifically new editors being reverted by much more experienced editors – under which reverts are particularly damaging. We propose that reducing the damage from reverts might be one effective path for Wikipedia to solve the newcomer retention problem. 0 2
Mentoring in Wikipedia: A clash of cultures Mentoring
User retention
Wikipedia
WikiSym 2011 Conference Proceedings - 7th Annual International Symposium on Wikis and Open Collaboration English 2011 The continuous success of Wikipedia depends upon its capability to recruit and engage new editors, especially those with new knowledge and perspectives. Yet Wikipedia over the years has become a complicated bureaucracy that may be difficult for newcomers to navigate. Mentoring is a practice that has been widely used in offline organizations to help new members adjust to their roles. In this paper, we draw insights from the offline mentoring literature to analyze mentoring practices in Wikipedia and how they influence editor behaviors. Our quantitative analysis of the Adopt-a-user program shows mixed success of the program. Communication between adopters and adoptees is correlated with the amount of article editing done by adoptees shortly after adoption. Our qualitative analysis of the communication between adopters and adoptees suggests that several key functions of mentoring are missing or not fulfilled consistently. Most adopters focus on establishing their legitimacy rather than acting proactively to guide, protect, and support the long-term growth of adoptees. We conclude with recommendations of how Wikipedia mentoring programs can evolve to take advantage of offline best practices. 0 2
Mentoring in Wikipedia: a clash of cultures Wikipedia
Mentoring
User retention
WikiSym English 2011 0 2
NICE: Social translucence through UI intervention Contribution
Experiment
Interface
Revert
Wikipedia
WikiSym 2011 Conference Proceedings - 7th Annual International Symposium on Wikis and Open Collaboration English 2011 Social production systems such as Wikipedia rely on attracting and motivating volunteer contributions to be successful. One strong demotivating factor can be when an editor's work is discarded, or "reverted", by others. In this paper we demonstrate evidence of this effect and design a novel interface aimed at improving communication between the reverting and reverted editors. We deployed the interface in a controlled experiment on the live Wikipedia site, and report on changes in the behavior of 487 contributors who were reverted by editors using our interface. Our results suggest that simple interface modifications (such as informing Wikipedians that the editor they are reverting is a newcomer) can have substantial positive effects in protecting against contribution loss in newcomers and improving the quality of work done by more experienced contributors. 0 0
The Past, Present, and Future of Wikipedia Social computing
Wikipedia
Nupedia
Citizendium
Computer English 2011 0 1
WP:clubhouse? An exploration of Wikipedia's gender imbalance Collaboration
Content coverage
Gender gap
Wikipedia
WikiSym 2011 Conference Proceedings - 7th Annual International Symposium on Wikis and Open Collaboration English 2011 Wikipedia has rapidly become an invaluable destination for millions of information-seeking users. However, media reports suggest an important challenge: only a small fraction of Wikipedia's legion of volunteer editors are female. In the current work, we present a scientific exploration of the gender imbalance in the English Wikipedia's population of editors. We look at the nature of the imbalance itself, its effects on the quality of the encyclopedia, and several conflict-related factors that may be contributing to the gender gap. Our findings confirm the presence of a large gender gap among editors and a corresponding gender-oriented disparity in the content of Wikipedia's articles. Further, we find evidence hinting at a culture that may be resistant to female participation. 0 1
WP:clubhouse?: an exploration of Wikipedia's gender imbalance Wikipedia
Collaboration
Content coverage
Gender gap
WikiSym English 2011 0 1
The effects of diversity on group productivity and member withdrawal in online volunteer groups Diversity
Online volunteer group
Performance
Wikipedia
Conference on Human Factors in Computing Systems - Proceedings English 2010 The "wisdom of crowds" argument emphasizes the importance of diversity in online collaborations, such as open source projects and Wikipedia. However, decades of research on diversity in offline work groups have painted an inconclusive picture. On the one hand, the broader range of insights from a diverse group can lead to improved outcomes. On the other hand, individual differences can lead to conflict and diminished performance. In this paper, we examine the effects of group diversity on the amount of work accomplished and on member withdrawal behaviors in the context of WikiProjects. We find that increased diversity in experience with Wikipedia increases group productivity and decreases member withdrawal - up to a point. Beyond that point, group productivity remains high, but members are more likely to withdraw. Strikingly, no such diminishing returns were observed for differences in member interest, which increases productivity and decreases member withdrawal in a linear fashion. Our results suggest that the low visibility of individual differences in online groups may allow them to harvest more of the benefits of diversity while bearing less of the cost. We discuss how our findings can inform further research of online collaboration. 0 0
The effects of group composition on decision quality in a social production community Collaboration
Decision making
Small groups
Wikipedia
Proceedings of the 16th ACM International Conference on Supporting Group Work, GROUP'10 English 2010 Online social production communities allow efficient construction of valuable and high-quality information sources. To be successful, community members must be effective at collaboration, including makink collective decisions in the presence of disagreement. We examined over 100,000 decisions made by small working groups in Wikipedia, and analyzed how decision quality in these online groups is influenced by four group composition factors: the size of the group, how members were invited to the group, the prior experience of group members, and apparent bias shown by the group administrator. Our findings lead us to recommendations for designers of social production communities. 0 1
A Jury of your Peers: Quality, Experience and Ownership in Wikipedia Wikipedia peer peer review wikiwork experience ownership quality WikiSym2009: Symposium on Wikis and Open Collaboration 2009 Wikipedia is a highly successful example of what mass collaboration in an informal peer review system can accomplish. In this paper, we examine the role that the quality of the contributions, the experience of the contributors and the ownership of the content play in the decisions over which contributions become part of Wikipedia and which ones are rejected by the community. We introduce and justify a versatile metric for automatically measuring the quality of a contribution. We find little evidence that experience helps contributors avoid rejection. In fact, as they gain experience, contributors are even more likely to have their work rejected. We also find strong evidence of ownership behaviors in practice despite the fact that ownership of content is discouraged within Wikipedia. 0 6
A jury of your peers: Quality, experience and ownership in Wikipedia Experience
Ownership
Peer
Peer review
Quality
Wikipedia
WikiWork
WikiSym English 2009 Wikipedia is a highly successful example of what mass collaboration in an informal peer review system can accomplish. In this paper, we examine the role that the quality of the contributions, the experience of the contributors and the ownership of the content play in the decisions over which contributions become part of Wikipedia and which ones are rejected by the community. We introduce and justify a versatile metric for automatically measuring the quality of a contribution. We find little evidence that experience helps contributors avoid rejection. In fact, as they gain experience, contributors are even more likely to have their work rejected. We also find strong evidence of ownership behaviors in practice despite the fact that ownership of content is discouraged within Wikipedia. Copyright 0 6
Altruism, selfishness, and destructiveness on the social web Lecture Notes in Computer Science English 2008 Many online communities are emerging that, like Wikipedia, bring people together to build community-maintained artifacts of lasting value (CALVs). What is the nature of people's participation in building these repositories? What are their motives? In what ways is their behavior destructive instead of constructive? Motivating people to contribute is a key problem because the quantity and quality of contributions ultimately determine a CALV's value. We pose three related research questions: 1) How does intelligent task routing-matching people with work-affect the quantity of contributions? 2) How does reviewing contributions before accepting them affect the quality of contributions? 3) How do recommender systems affect the evolution of a shared tagging vocabulary among the contributors? We will explore these questions in the context of existing CALVs, including Wikipedia, Facebook, and MovieLens. 0 0
Creating, Destroying, and Restoring Value in Wikipedia Wikipedia Department of Computer Science and Engineering University of Minnesota 2007 Wikipedia's brilliance and curse is that any user can edit any of the encyclopedia entries. We introduce the notion of the impact of an edit, measured by the number of times the edited version is viewed. Using several datasets, including recent logs of all article views, we show that an overwhelming majority of the viewed words were written by frequent editors and that this majority is increasing. Similarly, using the same impact measure, we show that the probability of a typical article view being damaged is small but increasing, and we present empirically grounded classes of damage. Finally, we make policy recommendations for Wikipedia and other wikis in light of these findings. 0 12
Creating, destroying, and restoring value in Wikipedia English 2007 Wikipedia's brilliance and curse is that any user can edit any of the encyclopedia entries. We introduce the notion of the impact of an edit, measured by the number of times the edited version is viewed. Using several datasets, including recent logs of all article views, we show that an overwhelming majority of the viewed words were written by frequent editors and that this majority is increasing. Similarly, using the same impact measure, we show that the probability of a typical article view being damaged is small but increasing, and we present empirically grounded classes of damage. Finally, we make policy recommendations for Wikipedia and other wikis in light of these findings. 0 12
SuggestBot: Using intelligent task routing to help people find work in wikipedia Intelligent task routing
Member-maintained communities
Online community
Recommender system
Wikipedia
International Conference on Intelligent User Interfaces, Proceedings IUI English 2007 Member-maintained communities ask their users to perform tasks the community needs. From Slashdot, to IMDb, to Wikipedia, groups with diverse interests create community-maintained artifacts of lasting value (CALV) that support the group's main purpose and provide value to others. Said communities don't help members find work to do, or do so without regard to individual preferences, such as Slashdot assigning meta-moderation randomly. Yet social science theory suggests that reducing the cost and increasing the personal value of contribution would motivate members to participate more.We present SuggestBot, software that performs intelligent task routing (matching people with tasks) in Wikipedia. SuggestBot uses broadly applicable strategies of text analysis, collaborative filtering, and hyperlink following to recommend tasks. SuggestBot's intelligent task routing increases the number of edits by roughly four times compared to suggesting random articles. Our contributions are: 1) demonstrating the value of intelligent task routing in a real deployment; 2) showing how to do intelligent task routing; and 3) sharing our experience of deploying a tool in Wikipedia, which offered both challenges and opportunities for research. Copyright 2007 ACM. 0 1
SuggestBot: using intelligent task routing to help people find work in Wikipedia Online community
Member-maintained communities
Wikipedia
Intelligent task routing
Recommender system
English 2007 Member-maintained communities ask their users to perform tasks the community needs. From Slashdot, to IMDb, to Wikipedia, groups with diverse interests create community-maintained artifacts of lasting value (CALV) that support the group's main purpose and provide value to others. Said communities don't help members find work to do, or do so without regard to individual preferences, such as Slashdot assigning meta-moderation randomly. Yet social science theory suggests that reducing the cost and increasing the personal value of contribution would motivate members to participate more. We present SuggestBot, software that performs intelligent task routing (matching people with tasks) in Wikipedia. SuggestBot uses broadly applicable strategies of text analysis, collaborative filtering, and hyperlink following to recommend tasks. SuggestBot's intelligent task routing increases the number of edits by roughly four times compared to suggesting random articles. Our contributions are: 1) demonstrating the value of intelligent task routing in a real deployment; 2) showing how to do intelligent task routing; and 3) sharing our experience of deploying a tool in Wikipedia, which offered both challenges and opportunities for research. 0 1
Using intelligent task rooting and contribution review to help communities build artifacts of lasting value Contribution models
Editorial review
Intelligent task routing
Member-maintained
Online community
Wikipedia
Conference on Human Factors in Computing Systems - Proceedings English 2006 Many online communities are emerging that, like Wikipedia, bring people together to build community-maintained artifacts of lasting value (CALVs). Motivating people to contribute is a key problem because the quantity and quality of contributions ultimately determine a CALVs value. We pose two related research questions: 1) How does intelligent task routing - matching people with work - affect the quantity of contributions? 2) How does reviewing contributions before accepting them affect the quality of contributions? A field experiment with 197 contributors shows that simple, intelligent task routing algorithms have large effects. We also model the effect of reviewing contributions on the value of CALVs. The model predicts, and experimental data shows, that value grows more slowly with review before acceptance. It also predicts, surprisingly, that a CALV will reach the same final value whether contributions are reviewed before or after they are made available to the community. Copyright 2006 ACM. 0 0