Research
Working Papers
- Personalized rankings and user engagement: an empirical evaluation of the Reddit news feedAlex MoehringJob market paper, 2023
Digital platforms increasingly curate their content through personalized algorithmic rankings. Given the limited attention of their users and reliance on advertising, platforms have an incentive to promote content that increases the predicted engagement of each user. However, managers must also balance maximizing total engagement with the quality of content promoted on the platform due to advertiser concerns over brand safety and to satisfy policymakers. This paper studies how maximizing engagement for each user affects the quality of content with which users engage to understand the extent to which engagement-maximizing algorithms promote and incentivize low-quality content. In addition, I evaluate how the ranking algorithm itself can be designed to promote and encourage engagement with high quality content. To do this, I study the Reddit politics community and exploit a novel discontinuity – revealed in Reddit’s code repository – in how the ranking algorithm orders posts to identify the effect of a post’s rank on the number of comments it receives. I use this discontinuity to identify a discrete choice model of user comment decisions and estimate the distribution of news that users are exposed to and comment on under a personalized algorithm that maximizes engagement. This counterfactual demonstrates that personalization drives a wedge between users in terms of the quality of content – the credibility rating of an article’s publisher – with which are exposed and engage. Under the personalized ranking algorithm, users who ordinarily engage with high-credibility publishers continue to do so. However, users who ordinarily engage with lower-credibility publishers are exposed to and engage with an even larger share of low-credibility publishers under the personalized engagement-maximizing algorithm. Finally, I evaluate a credibility-aware algorithm that explicitly promotes credible news publishers and find that moving to the credibility-maximizing algorithm reduces total engagement by 5.0%, a meaningful decline. Yet, platforms can increase the share of the average user’s engagement with high-credibility publishers by 6.8 percentage points for only a 2.0% decrease in engagement. These findings suggest that algorithmic interventions can be a useful tool to promote higher-quality content to help satisfy both advertisers and policy makers.
- Combining human expertise with artificial intelligence: Experimental evidence from radiologyNikhil Agarwal , Alex Moehring, Pranav Rajpurkar , and Tobias Salz2023
While Artificial Intelligence (AI) algorithms have achieved performance levels comparable to human experts on various predictive tasks, human experts can still access valuable contextual information not yet incorporated into AI predictions. Humans assisted by AI predictions could outperform both human-alone or AI-alone. We conduct an experiment with professional radiologists that varies the availability of AI assistance and contextual information to study the effectiveness of human-AI collaboration and to investigate how to optimize it. Our findings reveal that (i) providing AI predictions does not uniformly increase diagnostic quality, and (ii) providing contextual information does increase quality. Radiologists do not fully capitalize on the potential gains from AI assistance because of large deviations from the benchmark Bayesian model with correct belief updating. The observed errors in belief updating can be explained by radiologists’ partially underweighting the AI’s information relative to their own and not accounting for the correlation between their own information and AI predictions. In light of these biases, we design a collaborative system between radiologists and AI. Our results demonstrate that, unless the documented mistakes can be corrected, the optimal solution involves assigning cases either to humans or to AI, but rarely to a human assisted by AI.
- Social influence and news consumptionAlex Moehring, and Carlos Molina2023
Populations in several countries have become decidedly more polarized over the last decades. Many believe that social media, which facilitates interactions within echo chambers, is partly to blame. These interactions can trigger two distinct effects on the demand for biased news. First, individuals could be influenced by the news consumption of their peers, for example, because they value keeping an ideologically congruent news diet with their peers. Second, individuals might purposefully skew their news consumption preferences, anticipating that their choices will be observed by their peers. We design a field experiment on Twitter to separately identify the importance of both mechanisms. Our experiment induces variation in both an individual’s perceptions of the political leanings of their peers’ news consumption and the visibility of their own news preferences to their social media followers. We track participants’ sharing behavior and news consumption, proxied by the news outlets that the participants follow. Our main result is that participants alter their news diet when they believe these choices will be observed by their peers, as in the second channel above. We find that individuals primarily value presenting themselves as following a balanced set of news which attenuates the demand for biased news. In contrast, we do not find evidence in favor of the first channel: our experimental variation does influence beliefs about the news diets of their peers but participants do not respond by changing their own news diets. In sum, we find little evidence supporting the hypothesis that, at least through the lenses of these two mechanisms, online interactions with like-minded people are a major contributor to the demand for polarized news content. Instead, our paper uncovers one mechanism for which we should expect the opposite: as social media platforms amplify the visibility of user interactions, the importance of social image concerns increases and, as a result, users adjust their news consumption to be more balanced.
- Information frictions and heterogeneity in valuations of personal dataAvinash Collis , Alex Moehring, Ananya Sen , and Alessandro Acquisti2021
We investigate how consumer valuations of personal data are affected by real world information interventions. Proposals to compensate users for the information they disclose to online services have been advanced in both research and policy circles. These proposals may be hampered by information frictions that limit consumers’ ability to assess the value of their own data. We use an incentive-compatible mechanism to capture consumers’ willingness to share their social media data for monetary compensation and estimate distributions of valuations of social media data, before and after an information treatment. We find evidence of significant dispersion and heterogeneity in valuations before the information intervention, with women, Black, and low income individuals reporting systematically lower valuations than other groups. In both samples, the provision of information leads to a reduction in dispersion in data valuations. The reduction takes the form of increasing valuations by low-valuation individuals—in which women, low income, and Black users are over-represented. These participants with low valuations and high probability of revisions belong to groups that are traditionally associated with low levels of digital literacy. The findings suggest that strategies aimed at increasing information availability in markets for personal data may affect consumer welfare gains from data markets.
- News feeds and user engagement: Evidence from the reddit news tabAlex Moehring2022
We study how the introduction of a new non-personalized news feed impacts user engagement quantity, quality, and diversity on Reddit. In June 2018, Reddit introduced the News tab on iOS devices that surfaces popular content from a curated list of news-related communities. We leverage this natural experiment to identify the causal effects of the News tab on iOS user engagement in a difference-in-differences design. We find that the News tab increases the share of iOS devices that engage with news-related content and the new engagement is not meaningfully different in quality from existing engagement. Additionally, we find that the diversity of engagement within news categories and within articles from publishers across the political spectrum increases as a result of the News tab. These results suggest that non-personalized feeds can be an important tool to mitigate algorithmic filter bubbles.
Published Articles
- Providing normative information increases intentions to accept a COVID-19 vaccineAlex Moehring, Avinash Collis , Kiran Garimella , M Amin Rahimian , and 2 more authorsNature Communications, 2023
Despite the availability of multiple safe vaccines, vaccine hesitancy may present a challenge to successful control of the COVID-19 pandemic. As with many human behaviors, people’s vaccine acceptance may be affected by their beliefs about whether others will accept a vaccine (i.e., descriptive norms). However, information about these descriptive norms may have different effects depending on the actual descriptive norm, people’s baseline beliefs, and the relative importance of conformity, social learning, and free-riding. Here, using a pre-registered, randomized experiment (N = 484,239) embedded in an international survey (23 countries), we show that accurate information about descriptive norms can increase intentions to accept a vaccine for COVID-19. We find mixed evidence that information on descriptive norms impacts mask wearing intentions and no statistically significant evidence that it impacts intentions to physically distance. The effects on vaccination intentions are largely consistent across the 23 included countries, but are concentrated among people who were otherwise uncertain about accepting a vaccine. Providing normative information in vaccine communications partially corrects individuals’ underestimation of how many other people will accept a vaccine. These results suggest that presenting people with information about the widespread and growing acceptance of COVID-19 vaccines helps to increase vaccination intentions.
- Global survey on COVID-19 beliefs, behaviours and normsAvinash Collis , Kiran Garimella , Alex Moehring, M Amin Rahimian , and 6 more authorsNature Human Behaviour, 2022
Policy and communication responses to COVID-19 can benefit from better understanding of people’s baseline and resulting beliefs, behaviours and norms. From July 2020 to March 2021, we fielded a global survey on these topics in 67 countries yielding over 2 million responses. This paper provides an overview of the motivation behind the survey design, details the sampling and weighting designed to make the results representative of populations of interest and presents some insights learned from the survey. Several studies have already used the survey data to analyse risk perception, attitudes towards mask wearing and other preventive behaviours, as well as trust in information sources across communities worldwide. This resource can open new areas of enquiry in public health, communication and economic policy by leveraging large-scale, rich survey datasets on beliefs, behaviours and norms during a global pandemic.
- Interdependence and the cost of uncoordinated responses to COVID-19David Holtz , Michael Zhao , Seth G Benzell , Cathy Y Cao , and 7 more authorsProceedings of the National Academy of Sciences, 2020
Social distancing is the core policy response to coronavirus disease 2019 (COVID-19). But, as federal, state and local governments begin opening businesses and relaxing shelter-in-place orders worldwide, we lack quantitative evidence on how policies in one region affect mobility and social distancing in other regions and the consequences of uncoordinated regional policies adopted in the presence of such spillovers. To investigate this concern, we combined daily, county-level data on shelter-in-place policies with movement data from over 27 million mobile devices, social network connections among over 220 million Facebook users, daily temperature and precipitation data from 62,000 weather stations, and county-level census data on population demographics to estimate the geographic and social network spillovers created by regional policies across the United States. Our analysis shows that the contact patterns of people in a given region are significantly influenced by the policies and behaviors of people in other, sometimes distant, regions. When just one-third of a state’s social and geographic peer states adopt shelter-in-place policies, it creates a reduction in mobility equal to the state’s own policy decisions. These spillovers are mediated by peer travel and distancing behaviors in those states. A simple analytical model calibrated with our empirical estimates demonstrated that the “loss from anarchy” in uncoordinated state policies is increasing in the number of noncooperating states and the size of social and geographic spillovers. These results suggest a substantial cost of uncoordinated government responses to COVID-19 when people, ideas, and media move across borders.