Tag: field experiment

Narratives and Expert Information in Agenda-Setting: Experimental Evidence on State Legislator Engagement with Artificial Intelligence Policy

by Daniel S. Schiff & Kaylyn Jackson Schiff

Previous scholarship has investigated how policy entrepreneurs use narratives and expert information to influence policy agendas. In particular, narratives can be powerful tools for communicating policy problems and solutions, while expert information can help clarify complicated subject matters and increase confidence in policy proposals. This raises a question: can policy entrepreneurs effectively use narratives to influence policymakers even in complex, technical policy domains where we might think the technical details might be traditionally most important?

We explore this question in the context of artificial intelligence (AI) policy – an emerging policy domain that is highly technical and multi-faceted, with social, ethical, economic implications. Because the agenda for AI policy is still in the process of development, it presents a ripe case for understanding agenda setting and policy influence efforts. In partnership with a leading AI think tank, The Future Society (TFS), we conducted a field experiment on state legislators across the United States. Emails about AI policy were sent to 7,355 legislative offices. Legislators were randomly assigned to receive an email containing either a narrative strategy, an expertise strategy, or generic, neutral information. We also considered two ways of issue framing: ethical and economic/competition (see Figure 1). 

Legislators were presented with either a fact sheet or story, and invited to register for and attend a webinar about AI for state legislators, which we hosted in December 2021. For example, legislators (or their staffers) might read an email message about an individual falsely arrested due to facial recognition, or between a geopolitical contest between the US and China.

We measured link clicks and webinar registration and attendance as proxies for policymaker engagement. Using these data on engagement with the emails, we tested the following hypotheses:

  • Policy Entrepreneur Effectiveness Hypothesis: The provision of narratives or expertise by policy entrepreneurs will increase policymaker attention to and engagement with the policy issue at hand.
  • Dominance of Narratives Hypothesis: The provision of narratives will induce greater policymaker engagement than the provision of expertise.
  • Dominance of Expertise Hypothesis: The provision of expertise will induce greater policymaker engagement than the provision of narratives.
  • Strategies by Issue Framing Hypothesis: Policymakers will respond with greater engagement to narratives when they are provided issue frames emphasizing the ethical and social dimensions of AI as compared to issue frames emphasizing the economic and technological competitiveness dimensions of AI.
  • Prior Experience Hypothesis: Compared to legislators in states with greater prior experience in AI policymaking, legislators in states with less experience with AI will respond with greater engagement to the expertise treatment.

Consistent with the Policy Entrepreneur Effectiveness Hypothesis, we found that narrative strategies and expert information increased engagement with the emails (see Figure 3). Interestingly, comparing the narrative and expertise treatments, we found no statistically significant differences in their effects on engagement, suggesting that narratives are as effective as expert information even for this complex policy domain. 

Figure 3. Both expert information and narratives engaged state legislators as compared to a more generic ‘control’ message, with increased engagement of 30 or more percentage points.

Contrary to our expectations, framing the issue to emphasize ethical or economic dimensions of AI also did not affect engagement, suggesting that the use of strategies like narratives can be effective even when AI policy is framed in very different ways. We had hypothesized that narratives might be especially effective when an ethics-focused policy frame of AI is promoted, but it appears narratives are just as effective when geopolitical and strategic dimensions of AI policy are emphasized. 

Finally, legislators with no prior experience with AI policy were more likely to engage with the emails than legislators who had considered or passed AI policy in the past, and state legislatures with higher capacity (e.g., more staff, longer sessions) were far more likely to the email messages, an important note for those seeking to reach out to policymakers..

Our findings show that narratives can influence policymakers as much as expertise, even in complicated policy domains like AI. It is worth noting that our data was collected in 2021 before the introduction of large language models (LLMs), like OpenAI’s ChatGPT, which gained unprecedented public attention. This development has surely influenced the salience of AI policy. We suggest that future research should consider this development. Nevertheless, our work makes important contributions by extending the NPF to new contexts and investigating narratives using field experiments, a novel research approach in the field.

You can read the original article in Policy Studies Journal at

Schiff, Daniel S. and Kaylyn Jackson Schiff. 2023. “ Narratives and Expert Information in Agenda-setting: Experimental Evidence on State Legislator Engagement With Artificial Intelligence Policy.” Policy Studies Journal 51(4): 817–842. https://doi.org/10.1111/psj.12511.

About the Authors

Dr. Daniel Schiff is an Assistant Professor of Technology Policy at Purdue University’s Department of Political Science and the Co-Director of GRAIL, the Governance and Responsible AI Lab. He studies the formal and informal governance of AI through policy and industry, as well as AI’s social and ethical implications in domains like education, manufacturing, finance, and criminal justice.

Follow him on X/Twitter: @Dan_Schiff (@purduepolsci and @Purdue)

Kaylyn Jackson Schiff is an Assistant Professor in the Department of Political Science at Purdue University and Co-Director of the Governance and Responsible AI Lab (GRAIL). Her research addresses the impacts of emerging technologies on government and society. She studies how technological developments are changing citizen-government contact, and she explores public opinion on artificial intelligence in government.

Follow her on X/Twitter: @kaylynjackson

, , , , ,

Policy Feedback via Economic Behavior: A Model and Experimental Analysis of Consumption Behavior

by Gregory S. Schober

Policy feedback scholars have done extensive work to understand how public policies affect mass behavior (the feed) and subsequent policy outcomes (the back). Thus far, this literature has focused mainly on political behavior feeds. However, the impact of these policies extends beyond the realm of politics, influencing the economic behavior of individuals and, in turn, shaping future policy outcomes.

In my paper, I develop a policy feedback model of consumption behavior in mass publics. Illustrated in Figure 1, the model shows how policies influence consumption capacity and preferences, which in turn affect future policy decisions. For example, social assistance policies transfer resources to beneficiaries, thus altering their spending decisions and influencing government policy responses (see path A-C-F-H).

I use this theory to investigate how targeted cash assistance policies (TCAPs) influence not just the immediate consumption patterns but also the subsequent policies. To do this, I analyzed the effects of Progresa—a Mexican TCAP that aimed to reduce poverty—by utilizing data collected during a randomized field experiment. I performed downstream analysis on the data to estimate the effects of Progresa.

In the short term, Progresa positively influenced private consumption of basic utilities. When households received the cash transfers, they used them to purchase private access to drainage (via septic tanks). However, in the medium term, a startling shift occurred. In communities where Progresa was implemented and private access to drainage increased, the government began making less of an effort to maintain the public water system.

My work offers key insights into the complex relationship between short-term consumption changes and (unintended) medium-term policy outcomes. It emphasizes that while consumption effects did occur swiftly, leading to increased private access to drainage, the subsequent impact on government policy ultimately led to reductions in basic utility access.

An intriguing question arises when considering the medium-term results: are targeted cash assistance policies—which generally are administered at the national level—letting local governments off the hook in terms of basic utility provision? When program beneficiaries use cash transfers to invest in private access to basic utilities, they in turn may be disincentivizing local governments from investing in public utility systems.

The implications of this study reverberate across various domains of policy feedback research. It highlights the need to broaden the scope of policy feedback analysis beyond political spheres to include economic mechanisms. These findings prompt further exploration into how economic feeds could influence future political behavior and policy outcomes.

In conclusion, this research breaks new ground by unraveling the ripple effects of social assistance policies, shedding light on how they influence consumption patterns and government policies regarding basic utilities. Understanding these intricate dynamics between policy, consumption behavior, and subsequent governance decisions is crucial for designing effective, holistic policies that address poverty while ensuring sustained access to essential services for vulnerable populations.

You can read the original article in Policy Studies Journal at

Schober, Gregory S. 2023. “Policy feedback via economic behavior: A model and experimental analysis of consumption behavior.” Policy Studies Journal 51: 607–627. https://doi.org/10.1111/psj.12474.

About the Author

Gregory S. Schober is an Assistant Professor in the Rehabilitation Sciences Program at The University of Texas at El Paso. His research examines social policy, political and economic behavior, and health in developing countries and the United States.


, , , ,