policy agendas – Policy Studies Journal Blog

by Amnon Cavari, Akos Mate, & Miklós Sebők

In a representative democracy like the United States, we expect that the policy priorities expressed by politicians on the campaign trail or in stump speeches reflect the same priorities that they pursue while in office. We further expect that politicians would continue their policy commitment in their programmatic messages as well as in their daily activities and speeches. When they do that, we say that they stay on the democratic script.

We test this proposition, focusing on the relationship between the programmatic addresses of US presidents and the daily speeches by comparing the annual State of the Union address (SOU) with subsequent day-to-day speeches, which we refer to as occasional remarks (ORs). Occasional remarks are crucial because they allow the president to show the electorate that they are following through on their promises. They can also serve as testing grounds for new ideas or messages.

Using the American Presidency Project, we gathered all State of the Union addresses and occasional remarks for every president from Harry Truman to Donald Trump. We then coded the documents using the codebook of the Comparative Agendas Project, which defines 20 policy categories. Because of the large volume of documents that made up our dataset (16,523 speeches divided into nearly 2 million sentences), we used a large language model to conduct the bulk of our coding, supplemented with some manual coding used to train and refine our language model.

We used the coded data to test three hypotheses:

H1: The policy agenda of the most important programmatic speech (SOU) and of routine remarks (ORs) each year will be positively correlated.
H2: The correlation between the policy agenda of the most important programmatic speech (SOU) and that of the routine remarks (ORs) will steadily decline over the course of the year.
H3a: Major domestic and foreign events decrease the diversity of the presidents’ routine attention (measured in ORs) relative to that presented in strategic communication (based on SOU).
H3b: The effect of domestic and foreign events on the diversity of routine agenda would be conditioned on the diversity of the annual agenda in the SOU.

Figure 1. Correlations between speech types by policy topic.

As the above figure illustrates, across the 20 coded policy topics, there’s a strong correlation between the topics that are emphasized in State of the Union addresses and those that subsequently appear in occasional remarks, giving credence to Hypothesis 1. Applying regression analysis, we also found that, as time goes on, the policy topics addressed in occasional remarks diverge from those emphasized in the State of the Union, supporting Hypothesis 2.

As for Hypothesis 3, we found a positive correlation between the diversity of policy topics referenced in the State of the Union and those referenced in occasional remarks; but, in contrast to our expectation, we do not find that major events (e.g., foreign conflicts, natural disasters, etc.) have a major impact on shifting the focus of presidential remarks.

Our results show that, generally speaking, U.S. presidents are staying on the democratic script: The policy priorities that they outline in their State of the Union addresses are the same priorities to which they return in subsequent remarks. By comparing State of the Union addresses to occasional remarks, we’ve shown a link between programmatic and occasional communications that may have broader applicability beyond the presidency. We have also demonstrated the value of using large language models for parsing large volumes of policy texts, as our model’s coding displayed a higher accuracy rating than our manual coders, and at a fraction of the time. There are numerous avenues for building upon the insights outlined here, including examining the relationship between speechmaking and public opinion, how different speech types intersect with the policymaking process, and exploring the populations exposed to these speeches and how they respond to the speeches.

You can read the original article in Policy Studies Journal at

Cavari, Amnon, Akos Mate and Miklós Sebők. 2024. “ Staying on the Democratic Script? A Deep Learning Analysis of the Speechmaking of U.S. Presidents.” Policy Studies Journal 52(4): 709–729. https://doi.org/10.1111/psj.12534.

About the Authors

Amnon Cavari is Associate Professor and head of the Institute for Liberty and Responsibility at the Lauder School of Government, Diplomacy and Strategy at Reichman University, Israel. Prof. Cavari’s main research interests are in the interrelationship between actions of elected officials and public opinion in the United States and in Israel. He is the author of The Party Politics of Presidential Rhetoric.
Twitter/X: @ACavari

Akos Mate is a computational social scientist whose research interests are political economics and quantitative methodology. He is a research fellow at the Centre for Social Sciences, Budapest. He also teaches as visiting faculty at the Central European University, Vienna, and served as a consultant for the IMF’s Independent Evaluation Office.

Twitter/X: @aakos_m

Miklós Sebők is a Senior Research Fellow at the HUN-REN Centre for Social Sciences (CSS), Budapest. He serves as the research co-director of the Artificial Intelligence National Lab at CSS, the principal investigator of the V-SHIFT Momentum research project, and the convenor of the COMPTEXT conference. His main research interest lies at the intersection of policy studies and natural language processing.
Twitter/X: @Miklos_Sebok

content analysis, deep learning, large language models, policy agendas, state of the union addresses

by E. J. Fagan, Alexander Furnas, Chris Koski, Herschel Thomas, Samuel Workman, & Corinne Connor

The Policy Studies Journal (PSJ) is the premier destination for scholars who apply and advance theories of the public policy process. As such, the work published in the journal reflects important trends and priorities in the policy community. In our article, we examine the agenda of PSJ over the last three decades in an effort to understand the evolving focus of the discipline and contribute to the emerging “Science of Science” literature. To do this, we analyze over 1,300 abstracts from PSJ articles, using the Comparative Agendas Project (CAP) codebook to systematically categorize the policy topics covered.

One of the key insights from our analysis is the broad range of policy topics covered in PSJ. With the exception of a major focus on environmental policy, attention is roughly divided among a variety of different topics. While the substantive range of the scholarship in PSJ is encouraging, we do identify a recent decline in topic diversity and a lack of attention on areas like foreign policy.

Figure 1. Subfigure (a) shows the distribution of paper across policy topics from our coding of abstract text. Subfigure (b) shows the normalized Shannon’s H of the topic codings for papers published in PSJ over time. We exclude the “No Substantive Topic” category from the calculation of Shannon’s H so the measure reflects the diversity of PSJ papers across substantive policy topics. The years 1986–1990 are excluded from this plot because there are no coded papers due to the lack of available abstracts in OpenAlex.

We also examine the theoretical frameworks that have shaped policy process research within the PSJ. Notably, Punctuated Equilibrium Theory (PET) emerged as a dominant framework, appearing more frequently in the journal than other influential theories like the Advocacy Coalition Framework (ACF) and the Institutional Analysis and Development (IAD) Framework. We attribute PET’s extensive presence in the journal to the theory’s broad applicability across various policy domains. Additionally, we find that theories of the policy process tend to specialize in specific areas, such as ACF’s focus on environmental and energy policy.

In terms of the policy stages addressed in PSJ articles, our findings indicate a strong emphasis on the implementation and evaluation stages of the policy cycle. This is particularly interesting given that one might expect PET’s focus on agenda-setting and lawmaking to lead to greater attention to these stages. Instead, the journal’s content has increasingly shifted from stage-based analysis to a more theory-driven approach.

Figure 2. Subfigure (a) shows the distribution of PSJ papers across policy theories identified using keywords in the abstract. Subfigure (b) shows the stage of the policymaking cycle papers addressed and identified using keywords in the abstract.

Another important aspect of our study is the impact of PSJ articles on both academic research and policy-making. We find that articles addressing general policy processes or theoretical questions tend to receive more academic citations, while those focused on specific policy areas, such as education, are more likely to be cited in policy documents. This distinction highlights the dual role that the journal plays in both advancing theoretical understanding and informing practical policy decisions.

Figure 3. Subfigure (a) plots the share of PSJ papers within each topic compared to the share of CRS reports on each policy topic for 1997–2019. Subfigure (b) plots the share of PSJ papers received by papers within each topic compared to the share of think tank reports from four prominent think tanks on each policy topic for 2007–2017.

Finally, we compare the journal’s focus with the priorities of other policy experts, including those from the Congressional Research Service (CRS) and various think tanks. Our analysis reveals significant discrepancies in the attention given to certain issues. For instance, while PSJ articles emphasize environmental policy, CRS reports are more likely to focus on defense and government operations. This divergence suggests that policy process scholars sometimes prioritize different issues than those that dominate the agendas of policymakers and other experts.

Through this study, we aim to shed light on the dynamics of issue attention within the field of policy process scholarship. By doing so, we hope to contribute to a deeper understanding of how scholarly priorities evolve and how they align—or sometimes fail to align—with the broader needs of society.

You can read the original article in Policy Studies Journal at

Fagan, E. J., Alexander Furnas, Chris Koski, Herschel Thomas, Samuel Workman and Corinne Connor. 2024. “ The Dynamics of Issue Attention in Policy Process Scholarship.” Policy Studies Journal 52 (3): 481–492. https://doi.org/10.1111/psj.12548.

About the Authors

E.J. Fagan is an Assistant Professor of Political Science at the University of Illinois Chicago. He studies agenda setting, think tanks, political parties and policymaking in the U.S. Congress.

Alexander C. Furnas Ph.D. (Zander) is a Research Assistant Professor at the Center for Science of Science and Innovation at the Kellogg School of Management, Northwestern University, and Faculty Associate at the Institute for Policy Research and the Ryan Center on Complexity. He has a PhD in political science from the University of Michigan. He researches the political economy of information, with a focus on the production, dissemination and uptake of science and expertise in the policymaking process. His work has been published in American Political Science Review, American Journal of Political Science, Policy Studies Journal, Proceedings of the National Academy of Sciences, and Legislative Studies Quarterly, among others.

Chris Koski joined the Reed College faculty in Fall 2011 after four years as an assistant professor at James Madison University (2007–2011). His research interests include many aspects of the policy process, with a particular focus on agenda-setting, policy design, and implementation. Theoretically, much of his work is situated in punctuated equilibrium theory (PET) and the social construction framework (SCF). Substantively, the bulk of Chris’ research is focused on environmental policy, most recently the politics of climate change – mitigation, adaptation, and geoengineering. He has also published work on homeland security policy and the politics of state budgeting.

Herschel F. Thomas is an Associate Professor of Public Affairs in the LBJ School of Public Affairs at the University of Texas at Austin. He is a faculty fellow of the RGK Center for Philanthropy and Community Service and faculty affiliate of the Policy Agendas Project. His research examines US national institutions and policy processes, with an emphasis on the role of civil society in shaping public policy decision-making and outcomes. His work focuses on interest group politics, public health, and agenda-setting, and is published in journals such as the American Journal of Public Health, Policy Studies Journal, Public Administration, Political Research Quarterly, and Risk, Hazards & Crisis in Public Policy, among others. He is co-author with Timothy LaPira of Revolving Door Lobbying.

Samuel Workman is Professor of political science and Director of the Institute for Policy Research and Public Affairs at West Virginia University. His area of expertise is constructing large data infrastructures to answer fundamental questions about public policy across time and space. His previous work has addressed public policy, regulation, and how governments generate and use information. His work emphasizes text-as-data, machine learning, and statistical modeling, especially classification. His work appears in the top public policy and public administration journals, including Policy Studies Journal, Journal of Public Administration Research and Theory, and Policy and Politics. He is the author of The Dynamics of Bureaucracy (Cambridge, 2015), Punctuated Equilibrium Theory: A Policy Theory of Politics (Cambridge, Forthcoming), and Co-Editor of Methods of the Policy Process (Routledge, 2022).

Corinne Connor is a Program Analyst for The Heinz Endowments in Pittsburgh, PA. She is a former affiliate of the Institute for Policy Research and Public Affairs at West Virginia University and received her MA in Political Science from WVU in May 2023.

policy agendas, policy cycle, policy process, policy stages, policy theory, science of science