Topic Model For A ‘North African Role Model’ – Analysis

Using Latent Dirilecht Allocation (LDA) and UN General Debate Corpus (UNGDC) to Automate Moroccan Policy Analysis

Since 1946, world leaders have used the annual general debate of the United Nations General Assembly to address the challenges they deem most pertinent to their country and the world. In choosing between the possible topics to discuss, they offer their listeners insight into their government’s prioritization.

Officials from Morocco have spoken at every general debate since the country formally gained independence from France and joined the UN in 1956. From then until now, the kingdom has seen several regimes, increasingly participated on the global stage, and dealt with domestic and international issues including border conflicts and political riots. Morocco’s annual general debate speeches provide a window into the government’s internal thought process throughout those developments.

A corpus containing transcripts of the general debate speeches of every UN GA member nation between 1970 and 2020 has been made available and continually updated by a group of European researchers. Analyzing all of Morocco’s speeches in that timeframe by hand is an arduous process that can be sped up through topic modeling. Latent Dirilecht Allocation (LDA) is a generative probability algorithm that attempts to identify salient topics from a group of documents. The model aims to find topics as specific and distinct from others as possible using a linear discriminant function. The algorithm was first proposed in 2000 and has since become the most common method for topic modeling. Applied to time-stamped documents, it can also be used to extract trends in the distribution of topics over time.

Traditional analysis of government decision-making requires pouring through countless legislative documents, announcements, and interviews to summarize how lawmakers have shifted their priorities over time. Because of the revealing nature of general debate speeches, we wanted to see whether analysis with LDA can be an automated alternative.

Fitting LDA on the speech corpus, we can ask two important questions:

1. How coherently can LDA identify unique topics within Morocco’s general debate transcripts?

2. How well do the LDA-identified trends in Morocco’s general debate speeches align with traditional analysis of Moroccan policy changes?

On their own, the speeches show what the government intends to emphasize to the international community. Contextualizing the topic trends with changes in economic and political conditions as well as budgetary and policy decisions allows us to ask two more questions:

3. What events and trends have caused shifts in Morocco’s rhetoric towards the international community?

4. Has the government written policy and put money behind the issues it has emphasized?

Methodology

The speech transcripts were lengthy but already broken up into sections based on the speaker’s delivery. Each section was treated as a separate document, totaling 2,094 documents in all. As is recommended for topic modeling, pre-processing was limited to stopword removal and tokenization. The LDA model comes from the popular Gensim library.

The intertopic distance map provides a visual representation (fig. 1) from the fitted model of topic coherence and distinctiveness by simplifying the multidimensional organization of topics. Through parameterization, topic circles should be as small and separated as possible. Fifty years of speeches certainly cover more than the seven topics shown, but identifying more than seven led to excessive overlap. Determining a summary for each topic based on the words it contained was simple. This is evidence that the topics are sufficiently coherent, a claim also supported by UMass coherence scoring.

The topics can be summarized as follows:

1. The peaceful resolution of international disputes not involving Morocco

2. The current issues Morocco is working on in the given year (1)

3. Sustainability and sustainable development

4. Morocco’s vision and goals for the future

5. Multilateralism in the face of global threats, i.e., terrorism and COVID-19 (2)

6. Emphasis on practicality and a call for international realism

7. Climate change

Topic Trends

Timeseries analysis reveals patterns in the popularity of topics over time. Notable is which topics trend in tandem and which move in opposition. For most of the time span, ‘multilateralism’, ‘sustainability’, ‘current’, and ‘disputes’ seem to move together (fig. 2). The other three topics, ‘vision’, ‘realism’, and ‘climate change’, move as their own group in the opposite direction.

The past 20 years contained the most volatility in topic relevance. In 2002, ‘sustainability’ started to grow quickly in importance. It increasingly became Morocco’s primary topic until ‘disputes’, which started rising in 2006, slightly surpassed it in 2012. Both, however, fell in the second half of the period. ‘Vision’, ‘realism’, and ‘climate change’ moved in the opposite direction: losing importance to ‘sustainability’ and ‘disputes’ and then gaining back relevance when they fell. ‘Multilateralism’ and ‘current’ have made small movements since 2000, though ‘multilateralism’ has become the most popular topic in the past two years.

The drastic increase in ‘sustainability’ in the early-2000s is supported by traditional analysis of Moroccan domestic policy. Researchers focused on the launch of the Initiative Nationale pour le Développement Humain (INDH) in 2005 as Morocco’s primary poverty reduction program. From 2005 to 2010, the INDH operated with a budget of 10 million dirhams that largely went towards participatory revenue-generating projects. The impact of the program on poverty is debated, but it still represents a major push for sustainability coinciding with an increase in ‘sustainability’ in Morocco’s speeches – evidence of Morocco following through on its UN speeches.

Our model indicates that Morocco decided to permanently turn away from sustainable development in 2008. Traditional policy analysis, however, reveals that not to be the case. Researchers point to the expansion of the INDH as a sign of continued interest in sustainability. The first phase of the INDH concluded in 2010 and the second phase, operating from 2011 until 2018, had an 80% larger budget. This indicates a failure of our algorithm to automate analysis.

Two possible explanations can be made for the discrepancy:

1. Economic conditions impact Morocco’s international emphasis on sustainability but not its legislative priorities.

Moroccan fiscal policy in the early-2000s focused on maintaining stability in public debt after the volatility that characterized the prior 10 years. That stability was met with sustained growth in GDP per capita, rising from $1,334.90 in 2000 to $2,890.40 USD in 2008. That upward trend line, which is the steepest in Morocco’s recorded history, strongly correlates with the ‘sustainability’ topic growth in the same period (fig. 3).

In fact, GDP per capita seems to correlate with ‘sustainability’ popularity for years before this period. More interestingly, GDP per capita growth stagnated in 2008 with the financial crisis and ‘multilateralism’ collapsed. Correlation does not necessarily indicate causation, but the 2008 crisis was universally felt and certainly had an impact on Morocco’s international rhetoric. 

2. Morocco’s increased emphasis on ‘multilateralism’ caused ‘sustainability’ to fall.

Topic modeling speeches reveals the relative popularity of talking points. Speakers are held to 15 minutes, so if the government intends to increase its emphasis on a particular issue, the relative popularity of at least one other topic must decrease. That doesn’t necessarily indicate, however, that the government has curtailed its actions on the decreasing topic(s). In this case, the decrease in ‘sustainability’ was concurrent with an increase in ‘multilateralism’.

Responding to unrest during the Arab spring of 2011, the Moroccan government promised to decentralize its authority and increase political participation. Though much criticism has been leveled at the realization of those promises, decentralization has become an important framework within which the government communicates its goals. That change in language is one reason that ‘multilateralism’ and ‘vision’ replaced ‘sustainability’ in topic popularity after 2011 even as the government put more money into sustainable development.

The increased emphasis on multilateralism was in part due to Morocco’s ongoing response to increased terrorism in the North African region. The Arab Spring brought instability to much of the Middle East and North Africa (MENA) region, drastically increasing the frequency of domestic terror attacks (Kim & Sandler, 2020). Between 2007 and 2021, MENA accounted for the most terrorist attacks in the world by region at 49,674. That surpassed the second-highest region, South Asia, by over 10,000 attacks. 

Though Morocco didn’t suffer as many attacks as some of its neighbors, it became a leader and major advocate for multilateral cooperation in the region. In 2015, the Mohammed VI Institute was founded in Rabat to educate future imams in an effort to reduce radicalization. The institute has grown larger each year, taking in students from countries across North Africa and France. In 2016, Morocco successfully petitioned to become co-chair of the Global Counterterrorism Forum (GCTF). Regional cooperation, however, has been impeded by the continued dispute with Algeria over Western Sahara. 

Also a part of Morocco’s effort to expand foreign relations, trade has increased in tandem with the ‘multilateralism’ topic increase. More recently, Morocco has called for increased cooperation within MENA to combat the COVID-19 pandemic. Relative to the region, Morocco has done well at mitigating the spread within its borders and has sent medical supplies to nearby nations that have comparatively struggled.

Results

Our results indicate that LDA analysis is an effective tool for recognizing topics within the general debate transcripts. The topics our model identified after parameterization are coherent and have limited overlap. The ideas associated with each topic seem to cover what would be expected of the content of the speeches over the time span.

The changes in topic popularity contextualized with economic conditions and major events tell many stories of Moroccan decision-making and prioritization that largely align with traditional analysis from research groups and non-governmental organizations. Trends in ‘sustainability’ through the early-2000s and ‘multilateralism’ over the past decade are well-founded in relevant literature. 

However, our LDA time-series analysis often fails to establish when a trend in one topic is caused by a shift in the government’s attitudes or a trend in another topic. In doing so, it may incorrectly lead us to believe the government has changed stances on an issue. This is demonstrated by the decrease in ‘sustainability’ popularity after 2010 conflicting with an increase in funding into sustainable development. This indicates that topic modeling may best be used as a supplement to traditional political analysis rather than as a replacement. Still, it introduces automation to a task that typically requires multiple researchers and extensive reading. Solutions to investigate through further research include incorporating sentiment analysis to recognize when a topic is fading/rising in popularity but still looked upon positively/negatively. A more certain solution would be to analyze legislative documents where topics are more likely to maintain their popularity despite changes in others.

Still, our topic coherence indicates that LDA is an effective tool for the analysis of Moroccan general debate speeches. Through an automated process, we can accurately generate trends in Morocco’s rhetoric towards the international community. The rest of the UN corpus merits further analysis to analyze trends on a global scale and relationships between countries’ prioritizations. For example, training our model on all MENA countries would allow us to see how well Morocco has aligned with its neighbors while it has pushed for regional cooperation. The algorithm is not only faster than traditional analysis of foreign policy but also has the ability to recognize micro-trends that would likely go otherwise unnoticed.

Our success in topic modelling Morocco’s language to the international community allows us to pass judgement on the sincerity of its communication. The results largely show Morocco making an effort to bring its international promises to fruition. Increased emphasis on topics was matched by increased funding and policy changes on those topics. That consistent follow-through has allowed Morocco to continue to serve as a major non-NATO ally for the United States and many European nations. 

However, perhaps more thought should be put into maximizing program effectiveness rather than budget. The INDH has faced criticism for a lack of communication between stakeholders and minimal community-based participation compared to how the program was originally advertised. If Morocco wishes to speak on sustainability, it should make sure its programs can serve as a model for other nations. Otherwise, its language appears insincere and insignificant. And one area where the government has failed to do much beyond make promises: decentralization. Political repression has increased dramatically in recent years with numerous false allegations made against journalists critical of the government. Morocco promised to reduce authoritarianism after the 2011 uprisings but has instead used it repeatedly to silence opposition. As unrest continues to grow steadily at the unfulfilled vows, the government must make meaningful change, or the international community will have no reason to listen.

Amit Vallabh is a rising senior and Jefferson Scholar at the University of Virginia studying applied statistics and interning for the High Atlas Foundation in Marrakech, Morocco.

Endnotes:

1.  Morocco has long dealt with conflict surrounding the contested region of Western Sahara. This topic often contains that conflict. The ambiguity likely comes from Morocco’s cautious language regarding its stance, which historically has been at odds with much of the international community.
2.  Terrorism and the pandemic are difficult to disentangle as they have inspired very similar calls for international cooperation from Morocco in recent years.