2019 UCR Culture Survey

View the full results of the 2019 UCR Culture Survey here.


Methodology

140 individuals submitted entries from 135 different regions. In the event of duplicate submissions, we generally used the submission submitted by a Regional Officer as the ‘official’ submission from the region, though all data still exists. When looking at top-line results, all regions are included, however, when considering some comparisons that involved percentages (specifically when looking at how certain off-site properties impact regional health), we look only at regions with at least 20 nations.[su_tooltip]Hover me to open tooltip[/su_tooltip]

In 2018’s survey, which will be referenced periodically in this article, 52 individuals submitted entries for 47 different regions. This poses a challenge when making comparisons between 2018 and 2019, as 2019’s survey was advertised on the NS Gameplay forum and via telegram to all User-Created Region (UCR) WA Delegates, while 2018’s survey was only advertised on the NS Gameplay forum and in the NS Leaders Discord server. This difference in the survey population is worth keeping in mind as comparisons are made across the two different years.

There were four regions who answered the survey in both 2018 and 2019. When faced with questions of survey population disparity, I will occasionally look at the changes between these specific regions (who I’ll refer to as the ‘focus group’) between 2018 and 2019 as a sort of ‘gut check’ to see if the changes among these static regions mirror the changes we’re seeing among two questionably analogous datasets.

Results

In a game filled with countless communities all with unique characteristics and flavors, it seems like a gross oversimplification to ask regions to label themselves as either gameplay, roleplay, or social. The overwhelming majority of gameplay or roleplay regions are going to have social components, and roleplay regions may still care about gameplay elements like the World Assembly. Even regions that are highly social could hold regional elections or discuss surprising outcomes of issues — these labels are not black and white.

But while these labels may be a gross shorthand for more complicated realities, asking people for their perception of the core purpose of their regions does serve as an interesting introduction to these surveys from 2018 and 2019, and the state of UCRs today. In 2018, a slim plurality of UCRs identified as ‘almost entirely’ or ‘mostly’ gameplay. In 2019, a resounding plurality (nearly a majority) of regions identified as ‘almost entirely’ or ‘mostly’ social-oriented.


External link to graph image


It would be remiss to not point out the discrepancy of survey populations. Distribution of the survey to regional leaders in the NS Founders Discord in 2018 could have collected responses from a population more inclined toward gameplay. The focus group isn’t of much help parsing out the noise here, as 3 out of 4 regions identified as social in 2018, so it’s tough to test if gameplay and roleplay regions are trending toward socialization.

The sheer scale of these numbers, however, suggest that the trend is happening to one degree or another. I found the decline in roleplay regions particularly surprising, though, as I thought they would have more representation in 2019 with the telegram advertising of the survey. 

When looking at what platforms regions are using, this is where the survey population disparity between 2018 and 2019 becomes quite stark. Because Discord itself was a huge medium of recruiting participants in 2018, there was probably a false sense of its domination in that year. Plenty of regions do not participate on any offsite platforms, or participate on other platforms like WhatsApp, Instagram, or gathers in real-life between friends. That being said, we should be cautious to not ‘unskew’ these findings too much over survey technique. When looking at our focus group, they largely show the same trends.


External link to graph image

External link to graph image


If this data is to be believed, the story isn’t just a rising use of instant-chat Discord over threaded forums. The story is the continued decline of forums, but also an increased frequency of regions who don’t have Discord platforms either, perhaps due to a lack in the level of activity required to support it.

Regardless of platforms used, when it came to the type of government governing UCRs, there was remarkable consistency. Democracies remain the most popular regional governments, with a mixed system noticeably behind. More regions characterized themselves as having ‘no real government’ in 2019, but I suspect that variation is (once again) due to the difference in survey advertisements. All-in-all, an impressive amount of stability.


External link to graph image


But what about the performance and output of those governments? With all the talk about a decline in NationStates, how are individual UCRs faring? We posed two questions in numerical terms, asking respondents to estimate how many citizens have the ambition to serve in regional government, and what per cent of the region seemed interested in doing essential tasks to keep it alive. On both counts, we saw significant declines from 2018 to 2019.


External link to graph image

External link to graph image


When we asked respondents a more qualitative question, to characterize the culture of their regional government, we saw a 7% increase in the frequency of the most pessimistic statement, which is that virtually no members seem interested in building the region and members seem more interested in idle chatter and off-site activities, with only a fleeting desire to build up a region. Overall on the qualitative statement, we saw a ten-point swing from the two most positive characterizations to the two most negative ones.

Analysis

There is no perfect dataset in NationStates, and the data collected in the 2018 and 2019 UCR Surveys are certainly far from perfect. However, in broad strokes, the data supports the assertion of a more social-oriented gameplay, filled with UCRs that are less effective at being active and perpetuating themselves.

Finally, while much dialogue has been focused on the impact of Discord, there is early evidence that, in limited instances, some regions are struggling to even maintain offsite Discord servers because activity problems are so dire. None of this may be a surprise to many NS Gameplay observers, however, many of the causes of this downturn are up for debate. Much has been discussed of the role of Discord, and the changing nature of off-site platforms.


External link to graph image


One tool that we will use to make additional observations is the Pearson Correlation Coefficient, which measures the linear correlation of two variables between the numbers -1 (perfect negative relationship) and 1 (perfect positive relationship). We will be primarily examining positive relationships, where coefficients of 0.1-0.3 are seen as having a small association, coefficients of 0.3-0.5 are seen as having medium associations, and coefficient of 0.5+ are seen as having strong associations.

While there is no significant correlation between regional health metrics measured in the 2019 survey and merely having off-site platforms, there are some interesting trends when it comes to regional participation on those platforms. Regions with a forum participation rate (forum members as a proportion of the total population) of 10% or more had 8% more regional members doing vital tasks to keep the region alive than those with less than 10%, totalling 36% and 28% respectively. Comparatively, there was no significant disparity between regions above and below a 10% Discord participation rate.

There was a strong correlation coefficient (0.50) between a region’s total active member rate (total active members as a proportion of the total population), and the forum participation rate. The correlation was much weaker between total active member rate and Discord participation rate (0.20).

Finally, and notably, there was a small but significant positive correlation (0.30) between a region’s forum participation rate and share of a region performing tasks that keep the region alive. There was no significant correlation between the share of region performing essential tasks and the Discord participation rate. This may mean that UCRs get significant development benefits from getting game-side nations on an offsite forum, but not when pushing the same nations to a Discord server. 

Conclusion

There were more than a few comments on this survey making the point that this poll was measuring factors that would be relevant to only a certain subset of players. That’s to say that I come from a school of thought where ‘success’ in NationStates means building, growing a region, and increasing prestige and relevance across the game. That’s not how all UCRs operate. If your measure of success in this game is tracking your nation’s changes in response to issues, or chatting with old friends on the Regional Message Board, many themes in this survey may have been confusing or even frustrating. After all, there are many versions of this game that could never truly ‘die’ unless Max Barry decided to pull the plug and close up shop.

But I would argue that for those who play the game as I do — or even for passionate UCR-based roleplayers — there are certainly warning signs for how the UCR world is faring. New nations are being founded every day, but UCRs are increasingly struggling to get those nations to do much other than add one or two NationStates servers to their Discord Rolodex. Indicators of UCR health are down from 2018, and some regions are struggling to even maintain an offsite Discord server because of activity problems.

It is tough to imagine a world where Discord is ever put back into a box and, even if it could be, I highly doubt that would actually improve outcomes for UCRs. Too many passionate gamers find Discord to be a familiar tool after years of adoption, and it’s very possible that Discord is what has helped NationStates communities evolve into the mobile-first age.

But if regions are spending time and energy advertising for and maintaining their Discord servers without any immediate measurable benefits to the community, does that suggest there’s a better way to organize our tools and structure our communities? I don’t know, that’s totally up to you.

HEM

HEM has been playing NationStates since 2006, and has notably been the founder of Europeia and the former Vice Delegate of The South Pacific. Throughout his career, HEM has kept a special focus on journalism and has had various media outlets in Europeia, The South Pacific, The North Pacific, The Land of Kings and Emperors, and many others. He has penned controversial essays that have made waves, including 'Debunking Defenderism' and 'The Deceit of Defenderism', as well as biting satire, like 'Albion Excited For New Trollish Regional Outlook', that has (proudly) caused foreign relations crises.