How to automate your Lego Universe Server Backups and Updates

In the previous article, we learned how to set up a Lego Universe Server using the Darkflame Universe project. In this guide we will explore how we can automate our server’s backups and updates to run the server as autonomously as possible.

Repository Setup

First, and perhaps most importantly, we are going to handle server backups. This includes your users’ data, as well as your server configurations for a quick reboot after a server update.

An easy way of achieving this is using a Github repository (note: we will not be storing any of the server’s build files on our repository) — here the Github repository is called LegoUniverseServer. Therefor, go onto Github and create a new blank repository and then clone it onto your Lego Universe server’s machine.

Navigate into the cloned repository, and create the following directories (my repo is called LegoUniverseServer):

LegoUniverseServer/
├── build/
├── darkflame_database/
├── fix_database/
├── ini_files/
├── scripts/
└── server_status/

The scripts folder is unused in this guide, but can house the account management tools found on my Gist.

SQL Import/Export

To back up and restore our users’ data on the fly, were going to automate these operations first. Navigate into the fit_database/ directory and create a file export_database with the following contents:

rm ../darkflame_database/darkflame.sql
mysqldump -u dflame -pdflame darkflame > ../darkflame_database/darkflame.sql
cd ../darkflame_database/ && ./backup_to_discord

(*) The last line is optional, and allows uploading the exported database to discord, such that your users can anytime leave the server without having their data lost (user passwords are excluded from this, more on that later).

Then create another file called import_database with the following contents:

sudo service mysql start
mysql -u dflame -pdflame darkflame < ../darkflame_database/darkflame.sql

Then set both scripts to be marked as executable

parent_dir/LegoUniverseServer/fix_database ~> chmod +x ./export_database
parent_dir/LegoUniverseServer/fix_database ~> chmod +x ./import_database

With both of these scripts configured, you can now export the database using ./export_database and re-import it using ./import_database

Automating Server Updates

This part is a little bit more tricky, as the server automatically overwrites any files during the update process. ini files sometimes get updated and settings may not persist through updates. Therefore, were going to work around that.

Head back to the root of your new repository and create a file called updateServer with the following contents:

pkill MasterServer
cd ../Github/DarkflameServer
git pull
git submodule update --recursive
cd ../../LegoUniverseServer/build
sed -i -e 's/add_subdirectory(tests)/#add_subdirectory(tests)/g' ../CMakeLists.txt
cmake .. && cmake --build . --config Release -j4
cd ..
git checkout CMakeLists.txt
./server_status/discord_status_notify updated > /dev/null 2>&1

As you may see, this script assumes that the DarkflameServer repository is present within the parent directory of your current repository. To adjust for this, navigate out of your current repository, then create a child directory called Github and move the DarkflameServer directory into the Github directory. At the end, you should have the following folder structure:

parent_dir/
├── Github/
|   └── DarkflameServer/
|       └── ...
└── LegoUniverseServer/

Navigate back into your new repository and create a second file (at the root level) called fixlnks with the following contents:

rm CMakeLists.txt
rm CMakeVariables.txt
rm dAuthServer
rm dChatFilter
rm dChatServer
rm dCommon
rm dDatabase
rm dGame
rm dMasterServer
rm dNet
rm dPhysics
rm dScripts
rm dWorldServer
rm dZoneManager
rm migrations
rm resources
rm thirdparty
rm vanity
rm versions.txt
ln -s ../Github/DarkflameServer/CMakeLists.txt CMakeLists.txt
ln -s ../Github/DarkflameServer/CMakeVariables.txt CMakeVariables.txt
ln -s ../Github/DarkflameServer/dAuthServer dAuthServer
ln -s ../Github/DarkflameServer/dChatFilter dChatFilter
ln -s ../Github/DarkflameServer/dChatServer dChatServer
ln -s ../Github/DarkflameServer/dCommon dCommon
ln -s ../Github/DarkflameServer/dDatabase dDatabase
ln -s ../Github/DarkflameServer/dGame dGame
ln -s ../Github/DarkflameServer/dMasterServer dMasterServer
ln -s ../Github/DarkflameServer/dNet dNet
ln -s ../Github/DarkflameServer/dPhysics dPhysics
ln -s ../Github/DarkflameServer/dScripts dScripts
ln -s ../Github/DarkflameServer/dWorldServer dWorldServer
ln -s ../Github/DarkflameServer/dZoneManager dZoneManager
ln -s ../Github/DarkflameServer/migrations migrations
ln -s ../Github/DarkflameServer/resources resources
ln -s ../Github/DarkflameServer/thirdparty thirdparty
ln -s ../Github/DarkflameServer/vanity vanity
ln -s ../Github/DarkflameServer/versions.txt versions.txt

Then, mark both of your newly created files as executable

parent_dir/LegoUniverseServer ~> chmod +x ./updateServer
parent_dir/LegoUniverseServer ~> chmod +x ./fixlnks

The fixlnks script sets up symbolic links to the files within the DarkflameServer repository, allowing you to update it independently of your server files. The updateServer script automates the update process, first pulling the new files to the DarkflameServer repository, then re-executing the build process for the server.

Automating Server Configs

This section focuses mainly on maintaining the server settings when migrating from one (physical) server to another, allowing quick re-setup of the Lego Universe Server.

Navigate into the ini_files directory and create a file called fix_ini_files with the following contents:

cd ../build
ln -s ../ini_files/authconfig.ini authconfig.ini
ln -s ../ini_files/chatconfig.ini chatconfig.ini
ln -s ../ini_files/masterconfig.ini masterconfig.ini
ln -s ../ini_files/worldconfig.ini worldconfig.ini
ln -s ../ini_files/sharedconfig.ini sharedconfig.ini

Then, copy over the ini files you permanently use into this ini_files directory. The file we just created will allow the server to look for the configs in this directory in the future, rather than in the original build directory.

Finally, mark the file as executable

parent_dir/LegoUniverseServer/ini_files ~> chmod +x fix_ini_files

Automating Discord Backups

Finally, we add the discord backup. This allows your users to download their data and use it on their own if they ever decide to leave your community.

Navigate into the darkflame_database folder in your new repo, and create a file called backup_to_discord with the following contents

#!/usr/bin/env python3
import discord
import datetime

TOKEN = 'YOUR_API_KEY'

filename = 'darkflame.sql'

data = []
with open(filename, 'rb') as f:
    for data in f.readlines():
        if b'INSERT INTO `accounts` VALUES' in data:
            break

if data == []:
    print('Password hashes not found! Exiting!')
    exit(-1)

data = data.replace(b'INSERT INTO `accounts` VALUES (', b'').replace(b');\n', b'').split(b'),(')
hashes = []

i = 0
for entry in data:
    entry = entry.decode().split(',')
    entry[2] = "''"
    entry = ','.join(entry)
    data[i] = entry
    i += 1

statement = 'INSERT INTO `accounts` VALUES (' + '),('.join(data) + ');\n'

new_filename = 'darkflame_backup.sql'

with open(new_filename, 'wb+') as w:
    with open(filename, 'rb') as f:
        for data in f.readlines():
            if b'INSERT INTO `accounts` VALUES' in data:
                w.write(bytes(statement, 'utf-8'))
            else:
                w.write(data)

client = discord.Client(intents=discord.Intents.default())

@client.event
async def on_ready():
    channel = client.get_channel(YOUR_CHANNEL_ID)
    await channel.send(f'Latest SQL data dump ({datetime.datetime.now()})', file=discord.File(new_filename))
    exit(0)

client.run(TOKEN)

(*) Make sure to replace YOUR_API_KEY and YOUR_CHANNEL_ID with your Discord API key and Channel IDs.

This python script allows backing up the data to Discord, while removing user passwords for privacy reasons. To run this module, you need the discord python library which you can install using pip3 install discord

Next, just mark the file as executable chmod +x ./backup_to_discord

Putting It All Together

With all the above scripts, we just need to automate the entire process. We are going to leave server updates as a manual task, as it only needs to be carried out once in a while. Data backups however, are going to be scheduled as a daily task.

To automate the data backups (both to Github and Discord), we are going to create a new cron job that runs at 1 AM ever day. To create this job, execute the command crontab -e to edit the cron tab, then paste the following job at the bottom of the crontab (be sure to replace the path to your repository with the correct one)

0 1 * * * cd /path/to/LegoUniverseServer/fix_database/ && ./export_database && cd .. && git add . && git commit -m "database backup `date`" && git push

This will execute the export_database script we created earlier, which will automatically export the current darkflame database to an SQL file, store it within the darkflame_database subdirectory, push an updated copy without passwords into a Discord community channel and then upload the database backup to Github, tagging the commit with the current date and time. Viola, automation!

(*) It should go without saying, but pushing to git requires you to have the permission to access the repository. I would not recommend leaving a trusted ssh key on the server just for the sake of backups, but rather using an access token for pushing the backups (Repo settings > Deploy keys > Add the ssh key there and enable write access).

To update the server at any given time, terminate the current instance, then navigate into the root of your new repository and just execute the command ./updateServer and then re-run the server after the update is done.

Restoring/Migrating Server to New Server

With all these tools, it is quite straightforward to migrate your server to new infrastructure.

First, start off by shutting down your server and executing the ./export_database script. Then, push all the changes to Github.

On the new server, create a parent directory and clone your repository into it. Within the root of this parent directory, create a Github directory and clone into it the DarkflameServer repository (with all its submodules). After doing this, you can ignore the Github directory and proceed with your repository.

Navigate into the root of your repository and execute the fixlnks script. This will automatically set up the symbolic links to the DarkflameServer repository. Then, navigate into the ini_files directory and execute the fix_ini_files script.

To build the server, simply execute the updateServer script in the root of your repository.

Once done, navigate into the fix_database directory and execute the import_database script to re-import all the data from the old infrastructure.

With these steps done, all your previous infrastructure should be replicated on the new machine, and you can simply navigate into the build directory and execute the ./runserver script provided below (or however you choose to run the server).

Make sure to execute the script from their predefined directories. Since the scripts use relative paths, make sure to adhere to my above documentation as otherwise errors will occur during the execution.

Add-On: Server Run Script

To notify your Discord users of server restarts, you can add the following runserver file to your new repository’s build directory

/path/to/LegoUniverseServer/server_status/discord_status_notify start > /dev/null 2>&1
cd /path/to/LegoUniverseServer/build && \
sudo setcap 'cap_net_bind_service=+ep' AuthServer && \
./MasterServer
/path/to/LegoUniverseServer/server_status/discord_status_notify stop > /dev/null 2>&1

Under the directory server_status create the script discord_status_notify with the following contents

#!/usr/bin/env python3
import discord
import sys
import datetime

TOKEN = 'YOUR_API_KEY'

if len(sys.argv) < 2:
    print("Usage: './discord_status_notify start' or './discord_status_notify stop'")
    exit(-1)

mode = sys.argv[1]

client = discord.Client(intents=discord.Intents.default())

@client.event
async def on_ready():
    channel = client.get_channel(YOUR_CHAT_ID)
    if mode == 'start':
        await channel.send(f'Server started up! Time: ({datetime.datetime.now()})')
    elif mode == 'stop':
        await channel.send(f'Server shut down! Time: ({datetime.datetime.now()})')
    elif mode == 'updated':
        await channel.send(f'Server was updated! Time: ({datetime.datetime.now()})')
    exit(0)

client.run(TOKEN)

(*) Make sure to replace YOUR_API_KEY and YOUR_CHANNEL_ID with your Discord API key and Channel IDs.

Make sure to mark the script as executable using chmod +x ./discord_status_notify

The best way of running the server using this script is starting a screen instance, then executing ./runserver and then using the shortcut ctrl+a D to detach from the instance.

As a final add-on, here is the .gitignore file for the abovementioned automations

temp/
cmake-build-debug/
RelWithDebInfo/
build/CMakeFiles/
build/*
!build/res/
!build/locale/
!build/runserver
darkflame_database/*.ibd
darkflame_database/*.frm
darkflame_database/darkflame_backup.sql
# Third party libraries
thirdparty/mysql/
thirdparty/mysql_linux/
#CMakeVariables.txt
build/_deps/
# Build folders
linux_build/

# Codeblocks
*.layout

# User-specific files
*.rsuser
*.suo
*.user
*.userosscache
*.sln.docstates

# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs

# Build results
[Dd]ebug/
[Dd]ebugPublic/
[Rr]elease/
[Rr]eleases/
x64/
x86/
bld/
[Bb]in/
[Oo]bj/
[Ll]og/

# Visual Studio 2015/2017 cache/options directory
.vs/

# Files built by Visual Studio
*_i.c
*_p.c
*_h.h
*.ilk
*.meta
*.obj
*.iobj
*.pch
*.pdb
*.ipdb
*.idb
*.pgc
*.pgd
*.rsp
*.sbr
*.tlb
*.tli
*.tlh
*.tmp
*.tmp_proj
*_wpftmp.csproj
*.log
*.vspscc
*.vssscc
.builds
*.pidb
*.svclog
*.scc
*.lastbuildstate

# Visual C++ cache files
ipch/
*.aps
*.ncb
*.opendb
*.opensdf
*.sdf
*.cachefile
*.VC.db
*.VC.VC.opendb

# Visual Studio Log Files
*.tlog
*.log

# VS Code
.vscode

# Clion (IntelliJ)
.idea
.DS_Store

# Exceptions:
CMakeSettings.json
*.vcxproj
*.filters
*.cmake
CMakeCache.txt
#*.bin
CMakeFiles/3.17.3/CompilerIdC/CMakeCCompilerId.c
CMakeFiles/3.17.3/CompilerIdCXX/CMakeCXXCompilerId.cpp
CMakeFiles/3.17.3/VCTargetsPath.txt
CMakeFiles/3e6b105f924142e3e299a1a30e09566e/generate.stamp.rule
CMakeFiles/cmake.check_cache
CMakeFiles/generate.stamp
CMakeFiles/generate.stamp.depend
CMakeFiles/generate.stamp.list
CMakeFiles/TargetDirectories.txt
*.sln
*.recipe

# clangd
.cache
thirdparty/zlib-1.2.11/

Closing Words

I hope these tools help you in more efficiently handling your Lego Universe server. Should you have any questions, do not hesitate to leave them below this article or on my Gist.

If you like more cool tools, check out my Website which has a whole variety of useful tools and automations.