nev/minecraft-backup
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #15 from nicolaschan/rcon

Browse source 
Add RCON support
This commit is contained in:
Nicolas Chan 2021-03-04 13:25:46 -08:00 • committed by GitHub
commit e058250a33
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 468 additions and 86 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
.github/workflows/ci.yml vendored
View file

@ -12,7 +12,7 @@ on:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel # A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs: jobs:
# This workflow contains a single job called "build" # This workflow contains a single job called "build"
build: test:
# The type of runner that the job will run on # The type of runner that the job will run on
runs-on: ubuntu-latest runs-on: ubuntu-latest
@ -23,8 +23,8 @@ jobs:
with: with:
submodules: 'recursive' submodules: 'recursive'
- name: Install kcov - name: Install kcov and shellcheck
run: sudo apt-get install -y kcov run: sudo apt-get install -y kcov shellcheck
- name: Run tests - name: Run tests
run: test/test.sh run: test/test.sh
@ -38,3 +38,6 @@ jobs:
token: ${{ secrets.CODECOV_TOKEN }} token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage/test.sh/cov.xml file: ./coverage/test.sh/cov.xml
- name: shellcheck
run: shellcheck backup.sh

82
README.md
View file

@ -1,28 +1,40 @@
# Minecraft Backup # Minecraft Backup
Backup script for Linux servers running a Minecraft server in a GNU Screen or tmux ![GitHub Workflow Status](https://img.shields.io/github/workflow/status/nicolaschan/minecraft-backup/CI)
[![codecov](https://codecov.io/gh/nicolaschan/minecraft-backup/branch/master/graph/badge.svg?token=LCbVC4TbYJ)](https://codecov.io/gh/nicolaschan/minecraft-backup)
### Disclaimer Backup script for Minecraft servers on Linux.
Backups are essential to the integrity of your Minecraft world. You should automate regular backups and **check that your backups work**. While this script has been used in production for several years, it is up to you to make sure that your backups work and that you have a reliable backup policy. Supports servers running in [screen](https://en.wikipedia.org/wiki/GNU_Screen), [tmux](https://en.wikipedia.org/wiki/Tmux), or with [RCON](https://wiki.vg/RCON) enabled.
Please refer to the LICENSE (MIT License) for the full legal disclaimer.
## Features ## Features
- Create backups of your world folder - Create backups of your world folder
- Manage deletion of old backups - Manage deletion of old backups
- "thin" - keep last 24 hourly, last 30 daily, and use remaining space for monthly backups - "thin" - keep last 24 hourly, last 30 daily, and use remaining space for weekly backups
- "sequential" - delete oldest backup - "sequential" - delete oldest backup
- Choose your own compression algorithm (tested with: `gzip`, `xz`, `zstd`) - Works on vanilla (no plugins required)
- Able to print backup status and info to the Minecraft chat - Print backup status to the Minecraft chat
## Requirements ## Install
- Linux computer (tested on Ubuntu) ```bash
- GNU Screen (running your Minecraft server) wget https://raw.githubusercontent.com/nicolaschan/minecraft-backup/master/backup.sh
- Minecraft server (tested with Vanilla 1.10.2 only) chmod +x backup.sh
```
## Installation Make sure your system has `tar` and your chosen compression algorithm (`gzip` by default) installed.
1. Download the script: `$ wget https://raw.githubusercontent.com/nicolaschan/minecraft-backup/master/backup.sh` If using RCON, you will also need to have the [`xxd`](https://linux.die.net/man/1/xxd) command.
2. Mark as executable: `$ chmod +x backup.sh`
3. Use the command line options or configure default values at the top of `backup.sh`: ## Usage
```bash
# If connecting with RCON:
./backup.sh -c -i /home/user/server/world -o /mnt/storage/backups -s localhost:25575:secret -w rcon
# If running on screen called "minecraft":
./backup.sh -c -i /home/user/server/world -o /mnt/storage/backups -s minecraft
# If running on tmux session 0:
./backup.sh -c -i /home/user/server/world -o /mnt/storage/backups -s 0 -w tmux
```
This will show chat messages (`-c`) and save a backup of `/home/user/server/world` into `/mnt/storage/backups` using the default thinning deletion policy for old backups.
Command line options: Command line options:
```text ```text
@ -38,20 +50,17 @@ Command line options:
-o Output directory -o Output directory
-p Prefix that shows in Minecraft chat (default: Backup) -p Prefix that shows in Minecraft chat (default: Backup)
-q Suppress warnings -q Suppress warnings
-s Minecraft server screen name -s Screen name, tmux session name, or hostname:port:password for RCON
-v Verbose mode -v Verbose mode
-w Window manager: screen (default) or tmux -w Window manager: screen (default), tmux, RCON
``` ```
Example usage of command line options: ### Automate backups with cron
```bash - Edit the crontab with `crontab -e`
./backup.sh -c -i /home/server/minecraft-server/world -o /mnt/external-storage/minecraft-backups -s minecraft - Example for hourly backups:
```
00 * * * * /path/to/backup.sh -c -i /home/user/server/world -o /mnt/storage/backups -s minecraft
``` ```
This will use show chat messages (`-c`) in the screen called "minecraft" and save a backup of `/home/server/minecraft-server/world` into `/mnt/external-storage/minecraft-backups` using the default thinning delete policy for old backups.
4. Create a cron job to automatically backup:
- Edit the crontab: `$ crontab -e`
- Example for hourly backups: `00 * * * * /path/to/backup.sh`
## Retrieving Backups ## Retrieving Backups
Always test your backups! Backups are in the `tar` format and compressed depending on the option you choose. To restore, first decompress if necessary and then extract using tar. You may be able to do this in one command if `tar` supports your compression option, as is the case with `gzip`: Always test your backups! Backups are in the `tar` format and compressed depending on the option you choose. To restore, first decompress if necessary and then extract using tar. You may be able to do this in one command if `tar` supports your compression option, as is the case with `gzip`:
@ -65,11 +74,30 @@ tar -xzvf /path/to/backups/2019-04-09_02-15-01.tar.gz
Then you can move your restored world (`restored-world` in this case) to your Minecraft server folder and rename it (usually called `world`) so the Minecraft server uses it. Then you can move your restored world (`restored-world` in this case) to your Minecraft server folder and rename it (usually called `world`) so the Minecraft server uses it.
## Why not use `tar` directly?
If you use `tar` while the server is running, you will likely get an error like this because Minecraft autosaves the world periodically:
```
tar: /some/path/here/world/region/r.1.11.mca: file changed as we read it
```
To fix this problem, the backup script disables autosaving with the `save-off` Minecraft command before running `tar` and then re-enables autosaving after `tar` is done.
## Help ## Help
- Make sure the compression algorithm you specify is installed on your system. (zstd is not installed by default) - Make sure the compression algorithm you specify is installed on your system. (zstd is not installed by default)
- Make sure your compression algorithm is in the crontab's PATH - Make sure your compression algorithm is in the crontab's PATH
- Make sure cron has permissions for all the files involved and access to the Minecraft server's GNU Screen - Make sure cron has permissions for all the files involved and access to the Minecraft server's GNU Screen
- It's surprising how much space backups can take--make sure you have enough empty space - It's surprising how much space backups can take--make sure you have enough empty space
- `SERVER_DIRECTORY` should be the server directory, not the `world` directory
- Do not put trailing `/` in the `SERVER_DIRECTORY` or `BACKUP_DIRECTORY` - Do not put trailing `/` in the `SERVER_DIRECTORY` or `BACKUP_DIRECTORY`
- If "thin" delete method is behaving weirdly, try emptying your backup directory or switch to "sequential" - If "thin" delete method is behaving weirdly, try emptying your backup directory or switch to "sequential"
## Known Issues
There is a Minecraft bug [MC-217729](https://bugs.mojang.com/projects/MC/issues/MC-217729) in recent Minecraft server versions that cause them to automatically save the world even after receiving the `save-off` command. Until this is fixed, there is a chance that the backup will fail because the world files are modified by Minecraft in the process of creating the backup. This script will try to detect and report this problem if it does occur.
## Disclaimer
Backups are essential to the integrity of your Minecraft world. You should automate regular backups and **check that your backups work**. It is up to you to make sure that your backups work and that you have a reliable backup policy.
Some backup tips:
- Drives get corrupted or fail! Backup to a _different_ drive than the one your server is running on, so if your main drive fails then you have backups.
- _Automate_ backups so you never lose too much progress.
- Check that your backups work from time to time.
Please refer to the LICENSE (MIT License) for the full legal disclaimer.

243
backup.sh
View file

@ -8,7 +8,7 @@
# For most convenience, run automatically with cron. # For most convenience, run automatically with cron.
# Default Configuration # Default Configuration
SCREEN_NAME="" # Name of the GNU Screen or tmux pane your Minecraft server is running in SCREEN_NAME="" # Name of the GNU Screen, tmux session, or hostname:port:password for RCON
SERVER_WORLD="" # Server world directory SERVER_WORLD="" # Server world directory
BACKUP_DIRECTORY="" # Directory to save backups in BACKUP_DIRECTORY="" # Directory to save backups in
MAX_BACKUPS=128 # -1 indicates unlimited MAX_BACKUPS=128 # -1 indicates unlimited
@ -20,12 +20,24 @@ ENABLE_CHAT_MESSAGES=false # Tell players in Minecraft chat about backup status
PREFIX="Backup" # Shows in the chat message PREFIX="Backup" # Shows in the chat message
DEBUG=false # Enable debug messages DEBUG=false # Enable debug messages
SUPPRESS_WARNINGS=false # Suppress warnings SUPPRESS_WARNINGS=false # Suppress warnings
WINDOW_MANAGER="screen" # Choices: screen, tmux WINDOW_MANAGER="screen" # Choices: screen, tmux, RCON
# Other Variables (do not modify) # Other Variables (do not modify)
DATE_FORMAT="%F_%H-%M-%S" DATE_FORMAT="%F_%H-%M-%S"
TIMESTAMP=$(date +$DATE_FORMAT) TIMESTAMP=$(date +$DATE_FORMAT)
log-fatal () {
echo -e "\033[0;31mFATAL:\033[0m $*"
}
log-warning () {
echo -e "\033[0;33mWARNING:\033[0m $*"
}
debug-log () {
if "$DEBUG"; then
echo "$1"
fi
}
while getopts 'a:cd:e:f:hi:l:m:o:p:qs:vw:' FLAG; do while getopts 'a:cd:e:f:hi:l:m:o:p:qs:vw:' FLAG; do
case $FLAG in case $FLAG in
a) COMPRESSION_ALGORITHM=$OPTARG ;; a) COMPRESSION_ALGORITHM=$OPTARG ;;
@ -47,9 +59,9 @@ while getopts 'a:cd:e:f:hi:l:m:o:p:qs:vw:' FLAG; do
echo "-o Output directory" echo "-o Output directory"
echo "-p Prefix that shows in Minecraft chat (default: Backup)" echo "-p Prefix that shows in Minecraft chat (default: Backup)"
echo "-q Suppress warnings" echo "-q Suppress warnings"
echo "-s Minecraft server screen name" echo "-s Screen name, tmux session name, or hostname:port:password for RCON"
echo "-v Verbose mode" echo "-v Verbose mode"
echo "-w Window manager: screen (default) or tmux" echo "-w Window manager: screen (default), tmux, RCON"
exit 0 exit 0
;; ;;
i) SERVER_WORLD=$OPTARG ;; i) SERVER_WORLD=$OPTARG ;;
@ -61,14 +73,118 @@ while getopts 'a:cd:e:f:hi:l:m:o:p:qs:vw:' FLAG; do
s) SCREEN_NAME=$OPTARG ;; s) SCREEN_NAME=$OPTARG ;;
v) DEBUG=true ;; v) DEBUG=true ;;
w) WINDOW_MANAGER=$OPTARG ;; w) WINDOW_MANAGER=$OPTARG ;;
*) log-fatal "Invalid option -$FLAG"; exit 1 ;;
esac esac
done done
log-fatal () { rcon-command () {
echo -e "\033[0;31mFATAL:\033[0m $*" HOST="$(echo "$1" | cut -d: -f1)"
} PORT="$(echo "$1" | cut -d: -f2)"
log-warning () { PASSWORD="$(echo "$1" | cut -d: -f3-)"
echo -e "\033[0;33mWARNING:\033[0m $*" COMMAND="$2"
reverse-hex-endian () {
# Given a 4-byte hex integer, reverse endianness
while read -r -d '' -N 8 INTEGER; do
echo "$INTEGER" | sed -E 's/(..)(..)(..)(..)/\4\3\2\1/'
done
}
decode-hex-int () {
# decode little-endian hex integer
while read -r -d '' -N 8 INTEGER; do
BIG_ENDIAN_HEX=$(echo "$INTEGER" | reverse-hex-endian)
echo "$((16#$BIG_ENDIAN_HEX))"
done
}
stream-to-hex () {
xxd -ps
}
hex-to-stream () {
xxd -ps -r
}
encode-int () {
# Encode an integer as 4 bytes in little endian and return as hex
INT="$1"
# Source: https://stackoverflow.com/a/9955198
printf "%08x" "$INT" | sed -E 's/(..)(..)(..)(..)/\4\3\2\1/'
}
encode () {
# Encode a packet type and payload for the rcon protocol
TYPE="$1"
PAYLOAD="$2"
REQUEST_ID="$3"
PAYLOAD_LENGTH="${#PAYLOAD}"
TOTAL_LENGTH="$((4 + 4 + PAYLOAD_LENGTH + 1 + 1))"
OUTPUT=""
OUTPUT+=$(encode-int "$TOTAL_LENGTH")
OUTPUT+=$(encode-int "$REQUEST_ID")
OUTPUT+=$(encode-int "$TYPE")
OUTPUT+=$(echo -n "$PAYLOAD" | stream-to-hex)
OUTPUT+="0000"
echo -n "$OUTPUT" | hex-to-stream
}
read-response () {
# read next response packet and return the payload text
HEX_LENGTH=$(head -c4 <&3 | stream-to-hex | reverse-hex-endian)
LENGTH=$((16#$HEX_LENGTH))
RESPONSE_PAYLOAD=$(head -c $LENGTH <&3 | stream-to-hex)
echo -n "$RESPONSE_PAYLOAD"
}
response-request-id () {
echo -n "${1:0:8}" | decode-hex-int
}
response-type () {
echo -n "${1:8:8}" | decode-hex-int
}
response-payload () {
echo -n "${1:16:-4}" | hex-to-stream
}
login () {
PASSWORD="$1"
encode 3 "$PASSWORD" 12 >&3
RESPONSE=$(read-response "$IN_PIPE")
RESPONSE_REQUEST_ID=$(response-request-id "$RESPONSE")
if [ "$RESPONSE_REQUEST_ID" -eq -1 ] || [ "$RESPONSE_REQUEST_ID" -eq 4294967295 ]; then
log-warning "RCON connection failed: Wrong RCON password" 1>&2
return 1
fi
}
run-command () {
COMMAND="$1"
# encode 2 "$COMMAND" 13 >> "$OUT_PIPE"
encode 2 "$COMMAND" 13 >&3
RESPONSE=$(read-response "$IN_PIPE")
response-payload "$RESPONSE"
}
# Open a TCP socket
# Source: https://www.xmodulo.com/tcp-udp-socket-bash-shell.html
exec 3<>/dev/tcp/"$HOST"/"$PORT"
login "$PASSWORD" || return 1
debug-log "$(run-command "$COMMAND")"
# Close the socket
exec 3<&-
exec 3>&-
} }
if [[ $COMPRESSION_FILE_EXTENSION == "." ]]; then if [[ $COMPRESSION_FILE_EXTENSION == "." ]]; then
@ -78,7 +194,7 @@ fi
# Check for missing encouraged arguments # Check for missing encouraged arguments
if ! $SUPPRESS_WARNINGS; then if ! $SUPPRESS_WARNINGS; then
if [[ $SCREEN_NAME == "" ]]; then if [[ $SCREEN_NAME == "" ]]; then
log-warning "Minecraft screen name not specified (use -s)" log-warning "Minecraft screen/tmux/rcon location not specified (use -s)"
fi fi
fi fi
# Check for required arguments # Check for required arguments
@ -96,8 +212,8 @@ if $MISSING_CONFIGURATION; then
exit 1 exit 1
fi fi
ARCHIVE_FILE_NAME=$TIMESTAMP.tar$COMPRESSION_FILE_EXTENSION ARCHIVE_FILE_NAME="$TIMESTAMP.tar$COMPRESSION_FILE_EXTENSION"
ARCHIVE_PATH=$BACKUP_DIRECTORY/$ARCHIVE_FILE_NAME ARCHIVE_PATH="$BACKUP_DIRECTORY/$ARCHIVE_FILE_NAME"
# Minecraft server screen interface functions # Minecraft server screen interface functions
message-players () { message-players () {
@ -109,9 +225,11 @@ execute-command () {
local COMMAND=$1 local COMMAND=$1
if [[ $SCREEN_NAME != "" ]]; then if [[ $SCREEN_NAME != "" ]]; then
case $WINDOW_MANAGER in case $WINDOW_MANAGER in
"screen") screen -S $SCREEN_NAME -p 0 -X stuff "$COMMAND$(printf \\r)" "screen") screen -S "$SCREEN_NAME" -p 0 -X stuff "$COMMAND$(printf \\r)"
;; ;;
"tmux") tmux send-keys -t $SCREEN_NAME "$COMMAND" ENTER "tmux") tmux send-keys -t "$SCREEN_NAME" "$COMMAND" ENTER
;;
"RCON"|"rcon") rcon-command "$SCREEN_NAME" "$COMMAND"
;; ;;
esac esac
fi fi
@ -130,9 +248,7 @@ message-players-color () {
local MESSAGE=$1 local MESSAGE=$1
local HOVER_MESSAGE=$2 local HOVER_MESSAGE=$2
local COLOR=$3 local COLOR=$3
if $DEBUG; then debug-log "$MESSAGE ($HOVER_MESSAGE)"
echo "$MESSAGE ($HOVER_MESSAGE)"
fi
if $ENABLE_CHAT_MESSAGES; then if $ENABLE_CHAT_MESSAGES; then
execute-command "tellraw @a [\"\",{\"text\":\"[$PREFIX] \",\"color\":\"gray\",\"italic\":true},{\"text\":\"$MESSAGE\",\"color\":\"$COLOR\",\"italic\":true,\"hoverEvent\":{\"action\":\"show_text\",\"value\":{\"text\":\"\",\"extra\":[{\"text\":\"$HOVER_MESSAGE\"}]}}}]" execute-command "tellraw @a [\"\",{\"text\":\"[$PREFIX] \",\"color\":\"gray\",\"italic\":true},{\"text\":\"$MESSAGE\",\"color\":\"$COLOR\",\"italic\":true,\"hoverEvent\":{\"action\":\"show_text\",\"value\":{\"text\":\"\",\"extra\":[{\"text\":\"$HOVER_MESSAGE\"}]}}}]"
fi fi
@ -143,50 +259,67 @@ message-players "Starting backup..." "$ARCHIVE_FILE_NAME"
# Parse file timestamp to one readable by "date" # Parse file timestamp to one readable by "date"
parse-file-timestamp () { parse-file-timestamp () {
local DATE_STRING=$(echo $1 | awk -F_ '{gsub(/-/,":",$2); print $1" "$2}') local DATE_STRING
echo $DATE_STRING DATE_STRING="$(echo "$1" | awk -F_ '{gsub(/-/,":",$2); print $1" "$2}')"
echo "$DATE_STRING"
} }
# Delete a backup # Delete a backup
delete-backup () { delete-backup () {
local BACKUP=$1 local BACKUP=$1
rm $BACKUP_DIRECTORY/$BACKUP rm "$BACKUP_DIRECTORY"/"$BACKUP"
message-players "Deleted old backup" "$BACKUP" message-players "Deleted old backup" "$BACKUP"
} }
# Sequential delete method # Sequential delete method
delete-sequentially () { delete-sequentially () {
local BACKUPS=($(ls $BACKUP_DIRECTORY)) local BACKUPS=("$BACKUP_DIRECTORY"/*)
while [[ $MAX_BACKUPS -ge 0 && ${#BACKUPS[@]} -gt $MAX_BACKUPS ]]; do while [[ $MAX_BACKUPS -ge 0 && ${#BACKUPS[@]} -gt $MAX_BACKUPS ]]; do
delete-backup ${BACKUPS[0]} delete-backup "$(basename "${BACKUPS[0]}")"
BACKUPS=($(ls $BACKUP_DIRECTORY)) BACKUPS=("$BACKUP_DIRECTORY"/*)
done done
} }
# Functions to sort backups into correct categories based on timestamps # Functions to sort backups into correct categories based on timestamps
is-hourly-backup () { is-hourly-backup () {
local TIMESTAMP=$* local TIMESTAMP=$*
local MINUTE=$(date -d "$TIMESTAMP" +%M) local MINUTE
return $MINUTE MINUTE=$(date -d "$TIMESTAMP" +%M)
return "$MINUTE"
} }
is-daily-backup () { is-daily-backup () {
local TIMESTAMP=$* local TIMESTAMP=$*
local HOUR=$(date -d "$TIMESTAMP" +%H) local HOUR
return $HOUR HOUR=$(date -d "$TIMESTAMP" +%H)
return "$HOUR"
} }
is-weekly-backup () { is-weekly-backup () {
local TIMESTAMP=$* local TIMESTAMP=$*
local DAY=$(date -d "$TIMESTAMP" +%u) local DAY
return $((DAY - 1)) DAY=$(date -d "$TIMESTAMP" +%u)
return "$((DAY - 1))"
} }
# Helper function to sum an array # Helper function to sum an array
array-sum () { array-sum () {
SUM=0 SUM=0
for NUMBER in $*; do for NUMBER in "$@"; do
(( SUM += NUMBER )) (( SUM += NUMBER ))
done done
echo $SUM echo "$SUM"
}
# Given two exit codes, print a nonzero one if there is one
exit-code () {
if [ "$1" -ne 0 ]; then
echo "$1"
else
if [[ "$2" == "" ]]; then
echo 0
else
echo "$2"
fi
fi
} }
# Thinning delete method # Thinning delete method
@ -198,7 +331,7 @@ delete-thinning () {
local BLOCK_FUNCTIONS=("is-hourly-backup" "is-daily-backup" "is-weekly-backup") local BLOCK_FUNCTIONS=("is-hourly-backup" "is-daily-backup" "is-weekly-backup")
# Warn if $MAX_BACKUPS does not have enough room for all the blocks # Warn if $MAX_BACKUPS does not have enough room for all the blocks
TOTAL_BLOCK_SIZE=$(array-sum ${BLOCK_SIZES[@]}) TOTAL_BLOCK_SIZE=$(array-sum "${BLOCK_SIZES[@]}")
if [[ $MAX_BACKUPS != -1 ]] && [[ $TOTAL_BLOCK_SIZE -gt $MAX_BACKUPS ]]; then if [[ $MAX_BACKUPS != -1 ]] && [[ $TOTAL_BLOCK_SIZE -gt $MAX_BACKUPS ]]; then
if ! $SUPPRESS_WARNINGS; then if ! $SUPPRESS_WARNINGS; then
log-warning "MAX_BACKUPS ($MAX_BACKUPS) is smaller than TOTAL_BLOCK_SIZE ($TOTAL_BLOCK_SIZE)" log-warning "MAX_BACKUPS ($MAX_BACKUPS) is smaller than TOTAL_BLOCK_SIZE ($TOTAL_BLOCK_SIZE)"
@ -206,29 +339,29 @@ delete-thinning () {
fi fi
local CURRENT_INDEX=0 local CURRENT_INDEX=0
local BACKUPS=($(ls -r $BACKUP_DIRECTORY)) # List newest first local BACKUPS=("$BACKUP_DIRECTORY"/*) # List newest first
for BLOCK_INDEX in ${!BLOCK_SIZES[@]}; do for BLOCK_INDEX in "${!BLOCK_SIZES[@]}"; do
local BLOCK_SIZE=${BLOCK_SIZES[BLOCK_INDEX]} local BLOCK_SIZE=${BLOCK_SIZES[BLOCK_INDEX]}
local BLOCK_FUNCTION=${BLOCK_FUNCTIONS[BLOCK_INDEX]} local BLOCK_FUNCTION=${BLOCK_FUNCTIONS[BLOCK_INDEX]}
local OLDEST_BACKUP_IN_BLOCK_INDEX=$((BLOCK_SIZE + CURRENT_INDEX)) # Not an off-by-one error because a new backup was already saved local OLDEST_BACKUP_IN_BLOCK_INDEX=$((BLOCK_SIZE + CURRENT_INDEX)) # Not an off-by-one error because a new backup was already saved
local OLDEST_BACKUP_IN_BLOCK=${BACKUPS[OLDEST_BACKUP_IN_BLOCK_INDEX]} local OLDEST_BACKUP_IN_BLOCK
OLDEST_BACKUP_IN_BLOCK="$(basename "${BACKUPS[OLDEST_BACKUP_IN_BLOCK_INDEX]}")"
if [[ $OLDEST_BACKUP_IN_BLOCK == "" ]]; then if [[ "$OLDEST_BACKUP_IN_BLOCK" == "" ]]; then
break break
fi fi
local OLDEST_BACKUP_TIMESTAMP=$(parse-file-timestamp ${OLDEST_BACKUP_IN_BLOCK:0:19}) local OLDEST_BACKUP_TIMESTAMP
OLDEST_BACKUP_TIMESTAMP=$(parse-file-timestamp "${OLDEST_BACKUP_IN_BLOCK:0:19}")
local BLOCK_COMMAND="$BLOCK_FUNCTION $OLDEST_BACKUP_TIMESTAMP" local BLOCK_COMMAND="$BLOCK_FUNCTION $OLDEST_BACKUP_TIMESTAMP"
if $BLOCK_COMMAND; then if $BLOCK_COMMAND; then
# Oldest backup in this block satisfies the condition for placement in the next block # Oldest backup in this block satisfies the condition for placement in the next block
if $DEBUG; then debug-log "$OLDEST_BACKUP_IN_BLOCK promoted to next block"
echo "$OLDEST_BACKUP_IN_BLOCK promoted to next block"
fi
else else
# Oldest backup in this block does not satisfy the condition for placement in next block # Oldest backup in this block does not satisfy the condition for placement in next block
delete-backup $OLDEST_BACKUP_IN_BLOCK delete-backup "$OLDEST_BACKUP_IN_BLOCK"
break break
fi fi
@ -238,6 +371,9 @@ delete-thinning () {
delete-sequentially delete-sequentially
} }
# Ensure backup directory exists
mkdir -p "$(dirname "$ARCHIVE_PATH")"
# Disable world autosaving # Disable world autosaving
execute-command "save-off" execute-command "save-off"
@ -245,12 +381,16 @@ execute-command "save-off"
START_TIME=$(date +"%s") START_TIME=$(date +"%s")
case $COMPRESSION_ALGORITHM in case $COMPRESSION_ALGORITHM in
# No compression # No compression
"") tar -cf $ARCHIVE_PATH -C $SERVER_WORLD . "") tar -cf "$ARCHIVE_PATH" -C "$SERVER_WORLD" .
;; ;;
# With compression # With compression
*) tar -cf - -C $SERVER_WORLD . | $COMPRESSION_ALGORITHM -cv -$COMPRESSION_LEVEL - > $ARCHIVE_PATH 2>> /dev/null *) tar -cf - -C "$SERVER_WORLD" . | $COMPRESSION_ALGORITHM -cv -"$COMPRESSION_LEVEL" - > "$ARCHIVE_PATH" 2>> /dev/null
;; ;;
esac esac
ARCHIVE_EXIT_CODE="$(exit-code "${PIPESTATUS[0]}" "${PIPESTATUS[1]}")"
if [ "$ARCHIVE_EXIT_CODE" -ne 0 ]; then
log-fatal "Archive command exited with nonzero exit code $ARCHIVE_EXIT_CODE"
fi
sync sync
END_TIME=$(date +"%s") END_TIME=$(date +"%s")
@ -271,17 +411,22 @@ delete-old-backups () {
} }
# Notify players of completion # Notify players of completion
WORLD_SIZE_BYTES=$(du -b --max-depth=0 $SERVER_WORLD | awk '{print $1}') WORLD_SIZE_BYTES=$(du -b --max-depth=0 "$SERVER_WORLD" | awk '{print $1}')
ARCHIVE_SIZE_BYTES=$(du -b $ARCHIVE_PATH | awk '{print $1}') ARCHIVE_SIZE_BYTES=$(du -b "$ARCHIVE_PATH" | awk '{print $1}')
ARCHIVE_SIZE=$(du -h $ARCHIVE_PATH | awk '{print $1}') ARCHIVE_SIZE=$(du -h "$ARCHIVE_PATH" | awk '{print $1}')
BACKUP_DIRECTORY_SIZE=$(du -h --max-depth=0 $BACKUP_DIRECTORY | awk '{print $1}') BACKUP_DIRECTORY_SIZE=$(du -h --max-depth=0 "$BACKUP_DIRECTORY" | awk '{print $1}')
TIME_DELTA=$((END_TIME - START_TIME)) TIME_DELTA=$((END_TIME - START_TIME))
# Check that archive size is not null and at least 200 Bytes # Check that archive size is not null and at least 200 Bytes
if [[ "$WORLD_SIZE_BYTES" -gt 0 && "$ARCHIVE_SIZE" != "" && "$ARCHIVE_SIZE_BYTES" -gt 200 ]]; then if [[ "$ARCHIVE_EXIT_CODE" -eq 0 && "$WORLD_SIZE_BYTES" -gt 0 && "$ARCHIVE_SIZE" != "" && "$ARCHIVE_SIZE_BYTES" -gt 200 ]]; then
COMPRESSION_PERCENT=$(($ARCHIVE_SIZE_BYTES * 100 / $WORLD_SIZE_BYTES)) COMPRESSION_PERCENT=$((ARCHIVE_SIZE_BYTES * 100 / WORLD_SIZE_BYTES))
message-players-success "Backup complete!" "$TIME_DELTA s, $ARCHIVE_SIZE/$BACKUP_DIRECTORY_SIZE, $COMPRESSION_PERCENT%" message-players-success "Backup complete!" "$TIME_DELTA s, $ARCHIVE_SIZE/$BACKUP_DIRECTORY_SIZE, $COMPRESSION_PERCENT%"
delete-old-backups delete-old-backups
else else
message-players-error "Backup was not saved!" "Please notify an administrator" message-players-error "Backup was not saved!" "Please notify an administrator"
if [ "$ARCHIVE_EXIT_CODE" -ne 0 ]; then
exit "$ARCHIVE_EXIT_CODE"
else
exit 1
fi
fi fi

119
rcon.sh Executable file
View file

@ -0,0 +1,119 @@
#!/usr/bin/env bash
rcon-command () {
HOST="$(echo "$1" | cut -d: -f1)"
PORT="$(echo "$1" | cut -d: -f2)"
PASSWORD="$(echo "$1" | cut -d: -f3-)"
COMMAND="$2"
reverse-hex-endian () {
# Given a 4-byte hex integer, reverse endianness
while read -r -d '' -N 8 INTEGER; do
echo "$INTEGER" | sed -E 's/(..)(..)(..)(..)/\4\3\2\1/'
done
}
decode-hex-int () {
# decode little-endian hex integer
while read -r -d '' -N 8 INTEGER; do
BIG_ENDIAN_HEX=$(echo "$INTEGER" | reverse-hex-endian)
echo "$((16#$BIG_ENDIAN_HEX))"
done
}
stream-to-hex () {
xxd -ps
}
hex-to-stream () {
xxd -ps -r
}
encode-int () {
# Encode an integer as 4 bytes in little endian and return as hex
INT="$1"
# Source: https://stackoverflow.com/a/9955198
printf "%08x" "$INT" | sed -E 's/(..)(..)(..)(..)/\4\3\2\1/'
}
encode () {
# Encode a packet type and payload for the rcon protocol
TYPE="$1"
PAYLOAD="$2"
REQUEST_ID="$3"
PAYLOAD_LENGTH="${#PAYLOAD}"
TOTAL_LENGTH="$((4 + 4 + PAYLOAD_LENGTH + 1 + 1))"
OUTPUT=""
OUTPUT+=$(encode-int "$TOTAL_LENGTH")
OUTPUT+=$(encode-int "$REQUEST_ID")
OUTPUT+=$(encode-int "$TYPE")
OUTPUT+=$(echo -n "$PAYLOAD" | stream-to-hex)
OUTPUT+="0000"
echo -n "$OUTPUT" | hex-to-stream
}
read-response () {
# read next response packet and return the payload text
HEX_LENGTH=$(head -c4 <&3 | stream-to-hex | reverse-hex-endian)
LENGTH=$((16#$HEX_LENGTH))
RESPONSE_PAYLOAD=$(head -c $LENGTH <&3 | stream-to-hex)
echo -n "$RESPONSE_PAYLOAD"
}
response-request-id () {
echo -n "${1:0:8}" | decode-hex-int
}
response-type () {
echo -n "${1:8:8}" | decode-hex-int
}
response-payload () {
echo -n "${1:16:-4}" | hex-to-stream
}
login () {
PASSWORD="$1"
encode 3 "$PASSWORD" 12 >&3
RESPONSE=$(read-response "$IN_PIPE")
RESPONSE_REQUEST_ID=$(response-request-id "$RESPONSE")
if [ "$RESPONSE_REQUEST_ID" -eq -1 ] || [ "$RESPONSE_REQUEST_ID" -eq 4294967295 ]; then
echo "Authentication failed: Wrong RCON password" 1>&2
return 1
fi
}
run-command () {
COMMAND="$1"
# encode 2 "$COMMAND" 13 >> "$OUT_PIPE"
encode 2 "$COMMAND" 13 >&3
RESPONSE=$(read-response "$IN_PIPE")
response-payload "$RESPONSE"
}
# Open a TCP socket
# Source: https://www.xmodulo.com/tcp-udp-socket-bash-shell.html
exec 3<>/dev/tcp/"$HOST"/"$PORT"
login "$PASSWORD" || return 1
run-command "$COMMAND"
# Close the socket
exec 3<&-
exec 3>&-
}
HOST="$1"
PORT="$2"
PASSWORD="$3"
COMMAND="$4"
rcon-command "$HOST:$PORT:$PASSWORD" "$COMMAND"

31
test/mock_rcon.py Normal file
View file

@ -0,0 +1,31 @@
# Reference: https://docs.python.org/3/library/socketserver.html
import codecs
import socketserver
import sys
class Handler(socketserver.BaseRequestHandler):
def handle(self):
while True:
length = int.from_bytes(self.request.recv(4), 'little')
if not length:
continue
self.request.recv(4)
type_ = int.from_bytes(self.request.recv(4), 'little')
self.data = self.request.recv(length - 8)[:-2].decode('utf-8')
if self.data:
if type_ == 2:
print(self.data)
sys.stdout.flush()
try:
if type_ == 3 and self.data != sys.argv[2]:
self.request.sendall(codecs.decode('0a000000ffffffff020000000000', 'hex'))
else:
self.request.sendall(codecs.decode('0a00000010000000020000000000', 'hex'))
except:
break
if __name__ == "__main__":
HOST, PORT = "localhost", int(sys.argv[1])
with socketserver.ThreadingTCPServer((HOST, PORT), Handler) as server:
server.serve_forever()

70
test/test.sh
View file

@ -5,6 +5,8 @@
TEST_DIR="test" TEST_DIR="test"
TEST_TMP="$TEST_DIR/tmp" TEST_TMP="$TEST_DIR/tmp"
SCREEN_TMP="tmp-screen" SCREEN_TMP="tmp-screen"
RCON_PORT="8088"
RCON_PASSWORD="supersecret"
setUp () { setUp () {
rm -rf "$TEST_TMP" rm -rf "$TEST_TMP"
mkdir -p "$TEST_TMP/server/world" mkdir -p "$TEST_TMP/server/world"
@ -17,10 +19,17 @@ setUp () {
screen -S "$SCREEN_TMP" -X stuff "cat > $TEST_TMP/screen-output\n" screen -S "$SCREEN_TMP" -X stuff "cat > $TEST_TMP/screen-output\n"
tmux new-session -d -s "$SCREEN_TMP" tmux new-session -d -s "$SCREEN_TMP"
tmux send-keys -t "$SCREEN_TMP" "cat > $TEST_TMP/tmux-output" ENTER tmux send-keys -t "$SCREEN_TMP" "cat > $TEST_TMP/tmux-output" ENTER
sleep 0.5 python test/mock_rcon.py "$RCON_PORT" "$RCON_PASSWORD" > "$TEST_TMP/rcon-output" &
echo "$!" > "$TEST_TMP/rcon-pid"
while ! [[ (-f "$TEST_TMP/screen-output") && (-f "$TEST_TMP/tmux-output") && (-f "$TEST_TMP/rcon-output") ]]; do
sleep 0.1
done
} }
tearDown () { tearDown () {
RCON_PID="$(cat "$TEST_TMP/rcon-pid")"
kill "$RCON_PID" >/dev/null 2>&1 || true
screen -S "$SCREEN_TMP" -X quit >/dev/null 2>&1 || true screen -S "$SCREEN_TMP" -X quit >/dev/null 2>&1 || true
tmux kill-session -t "$SCREEN_TMP" >/dev/null 2>&1 || true tmux kill-session -t "$SCREEN_TMP" >/dev/null 2>&1 || true
} }
@ -35,12 +44,18 @@ assert-equals-directory () {
fi fi
} }
check-backup-full-paths () {
BACKUP_ARCHIVE="$1"
WORLD_DIR="$2"
mkdir -p "$TEST_TMP/restored"
tar --extract --file "$BACKUP_ARCHIVE" --directory "$TEST_TMP/restored"
assert-equals-directory "$WORLD_DIR" "$TEST_TMP/restored"
rm -rf "$TEST_TMP/restored"
}
check-backup () { check-backup () {
BACKUP_ARCHIVE="$1" BACKUP_ARCHIVE="$1"
mkdir -p "$TEST_TMP/restored" check-backup-full-paths "$TEST_TMP/backups/$BACKUP_ARCHIVE" "$TEST_TMP/server/world"
tar --extract --file "$TEST_TMP/backups/$BACKUP_ARCHIVE" --directory "$TEST_TMP/restored"
assert-equals-directory "$TEST_TMP/server/world" "$TEST_TMP/restored"
rm -rf "$TEST_TMP/restored"
} }
# Tests # Tests
@ -51,6 +66,17 @@ test-backup-defaults () {
check-backup "$TIMESTAMP.tar.gz" check-backup "$TIMESTAMP.tar.gz"
} }
test-backup-spaces-in-directory () {
TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01")"
WORLD_SPACES="$TEST_TMP/minecraft server/the world"
mkdir -p "$(dirname "$WORLD_SPACES")"
BACKUP_SPACES="$TEST_TMP/My Backups"
mkdir -p "$BACKUP_SPACES"
cp -r "$TEST_TMP/server/world" "$WORLD_SPACES"
./backup.sh -i "$WORLD_SPACES" -o "$BACKUP_SPACES" -s "$SCREEN_TMP" -f "$TIMESTAMP"
check-backup-full-paths "$BACKUP_SPACES/$TIMESTAMP.tar.gz" "$WORLD_SPACES"
}
test-backup-no-compression () { test-backup-no-compression () {
TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01")" TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01")"
./backup.sh -a "" -e "" -i "$TEST_TMP/server/world" -o "$TEST_TMP/backups" -s "$SCREEN_TMP" -f "$TIMESTAMP" ./backup.sh -a "" -e "" -i "$TEST_TMP/server/world" -o "$TEST_TMP/backups" -s "$SCREEN_TMP" -f "$TIMESTAMP"
@ -88,7 +114,7 @@ test-missing-options () {
OUTPUT="$(./backup.sh 2>&1)" OUTPUT="$(./backup.sh 2>&1)"
EXIT_CODE="$?" EXIT_CODE="$?"
assertEquals 1 "$EXIT_CODE" assertEquals 1 "$EXIT_CODE"
assertContains "$OUTPUT" "Minecraft screen name not specified" assertContains "$OUTPUT" "Minecraft screen/tmux/rcon location not specified (use -s)"
assertContains "$OUTPUT" "Server world not specified" assertContains "$OUTPUT" "Server world not specified"
assertContains "$OUTPUT" "Backup directory not specified" assertContains "$OUTPUT" "Backup directory not specified"
} }
@ -97,7 +123,14 @@ test-missing-options-suppress-warnings () {
OUTPUT="$(./backup.sh -q 2>&1)" OUTPUT="$(./backup.sh -q 2>&1)"
EXIT_CODE="$?" EXIT_CODE="$?"
assertEquals 1 "$EXIT_CODE" assertEquals 1 "$EXIT_CODE"
assertNotContains "$OUTPUT" "Minecraft screen name not specified" assertNotContains "$OUTPUT" "Minecraft screen/tmux/rcon location not specified (use -s)"
}
test-invalid-options () {
OUTPUT="$(./backup.sh -z 2>&1)"
EXIT_CODE="$?"
assertEquals 1 "$EXIT_CODE"
assertContains "$OUTPUT" "Invalid option"
} }
test-empty-world-warning () { test-empty-world-warning () {
@ -113,6 +146,14 @@ test-block-size-warning () {
assertContains "$OUTPUT" "is smaller than TOTAL_BLOCK_SIZE" assertContains "$OUTPUT" "is smaller than TOTAL_BLOCK_SIZE"
} }
test-nonzero-exit-warning () {
TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01")"
OUTPUT="$(./backup.sh -a _BLAH_ -i "$TEST_TMP/server/world" -o "$TEST_TMP/backups" -s "$SCREEN_TMP" -f "$TIMESTAMP" 2>&1)"
EXIT_CODE="$?"
assertNotEquals 0 "$EXIT_CODE"
assertContains "$OUTPUT" "Archive command exited with nonzero exit code"
}
test-screen-interface () { test-screen-interface () {
TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01")" TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01")"
./backup.sh -i "$TEST_TMP/server/world" -o "$TEST_TMP/backups" -s "$SCREEN_TMP" -f "$TIMESTAMP" ./backup.sh -i "$TEST_TMP/server/world" -o "$TEST_TMP/backups" -s "$SCREEN_TMP" -f "$TIMESTAMP"
@ -129,6 +170,20 @@ test-tmux-interface () {
assertEquals "$SCREEN_CONTENTS" "$EXPECTED_CONTENTS" assertEquals "$SCREEN_CONTENTS" "$EXPECTED_CONTENTS"
} }
test-rcon-interface () {
TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01")"
./backup.sh -w rcon -i "$TEST_TMP/server/world" -o "$TEST_TMP/backups" -s "localhost:$RCON_PORT:$RCON_PASSWORD" -f "$TIMESTAMP"
EXPECTED_CONTENTS=$(echo -e "save-off\nsave-on\nsave-all")
SCREEN_CONTENTS="$(head -n3 "$TEST_TMP/rcon-output")"
assertEquals "$SCREEN_CONTENTS" "$EXPECTED_CONTENTS"
}
test-rcon-interface-wrong-password () {
TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01")"
OUTPUT="$(./backup.sh -w RCON -i "$TEST_TMP/server/world" -o "$TEST_TMP/backups" -s "localhost:$RCON_PORT:wrong$RCON_PASSWORD" -f "$TIMESTAMP" 2>&1)"
assertContains "$OUTPUT" "Wrong RCON password"
}
test-sequential-delete () { test-sequential-delete () {
for i in $(seq 0 99); do for i in $(seq 0 99); do
TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01 +$i hour")" TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01 +$i hour")"
@ -138,6 +193,7 @@ test-sequential-delete () {
TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01 +$i hour")" TIMESTAMP="$(date +%F_%H-%M-%S --date="2021-01-01 +$i hour")"
check-backup "$TIMESTAMP.tar.gz" check-backup "$TIMESTAMP.tar.gz"
done done
assertEquals 10 "$(find "$TEST_TMP/backups" -type f | wc -l)"
} }
test-thinning-delete () { test-thinning-delete () {