starred/LinuxGSM

Fork 0

mirror of https://github.com/GameServerManagers/LinuxGSM.git synced 2026-04-24 21:56:01 +03:00

[GH-ISSUE #3298] [FEATURE] Add a blacklist file to prevent from backing up useless files #2277

New issue

Open

opened 2026-02-27 03:01:55 +03:00 by kerem · 4 comments

kerem commented

2026-02-27 03:01:55 +03:00

Owner

Originally created by @Sarrus1 on GitHub (Feb 23, 2021).
Original GitHub issue: https://github.com/GameServerManagers/LinuxGSM/issues/3298

My use case

I use LGSM to manage CS:GO servers, and I do backup every other weeks. However, each backup takes up to 13GB of space, and takes around 10 mins with the CPU running at 100% to complete.

The Idea

A way to reduce space taken up and time to run would be to implement a blacklist of the files we don't want to backup. In CS:GO's specific case, all .vpk files can be ignored for (i think) 99% of installation as they are only generic texture files that can be reobtained by running ./csgoserver validate.

A CS:GO server install is around 22Go without any custom maps. There are 166 .vpk files, each taking up more or less 120MB. The exact space they take up is around 20Go. Ignoring those files would shrink the backup file by around 90%.

My take on how to implement this

The simplest solution would be to whitelist files by extensions, (i.e ignore all .vpk files in my case)

However, for CS:GO and other source based games, it might be interesting to ignore some map files, while keeping others, for example not backing up de_dust2.bsp but backing up de_custommap.bsp.

Therefore, the best way to go around this would be to have a file called backup_blacklist.txt somewhere in lgsm/ and adding each file to be ignored, with its extension, line by line in this file.

I made a similar script using Python to generate a maplist.txt file which you can find here if that helps.

Thanks a lot for reading!

Originally created by @Sarrus1 on GitHub (Feb 23, 2021). Original GitHub issue: https://github.com/GameServerManagers/LinuxGSM/issues/3298 ## My use case I use LGSM to manage CS:GO servers, and I do backup every other weeks. However, each backup takes up to 13GB of space, and takes around 10 mins with the CPU running at 100% to complete. ## The Idea A way to reduce space taken up and time to run would be to implement a blacklist of the files we don't want to backup. In CS:GO's specific case, all `.vpk` files can be ignored for (i think) 99% of installation as they are only generic texture files that can be reobtained by running `./csgoserver validate`. A CS:GO server install is around 22Go without any custom maps. There are 166 `.vpk` files, each taking up more or less 120MB. The exact space they take up is around 20Go. Ignoring those files would shrink the backup file by around 90%. ## My take on how to implement this The simplest solution would be to whitelist files by extensions, (i.e ignore all `.vpk` files in my case) However, for CS:GO and other source based games, it might be interesting to ignore some map files, while keeping others, for example not backing up `de_dust2.bsp` but backing up `de_custommap.bsp`. Therefore, the best way to go around this would be to have a file called `backup_blacklist.txt` somewhere in `lgsm/` and adding each file to be ignored, with its extension, line by line in this file. I made a similar script using Python to generate a maplist.txt file which you can find [here](https://github.com/Sarrus1/maplistGenerator) if that helps. Thanks a lot for reading!

kerem added the

type: feature

command: backup

labels

2026-02-27 03:01:55 +03:00

kerem commented

2026-02-27 03:01:55 +03:00

Author

Owner

@MicLieg commented on GitHub (Feb 25, 2024):

link #2106

@MicLieg commented on GitHub (Feb 25, 2024): link #2106

kerem commented

2026-02-27 03:01:55 +03:00

Author

Owner

@Romaq commented on GitHub (Jan 18, 2025):

This issue is stale, but there appears to be no word on it.

Backups are 37G. 26G of that is "BlueMap," a mod that displays a current state of the map as a whole. It can be entirely reconstituted from the actual *.mca files used by the server itself. Ignoring the data in ./serverfiles/bluemap/web/maps/*/tiles would vastly free up resources.

Would anyone have a work-around or alternate solution until ./command_backup.sh is refactored for an IGNORE list?

@Romaq commented on GitHub (Jan 18, 2025): This issue is stale, but there appears to be no word on it. Backups are 37G. 26G of that is "BlueMap," a mod that displays a current state of the map as a whole. It can be entirely reconstituted from the actual *.mca files used by the server itself. Ignoring the data in `./serverfiles/bluemap/web/maps/*/tiles` would vastly free up resources. Would anyone have a work-around or alternate solution until `./command_backup.sh` is refactored for an IGNORE list?

kerem commented

2026-02-27 03:01:55 +03:00

Author

Owner

@MicLieg commented on GitHub (Jan 18, 2025):

Unfortunately, most issues are yes.

Would anyone have a work-around or alternate solution until ./command_backup.sh is refactored for an IGNORE list?

You should be able to add another excluded directory within the command_backup.sh file.

tar --use-compress-program=pigz -cf "${backupdir}/${backupname}.tar.gz" -C "${rootdir}" --exclude "${excludedir}" --exclude "${lockdir}" --exclude "${tmpdir}" --exclude "${rootdir}/serverfiles/bluemap/web/maps/" ./.

If you want to exclude only the tiles from each map, you must specify each map's tiles folder separately:

tar --use-compress-program=pigz -cf "${backupdir}/${backupname}.tar.gz" -C "${rootdir}" --exclude "${excludedir}" --exclude "${lockdir}" --exclude "${tmpdir}" --exclude "${rootdir}/serverfiles/bluemap/web/maps/map1/tiles" --exclude "${rootdir}/serverfiles/bluemap/web/maps/map2/tiles" ./.

Keep in mind that updating LGSM might overwrite that change.

Alternatively, you can choose to store the Bluemap tiles in a database to also speed up map rendering in the web application.

@MicLieg commented on GitHub (Jan 18, 2025): Unfortunately, most issues are yes. > Would anyone have a work-around or alternate solution until `./command_backup.sh` is refactored for an IGNORE list? You should be able to add another excluded directory within the [command_backup.sh](https://github.com/GameServerManagers/LinuxGSM/blob/master/lgsm/modules/command_backup.sh#L129) file. ```bash tar --use-compress-program=pigz -cf "${backupdir}/${backupname}.tar.gz" -C "${rootdir}" --exclude "${excludedir}" --exclude "${lockdir}" --exclude "${tmpdir}" --exclude "${rootdir}/serverfiles/bluemap/web/maps/" ./. ``` If you want to exclude only the tiles from each map, you must specify each map's tiles folder separately: ```bash tar --use-compress-program=pigz -cf "${backupdir}/${backupname}.tar.gz" -C "${rootdir}" --exclude "${excludedir}" --exclude "${lockdir}" --exclude "${tmpdir}" --exclude "${rootdir}/serverfiles/bluemap/web/maps/map1/tiles" --exclude "${rootdir}/serverfiles/bluemap/web/maps/map2/tiles" ./. ``` Keep in mind that updating LGSM might overwrite that change. Alternatively, you can choose to store the Bluemap tiles in a [database](https://bluemap.bluecolored.de/wiki/customization/Storages.html) to also speed up map rendering in the web application.

kerem commented

2026-02-27 03:01:55 +03:00

Author

Owner

@codingWithJimmy commented on GitHub (Jan 18, 2025):

The tar command allows two separate types of exclusions when it's invoked. One is single item exclusions like the command is written now. There are two other methods that use a file to maintain a list of exclusions that could be leveraged such as -X or --exclude-file.

       -X, --exclude-from=FILE
              Exclude files matching patterns listed in FILE.

For example, you could have a setting in the game's config file like backupexclusion.cfg where each line of the file is either an explicit file or a path like serverfiles/bluemap and it should exclude that directory.

serverfiles/bluemap

Then call to the file like this.

tar --use-compress-program=pigz -cf "${backupdir}/${backupname}.tar.gz" -C "${rootdir}" --exclude "${excludedir}" --exclude "${lockdir}" --exclude "${tmpdir}" -X  "${servercfgdir}/backupexclusion.cfg" ./.

This also allows for management of other games where you can exclude the entire installation in serverfiles and severely reduce your backup size.

@codingWithJimmy commented on GitHub (Jan 18, 2025): The tar command allows two separate types of exclusions when it's invoked. One is single item exclusions like the command is written now. There are two other methods that use a file to maintain a list of exclusions that could be leveraged such as `-X` or `--exclude-file`. ``` -X, --exclude-from=FILE Exclude files matching patterns listed in FILE. ``` For example, you could have a setting in the game's config file like `backupexclusion.cfg` where each line of the file is either an explicit file or a path like `serverfiles/bluemap` and it should exclude that directory. ``` serverfiles/bluemap ``` Then call to the file like this. ``` tar --use-compress-program=pigz -cf "${backupdir}/${backupname}.tar.gz" -C "${rootdir}" --exclude "${excludedir}" --exclude "${lockdir}" --exclude "${tmpdir}" -X "${servercfgdir}/backupexclusion.cfg" ./. ``` This also allows for management of other games where you can exclude the entire installation in `serverfiles` and severely reduce your backup size.

kerem referenced this issue

2026-02-27 03:54:58 +03:00

[PR #2278] [MERGED] fix(check): prevent restart from bypassing check_executable #3566