starred/LinuxGSM

Fork 0

mirror of https://github.com/GameServerManagers/LinuxGSM.git synced 2026-04-24 21:56:01 +03:00

[GH-ISSUE #1623] steam workshop collection downloader #1269

New issue

Open

opened 2026-02-27 02:56:12 +03:00 by kerem · 27 comments

kerem commented

2026-02-27 02:56:12 +03:00

Owner

Originally created by @shadow-absorber on GitHub (Sep 20, 2017).
Original GitHub issue: https://github.com/GameServerManagers/LinuxGSM/issues/1623

as requested by @dgibbs64 i am posting the script i made to download mod collections from steam workshop.

#!/bin/bash
curl https://steamcommunity.com/sharedfiles/filedetails/?id=1101449362 > source.html
echo "./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles \\" > moddownload.sh
cat source.html \
| grep -E "<div class=\"workshopItemPreviewHolder " \
| sed 's/"><div class=.*//' \
| sed 's/.*id=//' \
| sed -e 's/^/+workshop_download_item 211820 /' \
| sed -e 's/$/ \\/' >> moddownload.sh
echo "+quit" >> moddownload.sh
chmod +x moddownload.sh
./moddownload.sh
rm source.html
rm moddownload.sh

Originally created by @shadow-absorber on GitHub (Sep 20, 2017). Original GitHub issue: https://github.com/GameServerManagers/LinuxGSM/issues/1623 as requested by @dgibbs64 i am posting the script i made to download mod collections from steam workshop. ```bash #!/bin/bash curl https://steamcommunity.com/sharedfiles/filedetails/?id=1101449362 > source.html echo "./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles \\" > moddownload.sh cat source.html \ | grep -E "<div class=\"workshopItemPreviewHolder " \ | sed 's/"><div class=.*//' \ | sed 's/.*id=//' \ | sed -e 's/^/+workshop_download_item 211820 /' \ | sed -e 's/$/ \\/' >> moddownload.sh echo "+quit" >> moddownload.sh chmod +x moddownload.sh ./moddownload.sh rm source.html rm moddownload.sh ```

kerem added the

type: feature

info: steamcmd

game: Starbound

labels

2026-02-27 02:56:12 +03:00

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

this thing should help solve the issue #960

@shadow-absorber commented on GitHub (Sep 20, 2017): this thing should help solve the issue #960

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

so what the script does in detail is:
step 1. download the workshop collection's html page
step 2. start creating a bash script to download the mods using steamcmd with the login info
step 3. output the html text in a readable way
step 4. make grep search the html file for the mod ids
step 5. format the ids in a list way for steamcmd in the bash script
step 6. add a line to quit the steamcmd download when its done
step 7. make the scrip executable
step 8. run the steamcmd script to download all the mods and validate their files
step 9. delete the files create to reset the script to before it was run to not cause conflicts when it gets run again

@shadow-absorber commented on GitHub (Sep 20, 2017): so what the script does in detail is: step 1. download the workshop collection's html page step 2. start creating a bash script to download the mods using steamcmd with the login info step 3. output the html text in a readable way step 4. make grep search the html file for the mod ids step 5. format the ids in a list way for steamcmd in the bash script step 6. add a line to quit the steamcmd download when its done step 7. make the scrip executable step 8. run the steamcmd script to download all the mods and validate their files step 9. delete the files create to reset the script to before it was run to not cause conflicts when it gets run again

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

the thing the script needs is a way to replace the id part in this line:
curl https://steamcommunity.com/sharedfiles/filedetails/?id=1101449362
and also hook into the login for steam thats used when downloading the game
wich is specified currently in this line
echo "./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles \\" > moddownload.sh

@shadow-absorber commented on GitHub (Sep 20, 2017): the thing the script needs is a way to replace the id part in this line: `curl https://steamcommunity.com/sharedfiles/filedetails/?id=1101449362` and also hook into the login for steam thats used when downloading the game wich is specified currently in this line ` echo "./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles \\" > moddownload.sh `

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@phit commented on GitHub (Sep 20, 2017):

no reason to parse the html, you can get all collection objects with the webapi specifically ISteamRemoteStorage_GetCollectionDetails

example using curl:

curl --data "collectioncount=1&publishedfileids[0]=1101449362" https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/

output example: https://gist.github.com/phit/71cea6c43c7071a40589881966aa9fa4

@phit commented on GitHub (Sep 20, 2017): no reason to parse the html, you can get all collection objects with the webapi specifically [ISteamRemoteStorage_GetCollectionDetails](https://lab.xpaw.me/steam_api_documentation.html#ISteamRemoteStorage_GetCollectionDetails_v1) example using curl: ``` curl --data "collectioncount=1&publishedfileids[0]=1101449362" https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/ ``` output example: https://gist.github.com/phit/71cea6c43c7071a40589881966aa9fa4

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

thanks that is a huge improvement then probably.
the thing is it still needs to be able to split the mods up into the list for the downloads
and not include the collection number as that could cause heavy issues

so plz if you can improve the script post the improved version and we might be able to get this implemented
@phit

@shadow-absorber commented on GitHub (Sep 20, 2017): thanks that is a huge improvement then probably. the thing is it still needs to be able to split the mods up into the list for the downloads and not include the collection number as that could cause heavy issues so plz if you can improve the script post the improved version and we might be able to get this implemented @phit

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@phit commented on GitHub (Sep 20, 2017):

something like this should do, I just cut the first publishedfileid using inverted tail, since the first result will always be the collection, obviously this means you can always only have one collection

you could probably do this way nicer using jq to parse the json, but I don't think adding a dependency for this is really wanted

CMD="./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles "
DOWNLOADITEMS=`curl -s --data "collectioncount=1&publishedfileids[0]=1101449362" https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/ \
    | sed -n 's/.*"publishedfileid": "\(.*\)",/\1/p' \
    | tail -n +2 \
    | sed 's/^/+workshop_download_item 211820 /' \
    | tr '\n' ' '`
    
echo $CMD $DOWNLOADITEMS

@phit commented on GitHub (Sep 20, 2017): something like this should do, I just cut the first `publishedfileid` using inverted tail, since the first result will always be the collection, obviously this means you can always only have one collection you could probably do this way nicer using `jq` to parse the json, but I don't think adding a dependency for this is really wanted ```bash CMD="./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles " DOWNLOADITEMS=`curl -s --data "collectioncount=1&publishedfileids[0]=1101449362" https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/ \ | sed -n 's/.*"publishedfileid": "$.*$",/\1/p' \ | tail -n +2 \ | sed 's/^/+workshop_download_item 211820 /' \ | tr '\n' ' '` echo $CMD $DOWNLOADITEMS ```

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

yeah that could work.
tough i would prefer adding \n at the end of all of the +workshop downloads to make it more readable in output form

@shadow-absorber commented on GitHub (Sep 20, 2017): yeah that could work. tough i would prefer adding \n at the end of all of the +workshop downloads to make it more readable in output form

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@phit commented on GitHub (Sep 20, 2017):

the output will never be read by anyone in the end product, so I didn't think it was worthwhile

@phit commented on GitHub (Sep 20, 2017): the output will never be read by anyone in the end product, so I didn't think it was worthwhile

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

for example the output for how i wrote mine by myself before i made the first script.

#!/bin/bash
cd /home/sbserver/steamcmd
./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles \
+workshop_download_item 211820 850109963 \
+workshop_download_item 211820 821455287 \
+workshop_download_item 211820 731220462 \
+workshop_download_item 211820 882900100 \
+workshop_download_item 211820 945707062 \
+workshop_download_item 211820 958539829 \
+workshop_download_item 211820 729426722 \
+workshop_download_item 211820 729558042 \
+workshop_download_item 211820 729429063 \
+workshop_download_item 211820 729438381 \
+workshop_download_item 211820 764887546 \
+workshop_download_item 211820 764888606 \
+workshop_download_item 211820 796877993 \
+workshop_download_item 211820 730544933 \
+workshop_download_item 211820 734170655 \
+workshop_download_item 211820 730745660 \
+workshop_download_item 211820 729427436 \
+workshop_download_item 211820 947429656 \
+workshop_download_item 211820 947922190 \
+workshop_download_item 211820 774083065 \
+quit

@shadow-absorber commented on GitHub (Sep 20, 2017): for example the output for how i wrote mine by myself before i made the first script. ```bash #!/bin/bash cd /home/sbserver/steamcmd ./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles \ +workshop_download_item 211820 850109963 \ +workshop_download_item 211820 821455287 \ +workshop_download_item 211820 731220462 \ +workshop_download_item 211820 882900100 \ +workshop_download_item 211820 945707062 \ +workshop_download_item 211820 958539829 \ +workshop_download_item 211820 729426722 \ +workshop_download_item 211820 729558042 \ +workshop_download_item 211820 729429063 \ +workshop_download_item 211820 729438381 \ +workshop_download_item 211820 764887546 \ +workshop_download_item 211820 764888606 \ +workshop_download_item 211820 796877993 \ +workshop_download_item 211820 730544933 \ +workshop_download_item 211820 734170655 \ +workshop_download_item 211820 730745660 \ +workshop_download_item 211820 729427436 \ +workshop_download_item 211820 947429656 \ +workshop_download_item 211820 947922190 \ +workshop_download_item 211820 774083065 \ +quit ```

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

which is helpful if you have to debug it for some stupid reason

@shadow-absorber commented on GitHub (Sep 20, 2017): which is helpful if you have to debug it for some stupid reason

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

but then again i guess it would work your way also without problems

@shadow-absorber commented on GitHub (Sep 20, 2017): but then again i guess it would work your way also without problems

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Sep 20, 2017):

so now with the new api thing the scrip would look like this:

#!/bin/bash
echo "./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles \\" > moddownload.sh
curl -s --data "collectioncount=1&publishedfileids[0]=1101449362" https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/ \
| sed -n 's/.*"publishedfileid": "\(.*\)",/\1/p' \
| tail -n +2 \
| sed -e 's/^/+workshop_download_item 211820 /' \
| sed -e 's/$/ \\/' >> moddownload.sh
echo "+quit" >> moddownload.sh
chmod +x moddownload.sh
./moddownload.sh
rm moddownload.sh

that would use the api avoiding future problems because of html and should probably work forever
it also makes the resulting temporary file for steam cmd a lot easier to read if something messes up.

@shadow-absorber commented on GitHub (Sep 20, 2017): so now with the new api thing the scrip would look like this: ```bash #!/bin/bash echo "./steamcmd.sh +login "USERNAME" "PASSWORD" +force_install_dir ../serverfiles \\" > moddownload.sh curl -s --data "collectioncount=1&publishedfileids[0]=1101449362" https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/ \ | sed -n 's/.*"publishedfileid": "$.*$",/\1/p' \ | tail -n +2 \ | sed -e 's/^/+workshop_download_item 211820 /' \ | sed -e 's/$/ \\/' >> moddownload.sh echo "+quit" >> moddownload.sh chmod +x moddownload.sh ./moddownload.sh rm moddownload.sh ``` that would use the api avoiding future problems because of html and should probably work forever it also makes the resulting temporary file for steam cmd a lot easier to read if something messes up.

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Oct 11, 2017):

for those wondering this is being worked on to get implemented.....

@shadow-absorber commented on GitHub (Oct 11, 2017): for those wondering this is being worked on to get implemented.....

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@Fyb3roptik commented on GitHub (Nov 15, 2017):

Any updates? How can I download a map? I am unable to do this :(

@Fyb3roptik commented on GitHub (Nov 15, 2017): Any updates? How can I download a map? I am unable to do this :(

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shinji257 commented on GitHub (Nov 27, 2017):

Does this script work if you set +login to anonymous?

@shinji257 commented on GitHub (Nov 27, 2017): Does this script work if you set +login to anonymous?

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@LoadingSAa commented on GitHub (Dec 18, 2017):

Has there been anymore work put into this? I wanted to try adapt this for use with an ARMA3 server and was wondering if this has gotten any further?

@LoadingSAa commented on GitHub (Dec 18, 2017): Has there been anymore work put into this? I wanted to try adapt this for use with an ARMA3 server and was wondering if this has gotten any further?

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Dec 19, 2017):

@shinji257 the script does not work with anonymous logins
@LoadingSAa there has been some work on it but it seemingly stopped when another person helping me quited the project
and for other games then starbound you would change the game id(in starbounds case 211820) to whatever it is for that game. i also recall that the github wiki has guides for most of the games lgsm supports

@shadow-absorber commented on GitHub (Dec 19, 2017): @shinji257 the script does not work with anonymous logins @LoadingSAa there has been some work on it but it seemingly stopped when another person helping me quited the project and for other games then starbound you would change the game id(in starbounds case 211820) to whatever it is for that game. i also recall that the github wiki has guides for most of the games lgsm supports

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@shadow-absorber commented on GitHub (Dec 19, 2017):

for a link on the project info so far: WIP

@shadow-absorber commented on GitHub (Dec 19, 2017): for a link on the project info so far: [WIP](https://github.com/JustJinxed/command_shad/blob/master/command_shadmod.sh)

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@dgibbs64 commented on GitHub (Dec 19, 2017):

Decent effort so far. I have sadly been too busy to take a proper look :(. This is something that we should look at integrating in to LinuxGSM at some point.

@dgibbs64 commented on GitHub (Dec 19, 2017): Decent effort so far. I have sadly been too busy to take a proper look :(. This is something that we should look at integrating in to LinuxGSM at some point.

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@dgibbs64 commented on GitHub (Dec 19, 2017):

Decent effort so far. I have sadly been too busy to take a proper look :(. This is something that we should look at integrating in to LinuxGSM at some point.

@dgibbs64 commented on GitHub (Dec 19, 2017): Decent effort so far. I have sadly been too busy to take a proper look :(. This is something that we should look at integrating in to LinuxGSM at some point.

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@LoadingSAa commented on GitHub (Dec 19, 2017):

This is what I have so far for arma3 there is an issue with steam CMD timing out constantly though. I'm trying to find out why.

version1: https://ghostbin.com/paste/uvxf8

this is what I'm trying to make work now:

https://gist.github.com/marceldev89/12da69b95d010c8a810fd384cca8d02a

The above script works for me the only thing I can't get to work is collections.

@LoadingSAa commented on GitHub (Dec 19, 2017): This is what I have so far for arma3 there is an issue with steam CMD timing out constantly though. I'm trying to find out why. version1: https://ghostbin.com/paste/uvxf8 this is what I'm trying to make work now: https://gist.github.com/marceldev89/12da69b95d010c8a810fd384cca8d02a The above script works for me the only thing I can't get to work is collections.

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@phit commented on GitHub (May 13, 2018):

steam started stripping whitespace at some point which broke my script, switched to jq as its more reliable

curl -s --data "collectioncount=1&publishedfileids[0]=IDHERE" https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/ \
| jq '.response.collectiondetails[] | .children[] | .publishedfileid' \
| sed 's/^/+workshop_download_item 211820 /' | tr '\n' ' '

@phit commented on GitHub (May 13, 2018): steam started stripping whitespace at some point which broke my script, switched to jq as its more reliable ```bash curl -s --data "collectioncount=1&publishedfileids[0]=IDHERE" https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/ \ | jq '.response.collectiondetails[] | .children[] | .publishedfileid' \ | sed 's/^/+workshop_download_item 211820 /' | tr '\n' ' ' ```

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@rafaeljardim1660 commented on GitHub (Dec 9, 2018):

this script is for python?

@rafaeljardim1660 commented on GitHub (Dec 9, 2018): this script is for python?

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@linonetwo commented on GitHub (Dec 15, 2018):

And after that, you should edit the serverfiles/linux/sbinit.config

{
  "assetDirectories": [
    "../assets/",
    "../mods/",
    "../../Steam/steamapps/workshop/content/211820/939132577",
    "../../Steam/steamapps/workshop/content/211820/1507130642"
  ],
  "storageDirectory": "../storage/"
}

or steam/steamapps/common/Starbound/linux/sbinit.config if you are not using LinuxGSM.

{
  "assetDirectories" : [
    "../assets/",
    "../mods/",
    "/path/to/Steam/SteamApps/workshop/content/211820/733063633/",
    "/path/to/Steam/SteamApps/workshop/content/211820/743283539/",
    "/path/to/Steam/SteamApps/workshop/content/211820/729769049/",
    "/path/to/Steam/SteamApps/workshop/content/211820/736472581/",
    "/path/to/Steam/SteamApps/workshop/content/211820/732860513/"
  ],

  "storageDirectory" : "../storage/"
}

Don't forget to check whether this file is a good JSON.

@linonetwo commented on GitHub (Dec 15, 2018): And after that, you should edit the `serverfiles/linux/sbinit.config` ```json { "assetDirectories": [ "../assets/", "../mods/", "../../Steam/steamapps/workshop/content/211820/939132577", "../../Steam/steamapps/workshop/content/211820/1507130642" ], "storageDirectory": "../storage/" } ``` or `steam/steamapps/common/Starbound/linux/sbinit.config` if you are not using LinuxGSM. ```json { "assetDirectories" : [ "../assets/", "../mods/", "/path/to/Steam/SteamApps/workshop/content/211820/733063633/", "/path/to/Steam/SteamApps/workshop/content/211820/743283539/", "/path/to/Steam/SteamApps/workshop/content/211820/729769049/", "/path/to/Steam/SteamApps/workshop/content/211820/736472581/", "/path/to/Steam/SteamApps/workshop/content/211820/732860513/" ], "storageDirectory" : "../storage/" } ``` Don't forget to check whether this file is a good JSON.

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@Scassany commented on GitHub (Mar 11, 2020):

Is there any update on this? I'm interested in using a script like this to run my server and if there's already a working script or implementation in gsm, that would be awesome

@Scassany commented on GitHub (Mar 11, 2020): Is there any update on this? I'm interested in using a script like this to run my server and if there's already a working script or implementation in gsm, that would be awesome

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@dgibbs64 commented on GitHub (Mar 12, 2020):

@1285done there is task to review workshop support but cant give any eta

@dgibbs64 commented on GitHub (Mar 12, 2020): @1285done there is task to review workshop support but cant give any eta

kerem commented

2026-02-27 02:56:13 +03:00

Author

Owner

@ahoys commented on GitHub (Jul 27, 2022):

For those looking for a fast solution for this, I created a small standalone script that downloads mods based on a workshop collection or by a custom mod file. https://github.com/ahoys/useful-sh/tree/main/steam-workshop-collection-downloader

I use it with a LinuxGSM arma3server instance.

@ahoys commented on GitHub (Jul 27, 2022): For those looking for a fast solution for this, I created a small standalone script that downloads mods based on a workshop collection or by a custom mod file. [https://github.com/ahoys/useful-sh/tree/main/steam-workshop-collection-downloader](https://github.com/ahoys/useful-sh/tree/main/steam-workshop-collection-downloader) I use it with a LinuxGSM arma3server instance.