mirror of
https://github.com/librespot-org/librespot.git
synced 2026-04-27 08:15:50 +03:00
[GH-ISSUE #972] Suddenly unable to play any tracks #469
Labels
No labels
A-Alsa
SpotifyAPI
Tokio 1.0
audio
bug
can't reproduce
compilation
dependencies
duplicate
enhancement
good first issue
help wanted
high priority
imported
imported
invalid
new api
pull-request
question
reverse engineering
wiki
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/librespot#469
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @kosimst on GitHub (Mar 2, 2022).
Original GitHub issue: https://github.com/librespot-org/librespot/issues/972
I am using LibreSpot with the JACK Audio backend, and it worked perfectly, but since a few days I can't play any tracks. I didn't change any configurations on my side, so I assume there is a problem with spotify like it already occured here: #510
Setup
Custom compiled with
--no-default-featuresand--features jackaudio-backendrunning as a user service on a Raspberry Pi.Used Arguments:
LibreSpot allows connections from any device just fine, but can't play any song, no matter of its length. Mots of the time it just skips every song after a few seconds without any audio played, but sometimes it also crashes completely.
Logs:
@michaelherger commented on GitHub (Mar 2, 2022):
Where about on this planet are you located? I've received a few similar reports from users in the past few days.
@cvejlbo commented on GitHub (Mar 2, 2022):
Just in case you are gathering stats/data:
Same issue for me (in Denmark).
@jesperll commented on GitHub (Mar 2, 2022):
Same here (via Spotty integration in Logitech Media Server) (also Denmark)
@michaelherger commented on GitHub (Mar 2, 2022):
@jesperll - you're not JesperDB who's been reporting this similar issue to me, are you?
That then would be three Danish users. It's hard to believe this was a coincidence...
@kosimst commented on GitHub (Mar 2, 2022):
I am from Austria @michaelherger, so maybe the problem is in europe.
@ashthespy commented on GitHub (Mar 2, 2022):
Also saw similar channel errors from DK..
@JesperBarrit commented on GitHub (Mar 2, 2022):
No, that's me and I'm also in Denmark :)
I have the same issue here:
https://forums.slimdevices.com/showthread.php?111923-Announce-Spotty-4-0-integrate-local-library-with-your-Spotify-collection-(LMS-8-)&p=1049289&viewfull=1#post1049289
@cvejlbo commented on GitHub (Mar 3, 2022):
FWIW: A similar issue seem to happen if I try using
new-apibranch, except the spotify client/UI on my PC/phone does not repeatedly skip fwd after ~5 secs [still no playback, though]Console log with
-vfromnew-apibranch:However, if I start librespot with
-u <username> -p <password>with same build fromnew-apibranch, I am able to play back the audio.Maybe there were some recent changes to how the authentication is done with zeroconf?
@roderickvd commented on GitHub (Mar 3, 2022):
Put simply, we can't do anything about channel errors. They sometimes hit the Spotify infrastructure and then go away after some time.
The panic is due to trying to reload a track that couldn't be loaded the first time. This is fixed in
new-apimeaning that it won't panic anymore, but still won't be able to play the track.There is no difference in authentication to the Spotify infrastructure between zeroconf and command line, all that matters is the way it receives the credentials (either from the command line or over the LAN). From there it authenticates against Spotify in the same way.
It's more likely that the time you logged in with command line credentials, you connected to an access point that didn't suffer from channel errors. You can check the access point in your (verbose) logs when librespot first connects.
I'm closing this as this is not something we can do anything about.
@kingosticks commented on GitHub (Mar 3, 2022):
I do agree in general. But we could in theory provide an API to control what access point is used rather than always taking the first (or last, whatever it is, I find the comment there confusing). I am not saying that should then also be exposed to librespot application users (yet another program option) but it could.
@morphar commented on GitHub (Mar 3, 2022):
Just sharing info, in case it's helpful:
In DK.
2 official spotify clients working (1 not updated macOS, 1 latest releas iOS).
They play just fine, except for trying to use Spotify Connect, with anything derived from or using librespot to enable Spotify Connect (E.g. Volumio 3 and spotifyd).
Problem: Causes all tracks to skip.
@puerto-rico commented on GitHub (Mar 3, 2022):
i am seeing same thing here. also in denmark. skips all tracks in playlist - happend on 2 pi's at the same time.
but because there where running a older version of dietpi i upgrade to bulseye and installed from scratch. with no luck.
All tracks get skipped and from journalctl -f i can see the error.
@michaelherger commented on GitHub (Mar 4, 2022):
Would everybody mind sharing
Though there have been reports from Germany and Austria, too, I strongly believe this is either a geographical thing, or some network having issues or blocking things.
@morphar commented on GitHub (Mar 4, 2022):
Of course! :)
Here you go:
ISP: Hiper A/S (Bought by TDC, so they probably run on their network now)
http://apresolve.spotify.com/ gives:
I just did the test again and the client still does the skipping.
@roderickvd commented on GitHub (Mar 4, 2022):
You will also want to lookup those host names and report their IP. Same names could result in different IPs depending on geography, I’m not sure.
@morphar commented on GitHub (Mar 4, 2022):
ap-gew4.spotify.com: 34.158.0.131
ap-guc3.spotify.com: 104.154.127.126
ap-gae2.spotify.com: 104.199.240.237
ap-gew1.spotify.com: 104.199.65.124
@roderickvd commented on GitHub (Mar 4, 2022):
From The Netherlands (Ziggo) I get the same IP addresses for those hosts. My
apresolveanswer looks like this and I can play tracks fine:librespotpicks the first access point from the list, in my caseap-gew1.spotify.com:4070. Again you can see this in your verbose logs.@morphar commented on GitHub (Mar 4, 2022):
Interesting, mine chooses the ap-gew4:
And then starts skipping with these in the log:
@morphar commented on GitHub (Mar 4, 2022):
I just tried to force the
ap-gew4.spotify.comdomain to point to the IP of ap-gew1.spotify.com, by setting this in /etc/hosts:That works!! Sooo.... Is Spotify having an issue? Or are Spotify changing something?
@michaelherger commented on GitHub (Mar 4, 2022):
Ha, you beat me to it! I wanted to test the opposite (using
ap-gew4instead of what I receive).Yes, I believe there's a problem with that one host.
@morphar commented on GitHub (Mar 4, 2022):
That's annoying... Have you experienced this before? In that case: how long does this normally last? I'd rather not do this trick on my music streamer, but if it's normally a problem that takes a long time to fix, I might need to :)
Thanks for your time guys!
And thanks for this project! I love to have easy ability to get clean 320 bit directly into my DAC board :)
@michaelherger commented on GitHub (Mar 4, 2022):
I don't think it's a common problem. But it's probably still worth trying to report to Spotify?
And should librespot be smarter and not only rely on the first element in the list?
@morphar commented on GitHub (Mar 4, 2022):
I was thinking the same thing. Maybe there's some indicator that can be used to drop a host and try the other.
I guess in this case it could be difficult, since the server and API is responding on some calls.
@roderickvd commented on GitHub (Mar 4, 2022):
Channel errors are not new. If you use GitHub search on this project you will find a number of reports them. How long they'll last, only Spotify could say.
Programmatically detecting channel errors is not difficult, you literally see
channel error: 2 1in the log so we could act on that. But architecturally it's more difficult to letlibrespotreconnect to another access point. Innew-apiit's a bit easier to implement but still needs work.@morphar commented on GitHub (Mar 4, 2022):
Nice that there is something to detect it by :)
But yeah... I can imagine how many little details has to be thought through in order for a total re-connect to work seamlessly.
I wish I had learned Rust by now, so that I could try to help out.
@puerto-rico commented on GitHub (Mar 4, 2022):
i know it is not a beautiful solution but i can confirm that i works. as morphar suggested. :) thanks for the tip.
But instead of doing in all the pi's /etc/hosts we have implemented the redirect in our dns - 104.199.65.124 ponting to ap-gew4.spotify.com
@nhey commented on GitHub (Mar 6, 2022):
@roderickvd Would you accept a PR with a "fix" where the host is picked at random instead of picking the first one returned?
If the bad host is picked uniformly at random, natural trouble-shooting behaviour like restarting librespot would likely fix the issue.
For example, if one out of five hosts are bad, the probability of picking the bad host twice in a row is just 4% (or viewed differently, there is a 20% chance the restart does not fix the issue). Though, I should say, apresolve.spotify.com returns three out of five bad hosts for me (same host on different ports), so it may be less effective in practice...
@michaelherger commented on GitHub (Mar 6, 2022):
My reading of the host names is that the other two would be US Central and Asia East (Taiwan). Such choices could come at a cost in latency.
In Switzerland I received another set of European hosts. I therefore don’t think randomly picking one would be a good choice most of the time. I’d rather go with an option to force some host blame instead. But given my experience of the past 4-5 years would say it’s not worth the effort, as I can’t remember a similar incident.
@nhey commented on GitHub (Mar 6, 2022):
They do seem like regional names; I agree that it's not a desirable solution then.
@JGNi commented on GitHub (Mar 6, 2022):
{"ap_list":["ap-gew4.spotify.com:4070","ap-gew4.spotify.com:443","ap-gew4.spotify.com:80","ap-gae2.spotify.com:4070","ap-gew1.spotify.com:443","ap-guc3.spotify.com:80"]}
Denmark here, and not working.
@michaelherger commented on GitHub (Mar 7, 2022):
See @morphar's comment further up this report:
@Mads80 commented on GitHub (Mar 7, 2022):
Hello, I just want to say thank you for fixing my Spotify Connent issues on Volumio.
@roderickvd commented on GitHub (Mar 7, 2022):
@kingosticks's suggestion to offer an ABI to manually set the access point isn't a bad idea in this particular case. I am however hesitant to offer it as a command-line option because I don't want to get it misused.
What you can also try is blocking
apresolve.spotify.comaltogether, which will causelibrespotto default to pickap.spotify.comas a fallback access point. I assume that this is geobalanced and might offer some other, in working order, access point.I consider all of these as band-aids, ideally
librespotwould reconnect and drop the erroneous access point. Like I said I can investigate this as part ofnew-api. It's not on top of my list -- I'd rather have Spotify to fix their infrastructure -- but I should get around to it sometime. In the meantime PR's are more than welcome of course.@michaelherger commented on GitHub (Mar 7, 2022):
One way to enforce this behavior actually already is built in: set
ap-portto some random, invalid port. If librespot fails to connect to that port, it would use the fallback.@roderickvd commented on GitHub (Mar 7, 2022):
Ha I never thought about that, talk about emergent behaviour 😆
@michaelherger commented on GitHub (Mar 7, 2022):
I didn't know about
ap-portuntil I started looking into how an override could be implemented 😄.@michaelherger commented on GitHub (Mar 8, 2022):
I got reports that things started working again. Can anybody confirm? Remove workaround and try again?
Doesn't seem to be working for me.
@morphar commented on GitHub (Mar 8, 2022):
It doesn't work for me.
Maybe the reports are false positives? Maybe they can stream, but it's because they're not connecting to ap-gew4.spotify.com for some reason?
@michaelherger commented on GitHub (Mar 8, 2022):
Yes. It was a misunderstanding. The person was already using a workaround I implemented in my Spotty plugin (forcing fallback by providing an invalid
ap-portvalue). That seems to be working for people with the failingap-gew4.spotify.com.@knoer commented on GitHub (Mar 15, 2022):
Confirming that I also had this skipping issue here in Denmark for at least a week before finding the time to search for solutions.
Apresolve points to
ap-gew4.spotify.com- but through a VPN to Sweden/Stockholm I getap-gew1.spotify.comso this could be a country specific Spotify issue..Can confirm that using the workaround @michaelherger supplied with spotty version 4.8.1 solved my issues.
-for now at least. 🤔
Thanks Michael!
@jesperll commented on GitHub (Mar 15, 2022):
I've also updated to Spotty 4.8.1 but that didn't solve my issues. ;(
I've tried multiple ways of tricking my installation to resolve gew4 to a different IP but so far without success.
(Running LMS with Spotty as addon under home assistant)
@michaelherger commented on GitHub (Mar 15, 2022):
Did you check the new "fallback" option in the Spotty settings? No need to fiddle with DNS.
@jesperll commented on GitHub (Mar 15, 2022):
@michaelherger No, I didn't even look for a setting. I falsely assumed it "just did it"
It works now!
Thanks!
@radiorasmus commented on GitHub (Apr 8, 2022):
I have two things working against me, Not were good at programming and I'm from Denmark. Spottify Connect still doesn't work here ;(.
I have logged into my rasp4 with a ssh. OS is HiFiBerryOS. How do I get from here, to implementing :
"by setting this in /etc/hosts:
104.199.65.124 ap-gew4.spotify.com"
??
@Mads80 commented on GitHub (Apr 8, 2022):
@radiorasmus
Type "sudo nano /etc/hosts", and paste in "104.199.65.124 ap-gew4.spotify.com" exit and save and you should be up and running again. You might have to restart the system, not sure.
@radiorasmus commented on GitHub (Apr 8, 2022):
Hi Mads80,
I think I got it, omitted the "sudo" though. Is there any known and easy cmd that can confirm. Spotify connect went from skipping songs after 1 sec to stopping music after 1 min. It's an improvement ;)
@user7743 commented on GitHub (Apr 11, 2022):
Glad I found this thread. For me without this fix I couldnt play anything through Spotty plugin or librespot for weeks. Still very much an issue for me.
@kirenida commented on GitHub (May 8, 2022):
This problem started for me a few days ago in Norway.
After finding this thread, I blocked
apresolve.spotify.com, and now librespot works OK again. For now.@michaelherger commented on GitHub (May 8, 2022):
Yeah, got reports from Norwegian users in the past two days, too...
@spinningarrow commented on GitHub (May 12, 2022):
+1 here from Norway
@adriantaga commented on GitHub (May 15, 2022):
+1 here from Norway.
@willtonkin commented on GitHub (May 23, 2022):
+1 from here in Denmark 🇩🇰
@bjacobse commented on GitHub (Jun 6, 2022):
+1 from here in Denmark
Thank you for your advice
@Boxer66 commented on GitHub (Jun 10, 2022):
+1 from Norway. Fixed hosts in Arch and Win11 and working fine for now.(ncspot)
@orjanv commented on GitHub (Jun 18, 2022):
+1 from Norway, glad I found this thread, and the hosts entry did the job.
@fi5ch commented on GitHub (Jul 14, 2022):
+1 from Switzerland
With "104.199.65.124 ap-gew4.spotify.com" in the /etc/host file it works, thanks!
I am not sure what is causing the problem and how to properly fix it.
@Baktus79 commented on GitHub (Jul 14, 2022):
+1 from Norway
Adding
104.199.65.124 ap-gew4.spotify.comto /etc/hosts did solve the problem. Have had this issue for a while and suddenly came across this thread.Anyone got any answers from Spotify? This is lazy work at their end.
@agramner commented on GitHub (Jul 14, 2022):
+1 from Sweden
Used the "fallback ap hack". Set
--ap-port=[some random port]. Log then said Using fallback "ap.spotify.com:443" and it works!@Bettehem commented on GitHub (Jul 15, 2022):
+1 from Finland
Adding a line containing
104.199.65.124 ap-gew4.spotify.comto/etc/hostssolved the issue.@michaelherger commented on GitHub (Jul 15, 2022):
I've now had reports from Austria and Switzerland, too... are they shutting down old infrastructure?
@user7743 commented on GitHub (Jul 15, 2022):
I know its not quite AT & CH but I can confirm in Germany I have had this issue for quite some time.
@thomas-ah commented on GitHub (Jul 15, 2022):
+1 from The Netherlands (using
ncspotunder macos).Fixed by adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts :)
@westham commented on GitHub (Jul 15, 2022):
+1 from Sweden
Adding a line containing 104.199.65.124 ap-gew4.spotify.com to /etc/hosts solved the issue.
@emilBeBri commented on GitHub (Jul 15, 2022):
Nice that also worked for me, in denmark, thx
@philgzl commented on GitHub (Jul 15, 2022):
+1 from Denmark 🙏🙏🙏
@roderickvd commented on GitHub (Jul 15, 2022):
We have no lines of communication to Spotify.
It would surprise me because many legacy devices would suffer. I don’t think Spotify announced the EOL for such hardware.
On the other hand I don’t understand as this is just their own resolver, which last I checked was also used by their own clients. One difference is that I saw so when I was working on
new-apiwith HTTP. Who knows the Mercury is phased out in that server cluster, and legacy clients fall back?Seeing how this issue is only becoming worse we could consider blacklisting this host name.
@VB1987 commented on GitHub (Jul 16, 2022):
+1 from The Netherlands.
Fixed by adding
104.199.65.124 ap-gew4.spotify.comto/etc/hosts@andrzejj0 commented on GitHub (Jul 17, 2022):
+1 from 🇳🇱, the above fix helped me too
@CaptainAweesome commented on GitHub (Jul 17, 2022):
I am on Balena OS using balena sound to play tracks from Spotify.
I added
104.199.65.124 ap-gew4.spotify.comto/etc/hosts.But it is non persistant, since it is running in a container.
I tried adding it to pi.hole local DNS records. I can ping
ap-gew4.spotify.comfrom Spotify container and Host OS with reply.With no luck tracking down the real issue.
Log-snippet:
@agramner commented on GitHub (Jul 17, 2022):
@CaptainAweesome Maybe try extra_hosts in docker compose? See https://docs.docker.com/compose/compose-file/compose-file-v3/#extra_hosts
Or if possible modify https://github.com/balenalabs/balena-sound/blob/master/plugins/spotify/start.sh and add command line arg --ap-port=[some random port]
@kirenida commented on GitHub (Jul 17, 2022):
Try blocking
apresolve.spotify.comin pihole, that worked for me.@CaptainAweesome commented on GitHub (Jul 17, 2022):
This actually worked. I am astounded. Thank you very much!
@agramner I did not find any recommended ways to add parameters to containers in Balena. Since Balena-sound is a custom build of Balena OS with many containers, without any excessive customizability.
None the less, I thank you for your reply!
@CaptainAweesome commented on GitHub (Jul 19, 2022):
I'm now back to the same errors as before. Unable to load tracks from Spotify.
apresolve.spotify.comis blocked in pi.hole andnslookup apresolve.spotify.comgives the following address:0.0.0.0Even though I can access
http://apresolve.spotify.com/from my browser all of a sudden.@juhi24 commented on GitHub (Jul 19, 2022):
Had this problem today in Finland. Blocking
apresolve.spotify.comon pihole was a successful workaround.@andrzejj0 commented on GitHub (Jul 19, 2022):
@CaptainAweesome how do you precisely run librespot in a container? Do you use docker or docker-compose or something else?
Btw, it doesn't look like it's a librespot bug.
If you're using plain docker, try adding
--add-host apresolve.spotify.com:127.0.0.1to the docker command.the above will block apresolve.spotify.com.
@CaptainAweesome commented on GitHub (Jul 20, 2022):
@ajarmoniuk
It's a way of deploying docker containers to devices and managing said devices remotely.
Here is the balena-sound startup for Spotify: https://github.com/balenalabs/balena-sound/blob/master/plugins/spotify/start.sh
I have now instead of adding
apresolve.spotify.comto Blacklist, addedapresolve.spotify.com 127.0.0.1to Local DNS Records inside pi.hole. And strangely it works now, again. Thank you!@jelmerderonde commented on GitHub (Jul 20, 2022):
+1 from NL. Adding
104.199.65.124 ap-gew4.spotify.comto/etc/hostsfixed it. Also blockedapresolve.spotify.comin pihole to be on the safe side ;-).@sucama2022 commented on GitHub (Jul 20, 2022):
Hello, I am having the same problem with Moodeaudio version 8.1.2. with raspberry pi4.
I can't listen to Spotify. I am in Argentina, can someone help me with the dns of this area, to add: xxx.xxx.xxx.xxx ap-gew4.spotify.com to /etc/hosts? Thank you!
@sucama2022 commented on GitHub (Jul 20, 2022):
sucama@sucama-B250M-D3H:~$ ping ap-gew4.spotify.com
PING ap-gew4.spotify.com (34.158.0.131) 56(84) bytes of data.
64 bytes from 131.0.158.34.bc.googleusercontent.com (34.158.0.131): icmp_seq=1 ttl=101 time=246 ms
64 bytes from 131.0.158.34.bc.googleusercontent.com (34.158.0.131): icmp_seq=2 ttl=101 time=245 ms
64 bytes from 131.0.158.34.bc.googleusercontent.com (34.158.0.131): icmp_seq=3 ttl=101 time=246 ms
64 bytes from 131.0.158.34.bc.googleusercontent.com (34.158.0.131): icmp_seq=4 ttl=101 time=246 ms
@dsolano04 commented on GitHub (Jul 20, 2022):
Hi Sucama, I've just solved the issue by changing /etc/hosts as follows:
104.199.65.124 ap-gue1.spotify.com
Basically, ap-gue1.spotify.com is the host volumio/spotify try to resolve from Argentina but it doesn't work, so you need to point it to ap-gew4.spotify.com by adding its IP address (104.199.65.124) in the /etc/hosts file.
Make sure after the change to restart the network interfaces and restart volumio. You can also restart the whole server to make it easier.
Hope it helps.
@programamapache commented on GitHub (Jul 20, 2022):
Hi Sucama, I'm from Argentina too and I had been having this problem since a week ago. Yesterday I edited hosts file with:
0.0.0.0 apresolve.spotify.com
and now works fine. Tested spotify all day without problems.
I'm using HiFiBerry OS with Raspberry 3+.
Just for information, along all the trials I performed with no luck (updating Spotify in mac and iOS devices, clean install of HiFiBerry OS on the raspy, downgrading HiFiBerry OS to prior version) I installed spotify app in my LG TV and it worked fine. I could connect from spotify desktop and control the TV but not the raspberry. So I'm not sure if this is a DNS problem alone.
Hope it helps.
@EdoardoLaGreca commented on GitHub (Jul 21, 2022):
I had the same issue, I'm from Italy 🤌. I'm using a Raspberry Pi 3B with Raspberry Pi OS and Raspotify.
I added the following line to
/etc/hostsand now everything works fine:@dotto95 commented on GitHub (Jul 21, 2022):
Thank you @EdoardoLaGreca :) Same country same fix ;)
@kingcharles1666 commented on GitHub (Jul 21, 2022):
+1 from NL. Adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts fixed it for me
@d-ilievski commented on GitHub (Jul 21, 2022):
Same for me :) I was struggling for 3 days tinkering with my devices to find this thread and learn it's from Spotify's side. Thanks, guys, you are the best!
@kalj commented on GitHub (Jul 22, 2022):
Awesome workarounds! However, I am a bit curious what the difference is between defining
and
Both seem to fix the problem for me. But is either preferable for some reason?
@sucama2022 commented on GitHub (Jul 22, 2022):
Hi , Thanks To programamapache for this:
0.0.0.0 apresolve.spotify.com
Work for me in Argentina.. Moodeaudio versio 8.1.2 , Raspberry pi4
Thanks Very much!
@kirenida commented on GitHub (Jul 22, 2022):
Have you tried opening
apresolve.spotify.comin your browser?If you do that, you will get something like:
{"ap_list":["ap-gew1.spotify.com:4070","ap-gew1.spotify.com:443","ap-gew1.spotify.com:80","ap-gew1.spotify.com:4070","ap-guc3.spotify.com:443","ap-gae2.spotify.com:80"]}which is a list of Spotify content delivery servers most suitable for your location (as determined from Spotify's side)
As stated previously in this thread, it looks like Spotify has started rolling out some new method of serving content through some of these servers. That new method works with the official Spotify client, but not with librespot. (Which I don't consider "lazy" as stated in a previous comment - why should they care about unofficial clients?)
With
apresolve.spotify.comblocked, librespot defaults to some server that doesn't have that new method enabled (yet).If you just define a single server, there is a chance that the response from
apresolvecould change at some point in time, and then you would need to enter a new combination of IP address + URL.As I'm not experienced with the inner workings of librespot or Spotify, I'm just going with my basic knowledge and logic, so what I've just written may be wrong in some way. If that is so, somebody please correct me :)
@coaxial commented on GitHub (Jul 23, 2022):
@kirenida:
Because they deprecated the official libspotify so there isn't an "official client" anymore.
@kingosticks commented on GitHub (Jul 23, 2022):
Sorry to be pedantic but libspotify was deprecated in May 2015, they've not given a hoot for 7 years.
Edit:
And it is lazy since they promised a replacement back and then and didn't deliver. Useless? Lazy? Pick one!
Anyway, grumbling aside, what's interesting is there is another potential breakage coming in September 2022. This might be a deadline for getting new api branch merged (sorry, I know deadlines are not helpful @roderickvd and it's on everyone to step up and help). https://developer.spotify.com/community/news/2022/07/15/mobile-streaming-sdks-update/
@roderickvd commented on GitHub (Jul 23, 2022):
I was hoping to make some time to do a quick update release to blacklist this “broken” (to us) access point, but chances of me completely upgrading to the HTTP-based API before September are next to zero:
the current state of
new-apiis functional but needs bugs ironed out anddevmerged in. I don’t realistically have time for all of that before September. I have called multiple times for people to take on that chore but no one has answered the call.new-apinow retrieves files over audio files over HTTP but not encryption keys or commands.Now I don’t know what Spotify will be deprecating and what not, but:
the encryption keys over HTTP have not been reversed engineered yet (with decompilation attempts from some having been taken down).
the HTTP-based network commands (websockets) require implementing the dealer and this will take a lot of work as well as some advanced Rust skills and knowledge of librespot internals.
librespot-javahas the reverse engineering mostly down but it’s much more than just a matter of Java-2-Rust as the code architecture has diverged a lot.Again I don’t know what that Spotify SDK deprecation may bring but if this project wants to stay up to date then we really need more contributors.
@kingosticks commented on GitHub (Jul 23, 2022):
Good summary. I have a few Python bits to finish in my downstream project this weekend, after which I'm going to focus on getting gstspotifysrc where I need it to be. I'll switch it to the new-api (and worry about the release implications of that later) and then be some help to you here. I don't need all of librespot's features for my stuff but I'll do what I can.
However, I don't profess to having advanced Rust skills.
@TheN00r commented on GitHub (Jul 24, 2022):
Can confirm from the Netherlands that adding 104.199.65.124 ap-gew4.spotify.com to /storage/.config/hosts.conf did the trick as well!
Thanks!
@michaelherger commented on GitHub (Jul 25, 2022):
Just a little heads up. As outlined before there's no need to fiddle with DNS, hosts files, IP addresses etc. You can work around this issue using a librespot parameter:
@SuisChan commented on GitHub (Jul 25, 2022):
As a fallback - we can use widevine, but it will probably need a lot of re/writing code because it differs a lot from the current implementation.
@kingosticks commented on GitHub (Jul 25, 2022):
What do you mean by that @SuisChan? Widevine is a (commercial?) closed source library, isnt it? Reverse engineering that software would be like poking a (small, angry) bear.
@SuisChan commented on GitHub (Jul 25, 2022):
Everything is not quite right, since we can get a private key, we can use it accordingly, and the algorithms have been public for a long time (leaks and not only), besides, I have experience working with it, so I can confirm that it works.
@khink commented on GitHub (Jul 26, 2022):
+1 from NL. Adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts fixed it for me
@roderickvd commented on GitHub (Jul 26, 2022):
Please take #1026 for a spin. If this works (it does for me -- I also get
ap-gew4in the Netherlands usually) I can do a point release the coming days.@Boobybandit commented on GitHub (Jul 28, 2022):
For me adding ap-gew4 (manually) did not work for 2 out of 3 devices. Adding "0.0.0.0 apresolve.spotify.com" however did work on all 3 devices. Also Netherlands here, so that's a bit weird.
If I understand correctly then adding ap-gew4 is rerouting the stream while adding 0.0.0.0 is blocking some resolve api? Anyways, for now everything works here (faster than usual btw).
@roderickvd commented on GitHub (Jul 28, 2022):
Yes blocking
apresolveblocks the resolving mechanism altogether.It would be helpful if you could list those other access points besides
ap-gew4on those 2/3 devices that did not work for you?@HippieWithGuns commented on GitHub (Jul 28, 2022):
Fix worked for me aswell! Thank you so much <3
User from Switzerland here.
and just to be clear the fix for me was: add this line to etc/hosts
0.0.0.0 apresolve.spotify.com
@Boobybandit commented on GitHub (Jul 28, 2022):
Unfortunately I can't extract logs from my spotify implementation in Hifiberry (or don't know how). But, I just randomly tried adding other servers and apparently my 1 Hifiberry device uses "ap-guc3.spotify.com". Rerouting this to 104.199.65.124 did the trick. Seems completely random as I have 3 Hifiberry devices here and 1 uses the ap-gew4, the other ap-guc3 and the third I haven't checked yet. My ISP is KPN (glasvezel).
My (foolproof) suggestion would be to add multiple servers to the hosts file, pointing to 104.199.65.124 or blocking apresolve.
@roderickvd commented on GitHub (Jul 28, 2022):
Hmm. You’re the first to report defunct servers other than
ap-gew4but this may be because so many others are blocking the resolver completely.I’d like some confirmation on this because it matters for the PR I linked to. I don’t like blocking the resolver completely but if access point compatibility is dropping like flies then we might have to. Note that the GeoIP-based fallback might still direct you to a broken server so it’s not entirely foolproof.
HiFiBerryOS runs Spotifyd right? That’s a good crew but know that it’s still running on librespot 0.2 so I am unsure how fast they will pick up a librespot point release with a small fix.
@andrzejj0 commented on GitHub (Jul 28, 2022):
I have just made a test by using each gateway's IP address for all hosts in
/etc/hostsand it turns out only34.158.0.131(which isap-gew4) is using the new interface so far.@raabf commented on GitHub (Jul 28, 2022):
librespot also did stop working for me since this morning due to the problem described in this issue. I am located in Bavaria, Germany.
my
apresolve.spotify.comalso contains theap-gew4as first entry.Can confirm that adding
OR
to
/etc/hostsboth work for me.@piotrzieba commented on GitHub (Jul 28, 2022):
Hello, noob here – I added following line to /etc/hosts of Spotify service (via Balena dashboard -> Terminal -> Spotify target -> then editedd with vi, saved with :x), but the changes are not kept after rebooting.
Could you please explain step by step how to make this fix?
Context: unable to play songs since this morning, I'm also located in Germany. I'm using BalenaSound, not Librespot, but this issue has been seen across many platforms (also Volumio afaic)
@coaxial commented on GitHub (Jul 28, 2022):
It seems like you're running in a container. You have to make this change in the container's config or network wide (typically in your router). It very much depends on your particulars: ask the community support for your container image or check how to do it with your particular router model.
@roderickvd commented on GitHub (Jul 28, 2022):
I merged #1026 with a fix for
ap-gew4. If nothing else pops up I will do a0.4.2release the coming days.@marciogranzotto commented on GitHub (Jul 28, 2022):
@roderickvd also having the same problem with
ap-gue1@miguelprates commented on GitHub (Jul 29, 2022):
Here in Brazil
ap-gue1is the problem.I had to redirect dns from
ap-gue1toap-gew1on my router and now it's working.@mario-ruoff commented on GitHub (Jul 29, 2022):
I just temporary fixed the issue for BalenaSound by adding a command in the start.sh file of the Spotify Docker container:
echo "104.154.127.126 ap-gew4.spotify.com" >> /etc/hosts@roderickvd commented on GitHub (Jul 29, 2022):
Thanks @miguelprates and @marciogranzotto, I will add
ap-gue1to the blacklist and do a release. Hope this will cover most of the cases and otherwise people can use any of the workarounds mentioned in this issue.Will thereafter focus on development of
new-apiwhich is where the real fix is -- that new API doesn't run into trouble with any of these access points because it downloads files over HTTP instead of Mercury.@roderickvd commented on GitHub (Jul 29, 2022):
v0.4.2 has been released: https://github.com/librespot-org/librespot/releases/tag/v0.4.2
@nis65 commented on GitHub (Aug 12, 2022):
Just to help others that come here with the same error: I have a setup with a
dnsmasqin my internal network and multiple librespot devices in that. network. So if you add0.0.0.0 apresolve.spotify.comto your
dnsmasqconfig (e.g./etc/hosts) on your (wlan) router, all librespot devices profit immediately from that fix. Of course it's better to upgrade 😄@HippieWithGuns commented on GitHub (Aug 16, 2022):
May i ask what you mean by, of corse it's better to update? is there any newer client to stream spotify to a pi, or what do you mean? :)
@roderickvd commented on GitHub (Aug 16, 2022):
v0.4.2 works around the issue for two known bad access points.
You can also compile
0.5.0-devfrom source which fixes the issue entirely by using another API. It’s in actively development so expect other bugs (and we’d welcome the feedback).@JasonLG1979 commented on GitHub (Aug 25, 2022):
Not so much. I'm getting 403 errors for
hm://keymaster/token/authenticated?scope=playlist-readon0.5.0-devI have no problems withv0.4.2though.It looks like they're shutting down or at least scaling back that interface.
@roderickvd commented on GitHub (Aug 25, 2022):
Which commit of
0.5.0-devis this, the latest?I don't know,
0.4.2uses the old keymaster ID only, so if it works on that version then it would be evidence to the contrary...I'll try and fiddle with this tonight. With @SuisChan's code to solve the hashcash challenge, I can revert-and-update
github.com/librespot-org/librespot@cdf84925adand pass the real client ID everywhere. This should be the way forward, I think.@JasonLG1979 commented on GitHub (Aug 25, 2022):
Yep. I started having issues last night. My PR branch and the latest dev have the same problem (which stands to reason since I didn't mess with those bits).
I don't know either? I haven't really dug that deep in the network code?
@roderickvd commented on GitHub (Aug 25, 2022):
From the top of my head it might not send the
device_idand now that we do, some servers may not like when thatdevice_iddoes not correspond to some authenticatedclient_id, I don't know. No matter, let's focus on solving the challenge and then we can test that.I can't reproduce your issue here from The Netherlands. Let me see if I can get it working tonight, so you can still test on a "misbehaving" access point.
@JasonLG1979 commented on GitHub (Aug 25, 2022):
Sounds good. I'm pretty keen on providing reasonably full metadata over events. I think that would make users and upstream pretty happy to not rely on some other separate mechanism to retrieve it.
But again I think that should be a separate PR after my current one is merged.
@roderickvd commented on GitHub (Aug 25, 2022):
Working on it now. For posterity, the hashcash challenge is not presented when you say you're a Mac, but it is when you say you're on Linux 😕
@JasonLG1979 commented on GitHub (Aug 25, 2022):
Can we just lie?
@JasonLG1979 commented on GitHub (Aug 25, 2022):
I just tried it again after the revert. No go. Still not working.
@roderickvd commented on GitHub (Aug 25, 2022):
Yes I see it too on Linux (and not on macOS). It's not even that we're presented a hash cash challenge, it just fails outright with a HTTP/400 when requesting the client token. The Spotify servers are really picky about the header values that you send, it must be that we're sending some they don't like all of the sudden.
I tried "lying" by changing more than a few instances of Linux -> macOS but I haven't found it yet, argh...
@JasonLG1979 commented on GitHub (Aug 25, 2022):
It's very strange that I have no problem with
v0.4.2?@roderickvd commented on GitHub (Aug 25, 2022):
This concerns the HTTP client token in
spclient, v0.4.2 doesn't have that at all.@JasonLG1979 commented on GitHub (Aug 25, 2022):
See I told you I know basically nothing about the network stuff,lol!!! Kinda on that note I'm wondering if we can get the user's
display-nameI saw it somewhere in the web API docs, can't remember where but it would be nice to add that to theConnectedandDisconnectedevents. Currently what we call the "username" is really theuser-idand that's not super useful for the event for display purposes. It would be nice to have the "pretty" name.@JasonLG1979 commented on GitHub (Aug 25, 2022):
My idea for the events is to make it trivial for upstream to be able to display the user's
display-namealong with the client device's name and all the relevant metadata. I'm sure moode and such would like that. Last I checked they don't display anything when usinglibrespot?@JasonLG1979 commented on GitHub (Aug 25, 2022):
With all that info you could write a pretty basic Python script to push to the framebuffer on a Pi for example and not even need an event loop. It could just be driven purely by the events.
@roderickvd commented on GitHub (Aug 26, 2022):
You may want to try the latest commit, it works on Linux and macOS here. If it doesn't for you, please let me know your access point so I can manually try to connect to that.
It sure looks like Spotify is changing things on their end. Linux is now no longer presented a hash cash challenge. It was some weeks ago... to make Linux get a client ID again we still have to use the keymaster ID.
In case any hash cash challenge is thrown at us, @SuisChan's excellent example over at https://github.com/SuisChan/ctk-example shows how it is done for Android. I have ported this code, but not tested it, so at this point consider it experimental.
I was mixing up the client token (which I was working on) and the Mercury 403 error for the "normal" token you posted above. Again, let me know which access point is troubling you and I can try some stuff. Basically this is trial and error swapping between the keymaster ID and real client ID, and having the device ID there, removed, or truncated to 16 characters.
At this point it's clear that Spotify isn't consistent cross-platform and we should take care to test stuff like this on the major OSs.
I have not had the time to look at your event code but let's keep the discussion with that PR.
@JasonLG1979 commented on GitHub (Aug 26, 2022):
Still no go.
@roderickvd commented on GitHub (Aug 26, 2022):
Totally weird. I just connected to that AP manually and no problems here. But off to bed now...
@JasonLG1979 commented on GitHub (Aug 26, 2022):
Yep, I don't know what to say?
@JasonLG1979 commented on GitHub (Aug 26, 2022):
I have 2 other librespot devices both running
v0.4.2one wired and one on wifi and they both work fine. I even buildv0.4.2on my desktop and it works fine so it's not a network issue as far as I can tell?@roderickvd commented on GitHub (Aug 26, 2022):
I don't know either. I tried both on Linux and macOS, both successful. Can you try v0.5.0 on those other devices, see if that matters or if it sticks to a certain OS?
@JasonLG1979 commented on GitHub (Aug 26, 2022):
I already did. It didn't matter as they are all Linux, my desktop runs Fedora, my server runs Debian and and my Pi Zero v2 runs RaspberryPiOS.
It must be some kind of geo location blocking thing? I'm looking for a free proxy that's compatible with librespot, as in is http(s)://Host:Port and not IP:Port. Librespot errors out if you try to use a IP:Port proxy.
@JasonLG1979 commented on GitHub (Aug 26, 2022):
I would consider not being able to use a IP:Port style proxy a bug or at least a missing feature if you already have http(s)://Host:Port support. I will look into that and see what I can do.
@kingosticks commented on GitHub (Aug 26, 2022):
As a temporary workaround here, can you use an entry in /etc/hosts to do this?
@JasonLG1979 commented on GitHub (Aug 26, 2022):
Tried that already also. No dice.
I added this to my /etc/hosts:
Where 104.199.65.124 is the IP to ap-gew1.spotify.com and 2600:1901:1:5ca:: is the IP to gew1-spclient.spotify.com.
@JasonLG1979 commented on GitHub (Aug 26, 2022):
I even tried connecting to my phone via a hotshot so my IP was different.
@kingosticks commented on GitHub (Aug 26, 2022):
Sorry, maybe I misunderstood what you were trying to do, but I meant you could add an entry for the free hostname-only proxy you are trying to use in order to avoid the geo-restiction.
@JasonLG1979 commented on GitHub (Aug 26, 2022):
I tried setting up a proxy via GNOME's network settings but none of them actually worked that I could find.
I have never really had to mess with proxies and I just googled setting a proxy in /etc/hosts and everything I read says that you can't redirect to specific ports in /etc/hosts so that's not very useful.
@kingosticks commented on GitHub (Aug 26, 2022):
I was talking about a workaround for
/etc/hosts
Then use proxy http://foo.com:Port with librespot, which it should like, but you'll actually have a connection to some.proxy.you.only.have.an.ip.for.
Years ago when I tried random free proxies on the net, they never worked. Alternatively there are some free VPN services these days, that should Just Work.
@JasonLG1979 commented on GitHub (Aug 26, 2022):
The only free VPN's I can find are IPV4 only and *-spclient.spotify.com seems to be IPV6 only.
I even tried to power cycle my modem to get a different IP from my ISP which worked in the sense that I got a different IP, before my IP said I was in New York City, New York no I'm apparently now I'm in Hamilton Missouri. I ofc are in neither of those places,lol!!! But that was no good either.
@JasonLG1979 commented on GitHub (Aug 26, 2022):
Well as a workaround I just tried to use my Spotify creds and that worked without a hitch.
@JasonLG1979 commented on GitHub (Aug 26, 2022):
I feel kinda stupid because I didn't try that sooner 🤦♂️ but it still doesn't solve the problem of zeroconf auth not working.
@roderickvd commented on GitHub (Aug 26, 2022):
Still don't have a clue what's going on. Maybe you can sprinkle some traces around the Mercury sending code, to see the difference in request we're making depending on whether you're using zeroconf or command-line authentication?
@SuisChan commented on GitHub (Aug 27, 2022):
Hey! Are you having trouble getting an access token via hm://keymaster... or something else?
Why not (as an experiment) try to take it from login5 endpoint? (since it returns noy only the login credentials but also the token, but don't know what scopes it has since no information provided).
@mchccc commented on GitHub (Sep 4, 2022):
What if I just have a Facebook login? 🥲
@Foxtrod89 commented on GitHub (Dec 30, 2022):
what is workaround? macOS 11.6
@roderickvd commented on GitHub (Dec 30, 2022):
I don't think this is related to the subject of this issue.
But on Mac, just use Rodio instead of PortAudio. I don't know about this Homebrew recipe but Rodio should be the default backend anyway. Or specify
--devicewith whatever PortAudio needs, because that's what it's complaining about.@samflores commented on GitHub (Mar 22, 2023):
I've encountered this problem (I think) and don't know if it's better to create a new GH issue.
I installed the "0.5.0-dev" version yesterday and listened to a couple hours until it stopped and nothing I tried worked - changing backend, editing
/etc/hosts, running under a vpn, etc.The content of
http://apresolve.spotify.com/is:I'm starting librespot with
librespot -u "<elided>" -p "<elided>" --backend pulseaudio --disable-audio-cache --enable-volume-normalisation --initial-volume=100 -vand seeing this in the log when I try to play anything:@roderickvd commented on GitHub (Mar 22, 2023):
Weird. Have you tried blacklisting
ap-gue1.spotify.comin/etc/hosts? See if another access point will work?@samflores commented on GitHub (Mar 22, 2023):
If I point it to
0.0.0.0librespotjust closes and in the log I get:If I block
apresolve.spotify.com, though, I see this:but when I try to play anything I get the same error, with a different URL:
@smilg commented on GitHub (Mar 27, 2023):
FWIW I'm having the same issue, though I'm unable to modify
/etc/hostsdue to some details of my setup. Did you happen to find a fix?@bjorgvino commented on GitHub (Aug 30, 2024):
I ran into this on a Raspberry Pi running raspotify. I've been able to use it to play music from Spotify without problems, but all of the sudden I started getting this error when trying to play tracks - and tracks just skipped ahead immediately all the time:
The output of
http://apresolve.spotify.com/for me is:If I block
apresolve.spotify.com, i.e. add the following to/etc/hosts:0.0.0.0 apresolve.spotify.com, raspotify on my Raspberry Pi starts working again because it falls back toap.spotify.com:443:Yay!
@mrkickling commented on GitHub (Sep 1, 2024):
Had the same problem, can verify that @bjorgvino solution worked. Thanks!
EDIT: Stopped working sep 4th
@MitraMai commented on GitHub (Sep 1, 2024):
@bjorgvino 's solution worked for me! Many thanks!
@web-sash commented on GitHub (Sep 2, 2024):
@bjorgvino 's solution worked for me also! Thanks a lot..
The fix stoppped working today 04. September. ap.spotify.com doesn't seem to be reachable anymore
@kingosticks commented on GitHub (Sep 2, 2024):
Are you all using zeroconf to authenticate ?
@kingosticks commented on GitHub (Sep 2, 2024):
I can answer my own question: no, it's not directly due to authentication method. Please correct me if I am wrong, but it's presumably because you're all using v0.4.x. This problem doesn't exist for me in the dev branch, but does in the master branch.
@bjorgvino commented on GitHub (Sep 2, 2024):
Yeah I'm using the latest release, 0.4.2.
No authentication. It's just that the first access point doesn't seem to work for me (and others it seems), perhaps because of the port, or something completely different, and the fallback logic doesn't seem to kick in... i.e. it doesn't try the other access points.
Btw what I posted was not in any way "my solution", I just tried stuff already suggested in this thread and reported back how it went..
@kingosticks commented on GitHub (Sep 2, 2024):
It would only try the other access points it connecting fails. In the cases here, connection and authentication are seemingly work fine, it's the subsequent requesting of audio data that fails when using certain access points.
I appreciate not your solution, I was just trying to work out if further investigation was required. But given this all works in the latest code (that obtains audio data a different way), the answer is no.
@bjorgvino commented on GitHub (Sep 2, 2024):
Understood. Perhaps a new release is in order then?
@truppelito commented on GitHub (Sep 2, 2024):
Blocking
apresolve.spotify.comon my AdGuard Home server worked for me too.@bestlibre commented on GitHub (Sep 3, 2024):
It worked for me until this morning. I'm getting the same error with the ap.spotify.com fallback
@kingosticks commented on GitHub (Sep 3, 2024):
Then you'll have to switch to the real fix. which is to compile using the dev branch.
@mloskot commented on GitHub (Sep 4, 2024):
https://github.com/librespot-org/librespot/issues/972#issuecomment-2320943137 workaround does not work for me:
restart
spotify_playerand it keeps skipping as I show in@jaypi95 commented on GitHub (Sep 9, 2024):
I did this two days ago because of the zeroconf thing but as of this morning it stopped working again. Overriding the DNS entry also didn't work this time.
But blocking apresolve.spotify.com did the trick!
@ricardoreais commented on GitHub (Aug 10, 2025):
https://github.com/librespot-org/librespot/issues/972#issuecomment-2320943137 worked for me 👍 thanks!