mirror of
https://github.com/librespot-org/librespot.git
synced 2026-04-27 00:05:55 +03:00
[GH-ISSUE #650] Update protobuf definitions and endpoints to use current official client ones. #380
Labels
No labels
A-Alsa
SpotifyAPI
Tokio 1.0
audio
bug
can't reproduce
compilation
dependencies
duplicate
enhancement
good first issue
help wanted
high priority
imported
imported
invalid
new api
pull-request
question
reverse engineering
wiki
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/librespot#380
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @sashahilton00 on GitHub (Feb 24, 2021).
Original GitHub issue: https://github.com/librespot-org/librespot/issues/650
Over the course of the past few years, Spotify has updated it's protobufs several times, along with its endpoints. This was to be expected as they continued to add functionality, nonetheless librespot is now pretty significantly diverged from the protobuf definitions that are in use in the official clients, and possibly at higher risk of experiencing issues with endpoints being depreciated at short notice, such as the recent issues Facebook login flow (admittedly not due to protobuf definition issues though).
Furthermore, newer functionality such as canvas, context management, streaming groups, etc. is implemented in the newer
proto3definitions, attached below.Given that Spotify is slowly standardising to HTTP/Websockets/GRPC transports, and moving away from Hermes, it may be good for this project in the long run if we were to try to follow suit. To this end, further reverse engineering of the current endpoints that are in use will need to be carried out, and the existing code updated where necessary. As a side effect of this, my gut tells me that a number of the smaller issues/feature requests (eg. repeat functionality only partially working, more 'spotify-like' normalization, etc.) will be easier to implement once we are using their up to date APIs.
I will start investigating when I get some free time, and add to this comment accordingly with any findings that I turn up. Anyone else is also invited to crack out
spotify-dissectand start digging through the network traffic. If the log of notes starts to become unwieldy we can always form some sort of to-do list or stick the information in a wiki page temporarily.The latest protobufs: spotify_protobufs.zip
@Johannesd3 commented on GitHub (Feb 24, 2021):
One note: The best gRPC library in Rust is probably tonic (from the tokio stack). It uses prost instead of protobuf, so it might be better to switch.
@xou816 commented on GitHub (Mar 19, 2021):
Never done that so I can't promise I can help, but I might give it a try, it sounds interesting! Is there a place where you (and others) will be uploading your findings?
@sashahilton00 commented on GitHub (May 8, 2021):
Ok, so what I'm going to do is dedicate a comment to each endpoint that I reverse. I'll attach screenshots of the decoded protobufs (request/response) where it may be helpful, and reference the endpoint, along with the proto definitions.
The proto files I'm using I extracted earlier, available here.
The proto descriptor is useful for anyone looking to do their own poking around, uploaded here. Alternatively, you can compile if from the proto definitions above using a variant of the command:
protoc --descriptor_set_out=descriptor.desc **/*.proto *.proto@sashahilton00 commented on GitHub (May 8, 2021):
Extended Metadata Endpoint (Protobuf)
Endpoint:
https://spclient.wg.spotify.com/extended-metadata/v0/extended-metadataProto definitions (refer to
extended_metadata.proto):spotify.extendedmetadata.BatchedEntityRequest. It may also acceptspotify.extendedmetadata.EntityRequest, but untested.spotify.extendedmetadata.BatchedExtensionResponse.Decoded Request Screenshot
Decoded Response Screenshot
@sashahilton00 commented on GitHub (May 9, 2021):
User Customisation Endpoint (Protobuf)
Endpoint:
https://spclient.wg.spotify.com/extended-metadata/v0/extended-metadataProto definitions (refer to
ucs.proto(user customisation service)):spotify.remote_config.ucs.proto.UcsRequest.spotify.remote_config.ucs.proto.UcsResponseWrapper.Notes
I didn't think much of ths endpoint initially, it seemed to just be a bunch of A/B test flags, but on closer inspection, it appears to carry a bunch of possibly useful information. Besides feature flags (one of which is Spotify HiFi), it has buffer parameters, prefetch configuration, account status and permissions, api base urls, etc. Instead of a screenshot, I've posted the request/response json below (converted from protobuf)
@roderickvd
loudness-levelsmight be interesting for you. There's also some other related stuff.Request Payload
Response Payload
@sashahilton00 commented on GitHub (May 9, 2021):
Connect State Endpoint
Endpoint:
https://spclient.wg.spotify.com/connect-state/v1/devices/2e8...202Proto definitions (
connect.proto):connectstate.Clustermessage, though I want to double check this. It looks pretty similar to the web api connect state endpoint, there may be a few additional fields.Notes
To implement the repeat single/context functonality on the client, one needs to look at the
PlayerState(player.proto) entry ofconnectstate.Cluster. Within that there is aContextPlayerOptionsentry, which indicates whether repeat functionality is enabled and what type.ContextRestrictionscan also be seen in player state, and should be used to determine what functionality is enabled. For example, thedisallow_pausing_reasonscontext restriction sometimes appears if the player is paused, with the somewhat self-explanatoryalready_pausedmessage. Granted, not a very useful example, but we should probably pay attention to these context restrictions and pass through the reason for disabling either to stdout or the application using librespot. Example below:Capabilitiesalso has some new fields that indicate whether a device supports Spotify HiFi. There are some other fields withinconnectstate.CapabilitySupportDetailssuch asfully_supportedanduser_eligiblethat presumably need to be set for the option to appear. See below:Also, there are some other fields within the device message such as
metadata_mapthat look to be the local network details of the client, notablydevice_address_maskandtier1_port, both of which appear to correspond to the mDNS/SSDP discovery that spotify clients do. It might be worth investigating adding this information to boost speed and reliability of device discovery.The device-state endpoint needs quite a bit more exploration, and probably warrants a thread of its own, as it seems to be the bucket into which Spotify is dumping most of the player state used when syncing up devices.
@roderickvd commented on GitHub (May 11, 2021):
The comma-separated values probably map to Spotify's loudness levels. This is old documentation from the net:
Currently Spotify only documents the LUFS value and no longer the dB gain. Where dB is assumed to be dBFS and LUFS is k-weighted. So the documentation above is indicative but not necessarily normative.
If you add 3 dB to the comma-separated you're more-or-less meeting the indicative documented values.
I'd have to think harder what the first "1" and last "-2.0" indicate. Taking a guess: "1" for the scheme version number and "-2.0" for the -2 dB true peak that Spotify targets.
@sashahilton00 general question: so I understand
librespot-javahas implemented large parts of the new API. What can we reuse from their reverse engineering vs. your posts above? I'm just entirely new to these parts so trying to understand the lay of the land here.@devgianlu commented on GitHub (May 11, 2021):
Ping me if you need any help.
@sashahilton00 commented on GitHub (May 11, 2021):
@roderickvd that's a good question. From the issue emails I've been getting from librespot-java, it looks like @devgianlu and co. have put quite a bit of work into reverse engineering chunks of the new API, hence in some cases the stuff I post here has likely been reverse engineered in part or fully over at librespot-java. @devgianlu would be best placed to answer the question of what has been reverse engineered that we can use, something of note is the dealer endpoint, which if we want to support group listening at some point, will need to be added, as thisis where group listening context seems to be published to. Other than that, it just sends pingpong requests unless I am missing something.
My reverse engineering is simply me capturing and examining the network traffic between desktop <-> Spotify, so what I post should be useful enough as a reference, but in terms of what we focus on, we should probably look to discussions and their upvotes, along with features that we thnk are going to be increasingly important when working out what to prioritise.
In my mind, there are a few things that are must haves before the library hits
1.0.0, which are:removal of surplus audio backends. Currently my thinking is that retaining the
pipe,rodio(default),alsaandgstreamerbackends makes sense, on the understanding that thealsaandgstreamerbackends are being actively maintained. Given the processing that we are already doing with normalisation, dithering, shaping, etc. combined with the slow movement on issues upstream in rodio, I am inclined to think that it may be worth dropping rodio in favour of cpal as a lighter alternative, given that we're not using the functionality that rodio provides (multi format decoding, dynamic mixing, filters, etc.). If we feel that this is going to change, for example with the introduction of HiFi, then perhaps this is worth reevaluating.remove redundant decoders.
tremorandlibvorbisshould both be removed I think. The former because it was introduced iirc for the Pi rev 1, which didn't have a hardware FPU. The latter because it a) relies on bndings to a C library, and b) the crate appears unmaintained.support session reconnection. This is being tracked in #609 but requires a level of rust experience that exceeds my own. The reason for this being a requirement in my mind is that there's a trove of issues that appear to be connected to session reconnection not really being handled, which one wouldn't really consider acceptable in a 'stable' library.
retrieve files from the CDN. Again, not technically a requirement, but from what I've seen in recent additions such as canvas, podcasts, etc. these are defaulting to being supplied via the CDN, and there is clearly a push to move to use the CDN at Spotify for all new projects. Besides avoiding the nasty surprise when Spotify eventually decides to turn off the mercury endpoint one day, there should be a small performance boost in that audio files will be served from edge locations as opposed to from Spotify servers, which may help reduce overall load times and responsiveness.
there has been plenty of discussion in #648 around splitting out discovery functionality, moving audio playback and other features to Spotifyd, etc. If we are going to do that, it should be done before we hit stable. Personally I am in favour of moving most of the playback functionality to Spotifyd, and only retaining the
pipeandcpalbackends in librespot, though multiple people have said that they wantalsaandgstreamerin librespot and are willing to maintain them, n which case I don't think forcing migration is necessary.Besides the above, there are other things that would be 'nice to have', but are by no means essential. Things like extended metadata can be added in later versions, the above is just some of the stuff which I think is required for it to be released as stable. If people have a different view, I'm open to changing my mind. In the meantime, I will keep documenting the protocols when I have a moment and adding them here, since it's just grunt work that I can do when I have an hour or two spare. Anyone else feel free to do the same.
@sashahilton00 commented on GitHub (May 11, 2021):
Looking back, the above is a bit of a roaming response, but hopefully it provides some clarification on why I'm spending the time to examine the latest protocols and the overall direction that I'm working to as a basis.
@sashahilton00 commented on GitHub (May 11, 2021):
Client Token Endpoint (Protobuf)
Endpoint:
POST https://clienttoken.spotify.com/v1/clienttokenProto Definitions missing.
Notes
This endpoint is where the client token is retrieved from, which is subsequently used in a multitude of requests. The request/response are shown below. Further investigation is required to determne the proto definitions, which will be updated accordingly in the request/response once discovered.
65b7...appears to be a Spotify Client ID. This is also used in the login5 endpoint.Request
Response
@devgianlu commented on GitHub (May 11, 2021):
@sashahilton00 Should this become a discussion? I feel like I could comment on stuff, but it'll end up being an awful long thread.
@sashahilton00 commented on GitHub (May 11, 2021):
I thought it already was, just realised it's still in issues. Will move it.