[GH-ISSUE #342] Where does the reporting happen? #225

Closed
opened 2026-02-27 19:29:30 +03:00 by kerem · 24 comments
Owner

Originally created by @michaelherger on GitHub (Jun 29, 2019).
Original GitHub issue: https://github.com/librespot-org/librespot/issues/342

I've had a report of a user who happens to be an artist, too. He claims that when he's listening to his own music using my implementation of librespot, the tracks would not be reported. This is definitely not what I want to happen, therefore I'm a bit concerned.

And I must admit I've been wondering why my kids' listening habits hadn't polluted my profile yet... (but I was less concerned about that :-D).

Is there code in librespot which actively does report when a track has been listened to? I was assuming that telling librespot to play a track would do all the magic for me.

I'm using a slightly modified fork of librespot (https://github.com/michaelherger/librespot), and I'm not sure whether one of my modifications is causing this mis-behavior or not.

Originally created by @michaelherger on GitHub (Jun 29, 2019). Original GitHub issue: https://github.com/librespot-org/librespot/issues/342 I've had a report of a user who happens to be an artist, too. He claims that when he's listening to his own music using my implementation of librespot, the tracks would not be reported. This is definitely not what I want to happen, therefore I'm a bit concerned. And I must admit I've been wondering why my kids' listening habits hadn't polluted my profile yet... (but I was less concerned about that :-D). Is there code in librespot which actively does report when a track has been listened to? I was assuming that telling librespot to play a track would do all the magic for me. I'm using a slightly modified fork of librespot (https://github.com/michaelherger/librespot), and I'm not sure whether one of my modifications is causing this mis-behavior or not.
Author
Owner

@michaelherger commented on GitHub (Aug 19, 2019):

Anyone?

<!-- gh-comment-id:522421422 --> @michaelherger commented on GitHub (Aug 19, 2019): Anyone?
Author
Owner

@devgianlu commented on GitHub (Aug 19, 2019):

How can we test whether the playback history is being recorded or not? I am willing to fix this issue for librespot-java, but I need a way to test it properly.

<!-- gh-comment-id:522453492 --> @devgianlu commented on GitHub (Aug 19, 2019): How can we test whether the playback history is being recorded or not? I am willing to fix this issue for librespot-java, but I need a way to test it properly.
Author
Owner

@michaelherger commented on GitHub (Aug 19, 2019):

That's a good point and something I'll try to do: you can get the recently played tracks using the web API (the call is still beta... hopefully that won't confuse even more...).

https://developer.spotify.com/documentation/web-api/reference/player/get-recently-played/

<!-- gh-comment-id:522454794 --> @michaelherger commented on GitHub (Aug 19, 2019): That's a good point and something I'll try to do: you can get the recently played tracks using the web API (the call is still beta... hopefully that won't confuse even more...). https://developer.spotify.com/documentation/web-api/reference/player/get-recently-played/
Author
Owner

@devgianlu commented on GitHub (Aug 25, 2019):

@michaelherger I've checked and it seems that tracks played from librespot aren't reported by that endpoint. I will investigate this further.

<!-- gh-comment-id:524649734 --> @devgianlu commented on GitHub (Aug 25, 2019): @michaelherger I've checked and it seems that tracks played from librespot aren't reported by that endpoint. I will investigate this further.
Author
Owner

@michaelherger commented on GitHub (Aug 26, 2019):

Thanks for the confirmation!

<!-- gh-comment-id:524721448 --> @michaelherger commented on GitHub (Aug 26, 2019): Thanks for the confirmation!
Author
Owner

@devgianlu commented on GitHub (Aug 29, 2019):

Unluckily I don't see any specific call to report that a track is playing. I am afraid that it is done via the event-service endpoint which hasn't been reversed engineered at all. The only (mercury) packet that has the track uri in it is the following:

0078  # Changes every time, not incremental

5549496e746572616374696f6e # "UIInteraction"

0935  # Padding + "5"

0909  # Double padding
706c61796c697374  # "playlist"

09  # Padding
73706f746966793a6170703a706c61796c697374  # "spotify:app:playlist"

0909  # Double padding
32  # "2", could be track index in playlist

09  # Padding
73706f746966793a747261636b3a3367696379666945564d474f4e677a796770576a4e54  # Track uri

0909  # Double padding
6c6973742d6974656d2d73656c6563746564  # "list-item-selected"

09  # Padding
31353637303834303738323536  # Timestamp in millis

There's no reporting in the HTTP API either. Android doesn't seem to have any of this either.

I've also noticed that the Recently played API updates quite slowly, so maybe librespot-java works with reporting? Don't know.

<!-- gh-comment-id:526192016 --> @devgianlu commented on GitHub (Aug 29, 2019): Unluckily I don't see any specific call to report that a track is playing. I am afraid that it is done via the `event-service` endpoint which hasn't been reversed engineered at all. The only (mercury) packet that has the track uri in it is the following: ``` 0078 # Changes every time, not incremental 5549496e746572616374696f6e # "UIInteraction" 0935 # Padding + "5" 0909 # Double padding 706c61796c697374 # "playlist" 09 # Padding 73706f746966793a6170703a706c61796c697374 # "spotify:app:playlist" 0909 # Double padding 32 # "2", could be track index in playlist 09 # Padding 73706f746966793a747261636b3a3367696379666945564d474f4e677a796770576a4e54 # Track uri 0909 # Double padding 6c6973742d6974656d2d73656c6563746564 # "list-item-selected" 09 # Padding 31353637303834303738323536 # Timestamp in millis ``` There's no reporting in the HTTP API either. Android doesn't seem to have any of this either. I've also noticed that the Recently played API updates quite slowly, so maybe librespot-java works with reporting? Don't know.
Author
Owner

@ashthespy commented on GitHub (Aug 29, 2019):

@devgianlu There is also an activity-manager endpoint that I recall seeing, but didn't investigate further.

<!-- gh-comment-id:526197163 --> @ashthespy commented on GitHub (Aug 29, 2019): @devgianlu There is also an `activity-manager` endpoint that I recall seeing, but didn't investigate further.
Author
Owner

@devgianlu commented on GitHub (Aug 29, 2019):

@devgianlu There is also an activity-manager endpoint that I recall seeing, but didn't investigate further.

I did not see any of that. Maybe it has been deprecated when they upgraded the protocol.

<!-- gh-comment-id:526198082 --> @devgianlu commented on GitHub (Aug 29, 2019): > @devgianlu There is also an `activity-manager` endpoint that I recall seeing, but didn't investigate further. I did not see any of that. Maybe it has been deprecated when they upgraded the protocol.
Author
Owner

@ashthespy commented on GitHub (Aug 29, 2019):

This was from an older version indeed. I need to find my notes.
Also might be of interest:

hm://played-state/v1/items
hm://played-state/v1/updates
hm://played-state/v1/unsubscribe
hm://played-state/v1/items?fromTimestamp=%Lu
<!-- gh-comment-id:526199907 --> @ashthespy commented on GitHub (Aug 29, 2019): This was from an older version indeed. I need to find my notes. Also might be of interest: ``` hm://played-state/v1/items hm://played-state/v1/updates hm://played-state/v1/unsubscribe hm://played-state/v1/items?fromTimestamp=%Lu ```
Author
Owner

@devgianlu commented on GitHub (Aug 29, 2019):

I think that's for episodes only.

<!-- gh-comment-id:526200792 --> @devgianlu commented on GitHub (Aug 29, 2019): I think that's for episodes only.
Author
Owner

@ashthespy commented on GitHub (Aug 29, 2019):

You are probably right again, its been a while I looked at this!

<!-- gh-comment-id:526202667 --> @ashthespy commented on GitHub (Aug 29, 2019): You are probably right again, its been a while I looked at this!
Author
Owner

@caba8685 commented on GitHub (Nov 26, 2019):

Any updates on this?

<!-- gh-comment-id:558515915 --> @caba8685 commented on GitHub (Nov 26, 2019): Any updates on this?
Author
Owner

@devgianlu commented on GitHub (Nov 26, 2019):

We are stuck on trying to find a way to check if the changes on the client work.

<!-- gh-comment-id:558517151 --> @devgianlu commented on GitHub (Nov 26, 2019): We are stuck on trying to find a way to check if the changes on the client work.
Author
Owner

@sashahilton00 commented on GitHub (Nov 26, 2019):

It’s probably handled via the events service. I’d be surprised if Spotify does the play reporting client side, would make it too easy to abuse. Then again, I can’t see how they track it server side unless they’re doing some sort of calculation based on the number of chunks sent to a client

<!-- gh-comment-id:558529229 --> @sashahilton00 commented on GitHub (Nov 26, 2019): It’s probably handled via the events service. I’d be surprised if Spotify does the play reporting client side, would make it too easy to abuse. Then again, I can’t see how they track it server side unless they’re doing some sort of calculation based on the number of chunks sent to a client
Author
Owner

@ashthespy commented on GitHub (Nov 26, 2019):

I was thinking about this some time back, and I guess they probably have metrics based on client type, and since we report librespot they could just not be propagating those metrics?

<!-- gh-comment-id:558577218 --> @ashthespy commented on GitHub (Nov 26, 2019): I was thinking about this some time back, and I guess they probably have metrics based on client type, and since we report `librespot` they could just not be propagating those metrics?
Author
Owner

@devgianlu commented on GitHub (Nov 26, 2019):

From the get-recently-played docs:

Returns the most recent 50 tracks played by a user. Note that a track currently playing will not be visible in play history until it has completed. A track must be played for more than 30 seconds to be included in play history. Any tracks listened to while the user had “Private Session” enabled in their client will not be returned in the list of recently played tracks.

I can confirm that reporting doesn't work for librespot-java. When using the original client, as soon as the song ends, it appears in the list.

<!-- gh-comment-id:558638296 --> @devgianlu commented on GitHub (Nov 26, 2019): From the [get-recently-played](https://developer.spotify.com/documentation/web-api/reference/player/get-recently-played/) docs: > Returns the most recent 50 tracks played by a user. Note that a track currently playing will not be visible in play history until it has completed. A track must be played for more than 30 seconds to be included in play history. Any tracks listened to while the user had “Private Session” enabled in their client will not be returned in the list of recently played tracks. I can confirm that reporting doesn't work for `librespot-java`. When using the original client, as soon as the song ends, it appears in the list.
Author
Owner

@michaelherger commented on GitHub (Nov 26, 2019):

Could it be librespot downloading too fast, leaving Spotify to think we didn't listen to the track? (seems like an odd design, but who know?...)

FWIW: I've worked with many streaming services. Many of them would require you to manually report playback from the client's side. It's not uncommon.

<!-- gh-comment-id:558640651 --> @michaelherger commented on GitHub (Nov 26, 2019): Could it be librespot downloading too fast, leaving Spotify to think we didn't listen to the track? (seems like an odd design, but who know?...) FWIW: I've worked with many streaming services. Many of them would require you to manually report playback from the client's side. It's not uncommon.
Author
Owner

@ashthespy commented on GitHub (Nov 26, 2019):

Returns the most recent 50 tracks played by a user. Note that a track currently playing will not be visible in play history until it has completed. A track must be played for more than 30 seconds to be included in play history. Any tracks listened to while the user had “Private Session” enabled in their client will not be returned in the list of recently played tracks.

This duration seems to be passed to the client:

<ab_recently_played_feature_time_filter_threshold>
com.spotify.gaia=30,driving-mode=120,spotify%3Ainternal%3Astartpage=30
</ab_recently_played_feature_time_filter_threshold>
<!--Is this for the private session?-->
<incognito_mode_timeout>21600</incognito_mode_timeout>
<!-- gh-comment-id:558642341 --> @ashthespy commented on GitHub (Nov 26, 2019): > > Returns the most recent 50 tracks played by a user. Note that a track currently playing will not be visible in play history until it has completed. A track must be played for more than 30 seconds to be included in play history. Any tracks listened to while the user had “Private Session” enabled in their client will not be returned in the list of recently played tracks. This duration seems to be passed to the client: ```xml <ab_recently_played_feature_time_filter_threshold> com.spotify.gaia=30,driving-mode=120,spotify%3Ainternal%3Astartpage=30 </ab_recently_played_feature_time_filter_threshold> <!--Is this for the private session?--> <incognito_mode_timeout>21600</incognito_mode_timeout> ```
Author
Owner

@devgianlu commented on GitHub (Nov 26, 2019):

The following hm://event-service/v1/events packet seems interesting (number 15 in the attached dump):

.Z372	1	01a5ba17d8411f92a5b6601e872da877	spotify:track:3mof6Z6vz6gonsuIEQXank	0	[[0,254896]]

00000000  00 5a 33 37 32 09 31 09 30 31 61 35 62 61 31 37  |.Z372.1.01a5ba17|
00000010  64 38 34 31 31 66 39 32 61 35 62 36 36 30 31 65  |d8411f92a5b6601e|
00000020  38 37 32 64 61 38 37 37 09 73 70 6f 74 69 66 79  |872da877.spotify|
00000030  3a 74 72 61 63 6b 3a 33 6d 6f 66 36 5a 36 76 7a  |:track:3mof6Z6vz|
00000040  36 67 6f 6e 73 75 49 45 51 58 61 6e 6b 09 30 09  |6gonsuIEQXank.0.|
00000050  5b 5b 30 2c 32 35 34 38 39 36 5d 5d              |[[0,254896]]|

It has been sent once the track (spotify:track:3mof6Z6vz6gonsuIEQXank) finished. [[0,254896]] is the interval of the song that has been played in milliseconds. 00 5a is the length of the payload. 01a5ba17d8411f92a5b6601e872da877 is probably the device id.

01a5ba17d8411f92a5b6601e872da877 is the playback ID contained inside the player state (changes with every track), but no matter what I try, the client keeps sending 400.

reporting.zip

Another example:

.i372	1	019c4da947e968d572d76c8873c735a4	spotify:track:4K9xid96G3YmIvQZXN9SXg	0	[[0,15214],[164087,268000]]

00000000  00 69 33 37 32 09 31 09 30 31 39 63 34 64 61 39  |.i372.1.019c4da9|
00000010  34 37 65 39 36 38 64 35 37 32 64 37 36 63 38 38  |47e968d572d76c88|
00000020  37 33 63 37 33 35 61 34 09 73 70 6f 74 69 66 79  |73c735a4.spotify|
00000030  3a 74 72 61 63 6b 3a 34 4b 39 78 69 64 39 36 47  |:track:4K9xid96G|
00000040  33 59 6d 49 76 51 5a 58 4e 39 53 58 67 09 30 09  |3YmIvQZXN9SXg.0.|
00000050  5b 5b 30 2c 31 35 32 31 34 5d 2c 5b 31 36 34 30  |[[0,15214],[1640|
00000060  38 37 2c 32 36 38 30 30 30 5d 5d                 |87,268000]]|
<!-- gh-comment-id:558652818 --> @devgianlu commented on GitHub (Nov 26, 2019): The following `hm://event-service/v1/events` packet seems interesting (number 15 in the attached dump): ``` .Z372 1 01a5ba17d8411f92a5b6601e872da877 spotify:track:3mof6Z6vz6gonsuIEQXank 0 [[0,254896]] 00000000 00 5a 33 37 32 09 31 09 30 31 61 35 62 61 31 37 |.Z372.1.01a5ba17| 00000010 64 38 34 31 31 66 39 32 61 35 62 36 36 30 31 65 |d8411f92a5b6601e| 00000020 38 37 32 64 61 38 37 37 09 73 70 6f 74 69 66 79 |872da877.spotify| 00000030 3a 74 72 61 63 6b 3a 33 6d 6f 66 36 5a 36 76 7a |:track:3mof6Z6vz| 00000040 36 67 6f 6e 73 75 49 45 51 58 61 6e 6b 09 30 09 |6gonsuIEQXank.0.| 00000050 5b 5b 30 2c 32 35 34 38 39 36 5d 5d |[[0,254896]]| ``` It has been sent once the track (`spotify:track:3mof6Z6vz6gonsuIEQXank`) finished. `[[0,254896]]` is the interval of the song that has been played in milliseconds. `00 5a` is the length of the payload. <s>`01a5ba17d8411f92a5b6601e872da877` is probably the device id.</s> `01a5ba17d8411f92a5b6601e872da877` is the playback ID contained inside the player state (changes with every track), <s>but no matter what I try, the client keeps sending 400.</s> [reporting.zip](https://github.com/librespot-org/librespot/files/3892253/reporting.zip) Another example: ``` .i372 1 019c4da947e968d572d76c8873c735a4 spotify:track:4K9xid96G3YmIvQZXN9SXg 0 [[0,15214],[164087,268000]] 00000000 00 69 33 37 32 09 31 09 30 31 39 63 34 64 61 39 |.i372.1.019c4da9| 00000010 34 37 65 39 36 38 64 35 37 32 64 37 36 63 38 38 |47e968d572d76c88| 00000020 37 33 63 37 33 35 61 34 09 73 70 6f 74 69 66 79 |73c735a4.spotify| 00000030 3a 74 72 61 63 6b 3a 34 4b 39 78 69 64 39 36 47 |:track:4K9xid96G| 00000040 33 59 6d 49 76 51 5a 58 4e 39 53 58 67 09 30 09 |3YmIvQZXN9SXg.0.| 00000050 5b 5b 30 2c 31 35 32 31 34 5d 2c 5b 31 36 34 30 |[[0,15214],[1640| 00000060 38 37 2c 32 36 38 30 30 30 5d 5d |87,268000]]| ```
Author
Owner

@devgianlu commented on GitHub (Nov 27, 2019):

I am working on reverse engineering the event-service endpoint (https://github.com/librespot-org/librespot-java/pull/155). As always, I crawl for help.

<!-- gh-comment-id:559120109 --> @devgianlu commented on GitHub (Nov 27, 2019): I am working on reverse engineering the `event-service` endpoint (https://github.com/librespot-org/librespot-java/pull/155). As always, I crawl for help.
Author
Owner

@devgianlu commented on GitHub (Apr 14, 2020):

More work has been done and I finally got it working. There is still some work to do as they don't pick up tracks that end because of pressing some button (aka works only with trackdone).

There are also many fields that I don't know the purpose of. Here's a breakdown of the most interesting packet:

17 # Incremental ID
6cfc77983f62f1d2150a0eb7350ad303f8074ea5 # Device ID
012c7792bd5fe36573369d32966002c0 # Playback ID
00000000000000000000000000000000 # Parent playback ID (?)
library-collection appload # Where/how the track started
library-collection endplay # Where/how the track ended
648668 # Something related to the file size (may be equal to it)
3767776 # File size (twice)
29385 29385 # When the track ended (twice?)
191743 # Track duration
45 0 0 0 1 # No clue
75877 # When the track started
269 -1 context 146 0 0 0 1 0 # No clue
29385 29385 # When the track ended (twice, again?)
0 # No clue
160000 # Bitrate
spotify:user:11145089019:collection # Context URI
vorbis # Encoding
5988f2d40bcb442b92e324f7c486657f # Track ID
b5b84892efb54ddda3a24deb216f6b46 # Parent track ID (?) (usually absent) 
0 1586783725809 0 context # Some fixed values + timestamp
spotify:app:collection-songs 1.1.26 # Stuff from play_origin
com.spotify none none local na none # Fixed values

I've enhanced the dissect script to better reverse engineer the event service: https://github.com/devgianlu/spotify-analyze

<!-- gh-comment-id:613425917 --> @devgianlu commented on GitHub (Apr 14, 2020): More work has been done and I finally got it working. There is still some work to do as they don't pick up tracks that end because of pressing some button (aka works only with `trackdone`). There are also many fields that I don't know the purpose of. Here's a breakdown of the most interesting packet: ``` 17 # Incremental ID 6cfc77983f62f1d2150a0eb7350ad303f8074ea5 # Device ID 012c7792bd5fe36573369d32966002c0 # Playback ID 00000000000000000000000000000000 # Parent playback ID (?) library-collection appload # Where/how the track started library-collection endplay # Where/how the track ended 648668 # Something related to the file size (may be equal to it) 3767776 # File size (twice) 29385 29385 # When the track ended (twice?) 191743 # Track duration 45 0 0 0 1 # No clue 75877 # When the track started 269 -1 context 146 0 0 0 1 0 # No clue 29385 29385 # When the track ended (twice, again?) 0 # No clue 160000 # Bitrate spotify:user:11145089019:collection # Context URI vorbis # Encoding 5988f2d40bcb442b92e324f7c486657f # Track ID b5b84892efb54ddda3a24deb216f6b46 # Parent track ID (?) (usually absent) 0 1586783725809 0 context # Some fixed values + timestamp spotify:app:collection-songs 1.1.26 # Stuff from play_origin com.spotify none none local na none # Fixed values ``` I've enhanced the dissect script to better reverse engineer the event service: https://github.com/devgianlu/spotify-analyze
Author
Owner

@devgianlu commented on GitHub (Apr 18, 2020):

Just a quick question on the topic. Do you think it is better to send an event with some standard values that I didn't figure out or just not send it because it is not needed?

<!-- gh-comment-id:615876665 --> @devgianlu commented on GitHub (Apr 18, 2020): Just a quick question on the topic. Do you think it is better to send an event with some standard values that I didn't figure out or just not send it because it is not needed?
Author
Owner

@devgianlu commented on GitHub (Nov 1, 2020):

is something happened here?

What do you mean?

<!-- gh-comment-id:720074285 --> @devgianlu commented on GitHub (Nov 1, 2020): > is something happened here? What do you mean?
Author
Owner

@ashthespy commented on GitHub (Nov 1, 2020):

No, only librespot-java does it..

<!-- gh-comment-id:720119719 --> @ashthespy commented on GitHub (Nov 1, 2020): No, only `librespot-java` does it..
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/librespot#225
No description provided.