[GH-ISSUE #103] Reproducible builds

kerem commented

2026-02-27 19:26:11 +03:00

Owner

Originally created by @IzzySoft on GitHub (Nov 1, 2024).
Original GitHub issue: https://github.com/Lambada10/SongSync/issues/103

At IzzyOnDroid we support Reproducible Builds (see: Reproducible Builds, special client support and more at IzzyOnDroid). SongSync was established with RB here, and the previous release (v4.1.0) was successfully built as RB – but the current one (v4.2.0) failed, with a huge diff in classes.dex. Was that APK really built from a clean tree at the commit the tag points to?

Here's the diff of the APK:

  -rw-r--r--  0.0 unx      120 b-      116 defN 1981-01-01 01:01:02 a61f40a2 META-INF/version-control-info.textproto
- -rw-r--r--  0.0 unx     4873 b-     4873 stor 1981-01-01 01:01:02 96f4d314 assets/dexopt/baseline.prof
+ -rw-r--r--  0.0 unx     4873 b-     4873 stor 1981-01-01 01:01:02 69569fe3 assets/dexopt/baseline.prof
  -rw-r--r--  0.0 unx      707 b-      707 stor 1981-01-01 01:01:02 bcbb919c assets/dexopt/baseline.profm
- -rw-r--r--  0.0 unx  3763616 b-  1824191 defN 1981-01-01 01:01:02 de4bb8c4 classes.dex
+ -rw-r--r--  0.0 unx  3763440 b-  1824117 defN 1981-01-01 01:01:02 aa53cd34 classes.dex
  -rw-r--r--  0.0 unx    10096 b-     4386 defN 1981-01-01 01:01:02 9734baa0 lib/arm64-v8a/libandroidx.graphics.path.so
  -rw-r--r--  0.0 unx     7112 b-     2630 defN 1981-01-01 01:01:02 4f56ac38 lib/arm64-v8a/libdatastore_shared_counter.so
  -rw-r--r--  0.0 unx  1162816 b-   430991 defN 1981-01-01 01:01:02 eba4427b lib/arm64-v8a/libtaglib.so
@@ -102,8 +102,8 @@
  -rw----     2.0 fat        5 b-        7 defN 1981-01-01 01:01:02 6a2f0318 META-INF/kotlinx_coroutines_core.version
  -rw----     2.0 fat        5 b-        7 defN 1981-01-01 01:01:02 6a2f0318 META-INF/kotlinx_coroutines_jdk8.version
  -rw----     2.0 fat        5 b-        7 defN 1981-01-01 01:01:02 6a2f0318 META-INF/kotlinx_coroutines_slf4j.version
- -rw----     2.0 fat        6 b-        8 defN 1981-01-01 01:01:02 eef7bc49 META-INF/services/w3.x
- -rw----     2.0 fat        6 b-        8 defN 1981-01-01 01:01:02 ecb10210 META-INF/services/x3.a
+ -rw----     2.0 fat        5 b-        7 defN 1981-01-01 01:01:02 68d3808d META-INF/services/w3.x
+ -rw----     2.0 fat        5 b-        7 defN 1981-01-01 01:01:02 43fed34e META-INF/services/x3.a
  -rw----     2.0 fat      623 b-      280 defN 1981-01-01 01:01:02 beed7449 kotlin-tooling-metadata.json

The difference in META-INF/services/* are line endings, so this APK has probably NOT been built using your Github workflows but on a Windows machine. The differences in baseline.prof was to be expected if classes.dex differs. Which is too big to quote here (70+ kB) – but if you're interested, I can zip and attach it.

We'd appreciate if you could help making your build reproducible. We've prepared some hints on reproducible builds for that.

Looking forward to your reply!

Originally created by @IzzySoft on GitHub (Nov 1, 2024). Original GitHub issue: https://github.com/Lambada10/SongSync/issues/103 At IzzyOnDroid we support [Reproducible Builds](https://reproducible-builds.org/) (see: [Reproducible Builds, special client support and more at IzzyOnDroid](https://android.izzysoft.de/articles/named/iod-rbs-mirrors-clients)). SongSync was established with RB here, and the previous release (v4.1.0) was successfully built as RB – but the current one (v4.2.0) failed, with a huge diff in `classes.dex`. Was that APK really built from a clean tree at the commit the tag points to? Here's the diff of the APK: ```diff -rw-r--r-- 0.0 unx 120 b- 116 defN 1981-01-01 01:01:02 a61f40a2 META-INF/version-control-info.textproto - -rw-r--r-- 0.0 unx 4873 b- 4873 stor 1981-01-01 01:01:02 96f4d314 assets/dexopt/baseline.prof + -rw-r--r-- 0.0 unx 4873 b- 4873 stor 1981-01-01 01:01:02 69569fe3 assets/dexopt/baseline.prof -rw-r--r-- 0.0 unx 707 b- 707 stor 1981-01-01 01:01:02 bcbb919c assets/dexopt/baseline.profm - -rw-r--r-- 0.0 unx 3763616 b- 1824191 defN 1981-01-01 01:01:02 de4bb8c4 classes.dex + -rw-r--r-- 0.0 unx 3763440 b- 1824117 defN 1981-01-01 01:01:02 aa53cd34 classes.dex -rw-r--r-- 0.0 unx 10096 b- 4386 defN 1981-01-01 01:01:02 9734baa0 lib/arm64-v8a/libandroidx.graphics.path.so -rw-r--r-- 0.0 unx 7112 b- 2630 defN 1981-01-01 01:01:02 4f56ac38 lib/arm64-v8a/libdatastore_shared_counter.so -rw-r--r-- 0.0 unx 1162816 b- 430991 defN 1981-01-01 01:01:02 eba4427b lib/arm64-v8a/libtaglib.so @@ -102,8 +102,8 @@ -rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 6a2f0318 META-INF/kotlinx_coroutines_core.version -rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 6a2f0318 META-INF/kotlinx_coroutines_jdk8.version -rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 6a2f0318 META-INF/kotlinx_coroutines_slf4j.version - -rw---- 2.0 fat 6 b- 8 defN 1981-01-01 01:01:02 eef7bc49 META-INF/services/w3.x - -rw---- 2.0 fat 6 b- 8 defN 1981-01-01 01:01:02 ecb10210 META-INF/services/x3.a + -rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 68d3808d META-INF/services/w3.x + -rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 43fed34e META-INF/services/x3.a -rw---- 2.0 fat 623 b- 280 defN 1981-01-01 01:01:02 beed7449 kotlin-tooling-metadata.json ``` The difference in `META-INF/services/*` are line endings, so this APK has probably NOT been built using your Github workflows but on a Windows machine. The differences in `baseline.prof` was to be expected if `classes.dex` differs. Which is too big to quote here (70+ kB) – but if you're interested, I can zip and attach it. We'd appreciate if you could help making your build reproducible. We've prepared some [hints on reproducible builds](https://gitlab.com/IzzyOnDroid/repo/-/wikis/Reproducible-Builds) for that. Looking forward to your reply!

kerem closed this issue

2026-02-27 19:26:11 +03:00

kerem commented

2026-02-27 19:26:11 +03:00

Author

Owner

@Lambada10 commented on GitHub (Nov 1, 2024):

Yes, the release build was indeed built on a Windows machine. I do believe I built from a commit the tag points to, and checking my local tree right now indeed shows no differences with origin. Could you share the zip file?

@Lambada10 commented on GitHub (Nov 1, 2024): Yes, the release build was indeed built on a Windows machine. I do believe I built from a commit the tag points to, and checking my local tree right now indeed shows no differences with origin. Could you share the zip file?

kerem commented

2026-02-27 19:26:11 +03:00

Author

Owner

@IzzySoft commented on GitHub (Nov 1, 2024):

Yes, the release build was indeed built on a Windows machine.

But the previous release was not. Unfortunately, Windows inserts different line breaks (\r\n) then Linux (\n) – so while we can make both work with RB, we only can do it if you don't switch between one and the other (i.e. one release built on Linux, the next on Windows) as we can only set it for one variant – not for "alternating variants".

I do believe I built from a commit the tag points to, and checking my local tree right now indeed shows no differences with origin.

No artifacts from previous builds either? Gradle likes to cache them, and so does Android Studio. While the Github CI always builds from a "clean tree" as it starts with a "blank" image (as do our RBs), so there are no artifacts. So when not building with the CI, you'd have to run clean first.

TL;DR: Might be worth considering to have the release builds always created by the CI here. I can check such a "clean build" against ours then if you wish (i.e. if you "now" create one from the tag the way you'd do it in the future).

Could you share the zip file?

Sure, there you go: dex.zip

@IzzySoft commented on GitHub (Nov 1, 2024): > Yes, the release build was indeed built on a Windows machine. But the previous release was not. Unfortunately, Windows inserts different line breaks (`\r\n`) then Linux (`\n`) – so while we can make both work with RB, we only can do it if you don't switch between one and the other (i.e. one release built on Linux, the next on Windows) as we can only set it for one variant – not for "alternating variants". > I do believe I built from a commit the tag points to, and checking my local tree right now indeed shows no differences with origin. No artifacts from previous builds either? Gradle likes to cache them, and so does Android Studio. While the Github CI always builds from a "clean tree" as it starts with a "blank" image (as do our RBs), so there are no artifacts. So when not building with the CI, you'd have to run `clean` first. TL;DR: Might be worth considering to have the _release builds_ always created by the CI here. I can check such a "clean build" against ours then if you wish (i.e. if you "now" create one from the tag the way you'd do it in the future). > Could you share the zip file? Sure, there you go: [dex.zip](https://github.com/user-attachments/files/17604210/dex.zip)

kerem commented

2026-02-27 19:26:11 +03:00

Author

Owner

@Lambada10 commented on GitHub (Nov 2, 2024):

Might be worth considering to have the release builds always created by the CI here

Sure, I will only use the CI builds for release APKs from now on, should I update the APK for the current release?

@Lambada10 commented on GitHub (Nov 2, 2024): > Might be worth considering to have the release builds always created by the CI here Sure, I will only use the CI builds for release APKs from now on, should I update the APK for the current release?

kerem commented

2026-02-27 19:26:11 +03:00

Author

Owner

@IzzySoft commented on GitHub (Nov 2, 2024):

Sure, I will only use the CI builds for release APKs from now on,

Cool! That will make automated RB-updates a log easier, thanks!

should I update the APK for the current release?

One shouldn't replace what has already be distributed – so better have a "maintenance release" for that. Before doing so, may I suggest we first check such a build to make sure we're not in the same situation again with it? Could you build a release APK using the CI, rename it to *.zip so you can attach it here, then attach it here and name the commit you've built from? Then I could manually verify RB here to see if any other "breakers" need fixing for that "maintenance release". As I've kept the APKs from v4.2.0 here, the easiest way to do that would be a CI build from exactly that tag – so I'd only need to download the APK and compare, no need to create a local build here as I've retained that.

@IzzySoft commented on GitHub (Nov 2, 2024): > Sure, I will only use the CI builds for release APKs from now on, Cool! That will make automated RB-updates a log easier, thanks! > should I update the APK for the current release? One shouldn't replace what has already be distributed – so better have a "maintenance release" for that. Before doing so, may I suggest we first check such a build to make sure we're not in the same situation again with it? Could you build a release APK using the CI, **rename** it to `*.zip` so you can attach it here, then attach it here and name the commit you've built from? Then I could manually verify RB here to see if any other "breakers" need fixing for that "maintenance release". As I've kept the APKs from v4.2.0 here, the easiest way to do that would be a CI build from exactly that tag – so I'd only need to download the APK and compare, no need to create a local build here as I've retained that.

kerem commented

2026-02-27 19:26:11 +03:00

Author

Owner

@Lambada10 commented on GitHub (Nov 4, 2024):

Here's thee CI build which was built from v4.2.0 release
songsync-release.zip
https://github.com/Lambada10/SongSync/actions/runs/11629791434

@Lambada10 commented on GitHub (Nov 4, 2024): Here's thee CI build which was built from v4.2.0 release [songsync-release.zip](https://github.com/user-attachments/files/17616567/songsync-release.zip) https://github.com/Lambada10/SongSync/actions/runs/11629791434

kerem commented

2026-02-27 19:26:11 +03:00

Author

Owner

@IzzySoft commented on GitHub (Nov 4, 2024):

Thanks! That was not renamed to *.zip but but inside another zip (so I had to download, unzip and push it to the build server instead of just passing the URL to the build server), but meh… worked as well. And YES!! it's RB, wonderful! Looks like we found the culprit and solved the problem, so I'd say go ahead with the maintenance release then.

@IzzySoft commented on GitHub (Nov 4, 2024): Thanks! That was not **renamed** to `*.zip` but but inside another zip (so I had to download, unzip and push it to the build server instead of just passing the URL to the build server), but meh… worked as well. And **YES!!** it's RB, wonderful! Looks like we found the culprit and solved the problem, so I'd say go ahead with the maintenance release then.

kerem commented

2026-02-27 19:26:11 +03:00

Author

Owner

@Lambada10 commented on GitHub (Nov 5, 2024):

I'd say go ahead with the maintenance release then.

Done, I am waiting for confirmation if everything works, then I'm closing this

@Lambada10 commented on GitHub (Nov 5, 2024): > I'd say go ahead with the maintenance release then. Done, I am waiting for confirmation if everything works, then I'm closing this

kerem commented

2026-02-27 19:26:12 +03:00

Author

Owner

@IzzySoft commented on GitHub (Nov 5, 2024):

BUILD SUCCESSFUL in 4m 15s
45 actionable tasks: 45 executed
Configuration cache entry stored.
+ mv app/build/outputs/apk/release/app-release-unsigned.apk /outputs/unsigned.apk

--- END BUILD LOG ---
Keeping '2686bc5efc5770d78a6cd66a0942b1d4f7c349791b7bc3318751fa7b98f33494-pl.lambada.songsync-v4.2.1-upstream.apk'...
Keeping '4cf9a6dbd524f60e676bc7358370a521ca4573bdd55e1d5f6df27157861543e5-pl.lambada.songsync-v4.2.1-unsigned.apk'...
Reproducible: True
Tags built: 1.

Thanks a lot! Looks fine now 🤩

@IzzySoft commented on GitHub (Nov 5, 2024): ``` BUILD SUCCESSFUL in 4m 15s 45 actionable tasks: 45 executed Configuration cache entry stored. + mv app/build/outputs/apk/release/app-release-unsigned.apk /outputs/unsigned.apk --- END BUILD LOG --- Keeping '2686bc5efc5770d78a6cd66a0942b1d4f7c349791b7bc3318751fa7b98f33494-pl.lambada.songsync-v4.2.1-upstream.apk'... Keeping '4cf9a6dbd524f60e676bc7358370a521ca4573bdd55e1d5f6df27157861543e5-pl.lambada.songsync-v4.2.1-unsigned.apk'... Reproducible: True Tags built: 1. ``` Thanks a lot! Looks fine now :star_struck:

Rows
Columns

[GH-ISSUE #103] Reproducible builds #70