[GH-ISSUE #88] Deduplication configuration still not showing up for HDDs #21

Closed
opened 2026-03-07 19:10:04 +03:00 by kerem · 12 comments
Owner

Originally created by @Msprg on GitHub (Nov 16, 2024).
Original GitHub issue: https://github.com/007revad/Synology_enable_Deduplication/issues/88

I'm running DSM as a Proxmox VM with passed-trough SATA controller for the drives.

obrázok

I have run both enable deduplication with --hdd parameter and hdd_db script. Rebooted multiple times. Everything is healthy, performance analysis is enabled, and I've waited more than 10 minutes for the menu to show up. Like #79 but I still can't get it to work.

obrázok

Any ideas about what I'm missing??

Originally created by @Msprg on GitHub (Nov 16, 2024). Original GitHub issue: https://github.com/007revad/Synology_enable_Deduplication/issues/88 I'm running DSM as a Proxmox VM with passed-trough SATA controller for the drives. ![obrázok](https://github.com/user-attachments/assets/3c3201dd-d5b7-4a77-a636-300853c581b8) I have run both enable deduplication with --hdd parameter and hdd_db script. Rebooted multiple times. Everything is healthy, performance analysis is enabled, and I've waited more than 10 minutes for the menu to show up. Like #79 but I still can't get it to work. ![obrázok](https://github.com/user-attachments/assets/5b4db009-a875-406e-a043-431f1e21d290) Any ideas about what I'm missing??
kerem closed this issue 2026-03-07 19:10:04 +03:00
Author
Owner

@007revad commented on GitHub (Nov 17, 2024):

It is looking like it's a an issue with virtual disks. Xpenology, proxmox and virtual disks are all beyond my level of knowledge.

<!-- gh-comment-id:2480897401 --> @007revad commented on GitHub (Nov 17, 2024): It is looking like it's a an issue with virtual disks. Xpenology, proxmox and virtual disks are all beyond my level of knowledge.
Author
Owner

@Msprg commented on GitHub (Nov 17, 2024):

Definitely not. As I've mentioned, the sata controller is passed through, meaning that the drives are communicating with DSM exactly as they would in a real bare metal hardware DSM system.

obrázok

As far as virtual disks are concerned. Actually, the deduplication for those was ironically easily available even before running your scripts, despite me not needing it.

obrázok

<!-- gh-comment-id:2481216002 --> @Msprg commented on GitHub (Nov 17, 2024): Definitely not. As I've mentioned, the sata controller is passed through, meaning that the drives are communicating with DSM exactly as they would in a real bare metal hardware DSM system. ![obrázok](https://github.com/user-attachments/assets/e230ebf6-89e9-4d47-9ca0-534ade7a1f45) As far as virtual disks are concerned. Actually, the deduplication for those was ironically easily available even before running your scripts, despite me not needing it. ![obrázok](https://github.com/user-attachments/assets/8c9f46e5-e5db-4d24-841b-0424e9a09664)
Author
Owner

@Msprg commented on GitHub (Dec 3, 2024):

Would you mind sending me your patched StorageManager 1.0.0-00502 storage_component.js file to compare it to mine?

<!-- gh-comment-id:2515596008 --> @Msprg commented on GitHub (Dec 3, 2024): Would you mind sending me your patched `StorageManager 1.0.0-00502` storage_component.js file to compare it to mine?
Author
Owner

@Msprg commented on GitHub (Dec 15, 2024):

@007revad There's a different js file that's being sent to the DSM web clients than the one your script is modifying, or something like that. Because, even if the &&e.dedup_info.show_config_btn is deleted in the file correctly, the browser is still getting storage_panel.js that still contains the check. I have 100% confirmed that that's the culprit by using proxy and swapping the js Synology is sending with the modified js. And the deduplication menu was available right there and then.

It's not browser cache, and I have rebooted Synology - multiple times, and cold booted it as well.

Now I can't say what to do now. Maybe there's another caching going on in the Synology itself or something? I really don't feel like grepping entire file system just to find where is the other file / string hiding, but that's my last resort for now.
Let me know if you happen to have any ideas~!

<!-- gh-comment-id:2544174459 --> @Msprg commented on GitHub (Dec 15, 2024): @007revad There's a different js file that's being sent to the DSM web clients than the one your script is modifying, or something like that. Because, even if the `&&e.dedup_info.show_config_btn` is deleted in the file correctly, the browser is still getting `storage_panel.js` that still contains the check. I have 100% confirmed that that's the culprit by using proxy and swapping the js Synology is sending with the modified js. And the deduplication menu was available right there and then. It's not browser cache, and I have rebooted Synology - multiple times, and cold booted it as well. Now I can't say what to do now. Maybe there's another caching going on in the Synology itself or something? I really don't feel like grepping entire file system just to find where is the other file / string hiding, but that's my last resort for now. Let me know if you happen to have any ideas~!
Author
Owner

@007revad commented on GitHub (Dec 16, 2024):

After a reboot is the &&e.dedup_info.show_config_btn still deleted?

I just searched DSM 7.2.2 update 2 for files named storage_panel.js on a DS720+ and only found 2 storage_panel.js files

storage_panel.js               <-- the edited file
storage_panel.js.1.0.0-00502   <-- backup of the original file

I wonder if DSM is loading the backup storage_panel.js.1.0.0-00502 file as storage_panel.js. You could try moving, renaming or deleting storage_panel.js.1.0.0-00502.

<!-- gh-comment-id:2544225561 --> @007revad commented on GitHub (Dec 16, 2024): After a reboot is the `&&e.dedup_info.show_config_btn` still deleted? I just searched DSM 7.2.2 update 2 for files named `storage_panel.js` on a DS720+ and only found 2 storage_panel.js files ``` storage_panel.js <-- the edited file storage_panel.js.1.0.0-00502 <-- backup of the original file ``` I wonder if DSM is loading the backup storage_panel.js.1.0.0-00502 file as storage_panel.js. You could try moving, renaming or deleting storage_panel.js.1.0.0-00502.
Author
Owner

@Msprg commented on GitHub (Dec 17, 2024):

root@DSM1:/usr/local/packages/@appstore/StorageManager/ui# sha256sum storage_panel.js*
bdab05da7e550fd4696e9e9f7a54694d40ee71f1711799ec64078a7a601d0a94  storage_panel.js
bdab05da7e550fd4696e9e9f7a54694d40ee71f1711799ec64078a7a601d0a94  storage_panel.js.1.0.0-00502
9dec4942926cc8ec184c9575aa5d35cbb1ab10f7f6888d881b4c0f0ea5405e03  storage_panel.js.gz
cbf8f3709fcc43f1e5cc360e726bbb6e739e8176d3fd52b41f237a9400694280  storage_panel.js.gz.bak

I removed the check, and made both files the same. There has to be some other place where it's pulling it from still…

And btw, yes, the modifications do survive reboots.

<!-- gh-comment-id:2549526446 --> @Msprg commented on GitHub (Dec 17, 2024): ``` root@DSM1:/usr/local/packages/@appstore/StorageManager/ui# sha256sum storage_panel.js* bdab05da7e550fd4696e9e9f7a54694d40ee71f1711799ec64078a7a601d0a94 storage_panel.js bdab05da7e550fd4696e9e9f7a54694d40ee71f1711799ec64078a7a601d0a94 storage_panel.js.1.0.0-00502 9dec4942926cc8ec184c9575aa5d35cbb1ab10f7f6888d881b4c0f0ea5405e03 storage_panel.js.gz cbf8f3709fcc43f1e5cc360e726bbb6e739e8176d3fd52b41f237a9400694280 storage_panel.js.gz.bak ``` I removed the check, and made both files the same. There has to be some other place where it's pulling it from still… And btw, yes, the modifications do survive reboots.
Author
Owner

@Msprg commented on GitHub (Dec 17, 2024):

Oh, and maybe this will help, this is the file that Synology is sending despite having the files above modified: storage_panel_sentBySyno.zip

<!-- gh-comment-id:2549577979 --> @Msprg commented on GitHub (Dec 17, 2024): Oh, and maybe this will help, this is the file that Synology is sending despite having the files above modified: [storage_panel_sentBySyno.zip](https://github.com/user-attachments/files/18171125/storage_panel_sentBySyno.zip)
Author
Owner

@007revad commented on GitHub (Dec 18, 2024):

And btw, yes, the modifications do survive reboots.

Do you have syno_enable_dedupe.sh scheduled?

My real Synology NAS don't have the storage_panel.js.gz (or storage_panel.js.gz.bak) files. I assume they are created by the xpe loader. Does the storage-panel.js in storage_panel.js.gz contain &&e.dedup_info.show_config_btn?

I suspect what is happening is your xpe loader, or trcp-addons etc, is recreating storage-panel.js during the junior boot stage and then syno_enable_dedupe is editing it. But where the ui is loading an unedited copy from is a mystery.

A quick look at storagepanel.sh and install.sh shows they are creating the gzipped copy of storage-manager.js

<!-- gh-comment-id:2550083511 --> @007revad commented on GitHub (Dec 18, 2024): > And btw, yes, the modifications do survive reboots. Do you have syno_enable_dedupe.sh scheduled? My real Synology NAS don't have the storage_panel.js.gz (or storage_panel.js.gz.bak) files. I assume they are created by the xpe loader. Does the storage-panel.js in storage_panel.js.gz contain `&&e.dedup_info.show_config_btn`? I suspect what is happening is your xpe loader, or trcp-addons etc, is recreating storage-panel.js during the junior boot stage and then syno_enable_dedupe is editing it. But where the ui is loading an unedited copy from is a mystery. A quick look at [storagepanel.sh](https://github.com/PeterSuh-Q3/tcrp-addons/blob/main/storagepanel/src/storagepanel.sh) and [install.sh](https://github.com/PeterSuh-Q3/tcrp-addons/blob/main/storagepanel/src/install.sh) shows they are creating the gzipped copy of storage-manager.js
Author
Owner

@Msprg commented on GitHub (Dec 18, 2024):

Do you have syno_enable_dedupe.sh scheduled?

I don't.

My real Synology NAS don't have the storage_panel.js.gz (or storage_panel.js.gz.bak) files.

Oh, I didn't know that!
I've tried to modify the archives, and it seems to have worked!
I think they're there because the DSM loader allows choosing things such as HDD bays graphics etc., so I guess it's achieving that by modifying the storage_panel.js as well.

<!-- gh-comment-id:2551961662 --> @Msprg commented on GitHub (Dec 18, 2024): > Do you have syno_enable_dedupe.sh scheduled? I don't. > My real Synology NAS don't have the storage_panel.js.gz (or storage_panel.js.gz.bak) files. Oh, I didn't know that! I've tried to modify the archives, and it seems to have worked! I think they're there because the DSM loader allows choosing things such as HDD bays graphics etc., so I guess it's achieving that by modifying the storage_panel.js as well.
Author
Owner

@007revad commented on GitHub (Dec 18, 2024):

I've tried to modify the archives, and it seems to have worked!

If you reboot does the storage_panel.js.gz still contain your edits?

I might change the script to delete the existing storage_panel.js.gz and gzip storage_panel.js

But before I do that can you provide a zip file containing everything in /usr/local/packages/@appstore/StorageManager/ui

<!-- gh-comment-id:2552440158 --> @007revad commented on GitHub (Dec 18, 2024): > I've tried to modify the archives, and it seems to have worked! If you reboot does the storage_panel.js.gz still contain your edits? I might change the script to delete the existing storage_panel.js.gz and gzip storage_panel.js But before I do that can you provide a zip file containing everything in /usr/local/packages/@appstore/StorageManager/ui
Author
Owner

@Msprg commented on GitHub (Dec 19, 2024):

If you reboot does the storage_panel.js.gz still contain your edits?

So far, yes they do. However, I suspect that updates to the loader can easily break it / revert the changes.

But before I do that can you provide a zip file containing everything in /usr/local/packages/@appstore/StorageManager/ui

Sure can do. Here you go, should be in more-less unmodified (xpenology / loader) state:
StorageManagerui.tar.gz

<!-- gh-comment-id:2555101968 --> @Msprg commented on GitHub (Dec 19, 2024): > If you reboot does the storage_panel.js.gz still contain your edits? So far, yes they do. However, I suspect that updates to the loader can easily break it / revert the changes. > But before I do that can you provide a zip file containing everything in /usr/local/packages/@appstore/StorageManager/ui Sure can do. Here you go, should be in more-less unmodified (xpenology / loader) state: [StorageManagerui.tar.gz](https://github.com/user-attachments/files/18200954/StorageManagerui.tar.gz)
Author
Owner

@Msprg commented on GitHub (May 20, 2025):

I mean, this is technically solved for me, as well as anyone else that will just replace the file in the GZIP archive. Whether or not this will be natively implemented into this dedupe script is a separate matter, at least for me.

Thanks for the cooperation, I'm happy I got it working ^^

<!-- gh-comment-id:2895981749 --> @Msprg commented on GitHub (May 20, 2025): I mean, this is technically solved for me, as well as anyone else that will just replace the file in the GZIP archive. Whether or not this will be natively implemented into this dedupe script is a separate matter, at least for me. Thanks for the cooperation, I'm happy I got it working ^^
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/Synology_enable_Deduplication#21
No description provided.