[GH-ISSUE #91] to slow, takes to much cpu

kerem commented

2026-02-25 22:36:06 +03:00

Owner

Originally created by @akidburn on GitHub (May 2, 2018).
Original GitHub issue: https://github.com/floccusaddon/floccus/issues/91

version and config

Everything current (Firefox, floccus, nextcloud, chrome)
Nextcloud 13 installed on odroid c2 ( arm 1.5bghz quad-core, 2 gb ram) using nginx and php7-fpm
Nextcloud bruteforce protection disabled (had some 1500 entries there before I noticed)

About 3700 bookmarks previously synced by xmarks across 3 machines with Firefox and chrome each, so 6 clients.

current situation

First sync to upload bookmarks is relatively fast, If the tables are empty. Next download is not too bad on the server, but if you have a few hundred tabs open and a few extensions, you get a rate of one bookmarks per second added to Firefox. (Waaaay to slow)

But the real problem is, if you want to update the collection by adding new bookmarks. For one, each machine has to be fully synced from nextcloud before you start adding new bookmarks or you can delete everything again. But the actual problem with nextcloud on small raspberry-like hosts is the missing CPU power compared to desktop PC's. If I try to add a single bookmark to one client and sync it to server, I can't sync anything for at least 2 hours. There are at least 10 threads of php7-fpm running comparing the bookmarks, and that's just bad.

Originally created by @akidburn on GitHub (May 2, 2018). Original GitHub issue: https://github.com/floccusaddon/floccus/issues/91 ## version and config Everything current (Firefox, floccus, nextcloud, chrome) Nextcloud 13 installed on odroid c2 ( arm 1.5bghz quad-core, 2 gb ram) using nginx and php7-fpm Nextcloud bruteforce protection disabled (had some 1500 entries there before I noticed) About 3700 bookmarks previously synced by xmarks across 3 machines with Firefox and chrome each, so 6 clients. ## current situation First sync to upload bookmarks is relatively fast, If the tables are empty. Next download is not too bad on the server, but if you have a few hundred tabs open and a few extensions, you get a rate of one bookmarks per second added to Firefox. (Waaaay to slow) But the real problem is, if you want to update the collection by adding new bookmarks. For one, each machine has to be fully synced from nextcloud before you start adding new bookmarks or you can delete everything again. But the actual problem with nextcloud on small raspberry-like hosts is the missing CPU power compared to desktop PC's. If I try to add a single bookmark to one client and sync it to server, I can't sync anything for at least 2 hours. There are at least 10 threads of php7-fpm running comparing the bookmarks, and that's just bad.

kerem

2026-02-25 22:36:06 +03:00

closed this issue
added the
waiting for more information
label

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@marcelklehr commented on GitHub (May 2, 2018):

Hey @akidburn

Thank you for reporting this. So far I'm aware of performance problems with Firefox (which is due to implementation details in the browser), but not with Chrome.

version and config: Everything current

Generally, this doesn't help much, I'm afraid, as I don't know what your perception of 'current' is and if it is even correct. I need hard facts :)

For one, each machine has to be fully synced from nextcloud before you start adding new bookmarks or you can delete everything again.

Could you elaborate what you mean by this? It should be possible to add bookmarks in a stale local state and even while syncing.

But the actual problem with nextcloud on small raspberry-like hosts is the missing CPU power compared to desktop PC's. If I try to add a single bookmark to one client and sync it to server, I can't sync anything for at least 2 hours. There are at least 10 threads of php7-fpm running comparing the bookmarks, and that's just bad.

Do you sync any data on that machine other than bookmarks, with a comparable amount of data? I would be surprised if you saw better performance there. I'm afraid, this will hardly be improvable with the nextcloud bookmarks app as a backend, work is being done to develop a WebDAV backend, which may be more performant on low-grade server hardware.

Still, it's surprising to me that a single bookmark addition causes so much load on your server, as the server doesn't do any kind of comparison. It simply handles the CRUD operations necessary for the sync to work.

@marcelklehr commented on GitHub (May 2, 2018): Hey @akidburn Thank you for reporting this. So far I'm aware of performance problems with Firefox (which is due to implementation details in the browser), but not with Chrome. > version and config: Everything current Generally, this doesn't help much, I'm afraid, as I don't know what your perception of 'current' is and if it is even correct. I need hard facts :) > For one, each machine has to be fully synced from nextcloud before you start adding new bookmarks or you can delete everything again. Could you elaborate what you mean by this? It should be possible to add bookmarks in a stale local state and even while syncing. > But the actual problem with nextcloud on small raspberry-like hosts is the missing CPU power compared to desktop PC's. If I try to add a single bookmark to one client and sync it to server, I can't sync anything for at least 2 hours. There are at least 10 threads of php7-fpm running comparing the bookmarks, and that's just bad. Do you sync any data on that machine other than bookmarks, with a comparable amount of data? I would be surprised if you saw better performance there. I'm afraid, this will hardly be improvable with the nextcloud bookmarks app as a backend, work is being done to develop a WebDAV backend, which may be more performant on low-grade server hardware. Still, it's surprising to me that a single bookmark addition causes so much load on your *server*, as the server doesn't do any kind of comparison. It simply handles the CRUD operations necessary for the sync to work.

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@akidburn commented on GitHub (May 4, 2018):

Hi @marcelklehr

Thanks for the reply. In general, about Chrome, maybe I am biased compared to Xmarks, so this is mainly about Firefox and the Serverload.

Versions

Clients

Windos 10 Pro Version 1709
Google Chrome 66.0.3359.139 (mostly fine, speedwise)
Mozilla Firefox 59.0.3 (64-Bit)
Floccus 2.1.0 (both FF and Chrome)

Server

odroid-c2 3.14.79-117
Amlogic ARM® Cortex®-A53(ARMv8) 1.5Ghz quad core CPU
2Gbyte DDR3 SDRAM
Ubuntu 16.04
Nextcloud 13.0.2
Bookmarks App 0.11.0
nginx/1.13.12
php Version: 7.2.4
mariadb Version: 10.0.34

Speed Issue

As it is, i watched htop on the odroid-c2 during these issues.
there were up to 20 php-fpm processes open, all working, and the mariadb was using only 20 percent of a single core. i dont know what the php processes were doing, but there sure as hell was more going on than just CRUD into the db. also, whenever there was a sync error, meaning bookmark already existed, the cpu was dropping to zero immediately. could it be that the bookmark comparisons are done by php and not by sql?

akidburn

@akidburn commented on GitHub (May 4, 2018): Hi @marcelklehr Thanks for the reply. In general, about Chrome, maybe I am biased compared to Xmarks, so this is mainly about Firefox and the Serverload. ## Versions ### Clients Windos 10 Pro Version 1709 Google Chrome 66.0.3359.139 (mostly fine, speedwise) Mozilla Firefox 59.0.3 (64-Bit) Floccus 2.1.0 (both FF and Chrome) ### Server odroid-c2 3.14.79-117 Amlogic ARM® Cortex®-A53(ARMv8) 1.5Ghz quad core CPU 2Gbyte DDR3 SDRAM Ubuntu 16.04 Nextcloud 13.0.2 Bookmarks App 0.11.0 nginx/1.13.12 php Version: 7.2.4 mariadb Version: 10.0.34 ## Speed Issue As it is, i watched htop on the odroid-c2 during these issues. there were up to 20 php-fpm processes open, all working, and the mariadb was using only 20 percent of a single core. i dont know what the php processes were doing, but there sure as hell was more going on than just CRUD into the db. also, whenever there was a sync error, meaning bookmark already existed, the cpu was dropping to zero immediately. could it be that the bookmark comparisons are done by php and not by sql? akidburn

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@marcelklehr commented on GitHub (May 4, 2018):

As I maintain the server-side bookmarks app as well, I'm fairly sure about what's going on. Let me give you a rough overview:

Floccus synchronization has 4 phases:

Create new bookmarks on the server (this is the only part where the server tries to match incoming URLs with existing ones, so as not to create duplicates). When syncing for the first time, this will take the longest, even though floccus sends multiple requests at once.
Delete removed bookmarks from the server.
Get all bookmarks from the server and compare them to the local ones, deciding whether to
a) update the matching local one
b) update the server-side one
c) create a new local bookmark in case there is none that matches
Delete local bookmarks that no longer exist on the server.

Floccus makes a maximum of 10 requests in parallel to mitigate network latency. If the server is not that fast this may put it under stress, but floccus will wait for the running requests to finish before starting new ones. Downloading all bookmarks is a problem, but can't be avoided currently, as the bookmarks app doesn't have folders that could be used to make a tree comparison, which might terminate early.

@marcelklehr commented on GitHub (May 4, 2018): As I maintain the server-side bookmarks app as well, I'm fairly sure about what's going on. Let me give you a rough overview: Floccus synchronization has 4 phases: 1. Create new bookmarks on the server (this is the only part where the server tries to match incoming URLs with existing ones, so as not to create duplicates). When syncing for the first time, this will take the longest, even though floccus sends multiple requests at once. 2. Delete removed bookmarks from the server. 3. Get all bookmarks from the server and compare them to the local ones, deciding whether to a) update the matching local one b) update the server-side one c) create a new local bookmark in case there is none that matches 4. Delete local bookmarks that no longer exist on the server. Floccus makes a maximum of 10 requests in parallel to mitigate network latency. If the server is not that fast this may put it under stress, but floccus will wait for the running requests to finish before starting new ones. Downloading *all* bookmarks is a problem, but can't be avoided currently, as the bookmarks app doesn't have folders that could be used to make a tree comparison, which might terminate early.

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@akidburn commented on GitHub (May 4, 2018):

just tried to sync bookmarks into a chrome directory. no server load, though chrome nearly fries my old notebook with constant 100 % load, and now...

Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined

could the reason be if the bookmarks have no name, only an url?

EDIT:

another error:

Removing a bookmark on the server failed. remoteId=11451 Removing a bookmark on the server failed. remoteId=11057 Removing a bookmark on the server failed. remoteId=11056 Removing a bookmark on the server failed. remoteId=11043 Removing a bookmark on the server failed. remoteId=11054 Removing a bookmark on the server failed. remoteId=11053 Removing a bookmark on the server failed. remoteId=11450 Removing a bookmark on the server failed. remoteId=11045 Removing a bookmark on the server failed. remoteId=11058 Removing a bookmark on the server failed. remoteId=11055

how to reproduce

sync a folder with subfolders between two chrome clients
delete all bookmarks from the synced folder on one client and wait until the bookmarks are gone on nc
check client 2, if necessary click sync and watch errors

@akidburn commented on GitHub (May 4, 2018): just tried to sync bookmarks into a chrome directory. no server load, though chrome nearly fries my old notebook with constant 100 % load, and now... `Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined Cannot read property 'hash' of undefined` could the reason be if the bookmarks have no name, only an url? EDIT: another error: `Removing a bookmark on the server failed. remoteId=11451 Removing a bookmark on the server failed. remoteId=11057 Removing a bookmark on the server failed. remoteId=11056 Removing a bookmark on the server failed. remoteId=11043 Removing a bookmark on the server failed. remoteId=11054 Removing a bookmark on the server failed. remoteId=11053 Removing a bookmark on the server failed. remoteId=11450 Removing a bookmark on the server failed. remoteId=11045 Removing a bookmark on the server failed. remoteId=11058 Removing a bookmark on the server failed. remoteId=11055` ## how to reproduce 1. sync a folder with subfolders between two chrome clients 2. delete all bookmarks from the synced folder on one client and wait until the bookmarks are gone on nc 3. check client 2, if necessary click sync and watch errors

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@marcelklehr commented on GitHub (May 8, 2018):

"Removing a bookmark on the server failed." Is an upstream bug with nextcloud bookmarks, which will be resolved with the next release.

"Cannot read property 'hash' of undefined" is a new one. Does it persist?

@marcelklehr commented on GitHub (May 8, 2018): "Removing a bookmark on the server failed." Is an upstream bug with nextcloud bookmarks, which will be resolved with the next release. "Cannot read property 'hash' of undefined" is a new one. Does it persist?

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@akidburn commented on GitHub (May 19, 2018):

No it doesnt persist. May be a chrome issue during high cpu loads.
Floccus 2.2.2 works now, though if any two machines are synchronizing at the same time, errors occur.

@akidburn commented on GitHub (May 19, 2018): No it doesnt persist. May be a chrome issue during high cpu loads. Floccus 2.2.2 works now, though if any two machines are synchronizing at the same time, errors occur.

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@marcelklehr commented on GitHub (May 19, 2018):

Which kind of errors?

@marcelklehr commented on GitHub (May 19, 2018): Which kind of errors?

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@akidburn commented on GitHub (May 21, 2018):

Either the "Duplicate Bookmark" error or the "Removing a bookmark on the server failed" error, though in watching the oc_bookmark table i found, that just attempting to add a single new bookmark sometimes creates duplicates in the table, that dont produce errors until you force sync.

The Duplicate Error also leads directly or indirectly to the Removing error, if more then one client is in a failure state. As in, if you remove the duplicate client side, at least one client is going to display the Removing Error.

Though i found a little sql script that removes the duplicates from the tables, could be integrated in the bookmarks app as a cronjob if the limitation of no duplicates isnt going away.

DELETE a FROM oc_bookmarks a, oc_bookmarks b
WHERE a.id < b.id
AND a.url <=> b.url;

DELETE c FROM oc_bookmarks_tags c
LEFT JOIN oc_bookmarks f ON f.id = c.bookmark_id
WHERE f.id IS NULL;

@akidburn commented on GitHub (May 21, 2018): Either the "Duplicate Bookmark" error or the "Removing a bookmark on the server failed" error, though in watching the oc_bookmark table i found, that just attempting to add a single new bookmark sometimes creates duplicates in the table, that dont produce errors until you force sync. The Duplicate Error also leads directly or indirectly to the Removing error, if more then one client is in a failure state. As in, if you remove the duplicate client side, at least one client is going to display the Removing Error. Though i found a little sql script that removes the duplicates from the tables, could be integrated in the bookmarks app as a cronjob if the limitation of no duplicates isnt going away. >DELETE a FROM oc_bookmarks a, oc_bookmarks b WHERE a.id < b.id AND a.url <=> b.url; >DELETE c FROM oc_bookmarks_tags c LEFT JOIN oc_bookmarks f ON f.id = c.bookmark_id WHERE f.id IS NULL;

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@marcelklehr commented on GitHub (May 31, 2018):

Try v2.2.6 which solves a bug which I could verify causes some if not all of these errors.

@marcelklehr commented on GitHub (May 31, 2018): Try v2.2.6 which solves a bug which I could verify causes some if not all of these errors.

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@marcelklehr commented on GitHub (Jun 4, 2018):

All issues resolved?! 🎉

@marcelklehr commented on GitHub (Jun 4, 2018): All issues resolved?! :tada:

kerem commented

2026-02-25 22:36:07 +03:00

Author

Owner

@github-actions[bot] commented on GitHub (Mar 21, 2023):

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions[bot] commented on GitHub (Mar 21, 2023): This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Rows
Columns

[GH-ISSUE #91] to slow, takes to much cpu #90

version and config

current situation

Versions

Clients

Server

Speed Issue

how to reproduce