[GH-ISSUE #203] META: Duplicate bookmarks causing issues

kerem commented

2026-02-25 22:36:26 +03:00

Owner

Originally created by @jlbprof on GitHub (Nov 27, 2018).
Original GitHub issue: https://github.com/floccusaddon/floccus/issues/203

I intend this to be more of a discussion than a specific bug.

A month ago or so, I got an error "missing children in folder" and it turned out there was a few duplicates in my bookmark list.

I had the same issue today, there were still duplicates in my bookmarks. They have been there all along. So sometimes duplicates can cause issues sometimes they don't.

So the discussion: What about a duplicate could cause that issue?

To solve this problem I wrote a Perl script that read and modified my bookmarks.html file (rather than the .xbel file as I originally thought it was the merge between what was in my browser and what was in the xbel file).

Once ALL duplicates were removed it synced.

So what about a duplicate remover in the extension, not as a regular option but perhaps as another option that could be clicked?

It would read all the local bookmarks in a tree, and descends it looking for duplicates. My perl script used the URL as the key for finding a duplicate, but the title might trigger it also. After removing the duplicates from the tree it would rewrite it out to the bookmarks in the browser.

How hard would that be?

Is that a good idea?

Another way this could be addressed if the console log when the error occurs prints out the information about the folder or bookmark it is tripping on. I looked at the console output and it shows the id of the folder or bookmark but not the title or url that I could search for in the bookmarks manager. Some better output may work as well.

Thanx

Julian

Originally created by @jlbprof on GitHub (Nov 27, 2018). Original GitHub issue: https://github.com/floccusaddon/floccus/issues/203 I intend this to be more of a discussion than a specific bug. A month ago or so, I got an error "missing children in folder" and it turned out there was a few duplicates in my bookmark list. I had the same issue today, there were still duplicates in my bookmarks. They have been there all along. So sometimes duplicates can cause issues sometimes they don't. So the discussion: What about a duplicate could cause that issue? To solve this problem I wrote a Perl script that read and modified my bookmarks.html file (rather than the .xbel file as I originally thought it was the merge between what was in my browser and what was in the xbel file). Once ALL duplicates were removed it synced. So what about a duplicate remover in the extension, not as a regular option but perhaps as another option that could be clicked? It would read all the local bookmarks in a tree, and descends it looking for duplicates. My perl script used the URL as the key for finding a duplicate, but the title might trigger it also. After removing the duplicates from the tree it would rewrite it out to the bookmarks in the browser. How hard would that be? Is that a good idea? Another way this could be addressed if the console log when the error occurs prints out the information about the folder or bookmark it is tripping on. I looked at the console output and it shows the id of the folder or bookmark but not the title or url that I could search for in the bookmarks manager. Some better output may work as well. Thanx Julian

kerem closed this issue

2026-02-25 22:36:26 +03:00

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@marcelklehr commented on GitHub (Nov 27, 2018):

Hey! I'm working on this problem at the moment. The last few bugfixes were related to deduplication and handling of duplicates. These didn't pose any problems when floccus still didn't care about an exact match of folder contents between the server and the browser. Since I introduced sort order preservation however, this is no longer the case: The operation that imposes the sort order is hardened to only carry it out if the two folders have the same content size.

What about a duplicate could cause that issue?

Sometimes it's two different encodings of the same URL that cause problems (see github.com/marcelklehr/floccus@09607e6d8f), sometimes the server and the client use different algorithms to normalize URLs (see https://github.com/nextcloud/bookmarks/issues/607), sometimes it's bookmarks that have a scheme that the server doesn't accept (see github.com/marcelklehr/floccus@c577a175f1), sometimes the developer forgets edge cases (see github.com/marcelklehr/floccus@b5bf6b14e6, github.com/marcelklehr/floccus@9aa50abfac, github.com/marcelklehr/floccus@3c1b304af9, github.com/marcelklehr/floccus@f2ec29edb6).

Once ALL duplicates were removed it synced.

My requirement is that floccus be able to sync without making deduplication necessary.

The main problem at the moment is that we need more tests, in my opinion.

@marcelklehr commented on GitHub (Nov 27, 2018): Hey! I'm working on this problem at the moment. The last few bugfixes were related to deduplication and handling of duplicates. These didn't pose any problems when floccus still didn't care about an exact match of folder contents between the server and the browser. Since I introduced sort order preservation however, this is no longer the case: The operation that imposes the sort order is hardened to only carry it out if the two folders have the same content size. > What about a duplicate could cause that issue? Sometimes it's two different encodings of the same URL that cause problems (see https://github.com/marcelklehr/floccus/commit/09607e6d8f45d0c5ebce12f50eedfb2626516c1d), sometimes the server and the client use different algorithms to normalize URLs (see https://github.com/nextcloud/bookmarks/issues/607), sometimes it's bookmarks that have a scheme that the server doesn't accept (see https://github.com/marcelklehr/floccus/commit/c577a175f1c122158d3b612fe1668f07d4ea83e8), sometimes the developer forgets edge cases (see https://github.com/marcelklehr/floccus/commit/b5bf6b14e61e07bfb6ed96151cfe6d17af0c958f, https://github.com/marcelklehr/floccus/commit/9aa50abfacf2d37ce136687532b833f795b4a1ac, https://github.com/marcelklehr/floccus/commit/3c1b304af9ae613b0e117f818a2818dc644b5b33, https://github.com/marcelklehr/floccus/commit/f2ec29edb618e45c79ffa7c819a68857400851ca). > Once ALL duplicates were removed it synced. My requirement is that floccus be able to sync without making deduplication necessary. The main problem at the moment is that we need more tests, in my opinion.

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@marcelklehr commented on GitHub (Nov 27, 2018):

Do you happen to still have the logs from the recent error?

@marcelklehr commented on GitHub (Nov 27, 2018): Do you happen to still have the logs from the recent error?

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@jlbprof commented on GitHub (Nov 27, 2018):

I am not familiar with Javascript testing. But perhaps using bookmarks.html style bookmark sets that look out for some of those issues would be useful. We could load the bookmarks and run a sync. For automated tests I am not sure how we could do that considering either form needs a backend either a NC server or a WebDAV server.

And no, I did not keep the logs, I just went after the duplicates. I will remember next time.

@jlbprof commented on GitHub (Nov 27, 2018): I am not familiar with Javascript testing. But perhaps using bookmarks.html style bookmark sets that look out for some of those issues would be useful. We could load the bookmarks and run a sync. For automated tests I am not sure how we could do that considering either form needs a backend either a NC server or a WebDAV server. And no, I did not keep the logs, I just went after the duplicates. I will remember next time.

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@marcelklehr commented on GitHub (Nov 27, 2018):

Testing infrastructure is not a problem, actually -- I've obsessed over this enough to make it quite reliable: https://travis-ci.org/marcelklehr/floccus/ 😄

If you like you can have a look at the tests and add some more in a new branch: https://github.com/marcelklehr/floccus/blob/master/src/test/test.js (any branches in this repo will run through travis, but you can also test locally by simply running a local nextcloud instance. I use docker for this)

@marcelklehr commented on GitHub (Nov 27, 2018): Testing infrastructure is not a problem, actually -- I've obsessed over this enough to make it quite reliable: https://travis-ci.org/marcelklehr/floccus/ :smile: If you like you can have a look at the tests and add some more in a new branch: https://github.com/marcelklehr/floccus/blob/master/src/test/test.js (any branches in this repo will run through travis, but you can also test locally by simply running a local nextcloud instance. I use docker for this)

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@jlbprof commented on GitHub (Nov 27, 2018):

I will look at the testing infrastructure.

In the meantime, I restored my old bookmarks (that is the set that caused the error) and deleted my bookmarks.xbel, and it synced correctly. It may have been a merge problem then, I did add a new browser, yesterday, to my mix of browsers. So I cannot duplicate the issue anymore sorry and get the logs.

@jlbprof commented on GitHub (Nov 27, 2018): I will look at the testing infrastructure. In the meantime, I restored my old bookmarks (that is the set that caused the error) and deleted my bookmarks.xbel, and it synced correctly. It may have been a merge problem then, I did add a new browser, yesterday, to my mix of browsers. So I cannot duplicate the issue anymore sorry and get the logs.

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@jlbprof commented on GitHub (Nov 27, 2018):

floccus-3.1.13-2018-11-27.log
OK I got a log when I used my home computer's browser.

@jlbprof commented on GitHub (Nov 27, 2018): [floccus-3.1.13-2018-11-27.log](https://github.com/marcelklehr/floccus/files/2622224/floccus-3.1.13-2018-11-27.log) OK I got a log when I used my home computer's browser.

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@jlbprof commented on GitHub (Nov 27, 2018):

The log does not indicate what or why it failed but it looks like it has my bookmarks listed. Are you able to recreate the issue with those logs?

@jlbprof commented on GitHub (Nov 27, 2018): The log does not indicate what or why it failed but it looks like it has my bookmarks listed. Are you able to recreate the issue with those logs?

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@jlbprof commented on GitHub (Nov 27, 2018):

✘ Error!
Folder ordering is missing some of the folder's children

@jlbprof commented on GitHub (Nov 27, 2018): ✘ Error! Folder ordering is missing some of the folder's children

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@marcelklehr commented on GitHub (Nov 28, 2018):

In your case, this appears to have been the problem: github.com/marcelklehr/floccus@9d90401b18

@marcelklehr commented on GitHub (Nov 28, 2018): In your case, this appears to have been the problem: https://github.com/marcelklehr/floccus/commit/9d90401b182b1674354da8650cb56d252f677645

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@erdnussflips commented on GitHub (Nov 29, 2018):

Same problem here, account with xbel: Folder ordering is missing some of the folder's children

@erdnussflips commented on GitHub (Nov 29, 2018): Same problem here, account with xbel: `Folder ordering is missing some of the folder's children`

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@marcelklehr commented on GitHub (Nov 30, 2018):

Should be fixed in v3.1.14

@marcelklehr commented on GitHub (Nov 30, 2018): Should be fixed in v3.1.14

kerem commented

2026-02-25 22:36:27 +03:00

Author

Owner

@github-actions[bot] commented on GitHub (Mar 21, 2023):

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions[bot] commented on GitHub (Mar 21, 2023): This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Rows
Columns

[GH-ISSUE #203] META: Duplicate bookmarks causing issues #195