[GH-ISSUE #238] Error: Request timed out. Check your server configuration #231

Closed
opened 2026-02-25 22:36:33 +03:00 by kerem · 48 comments
Owner

Originally created by @marcelklehr on GitHub (Mar 3, 2019).
Original GitHub issue: https://github.com/floccusaddon/floccus/issues/238

@bhlevca

One year later and I have the same problem. With nextcloud behind haproxy floccus cannot connect.
Everything else works fine. I tried all the suggestion form above . I don't have the brute force app , but I use fail2ban instead.
My nextcloud server runs in virtual domain of an apache2 server and it is difficult for me to connect directly to that apache virtual domain using the IP.

Is there any way to debug this i looked in haproxy.logs and in the nextcloud logsand I found nothing relevant.

Originally created by @marcelklehr on GitHub (Mar 3, 2019). Original GitHub issue: https://github.com/floccusaddon/floccus/issues/238 @bhlevca > One year later and I have the same problem. With nextcloud behind haproxy floccus cannot connect. Everything else works fine. I tried all the suggestion form above . I don't have the brute force app , but I use fail2ban instead. > My nextcloud server runs in virtual domain of an apache2 server and it is difficult for me to connect directly to that apache virtual domain using the IP. > > Is there any way to debug this i looked in haproxy.logs and in the nextcloud logsand I found nothing relevant.
kerem closed this issue 2026-02-25 22:36:33 +03:00
Author
Owner

@marcelklehr commented on GitHub (Mar 3, 2019):

It could help to look at the network activity in the debugger:

Firefox

  • Go to about:debugging
  • enable debugging
  • click on debug button next to floccus entry
  • go to the network tab in the new window
  • trigger a sync by clicking on force Sync for the floccus account of your choice.
  • go back to the inspector window and check what's happening

Chrome

  • Go to chrome://extensions
  • enable Developer mode
  • click on dist/html/background.html button in floccus' entry next to "Inspect views: "
  • go to the network tab in the new window
  • trigger a sync by clicking on force Sync for the floccus account of your choice
  • go back to the inspector window and check what's happening
<!-- gh-comment-id:469015575 --> @marcelklehr commented on GitHub (Mar 3, 2019): It could help to look at the network activity in the debugger: #### Firefox * Go to `about:debugging` * enable debugging * click on `debug` button next to floccus entry * go to the network tab in the new window * trigger a sync by clicking on `force Sync` for the floccus account of your choice. * go back to the inspector window and check what's happening #### Chrome * Go to `chrome://extensions` * enable `Developer mode` * click on `dist/html/background.html` button in floccus' entry next to "Inspect views: " * go to the network tab in the new window * trigger a sync by clicking on `force Sync` for the floccus account of your choice * go back to the inspector window and check what's happening
Author
Owner

@marcelklehr commented on GitHub (Mar 3, 2019):

Also, please provide the following:

  • Browser:
  • Nextcloud:
  • Nextcloud Bookmarks app:
  • Floccus:
  • Floccus sync method:

and the exact error message :)

<!-- gh-comment-id:469017840 --> @marcelklehr commented on GitHub (Mar 3, 2019): Also, please provide the following: - Browser: - Nextcloud: - Nextcloud Bookmarks app: - Floccus: - Floccus sync method: and the exact error message :)
Author
Owner

@bhlevca commented on GitHub (Mar 3, 2019):

I did the debugging in firefox as it was your firs choice:

I attached the har file that contains the debuging messages but the reply was rejected so I am doing it again on the github site.

Broswer                        Firefox v64.0.2 ( can be ssen in the file
Nexcloud                      15.0.5
NC Bookmak app       0.16.2
Floccus                         3.2.7
Floccus sync method  Nextcloud bookmarks with folders (I tried legacy too, same thing)

Error: Request timed out. Check your server configuration

it takes about 67 sec to return info for the GET method in the debugger window so it is obviously a time out because I set the timeout at 1min in haproxy so I asume ha proxy gets it

I find this line in the nextcloud access log:

10.0.0.13 - bogdan [03/Mar/2019:11:17:59 -0500] "GET /index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1 HTTP/1.1" 200 1043851

and in the haproxy log :

10.0.0.13 - bogdan [03/Mar/2019:11:17:59 -0500] "GET index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1 HTTP/1.1" 200
1043851

Interesting to note that the logs only appear after 1 min, which it appears is the floccus default timeout, wheres in the debug windows of floccus in both firefox and chrome the request is shown immediately after the trigger

On Chrome I get this additional information:

504 Gateway Time-out The server didn't respond in time.

What do you make of all this?
Thanks
Bogdan

<!-- gh-comment-id:469042434 --> @bhlevca commented on GitHub (Mar 3, 2019): I did the debugging in firefox as it was your firs choice: I attached the har file that contains the debuging messages but the reply was rejected so I am doing it again on the github site. ``` Broswer Firefox v64.0.2 ( can be ssen in the file Nexcloud 15.0.5 NC Bookmak app 0.16.2 Floccus 3.2.7 Floccus sync method Nextcloud bookmarks with folders (I tried legacy too, same thing) ``` > Error: Request timed out. Check your server configuration it takes about 67 sec to return info for the GET method in the debugger window so it is obviously a time out because I set the timeout at 1min in haproxy so I asume ha proxy gets it I find this line in the nextcloud access log: > 10.0.0.13 - bogdan [03/Mar/2019:11:17:59 -0500] "GET /index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1 HTTP/1.1" 200 1043851 and in the haproxy log : > 10.0.0.13 - bogdan [03/Mar/2019:11:17:59 -0500] "GET index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1 HTTP/1.1" 200 > 1043851 Interesting to note that the logs only appear after 1 min, which it appears is the floccus default timeout, wheres in the debug windows of floccus in both firefox and chrome the request is shown immediately after the trigger On Chrome I get this additional information: > 504 Gateway Time-out The server didn't respond in time. What do you make of all this? Thanks Bogdan
Author
Owner

@marcelklehr commented on GitHub (Mar 3, 2019):

Alright, now we have data :)

504 Gateway Time-out The server didn't respond in time.

504 means that the ha proxy has decided to shut the request down and return nothing.

Interesting to note that the logs only appear after 1 min, which it appears is the floccus default timeout, wheres in the debug windows of floccus in both firefox and chrome the request is shown immediately after the trigger

This is just due to the way logs are written on the server. The browsers display them while the requests are running, but the server only writes a log entry once the request is done (whatever that means in this case...).

So, judging by the time out, it appears that this is a performance problem. Sometimes the ha proxy shuts down the request, sometimes floccus does. How many bookmarks do you have on the server? Which database are you using and what is your estimation of the general performance expectation for the server hardware (i.e. is it a Raspi or some high end server)?

<!-- gh-comment-id:469048466 --> @marcelklehr commented on GitHub (Mar 3, 2019): Alright, now we have data :) > 504 Gateway Time-out The server didn't respond in time. 504 means that the ha proxy has decided to shut the request down and return nothing. > Interesting to note that the logs only appear after 1 min, which it appears is the floccus default timeout, wheres in the debug windows of floccus in both firefox and chrome the request is shown immediately after the trigger This is just due to the way logs are written on the server. The browsers display them while the requests are running, but the server only writes a log entry once the request is done (whatever that means in this case...). So, judging by the time out, it appears that this is a performance problem. Sometimes the ha proxy shuts down the request, sometimes floccus does. How many bookmarks do you have on the server? Which database are you using and what is your estimation of the general performance expectation for the server hardware (i.e. is it a Raspi or some high end server)?
Author
Owner

@bhlevca commented on GitHub (Mar 3, 2019):

My turn : -) Some things seem to become more clear now

Yes it is a small computer a raspberry Pi so, so small processing power. I have a cluster of them behind the haproxy for load balancing and I synchronize the databases (mysql) with circular master/slave.

However, to make things simpler for nextcloud the nextcloud backend of the haproxy has only one server, which means that the requests for nextcloud go always to the same server/db

I can play with the timeouts on the haproxy side to make them longer if needed , but if the 1 min timeout is on the floccus side I cannot do anything.

This is the first time I try to sync the bookmarks so it is a huge number, over 4000.

Any suggestions?

<!-- gh-comment-id:469050765 --> @bhlevca commented on GitHub (Mar 3, 2019): My turn : -) Some things seem to become more clear now Yes it is a small computer a raspberry Pi so, so small processing power. I have a cluster of them behind the haproxy for load balancing and I synchronize the databases (mysql) with circular master/slave. However, to make things simpler for nextcloud the nextcloud backend of the haproxy has only one server, which means that the requests for nextcloud go always to the same server/db I can play with the timeouts on the haproxy side to make them longer if needed , but if the 1 min timeout is on the floccus side I cannot do anything. This is the first time I try to sync the bookmarks so it is a huge number, over 4000. Any suggestions?
Author
Owner

@bhlevca commented on GitHub (Mar 3, 2019):

I tried with a smaller folder with only a few bookmarks, It fails the same way so it is not a problem with the size of the folder of bookmarks

Also , I increased all the possible timeouts on the proxy, so the timeout is either on the floccus side or on the nextcloud side and is always around 60s, in firefox debug window it gave me 67 sec.

<!-- gh-comment-id:469051684 --> @bhlevca commented on GitHub (Mar 3, 2019): I tried with a smaller folder with only a few bookmarks, It fails the same way so it is not a problem with the size of the folder of bookmarks Also , I increased all the possible timeouts on the proxy, so the timeout is either on the floccus side or on the nextcloud side and is always around 60s, in firefox debug window it gave me 67 sec.
Author
Owner

@marcelklehr commented on GitHub (Mar 3, 2019):

This is the first time I try to sync the bookmarks so it is a huge number, over 4000.
I tried with a smaller folder with only a few bookmarks, It fails the same way so it is not a problem with the size of the folder of bookmarks

So, if this is the first time you're syncing, do you have all of your bookmarks on the server already?

<!-- gh-comment-id:469054144 --> @marcelklehr commented on GitHub (Mar 3, 2019): > This is the first time I try to sync the bookmarks so it is a huge number, over 4000. > I tried with a smaller folder with only a few bookmarks, It fails the same way so it is not a problem with the size of the folder of bookmarks So, if this is the first time you're syncing, do you have all of your bookmarks on the server already?
Author
Owner

@bhlevca commented on GitHub (Mar 3, 2019):

No, there is nothing on the server although I can import them from a html file if that helps

<!-- gh-comment-id:469054655 --> @bhlevca commented on GitHub (Mar 3, 2019): No, there is nothing on the server although I can import them from a html file if that helps
Author
Owner

@bhlevca commented on GitHub (Mar 3, 2019):

One more thing adding bookmarks with "Add to Nextcloud" JS button works well. Just syncing the browser existing bookmarks to the nextcloud server seems to an issue

<!-- gh-comment-id:469057521 --> @bhlevca commented on GitHub (Mar 3, 2019): One more thing adding bookmarks with "Add to Nextcloud" JS button works well. Just syncing the browser existing bookmarks to the nextcloud server seems to an issue
Author
Owner

@marcelklehr commented on GitHub (Mar 3, 2019):

So, with nothing on the server GET /index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1 takes more than one minute?

<!-- gh-comment-id:469057601 --> @marcelklehr commented on GitHub (Mar 3, 2019): So, with nothing on the server `GET /index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1` takes more than one minute?
Author
Owner

@bhlevca commented on GitHub (Mar 3, 2019):

So, with nothing on the server GET /index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1 takes more than one minute?

It appears so. I tried to sync only one leaf of my browser's bookmar tree, which has only 4 bookmarks and it takes the same amount of time to timeout.

As I mentioned above (cross posting) adding a bookmark with the "Add to Nextcloud" JS button works well. Only sync seems to be a problem.

<!-- gh-comment-id:469057948 --> @bhlevca commented on GitHub (Mar 3, 2019): > So, with nothing on the server GET /index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1 takes more than one minute? It appears so. I tried to sync only one leaf of my browser's bookmar tree, which has only 4 bookmarks and it takes the same amount of time to timeout. As I mentioned above (cross posting) adding a bookmark with the "Add to Nextcloud" JS button works well. Only sync seems to be a problem.
Author
Owner

@marcelklehr commented on GitHub (Mar 3, 2019):

If no other nextcloud URLs time out like that, the question is, why does this one.

<!-- gh-comment-id:469058491 --> @marcelklehr commented on GitHub (Mar 3, 2019): If no other nextcloud URLs time out like that, the question is, why does this one.
Author
Owner

@marcelklehr commented on GitHub (Mar 3, 2019):

Can you check nextcloud's own logging? It's at admin -> logging (or data/nextcloud.log)

<!-- gh-comment-id:469058605 --> @marcelklehr commented on GitHub (Mar 3, 2019): Can you check nextcloud's own logging? It's at `admin -> logging` (or `data/nextcloud.log`)
Author
Owner

@bhlevca commented on GitHub (Mar 3, 2019):

i have the log and I looked into it I saw nothing strange except mentioning the intl php lib not being native , but that was from yesterday, and i installed the right one. Can I add an attachment here somehow

If not I can share it on my googledrive with you or send t my email it is small . Do you have an email address I can send it to ?

<!-- gh-comment-id:469059543 --> @bhlevca commented on GitHub (Mar 3, 2019): i have the log and I looked into it I saw nothing strange except mentioning the intl php lib not being native , but that was from yesterday, and i installed the right one. Can I add an attachment here somehow If not I can share it on my googledrive with you or send t my email it is small . Do you have an email address I can send it to ?
Author
Owner

@marcelklehr commented on GitHub (Mar 3, 2019):

Do you have an email address I can send it to ?
Yes, on my profile page https://github.com/marcelklehr

We can also chat over here, if you like https://gitter.im/marcelklehr/floccus :)

<!-- gh-comment-id:469059731 --> @marcelklehr commented on GitHub (Mar 3, 2019): > Do you have an email address I can send it to ? Yes, on my profile page https://github.com/marcelklehr We can also chat over here, if you like https://gitter.im/marcelklehr/floccus :)
Author
Owner

@bhlevca commented on GitHub (Mar 4, 2019):

Marcel, just out of curiosity I stopped the long sync, switched log level from 0 to 3 and restarted apache and started sync again. Guess what? It does not timeout !!! It drives me crazy. I don't understand the only thing I did between getting stuck and working was playing with the log level in nextcloud.

I am glad that it is working now but I am scared like heck because I don't understand what has caused that timeout.

<!-- gh-comment-id:469086202 --> @bhlevca commented on GitHub (Mar 4, 2019): Marcel, just out of curiosity I stopped the long sync, switched log level from 0 to 3 and restarted apache and started sync again. Guess what? It does not timeout !!! It drives me crazy. I don't understand the only thing I did between getting stuck and working was playing with the log level in nextcloud. I am glad that it is working now but I am scared like heck because I don't understand what has caused that timeout.
Author
Owner

@marcelklehr commented on GitHub (Mar 4, 2019):

@bhlevca I suspected exactly this when thinking about it this morning :D Changing the loglevel is not the only thing we did, though. You also added that one bookmark via the bookmarklet. So, it could be that, when there are no bookmarks at all that the request times out, for whatever reason.

<!-- gh-comment-id:469113214 --> @marcelklehr commented on GitHub (Mar 4, 2019): @bhlevca I suspected exactly this when thinking about it this morning :D Changing the loglevel is not the only thing we did, though. You also added that one bookmark via the bookmarklet. So, it could be that, when there are no bookmarks at all that the request times out, for whatever reason.
Author
Owner

@bhlevca commented on GitHub (Mar 4, 2019):

@marcelklehr If it wouldn't take so much time to upload I could easily verify this by deleting all the bookmarks and trying again. Yes I am still up! Going to bed soon though. By the way the sync finished 2 min ago. Is my raspberry pi the culprit or is it a slow process? Could it be made fast by compression/decompression at both ends?

<!-- gh-comment-id:469114705 --> @bhlevca commented on GitHub (Mar 4, 2019): @marcelklehr If it wouldn't take so much time to upload I could easily verify this by deleting all the bookmarks and trying again. Yes I am still up! Going to bed soon though. By the way the sync finished 2 min ago. Is my raspberry pi the culprit or is it a slow process? Could it be made fast by compression/decompression at both ends?
Author
Owner

@marcelklehr commented on GitHub (Mar 4, 2019):

It's quite normal for this amount of bookmarks to take that long with floccus, at least on the first run. There is an option for parallel syncing, which uses more bandwidth and might be faster, though. Also, you can try selecting only a subfolder for testing purposes again (in this case, make sure to remove all server-side bookmarks beforehand...).

<!-- gh-comment-id:469115624 --> @marcelklehr commented on GitHub (Mar 4, 2019): It's quite normal for this amount of bookmarks to take that long with floccus, at least on the first run. There is an option for parallel syncing, which uses more bandwidth and might be faster, though. Also, you can try selecting only a subfolder for testing purposes again (in this case, make sure to remove all server-side bookmarks beforehand...).
Author
Owner

@bhlevca commented on GitHub (Mar 4, 2019):

I would try again but I need all my 4000 bookmarks and another 7 hours of loading is not a pretty perspective. In retrospect I should have done the test as you suggested with only a subfolder, but I had no expectations when we started the debugging. Maybe next time when something needs to be changed. Good night

<!-- gh-comment-id:469117778 --> @bhlevca commented on GitHub (Mar 4, 2019): I would try again but I need all my 4000 bookmarks and another 7 hours of loading is not a pretty perspective. In retrospect I should have done the test as you suggested with only a subfolder, but I had no expectations when we started the debugging. Maybe next time when something needs to be changed. Good night
Author
Owner

@marcelklehr commented on GitHub (Mar 4, 2019):

Good night :)

<!-- gh-comment-id:469118647 --> @marcelklehr commented on GitHub (Mar 4, 2019): Good night :)
Author
Owner

@bhlevca commented on GitHub (Mar 5, 2019):

Bad news, today the sync failed again with exactly the same problem. I made no changes to anything so I don't understand and don't believe in magic.
I did tried to find the culprit:

- disabled  all the other extensions in the browser
- monitoring the debug of floccus                    - Blocked for a min , nothing happens
- looked into /data/nextcloud.log                     -  The request for the bookmarks is not logged ( so it does not get through) 
- haproxy.log                                                    - gets the request immediately
- looked into /log/apache2/nextcloud-access - gets the request after 1 min.

Do you have any idea how to attack this? It is not the level of logging the problem as I tries to change it back and forth, restarted the haproxy, apache and even teh entire server. The problem is still there.
You know floccus and the bookmarks app and I hope that you can give me some direction how to investigate.

I knew that something that I don't understand will not fix the problem.

Thanks
Bogdan

<!-- gh-comment-id:469534518 --> @bhlevca commented on GitHub (Mar 5, 2019): Bad news, today the sync failed again with exactly the same problem. I made no changes to anything so I don't understand and don't believe in magic. I did tried to find the culprit: ``` - disabled all the other extensions in the browser - monitoring the debug of floccus - Blocked for a min , nothing happens - looked into /data/nextcloud.log - The request for the bookmarks is not logged ( so it does not get through) - haproxy.log - gets the request immediately - looked into /log/apache2/nextcloud-access - gets the request after 1 min. ``` Do you have any idea how to attack this? It is not the level of logging the problem as I tries to change it back and forth, restarted the haproxy, apache and even teh entire server. The problem is still there. You know floccus and the bookmarks app and I hope that you can give me some direction how to investigate. I knew that something that I don't understand will not fix the problem. Thanks Bogdan
Author
Owner

@marcelklehr commented on GitHub (Mar 5, 2019):

Alright, so what do we know? It's a fluctuating error, that probably doesn't depend on the database state. It's most definitely a server-side problem. You've been experiencing it for quite some time, yet you're the only person to have reported it, so there is likely something special about your setup -- whether it's an actual setup issue or just a rare bug, we don't know, though.

This is the controller method that should handle the URL in question. If you're up for it, I think it would be a good idea to litter that method with log statements (for starters mainly here: 1, 2, 3, 4, 5).

You can use the following line as a template:

$this->logger->warn("Your log message here (probably best to use the line number)", ['app' => 'bookmarks']);
<!-- gh-comment-id:469624987 --> @marcelklehr commented on GitHub (Mar 5, 2019): Alright, so what do we know? It's a fluctuating error, that probably doesn't depend on the database state. It's most definitely a server-side problem. You've been experiencing it for quite some time, yet you're the only person to have reported it, so there is likely something special about your setup -- whether it's an actual setup issue or just a rare bug, we don't know, though. [This](https://github.com/nextcloud/bookmarks/blob/master/lib/Controller/Rest/BookmarkController.php#L114) is the controller method that should handle the URL in question. If you're up for it, I think it would be a good idea to litter that method with log statements (for starters mainly here: [1](https://github.com/nextcloud/bookmarks/blob/master/lib/Controller/Rest/BookmarkController.php#L128), [2](https://github.com/nextcloud/bookmarks/blob/master/lib/Controller/Rest/BookmarkController.php#L172), [3](https://github.com/nextcloud/bookmarks/blob/master/lib/Controller/Rest/BookmarkController.php#L173), [4](https://github.com/nextcloud/bookmarks/blob/master/lib/Controller/Rest/BookmarkController.php#L226), [5](https://github.com/nextcloud/bookmarks/blob/master/lib/Controller/Rest/BookmarkController.php#L239)). You can use the following line as a template: ``` $this->logger->warn("Your log message here (probably best to use the line number)", ['app' => 'bookmarks']); ```
Author
Owner

@bhlevca commented on GitHub (Mar 6, 2019):

I assume from the beginning that it is a server side. I will do it closer to the end of the week when I get some time. Thanks for the advice

<!-- gh-comment-id:469968613 --> @bhlevca commented on GitHub (Mar 6, 2019): I assume from the beginning that it is a server side. I will do it closer to the end of the week when I get some time. Thanks for the advice
Author
Owner

@bhlevca commented on GitHub (Mar 7, 2019):

OK I did some research, I added some debug statements (by the way -> warn() did not work, I had to replace it with ->debug() )

the method is processed quickly until it reaches findBookmaks() where it times out afte about 1.5 min and I get the 503 return.

So the question is what am I passing to the findBookmarks function that gets it confused.Now I even have lots of bookmarks on the server.
Please look at the nextcloud.log last below, you can see the 5= w and 6= at 1.5min apart, and immediately below is the modified code.

Please let me know what you think is to investigate next.
Bogdan

                $this->logger->debug("5= before findBookmarks", ['app' => 'bookmarks']);
                $bookmarks = $this->bookmarks->findBookmarks(
                        $user,
                        $offset,
                        $sqlSortColumn,
                        $filterTag,
                        $tagsOnly,
                        $limit,
                        $publicOnly,
                        $attributesToSelect,
                        $conjunction,
                        $untagged,
                        $folder
                );
                $this->logger->debug("6= before return DataResponse", ['app' => 'bookmarks']);

{"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"/index.php/apps/b
ookmarks/public/rest/v2/bookmark?page=-1","message":"Enter getBookmarks","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.36
26.121 Safari/537.36","version":"15.0.5.3"}
{"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"/index.php/apps/b
ookmarks/public/rest/v2/bookmark?page=-1","message":"1= after registerResponder","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome\
/72.0.3626.121 Safari/537.36","version":"15.0.5.3"}
{"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"/index.php/apps/b
ookmarks/public/rest/v2/bookmark?page=-1","message":"2= after get Header Autorization","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) C
hrome/72.0.3626.121 Safari/537.36","version":"15.0.5.3"}
{"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"/index.php/apps/b
ookmarks/public/rest/v2/bookmark?page=-1","message":"3= after userExists","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3
626.121 Safari/537.36","version":"15.0.5.3"}
{"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"/index.php/apps/b
ookmarks/public/rest/v2/bookmark?page=-1","message":"4= after analyzeTagRequest","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome\
/72.0.3626.121 Safari/537.36","version":"15.0.5.3"}
{"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"/index.php/apps/b
ookmarks/public/rest/v2/bookmark?page=-1","message":"5= before findBookmarks","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72
.0.3626.121 Safari/537.36","version":"15.0.5.3"}
{"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:04:53+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"/index.php/apps/b
ookmarks/public/rest/v2/bookmark?page=-1","message":"6= before return DataResponse","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chro
me/72.0.3626.121 Safari/537.36","version":"15.0.5.3"}
`

<!-- gh-comment-id:470357991 --> @bhlevca commented on GitHub (Mar 7, 2019): OK I did some research, I added some debug statements (by the way -> warn() did not work, I had to replace it with ->debug() ) the method is processed quickly until it reaches findBookmaks() where it times out afte about 1.5 min and I get the 503 return. So the question is what am I passing to the findBookmarks function that gets it confused.Now I even have lots of bookmarks on the server. Please look at the nextcloud.log last below, you can see the 5= w and 6= at 1.5min apart, and immediately below is the modified code. Please let me know what you think is to investigate next. Bogdan ``` $this->logger->debug("5= before findBookmarks", ['app' => 'bookmarks']); $bookmarks = $this->bookmarks->findBookmarks( $user, $offset, $sqlSortColumn, $filterTag, $tagsOnly, $limit, $publicOnly, $attributesToSelect, $conjunction, $untagged, $folder ); $this->logger->debug("6= before return DataResponse", ['app' => 'bookmarks']); ``` > {"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"\/index.php\/apps\/b\ > ookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"Enter getBookmarks","userAgent":"Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/72.0.36\ > 26.121 Safari\/537.36","version":"15.0.5.3"} > {"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"\/index.php\/apps\/b\ > ookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"1= after registerResponder","userAgent":"Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\\ > /72.0.3626.121 Safari\/537.36","version":"15.0.5.3"} > {"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"\/index.php\/apps\/b\ > ookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"2= after get Header Autorization","userAgent":"Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) C\ > hrome\/72.0.3626.121 Safari\/537.36","version":"15.0.5.3"} > {"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"\/index.php\/apps\/b\ > ookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"3= after userExists","userAgent":"Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/72.0.3\ > 626.121 Safari\/537.36","version":"15.0.5.3"} > {"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"\/index.php\/apps\/b\ > ookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"4= after analyzeTagRequest","userAgent":"Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\\ > /72.0.3626.121 Safari\/537.36","version":"15.0.5.3"} > {"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:03:26+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"\/index.php\/apps\/b\ > ookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"5= before findBookmarks","userAgent":"Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/72\ > .0.3626.121 Safari\/537.36","version":"15.0.5.3"} > {"reqId":"xMM9mTUxThyajb3WyIHM","level":0,"time":"2019-03-07T02:04:53+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"GET","url":"\/index.php\/apps\/b\ > ookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"6= before return DataResponse","userAgent":"Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chro\ > me\/72.0.3626.121 Safari\/537.36","version":"15.0.5.3"} > `
Author
Owner

@marcelklehr commented on GitHub (Mar 7, 2019):

This is progress! Very nice! :)
So, we know now, that it's not an infinite loop or some kind of black hole that swallows the response, but the database query just takes reaally long -- longer at least than the reverse proxy timeout. To be sure, you may try to time this statement: 1, but there's not much more going on in findBookmarks other than preparing the query and executing it.
So, this would indicate that there's something going on with your MySQL instance. I'm not sure how much help I can be with debugging that, especially with your intricate setup, but I found some options allowing investigation into slow queries. The normal MySQL log might help as well.

<!-- gh-comment-id:470482505 --> @marcelklehr commented on GitHub (Mar 7, 2019): This is progress! Very nice! :) So, we know now, that it's not an infinite loop or some kind of black hole that swallows the response, but the database query just takes *reaally* long -- longer at least than the reverse proxy timeout. To be sure, you may try to time this statement: [1](https://github.com/nextcloud/bookmarks/blob/master/lib/Bookmarks.php#L549), but there's not much more going on in findBookmarks other than preparing the query and executing it. So, this would indicate that there's something going on with your MySQL instance. I'm not sure how much help I can be with debugging that, especially with your intricate setup, but I found some [options allowing investigation into slow queries](https://dev.mysql.com/doc/refman/5.5/en/slow-query-log.html). The normal MySQL log might help as well.
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

Yes slow progress, below more. I identified the problem in the file Bookmarks.php in function findBookmarks there is the code $qb->execute()->fetchAll(); which takes more that 1 min see below timestamps

                $this->logger->debug("** 22=  before fetchAll", ['app' => 'bookmarks']);
                $results = $qb->execute()->fetchAll();
                $this->logger->debug("** 23=  after fetchAll", ['app' => 'bookmarks']);

However after that fetchAll method, getBookmarkParentFolders() gets called in a loop and I see the entries called several times even though the request was abandoned.

I extended all timeout on the proxy to 10 minutes and the timeout still happens at about 1.5 min. Are you sure that is not floccus itself which times-out?

I will try more debugging when I get another chance, but if you have ideas please share them with me.

{"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:18:50+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"
GET","url":"/index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1","message":"** 22= before fetchAll","userAgent":"Mozilla/5.0 (X11;
Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0","version":"15.0.5.3"}
{"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"
GET","url":"/index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1","message":"** 23= after fetchAll","userAgent":"Mozilla/5.0 (X11; L
inux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0","version":"15.0.5.3"}
{"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"
GET","url":"/index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1","message":"** 24= before getBookmarkParentFolders","userAgent":"Moz
illa/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0","version":"15.0.5.3"}
{"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"
GET","url":"/index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1","message":"** 25= after getBookmarkParentFolders","userAgent":"Mozi
lla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0","version":"15.0.5.3"}
{"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"
GET","url":"/index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1","message":"** 24= before getBookmarkParentFolders","userAgent":"Moz
illa/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0","version":"15.0.5.3"}
{"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"
GET","url":"/index.php/apps/bookmarks/public/rest/v2/bookmark?page=-1","message":"** 25= after getBookmarkParentFolders","userAgent":"Mozi
lla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0","version":"15.0.5.3"}

<!-- gh-comment-id:470794097 --> @bhlevca commented on GitHub (Mar 8, 2019): Yes slow progress, below more. I identified the problem in the file Bookmarks.php in function findBookmarks there is the code $qb->execute()->fetchAll(); which takes more that 1 min see below timestamps ``` $this->logger->debug("** 22= before fetchAll", ['app' => 'bookmarks']); $results = $qb->execute()->fetchAll(); $this->logger->debug("** 23= after fetchAll", ['app' => 'bookmarks']); ``` However after that fetchAll method, getBookmarkParentFolders() gets called in a loop and I see the entries called several times even though the request was abandoned. I extended all timeout on the proxy to 10 minutes and the timeout still happens at about 1.5 min. Are you sure that is not floccus itself which times-out? I will try more debugging when I get another chance, but if you have ideas please share them with me. > {"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:18:50+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"\ GET","url":"\/index.php\/apps\/bookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"** 22= before fetchAll","userAgent":"Mozilla\/5.0 (X11; \ Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.5.3"} {"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"\ GET","url":"\/index.php\/apps\/bookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"** 23= after fetchAll","userAgent":"Mozilla\/5.0 (X11; L\ inux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.5.3"} {"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"\ GET","url":"\/index.php\/apps\/bookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"** 24= before getBookmarkParentFolders","userAgent":"Moz\ illa\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.5.3"} {"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"\ GET","url":"\/index.php\/apps\/bookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"** 25= after getBookmarkParentFolders","userAgent":"Mozi\ lla\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.5.3"} {"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"\ GET","url":"\/index.php\/apps\/bookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"** 24= before getBookmarkParentFolders","userAgent":"Moz\ illa\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.5.3"} {"reqId":"T15DHc5XIhZFIX7w2Uao","level":0,"time":"2019-03-08T03:20:13+00:00","remoteAddr":"10.0.0.13","user":"bogdan","app":"bookmarks","method":"\ GET","url":"\/index.php\/apps\/bookmarks\/public\/rest\/v2\/bookmark?page=-1","message":"** 25= after getBookmarkParentFolders","userAgent":"Mozi\ lla\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.5.3"}
Author
Owner

@marcelklehr commented on GitHub (Mar 8, 2019):

Sorry if I have been unclear: Yes, floccus employs a timeout as well. So, symptomatically, the timeouts on the server and client abandon the request before it is finished. However, I don't think increasing the timeouts is an adequate response, because the time the SQL query takes is much too long for my taste.

<!-- gh-comment-id:470901684 --> @marcelklehr commented on GitHub (Mar 8, 2019): Sorry if I have been unclear: Yes, floccus employs a timeout as well. So, symptomatically, the timeouts on the server and client abandon the request before it is finished. However, I don't think increasing the timeouts is an adequate response, because the time the SQL query takes is much too long for my taste.
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

So is there a way to increase the timeout on floccos side? 4000 bookmarks will always take more than a minute to download.

<!-- gh-comment-id:470936082 --> @bhlevca commented on GitHub (Mar 8, 2019): So is there a way to increase the timeout on floccos side? 4000 bookmarks will always take more than a minute to download.
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

Could try to sync only a btanch with a few bookmarks to see if this solves the problem. In case it works but it does not for more bookmarks I am not sure if I can use focus in it's current state. However if it still does not I will have ve to investigate why fetchAllvtakes so long

<!-- gh-comment-id:470937282 --> @bhlevca commented on GitHub (Mar 8, 2019): Could try to sync only a btanch with a few bookmarks to see if this solves the problem. In case it works but it does not for more bookmarks I am not sure if I can use focus in it's current state. However if it still does not I will have ve to investigate why fetchAllvtakes so long
Author
Owner

@marcelklehr commented on GitHub (Mar 8, 2019):

This would be the last option, yes. However, you described that the request took one minute even when the database was still empty!

<!-- gh-comment-id:470953090 --> @marcelklehr commented on GitHub (Mar 8, 2019): This would be the last option, yes. However, you described that the request took one minute even when the database was still empty!
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

Sorry for the typos, I wrote the text from my phone

<!-- gh-comment-id:470955095 --> @bhlevca commented on GitHub (Mar 8, 2019): Sorry for the typos, I wrote the text from my phone
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

OK, I tried with fewer bookmarks and it works fine. I left about 100 bookmarks and it completes in a few seconds even on a raspberry pi server.

I guess in my situation with so many bookmarks I will need an option to control the timeout on the floccus side. Otherwise, I think that everything works ... almost.

It appears to me that after loading the bookmarks on the server and we try sync it tries to get them ALL back in one request and that's the reason for failure. It is possible that a better design on the app side (instead getting them all in one shot to do it in a similar way as it did when it uploaded all the 4000 bookmarks without failing?

Thanks

<!-- gh-comment-id:470993986 --> @bhlevca commented on GitHub (Mar 8, 2019): OK, I tried with fewer bookmarks and it works fine. I left about 100 bookmarks and it completes in a few seconds even on a raspberry pi server. I guess in my situation with so many bookmarks I will need an option to control the timeout on the floccus side. Otherwise, I think that everything works ... almost. It appears to me that after loading the bookmarks on the server and we try sync it tries to get them ALL back in one request and that's the reason for failure. It is possible that a better design on the app side (instead getting them all in one shot to do it in a similar way as it did when it uploaded all the 4000 bookmarks without failing? Thanks
Author
Owner

@marcelklehr commented on GitHub (Mar 8, 2019):

I left about 100 bookmarks and it completes in a few seconds even on a raspberry pi server.

Ok, this is new. But in this case I agree, it makes sense to use pagination.

<!-- gh-comment-id:471031322 --> @marcelklehr commented on GitHub (Mar 8, 2019): > I left about 100 bookmarks and it completes in a few seconds even on a raspberry pi server. Ok, this is new. But in this case I agree, it makes sense to use pagination.
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

I have been monitoring it for a while on 2 browsers Brave and Firefox and both happily sync every 15 minutes.

My conclusion is that with small number of bookmarks it works fine as it is, but with a large number a pagination method as you call it should be implemented.

Also, it would be nice to make timeout and sync period configurable with the defaults and recommendations as you consider.

<!-- gh-comment-id:471071974 --> @bhlevca commented on GitHub (Mar 8, 2019): I have been monitoring it for a while on 2 browsers Brave and Firefox and both happily sync every 15 minutes. My conclusion is that with small number of bookmarks it works fine as it is, **but with a large number a pagination method as you call it should be implemented.** Also, it would be nice to make timeout and sync period configurable with the defaults and recommendations as you consider.
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

On another thought, I did not look inside the packet, but are only the differences sent over on both sides, I mean if there any information exchanged between floccus and the server about the bookmarks on each side?
I believe that is is done because if the sides are identical no further sync should take place.

I have an eversync bookmarks plugin I am wondering how can they sync from scratch 4000 bookmarks in 40 sec. I know they have probably powerful servers but they must use some compression trick to do it that fast.. Also, if there are no changes the sync process takes less then a second. I presume that tehy have some version code somewhere and do not check all the 4000 bookmarks to verify if there was a change.

Anyway these just were a few thoughts that I hope that will help with speeding up the code.

<!-- gh-comment-id:471074952 --> @bhlevca commented on GitHub (Mar 8, 2019): On another thought, I did not look inside the packet, but are only the differences sent over on both sides, I mean if there any information exchanged between floccus and the server about the bookmarks on each side? I believe that is is done because if the sides are identical no further sync should take place. I have an eversync bookmarks plugin I am wondering how can they sync from scratch 4000 bookmarks in 40 sec. I know they have probably powerful servers but they must use some compression trick to do it that fast.. Also, if there are no changes the sync process takes less then a second. I presume that tehy have some version code somewhere and do not check all the 4000 bookmarks to verify if there was a change. Anyway these just were a few thoughts that I hope that will help with speeding up the code.
Author
Owner

@marcelklehr commented on GitHub (Mar 8, 2019):

I have an eversync bookmarks plugin I am wondering how can they sync from scratch 4000 bookmarks in 40 sec.

I think they might be sending multiple bookmarks per request. Floccus, unfortunately, due to REST API constraints, sends one bookmark per request. Floccus has the parallel sync option, though, as mentioned earlier, which might speed up your syncs by sending as much as possible in parallel.

One further possible optimization would be to make the server provide hashes of the folders, so a quick hash comparison is sufficient to determine if something has changed.

<!-- gh-comment-id:471078820 --> @marcelklehr commented on GitHub (Mar 8, 2019): > I have an eversync bookmarks plugin I am wondering how can they sync from scratch 4000 bookmarks in 40 sec. I think they might be sending multiple bookmarks per request. Floccus, unfortunately, due to REST API constraints, sends one bookmark per request. Floccus has the parallel sync option, though, as mentioned earlier, which might speed up your syncs by sending as much as possible in parallel. One further possible optimization would be to make the server provide hashes of the folders, so a quick hash comparison is sufficient to determine if something has changed.
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

One further possible optimization would be to make the server provide hashes of the folders, so a quick hash comparison is sufficient to determine if something has changed.

Yes please do that, this is what was I referring to when I said version ID. This would be the single most worthy speed improvement for large sets of bookmarks

<!-- gh-comment-id:471080902 --> @bhlevca commented on GitHub (Mar 8, 2019): > One further possible optimization would be to make the server provide hashes of the folders, so a quick hash comparison is sufficient to determine if something has changed. Yes please do that, this is what was I referring to when I said version ID. This would be the single most worthy speed improvement for large sets of bookmarks
Author
Owner

@bhlevca commented on GitHub (Mar 8, 2019):

Most likely it should be a hash on each subfolder? Because if one subfolder has changed and the other not you don't want to sync all 5000 bookmarks again. Sounds tricky

;

<!-- gh-comment-id:471095735 --> @bhlevca commented on GitHub (Mar 8, 2019): Most likely it should be a hash on each subfolder? Because if one subfolder has changed and the other not you don't want to sync all 5000 bookmarks again. Sounds tricky ;
Author
Owner

@marcelklehr commented on GitHub (Mar 8, 2019):

It's not that tricky actually, I have this mechanism in place already, only I download every thing and then calculate the hashes. Doing the hashing on the server would avoid the trouble of downloading everything unconditionally. :)

<!-- gh-comment-id:471104415 --> @marcelklehr commented on GitHub (Mar 8, 2019): It's not that tricky actually, I have this mechanism in place already, only I download every thing and then calculate the hashes. Doing the hashing on the server would avoid the trouble of downloading everything unconditionally. :)
Author
Owner

@marcelklehr commented on GitHub (Mar 8, 2019):

If you like you could give the server implementation a go!? :)
https://github.com/nextcloud/bookmarks/issues/669

<!-- gh-comment-id:471104636 --> @marcelklehr commented on GitHub (Mar 8, 2019): If you like you could give the server implementation a go!? :) https://github.com/nextcloud/bookmarks/issues/669
Author
Owner

@bhlevca commented on GitHub (Mar 9, 2019):

If you like you could give the server implementation a go!?

I would but I am not fluent in PHP, if it was Python it would be a different story. Secondly, I am past the age when I burned nights and I have a full time job.

However, I'll be happy to help with testing if it is of any help

<!-- gh-comment-id:471139098 --> @bhlevca commented on GitHub (Mar 9, 2019): > If you like you could give the server implementation a go!? I would but I am not fluent in PHP, if it was Python it would be a different story. Secondly, I am past the age when I burned nights and I have a full time job. However, I'll be happy to help with testing if it is of any help
Author
Owner

@marcelklehr commented on GitHub (Mar 9, 2019):

I see, no worries :)

v3.2.8 now has the fix that should prevent the timeout from kicking in.

<!-- gh-comment-id:471172754 --> @marcelklehr commented on GitHub (Mar 9, 2019): I see, no worries :) v3.2.8 now has the fix that should prevent the timeout from kicking in.
Author
Owner

@bhlevca commented on GitHub (Mar 11, 2019):

Time out is OK now, I don't get the error anymore.

If the speed issue can be resolved, with the hashtags, floccus+nextcloud would be an excellent solution
Even with the parallel processing, the speed is really slow. If for the first time when you sync an empty bookmark folder it is OK, subsequent requests with just a few bookmarks changed should last a minute max as you said you like it.

Anyway, it is getting better by the day :-)

One more thing that I discovered recently, the timeout still happens when the parallel processing is checked.

<!-- gh-comment-id:471372228 --> @bhlevca commented on GitHub (Mar 11, 2019): Time out is OK now, I don't get the error anymore. If the speed issue can be resolved, with the hashtags, floccus+nextcloud would be an excellent solution Even with the parallel processing, the speed is really slow. If for the first time when you sync an empty bookmark folder it is OK, subsequent requests with just a few bookmarks changed should last a minute max as you said you like it. Anyway, it is getting better by the day :-) One more thing that I discovered recently, the timeout still happens when the parallel processing is checked.
Author
Owner

@bhlevca commented on GitHub (Mar 11, 2019):

On the same note of speeding up and control it would be useful that beside Merge, two more options should be available:

  1. Upload - overwrite the data on the server with the local data
  2. Download - erase the local data and download the server data
<!-- gh-comment-id:471594442 --> @bhlevca commented on GitHub (Mar 11, 2019): On the same note of speeding up and control it would be useful that beside Merge, two more options should be available: 1. Upload - overwrite the data on the server with the local data 2. Download - erase the local data and download the server data
Author
Owner

@marcelklehr commented on GitHub (Mar 19, 2019):

it would be useful that beside Merge, two more options should be available

This has already been requested :) See #169

<!-- gh-comment-id:474400987 --> @marcelklehr commented on GitHub (Mar 19, 2019): > it would be useful that beside Merge, two more options should be available This has already been requested :) See #169
Author
Owner

@bernd-wechner commented on GitHub (Oct 1, 2019):

I'm getting heaps of timeouts again. Something is up with my nextcloudpi definitely and I just fixed a few things, and improved response a bit, but still it is just a Raspberry Pi. Still given that's gotta be a common Nextloud backend (NextCloudPi is popular, easy cheap to put into service) so I'd love to see both a way to set a longer timeout, and perhaps more importantly on a timeout see reported even if only in debug log, the exact time that was waited (so the current timeout) and the URL it was waiting along with a timestamp so I can check server logs at that time for clues.

<!-- gh-comment-id:537008442 --> @bernd-wechner commented on GitHub (Oct 1, 2019): I'm getting heaps of timeouts again. Something is up with my nextcloudpi definitely and I just fixed a few things, and improved response a bit, but still it is just a Raspberry Pi. Still given that's gotta be a common Nextloud backend (NextCloudPi is popular, easy cheap to put into service) so I'd love to see both a way to set a longer timeout, and perhaps more importantly on a timeout see reported even if only in debug log, the exact time that was waited (so the current timeout) and the URL it was waiting along with a timestamp so I can check server logs at that time for clues.
Author
Owner

@github-actions[bot] commented on GitHub (Mar 21, 2023):

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

<!-- gh-comment-id:1477332119 --> @github-actions[bot] commented on GitHub (Mar 21, 2023): This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/floccus#231
No description provided.