starred/cypht

Fork 0

mirror of https://github.com/cypht-org/cypht.git synced 2026-04-25 13:05:53 +03:00

[GH-ISSUE #1402] 🐛 [Bug] Gmail messages not shown #648

New issue

Closed

opened 2026-02-25 21:35:35 +03:00 by kerem · 12 comments

kerem commented

2026-02-25 21:35:35 +03:00

Owner

Originally created by @indridieinarsson on GitHub (Dec 9, 2024).
Original GitHub issue: https://github.com/cypht-org/cypht/issues/1402

Originally assigned to: @christer77 on GitHub.

🐛 Bug

I have an install using sqlite database, redis and memcached enabled, running as a docker image.

I have 3 different accounts, one of them gmail. The other two accounts (on two separate and different servers) work as expected, both in single-mailbox mode (under the Email tab) and in the combined views.

Gmail, on the other hand has problems. After restarting cypht, it works, showing me mails in single-mode and in the combined views.
After just a few navigations, cypht stops showing any messages from my gmail account.
That goes for both viewing the gmail INBOX directly (which shows no messages, although there are thousands) and also the combined views, which show no message from the gmail account, only from the other two.

It may be of importance here, that my gmail inbox is pretty huge, containing my entire email history for years.

Version & Environment

Rev: 412b5adbd9
OS: Linux x86 / docker

Originally created by @indridieinarsson on GitHub (Dec 9, 2024). Original GitHub issue: https://github.com/cypht-org/cypht/issues/1402 Originally assigned to: @christer77 on GitHub. ## 🐛 Bug I have an install using sqlite database, redis and memcached enabled, running as a docker image. I have 3 different accounts, one of them gmail. The other two accounts (on two separate and different servers) work as expected, both in single-mailbox mode (under the Email tab) and in the combined views. Gmail, on the other hand has problems. After restarting cypht, it works, showing me mails in single-mode and in the combined views. After just a few navigations, cypht stops showing any messages from my gmail account. That goes for both viewing the gmail INBOX directly (which shows no messages, although there are thousands) and also the combined views, which show no message from the gmail account, only from the other two. It may be of importance here, that my gmail inbox is pretty huge, containing my entire email history for years. ### Version & Environment Rev: 412b5adbd9ada OS: Linux x86 / docker

kerem

2026-02-25 21:35:35 +03:00

closed this issue
added the
bug
label

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@indridieinarsson commented on GitHub (Dec 10, 2024):

I analysed this a bit. Cypth (modules/imap/hm-imap-base.php) is successfully communicating with gmail. Getting settings, and the list of messages. All of it, the first timestamp is 2006. My guess is that it then goes on to fetch a whole lot of messages and something times out, but I haven't confirmed that yet. My feeling is that gmail is not answering imap requests very fast (throttling them even).

Now, perhaps, I should tidy up my inbox, but then again, I'm hardly the only one using the search function instead of organized folders...
It's also worth noting that (at least a couple of weeks ago) everything worked ok on a mailbox which is a mirror of my gmail box. But that one is faster.

@indridieinarsson commented on GitHub (Dec 10, 2024): I analysed this a bit. Cypth (modules/imap/hm-imap-base.php) is successfully communicating with gmail. Getting settings, and the list of messages. All of it, the first timestamp is 2006. My guess is that it then goes on to fetch a whole lot of messages and something times out, but I haven't confirmed that yet. My feeling is that gmail is not answering imap requests very fast (throttling them even). Now, perhaps, I should tidy up my inbox, but then again, I'm hardly the only one using the search function instead of organized folders... It's also worth noting that (at least a couple of weeks ago) everything worked ok on a mailbox which is a mirror of my gmail box. But that one is faster.

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@indridieinarsson commented on GitHub (Dec 11, 2024):

I looked a bit into this. I see the following imap command being sent to gmail imap server :

A8 UID FETCH 53418,53419,53420,53421,53422,53423,53424,53425,53429,53432,53436,53437,53438,53439,53440,53441,53442,5 3443,53444,53445 (FLAGS INTERNALDATE RFC822.SIZE X-GM-MSGID X-GM-THRID X-GM-LABELS BODY.PEEK[HEADER.FIELDS (SUBJECT X-AUTO-BCC FROM DATE CONTENT-TYPE X-PRIORITY TO LIST-ARCHIVE REFERENCES ME SSAGE-ID X-SNOOZED)] BODY.PEEK[0.1])

which then yields the answer :
A8 BAD Could not parse command

@indridieinarsson commented on GitHub (Dec 11, 2024): I looked a bit into this. I see the following imap command being sent to gmail imap server : `A8 UID FETCH 53418,53419,53420,53421,53422,53423,53424,53425,53429,53432,53436,53437,53438,53439,53440,53441,53442,5 3443,53444,53445 (FLAGS INTERNALDATE RFC822.SIZE X-GM-MSGID X-GM-THRID X-GM-LABELS BODY.PEEK[HEADER.FIELDS (SUBJECT X-AUTO-BCC FROM DATE CONTENT-TYPE X-PRIORITY TO LIST-ARCHIVE REFERENCES ME SSAGE-ID X-SNOOZED)] BODY.PEEK[0.1])` which then yields the answer : `A8 BAD Could not parse command`

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@indridieinarsson commented on GitHub (Dec 11, 2024):

Ok, it's BODY.PEEK[0.1] that's causing this. Apparently not supported in this form by gmail. If I change it to BODY.PEEK[1], then it works.
And also if I disable the preview mail option (which controls the BODY.PEEK imap option), then i get gmail messages again.

@indridieinarsson commented on GitHub (Dec 11, 2024): Ok, it's `BODY.PEEK[0.1]` that's causing this. Apparently not supported in this form by gmail. If I change it to `BODY.PEEK[1]`, then it works. And also if I disable the preview mail option (which controls the `BODY.PEEK` imap option), then i get gmail messages again.

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@indridieinarsson commented on GitHub (Dec 17, 2024):

Follow up : after disabling preview mail option (and also locally changing BODY.PEEK[0.1] to BODY.PEEK[1]), things only sort-of work. If I nuke local cache in the browser, I get gmail message in the "Everything" view.
However, after a while, when I refresh the view, the gmail messages are no longer there (but the others are). The dedicated gmail Inbox still has all the gmail messages (but the "Everything" view is sort of the best feature of cypht...)

@indridieinarsson commented on GitHub (Dec 17, 2024): Follow up : after disabling preview mail option (and also locally changing BODY.PEEK[0.1] to BODY.PEEK[1]), things only sort-of work. If I nuke local cache in the browser, I get gmail message in the "Everything" view. However, after a while, when I refresh the view, the gmail messages are no longer there (but the others are). The dedicated gmail Inbox still has all the gmail messages (but the "Everything" view is sort of the best feature of cypht...)

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@christer77 commented on GitHub (Dec 19, 2024):

Hello @indridieinarsson,

thank you for raising the issue,
thank you also for the proposed solution.
We get to it and get back to you early.

@christer77 commented on GitHub (Dec 19, 2024): Hello @indridieinarsson, thank you for raising the issue, thank you also for the proposed solution. We get to it and get back to you early.

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@indridieinarsson commented on GitHub (Dec 19, 2024):

Hello @indridieinarsson,

thank you for raising the issue, thank you also for the proposed solution. We get to it and get back to you early.

Hi,

thanks for looking into it. And my "solution" isn't really a solution yet, as it goes only from "never works" to "works for an hour then no more".

@indridieinarsson commented on GitHub (Dec 19, 2024): > Hello @indridieinarsson, > > thank you for raising the issue, thank you also for the proposed solution. We get to it and get back to you early. Hi, thanks for looking into it. And my "solution" isn't really a solution yet, as it goes only from "never works" to "works for an hour then no more".

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@indridieinarsson commented on GitHub (Dec 19, 2024):

Now, I'm tracking this a bit. Seems that the xoauth2 logon is unsuccessful. In hm-imap.php, I am landing here:

      if (mb_stristr($response, 'A' . $this->command_count . ' OK')) {
                $authed = true;
                $this->state = 'authenticated';
            } elseif (mb_strtolower($this->auth) == 'xoauth2' && preg_match("/^\+ ([a-zA-Z0-9=]+)$/", $response, $matches)) {
                    $jsonString = base64_decode(mb_substr($response,2));
                    Hm_Functions::error_log(sprintf(' ' . $this->server . ' got response ' . $jsonString));
                $this->send_command("\r\n", true);
                $this->get_response();
            }

the base64 encoded json string in the reply says : {"status":"400","schemes":"Bearer","scope":"https://mail.google.com/"}
which is a bit weird, since I'm successfully logged into gmail imap server if I go to the dedicated gmail inbox. Also "Everything" view successfully logged in an hour earlier.

@indridieinarsson commented on GitHub (Dec 19, 2024): Now, I'm tracking this a bit. Seems that the xoauth2 logon is unsuccessful. In hm-imap.php, I am landing here: > if (mb_stristr($response, 'A' . $this->command_count . ' OK')) { > $authed = true; > $this->state = 'authenticated'; > } elseif (mb_strtolower($this->auth) == 'xoauth2' && preg_match("/^\+ ([a-zA-Z0-9=]+)$/", $response, $matches)) { > $jsonString = base64_decode(mb_substr($response,2)); > Hm_Functions::error_log(sprintf(' ' . $this->server . ' got response ' . $jsonString)); > $this->send_command("\r\n", true); > $this->get_response(); > } the base64 encoded json string in the reply says : `{"status":"400","schemes":"Bearer","scope":"https://mail.google.com/"}` which is a bit weird, since I'm successfully logged into gmail imap server if I go to the dedicated gmail inbox. Also "Everything" view successfully logged in an hour earlier.

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@indridieinarsson commented on GitHub (Dec 19, 2024):

So, it appears that this is caused by the oauth2 bearer token expiring.
When I'm in the "gmail dedicated" inbox, there is a function imap_refresh_oauth2_token which is regularly being called to check validity of the oauth2 token, and refresh it if needed.
When in the "Everything" view, this function never gets called, so the token expires (after an hour) and subsequent login attempts are refused.
If I navigate to the gmail-inbox, the token is refreshed, and I can enjoy one more hour of "Everything" view with everything actually in it, but then it expires again.

@indridieinarsson commented on GitHub (Dec 19, 2024): So, it appears that this is caused by the oauth2 bearer token expiring. When I'm in the "gmail dedicated" inbox, there is a function `imap_refresh_oauth2_token` which is regularly being called to check validity of the oauth2 token, and refresh it if needed. When in the "Everything" view, this function never gets called, so the token expires (after an hour) and subsequent login attempts are refused. If I navigate to the gmail-inbox, the token is refreshed, and I can enjoy one more hour of "Everything" view with everything actually in it, but then it expires again.

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@christer77 commented on GitHub (Dec 20, 2024):

Hello @indridieinarsson,
thank you for raising the issue, thank you also for the proposed solution. We get to it and get back to you early.

Hi,

thanks for looking into it. And my "solution" isn't really a solution yet, as it goes only from "never works" to "works for an hour then no more".

seen somewhere

@christer77 commented on GitHub (Dec 20, 2024): > > Hello @indridieinarsson, > > thank you for raising the issue, thank you also for the proposed solution. We get to it and get back to you early. > > Hi, > > thanks for looking into it. And my "solution" isn't really a solution yet, as it goes only from "never works" to "works for an hour then no more". [seen somewhere](https://github.com/curl/curl/discussions/10076#discussioncomment-4557455)

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@christer77 commented on GitHub (Dec 20, 2024):

Bonjour,
Merci d’avoir soulevé la question, merci aussi pour la solution proposée. Nous nous y rendons et vous revenons plus tôt.

Salut

Merci de vous y être penché. Et ma « solution » n’en est pas encore vraiment une, car elle va seulement de « ne fonctionne jamais » à « fonctionne pendant une heure puis plus maintenant ».

RFC

@christer77 commented on GitHub (Dec 20, 2024): > > Bonjour, > > Merci d’avoir soulevé la question, merci aussi pour la solution proposée. Nous nous y rendons et vous revenons plus tôt. > > Salut > > Merci de vous y être penché. Et ma « solution » n’en est pas encore vraiment une, car elle va seulement de « ne fonctionne jamais » à « fonctionne pendant une heure puis plus maintenant ». [RFC](https://www.rfc-editor.org/rfc/rfc3501#page-57)

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner

@christer77 commented on GitHub (Mar 20, 2025):

HI @indridieinarsson ,

can you mark this issue as resolved please?

@christer77 commented on GitHub (Mar 20, 2025): HI @indridieinarsson , can you mark this issue as resolved please?

kerem commented

2026-02-25 21:35:35 +03:00

Author

Owner