[GH-ISSUE #544] Set disableClusterDiscovery=true when using rqlite and Kubernetes #349

Closed
opened 2026-03-15 14:02:33 +03:00 by kerem · 7 comments
Owner

Originally created by @otoolep on GitHub (Jul 29, 2025).
Original GitHub issue: https://github.com/axllent/mailpit/issues/544

As per https://github.com/rqlite/gorqlite it's recommended to disable set this flag when talking to Kubernetes-based rqlite. Why? Because not doing so will mean that gorqlite will attempt to circumvent the load-balancing done by Kubernetes. This is because gorqlite was not built with Kubernetes-based deployments in mind.

I expect that users who have Mailpit talking to Kubernetes-based rqlite deployments may encounter connection issues.

@axllent -- I see you are using the stdlib version of gorqlite. How would you expect to configure it, so that this flag is set when necessary? There are a few issues here that I see:

  • the flag disableClusterDiscovery=true is not set if needed by Mailpit
  • Mailpit would need to way to know to set this flag. Since Mailpit is (presumably) unaware of Kubernetes it would need users in turn to pass a flag to Mailpit

WDYT?

(I'm going to look into a way for gorqlite to handle this situation automatically, BTW, so I might have solution -- this is just about the discussion right now)

Originally created by @otoolep on GitHub (Jul 29, 2025). Original GitHub issue: https://github.com/axllent/mailpit/issues/544 As per https://github.com/rqlite/gorqlite it's recommended to disable set this flag when talking to Kubernetes-based rqlite. Why? Because not doing so will mean that gorqlite will attempt to circumvent the load-balancing done by Kubernetes. This is because gorqlite was not built with Kubernetes-based deployments in mind. I expect that users who have Mailpit talking to Kubernetes-based rqlite deployments may encounter connection issues. @axllent -- I see you are using the stdlib version of gorqlite. How would you expect to configure it, so that this flag is set when necessary? There are a few issues here that I see: - the flag disableClusterDiscovery=true is not set if needed by Mailpit - Mailpit would need to way to *know* to set this flag. Since Mailpit is (presumably) unaware of Kubernetes it would need users in turn to pass a flag to Mailpit WDYT? (I'm going to look into a way for gorqlite to handle this situation automatically, BTW, so I might have solution -- this is just about the discussion right now)
kerem closed this issue 2026-03-15 14:02:38 +03:00
Author
Owner

@otoolep commented on GitHub (Jul 29, 2025):

The key section from the gorqlite readme:

  • Depending on your deployment, you may wish to disable Cluster Discovery. If you do disable it only the provided URL will be used to communicate with the API instead of discovering the leader and peers and retrying failed requests with different peers. To disable Cluster Discovery add disableClusterDiscovery=true as a URL Query Parameter when connecting to rqlite e.g. http://localhost:14001?disableClusterDiscovery=true.
    • This is helpful, for example, when using a Kubernetes service to handle the load balancing of the requests across healthy nodes. In such a setup Kubernetes takes care of finding a node to communicate with.
<!-- gh-comment-id:3132677002 --> @otoolep commented on GitHub (Jul 29, 2025): The key section from the gorqlite readme: * Depending on your deployment, **you may wish to disable _Cluster Discovery_**. If you do disable it only the provided URL will be used to communicate with the API instead of discovering the leader and peers and retrying failed requests with different peers. To disable _Cluster Discovery_ add `disableClusterDiscovery=true` as a URL Query Parameter when connecting to rqlite e.g. `http://localhost:14001?disableClusterDiscovery=true`. * This is helpful, for example, when using a Kubernetes service to handle the load balancing of the requests across healthy nodes. In such a setup Kubernetes takes care of finding a node to communicate with.
Author
Owner

@otoolep commented on GitHub (Jul 29, 2025):

https://github.com/axllent/mailpit/issues/543 is what triggered this issue.

<!-- gh-comment-id:3132678116 --> @otoolep commented on GitHub (Jul 29, 2025): https://github.com/axllent/mailpit/issues/543 is what triggered this issue.
Author
Owner

@otoolep commented on GitHub (Jul 29, 2025):

(BTW I never liked the way gorqlite tried to find other nodes in the cluster. It was trying to help, but it ended up confusing things. But that behaviour is now relied on, most likely, so I can't change its default)

<!-- gh-comment-id:3132711305 --> @otoolep commented on GitHub (Jul 29, 2025): (BTW I never liked the way gorqlite tried to find other nodes in the cluster. It was trying to help, but it ended up confusing things. But that behaviour is now relied on, most likely, so I can't change its default)
Author
Owner

@otoolep commented on GitHub (Jul 29, 2025):

At this point if we just update the Mailpit docs we're probably covered until I come up with a better solution.

<!-- gh-comment-id:3133064941 --> @otoolep commented on GitHub (Jul 29, 2025): At this point if we just update the Mailpit docs we're probably covered until I come up with a better solution.
Author
Owner

@axllent commented on GitHub (Jul 29, 2025):

Hey @otoolep - thanks for the info, appreciated. Yeah, the way to configure it would be via the --database flag/environment, and I don't think there is (or should be) any way I try detect that automatically and set it from within Mailpit, as a kubernetes deployment can be set up multiple ways such as number of nodes, auto-scaling etc. I'm not even sure an app could auto-detect if it's being run on k8s (which would be the other potential way). I think just a note in the docs should be sufficient (thank you for the PR btw), as anyone wanting to deploy to such an environment would likely (hopefully) have done some research beforehand and understand what they are doing, or at least investigate it further if it goes wrong.

I'll hopefully look into your PR later today (I'm away for work at the moment but will be back tonight [NZ time]).

<!-- gh-comment-id:3133855884 --> @axllent commented on GitHub (Jul 29, 2025): Hey @otoolep - thanks for the info, appreciated. Yeah, the way to configure it would be via the `--database` flag/environment, and I don't think there is (or should be) any way I try detect that automatically and set it from within Mailpit, as a kubernetes deployment can be set up multiple ways such as number of nodes, auto-scaling etc. I'm not even sure an app could auto-detect if it's being run on k8s (which would be the other potential way). I think just a note in the docs should be sufficient (thank you for the PR btw), as anyone wanting to deploy to such an environment would likely (hopefully) have done some research beforehand and understand what they are doing, or at least investigate it further if it goes wrong. I'll hopefully look into your PR later today (I'm away for work at the moment but will be back tonight [NZ time]).
Author
Owner

@otoolep commented on GitHub (Jul 29, 2025):

Yeah, I would merge the docs PR now and leave it at that. Don't worry about any code changes to Mailpit. I'm thinking about ways to have gorqlite to just "do the right thing" automatically.

<!-- gh-comment-id:3133866877 --> @otoolep commented on GitHub (Jul 29, 2025): Yeah, I would merge the docs PR now and leave it at that. Don't worry about any code changes to Mailpit. I'm thinking about ways to have gorqlite to just "do the right thing" automatically.
Author
Owner

@axllent commented on GitHub (Jul 30, 2025):

Great, and thanks. I've merged in that PR and deployed to the website. I agree, there are a couple of things that could be improved in gorqlite, but (as far as I'm aware), it is for the most part a pretty robust integration with Mailpit which is being used by a (growing?) number of users.

Anyway, nice to see you're keeping tabs on rqlite topics, and always good to get your input in times like this 👍

<!-- gh-comment-id:3134803390 --> @axllent commented on GitHub (Jul 30, 2025): Great, and thanks. I've merged in that PR and deployed to the website. I agree, there are a couple of things that could be improved in gorqlite, but (as far as I'm aware), it is for the most part a pretty robust integration with Mailpit which is being used by a (growing?) number of users. Anyway, nice to see you're keeping tabs on rqlite topics, and always good to get your input in times like this 👍
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/mailpit#349
No description provided.