[GH-ISSUE #101] Synchronous requests connection pool warnings (Home Assistant) #52

Closed
opened 2026-02-27 15:46:11 +03:00 by kerem · 1 comment
Owner

Originally created by @Corbeno on GitHub (Jun 13, 2022).
Original GitHub issue: https://github.com/proxmoxer/proxmoxer/issues/101

Originally assigned to: @jhollowe on GitHub.

If more then 10 vms/containers are polled concurrently, a warning occurs. This doesn't normally happen, but the current way the Home Assistant integration works is by queueing up multiple requests all at the same time.

Here is an example of the logs if a single node has 11+ vms/containers added to it in Home Assistant.

2022-06-13 02:11:39 WARNING (SyncWorker_4) [urllib3.connectionpool] Connection pool is full, discarding connection: 192.168.0.49. Connection pool size: 10
2022-06-13 02:12:39 WARNING (SyncWorker_5) [urllib3.connectionpool] Connection pool is full, discarding connection: 192.168.0.49. Connection pool size: 10
2022-06-13 02:13:39 WARNING (SyncWorker_8) [urllib3.connectionpool] Connection pool is full, discarding connection: 192.168.0.49. Connection pool size: 10

This has been reported here: https://github.com/home-assistant/core/issues/68487

According to this stack overflow post, no data is actually being lost though, so I think the warning can be disabled without any worry.

Another solution would be to increase the pool size, but this issue will just come up again when somebody with even more vms/containers breaks the limit again.

I'm not sure how to go about either solutions for this project though. Is this something that can be fixed in this library?

Originally created by @Corbeno on GitHub (Jun 13, 2022). Original GitHub issue: https://github.com/proxmoxer/proxmoxer/issues/101 Originally assigned to: @jhollowe on GitHub. If more then 10 vms/containers are polled concurrently, a warning occurs. This doesn't normally happen, but the current way the Home Assistant integration works is by queueing up multiple requests all at the same time. Here is an example of the logs if a single node has 11+ vms/containers added to it in Home Assistant. ```2022-06-13 02:10:39 WARNING (SyncWorker_4) [urllib3.connectionpool] Connection pool is full, discarding connection: 192.168.0.49. Connection pool size: 10 2022-06-13 02:11:39 WARNING (SyncWorker_4) [urllib3.connectionpool] Connection pool is full, discarding connection: 192.168.0.49. Connection pool size: 10 2022-06-13 02:12:39 WARNING (SyncWorker_5) [urllib3.connectionpool] Connection pool is full, discarding connection: 192.168.0.49. Connection pool size: 10 2022-06-13 02:13:39 WARNING (SyncWorker_8) [urllib3.connectionpool] Connection pool is full, discarding connection: 192.168.0.49. Connection pool size: 10 ``` This has been reported here: https://github.com/home-assistant/core/issues/68487 According to [this](https://stackoverflow.com/questions/53765366/urllib3-connectionpool-connection-pool-is-full-discarding-connection) stack overflow post, no data is actually being lost though, so I think the warning can be disabled without any worry. Another solution would be to increase the pool size, but this issue will just come up again when somebody with even more vms/containers breaks the limit again. I'm not sure how to go about either solutions for this project though. Is this something that can be fixed in this library?
Author
Owner

@jhollowe commented on GitHub (Jun 16, 2022):

The connection pool is something provided by urllib3 that reuses the TCP (and TLS) connection for subsequent requests. Like you said, this error appears only after a request and response has been made successfully (at least successful at the TCP level). It just prevents you from having to go to all the work of tearing down the connection just to create it again moments later.

I would suggest just disabling the error message (I assume it is using the logging library which makes this easy). You could also limit the number of workers to the size of the connection pool so that there are only 10 connection requests at a time and thus connections are never dropped and the polling is the most efficient use of the connections.

This is something outside the scope of proxmoxer so I'm going to close this issue. Feel free to still ask any questions you still have.

<!-- gh-comment-id:1158155173 --> @jhollowe commented on GitHub (Jun 16, 2022): The connection pool is something provided by urllib3 that reuses the TCP (and TLS) connection for subsequent requests. Like you said, this error appears only after a request and response has been made successfully (at least successful at the TCP level). It just prevents you from having to go to all the work of tearing down the connection just to create it again moments later. I would suggest just disabling the error message (I assume it is using the `logging` library which makes this easy). You could also limit the number of workers to the size of the connection pool so that there are only 10 connection requests at a time and thus connections are never dropped and the polling is the most efficient use of the connections. This is something outside the scope of proxmoxer so I'm going to close this issue. Feel free to still ask any questions you still have.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/proxmoxer#52
No description provided.