[PR #2155] [CLOSED] Dynamic uWSGI Configuration Optimization Based on System Resources #3855

New issue

Closed

opened 2026-03-14 07:42:37 +03:00 by kerem · 0 comments

kerem commented

2026-03-14 07:42:37 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/amidaware/tacticalrmm/pull/2155
Author: @Lordkaly
Created: 2/25/2025
Status: ❌ Closed

Base: develop ← Head: Lordkaly-Dynamic-uWSGI-1

📝 Commits (1)

2b7eb7f Dynamic uWSGI Configuration Optimization Based on System Resources

📊 Changes

1 file changed (+55 additions, -34 deletions)

View changed files

📝 api/tacticalrmm/core/management/commands/create_uwsgi_conf.py (+55 -34)

📄 Description

Overview

This pull request enhances the uWSGI configuration generated by tacticalrmm/management/commands/create_uwsgi_conf.py by dynamically adjusting key parameters (workers, threads, cheaper, socket-timeout, and harakiri) based on the system's CPU count and available RAM. The changes improve scalability and performance, particularly under high agent loads, while maintaining compatibility with smaller systems.

Motivation

In a production environment with 10 CPUs and 16 GB of RAM, running Tactical RMM with many active agents (like 2.000), I encountered frequent connect() failed (11: Resource temporarily unavailable) errors from NGINX due to uWSGI socket saturation. The original static configuration struggled to handle the load, leading to dropped connections and degraded performance. Through extensive testing, I identified optimized settings that resolved these issues.

Changes

Resource Detection:
- CPU count retrieved via multiprocessing.cpu_count() (unchanged from original).
- RAM detected by parsing /proc/meminfo for MemTotal, converted to GB using math.ceil.
Dynamic Parameter Calculation:
- max_workers: Scales from 4 to 8 workers per CPU, capped by RAM (100 MB per worker):
  - ≤ 4 GB: Up to 4 per CPU, max 20.
  - 4-8 GB: Up to 6 per CPU, max 40.
  - 8 GB: Up to 8 per CPU, max 80.
- threads: Set to 4 for ≤ 8 CPUs, reduced to 3 for > 8 CPUs to balance CPU usage and concurrency.
- cheaper and cheaper-initial: Set to ~33% of max_workers (minimum 2) for efficient initial scaling.
- max_requests: Adjusted by RAM (500 for ≤ 4 GB, 2000 for 4-8 GB, 5000 for > 8 GB) to optimize worker recycling.
- socket-timeout: Increased to 600 seconds to handle high concurrency.
- harakiri: Raised to 900 seconds to prevent premature worker termination under load.
- Busyness settings: cheaper-step = 2, cheaper-overload = 1, busyness-min = 20, busyness-max = 50 for faster scaling and stability.

Testing

Tested on a server with 10 CPUs and 16 GB of RAM, running Tactical RMM with numerous agents.
Initial static settings resulted in frequent NGINX errors. After iterative optimization (up to 80 workers, 3 threads, adjusted timeouts), the system stabilized with "busyness" levels dropping below 100% and no connection errors.
Fallback values ensure safe operation on systems where resource detection fails.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/amidaware/tacticalrmm/pull/2155 **Author:** [@Lordkaly](https://github.com/Lordkaly) **Created:** 2/25/2025 **Status:** ❌ Closed **Base:** `develop` ← **Head:** `Lordkaly-Dynamic-uWSGI-1` --- ### 📝 Commits (1) - [`2b7eb7f`](https://github.com/amidaware/tacticalrmm/commit/2b7eb7fef5d39a845fdd2d2f78bec7a427703f44) Dynamic uWSGI Configuration Optimization Based on System Resources ### 📊 Changes **1 file changed** (+55 additions, -34 deletions) <details> <summary>View changed files</summary> 📝 `api/tacticalrmm/core/management/commands/create_uwsgi_conf.py` (+55 -34) </details> ### 📄 Description ### Overview This pull request enhances the uWSGI configuration generated by `tacticalrmm/management/commands/create_uwsgi_conf.py` by dynamically adjusting key parameters (`workers`, `threads`, `cheaper`, `socket-timeout`, and `harakiri`) based on the system's CPU count and available RAM. The changes improve scalability and performance, particularly under high agent loads, while maintaining compatibility with smaller systems. ### Motivation In a production environment with 10 CPUs and 16 GB of RAM, running Tactical RMM with many active agents (like 2.000), I encountered frequent `connect() failed (11: Resource temporarily unavailable)` errors from NGINX due to uWSGI socket saturation. The original static configuration struggled to handle the load, leading to dropped connections and degraded performance. Through extensive testing, I identified optimized settings that resolved these issues. ### Changes - **Resource Detection**: - CPU count retrieved via `multiprocessing.cpu_count()` (unchanged from original). - RAM detected by parsing `/proc/meminfo` for `MemTotal`, converted to GB using `math.ceil`. - **Dynamic Parameter Calculation**: - `max_workers`: Scales from 4 to 8 workers per CPU, capped by RAM (100 MB per worker): - ≤ 4 GB: Up to 4 per CPU, max 20. - 4-8 GB: Up to 6 per CPU, max 40. - > 8 GB: Up to 8 per CPU, max 80. - `threads`: Set to 4 for ≤ 8 CPUs, reduced to 3 for > 8 CPUs to balance CPU usage and concurrency. - `cheaper` and `cheaper-initial`: Set to ~33% of `max_workers` (minimum 2) for efficient initial scaling. - `max_requests`: Adjusted by RAM (500 for ≤ 4 GB, 2000 for 4-8 GB, 5000 for > 8 GB) to optimize worker recycling. - `socket-timeout`: Increased to 600 seconds to handle high concurrency. - `harakiri`: Raised to 900 seconds to prevent premature worker termination under load. - Busyness settings: `cheaper-step = 2`, `cheaper-overload = 1`, `busyness-min = 20`, `busyness-max = 50` for faster scaling and stability. ### Testing - Tested on a server with 10 CPUs and 16 GB of RAM, running Tactical RMM with numerous agents. - Initial static settings resulted in frequent NGINX errors. After iterative optimization (up to 80 workers, 3 threads, adjusted timeouts), the system stabilized with "busyness" levels dropping below 100% and no connection errors. - Fallback values ensure safe operation on systems where resource detection fails. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>