[PR #24] fix: proxy memory leak, domain pagination, and DomainSelector component #25

New issue

Open

opened 2026-03-02 05:12:35 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 05:12:35 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/gotempsh/temps/pull/24
Author: @dviejokfs
Created: 2/28/2026
Status: 🔄 Open

Base: main ← Head: feat/proxy-log-batch-writer

📝 Commits (8)

1beb0d0 feat(domains): implement paginated domain listing with search functionality
07e9b6e chore: merge remote-tracking branch 'origin/main' into feat/proxy-log-batch-writer
40d0587 feat(domains): enhance domain management with wildcard support and pagination
48c8744 docs: update CHANGELOG with proxy batch writer, domain pagination, and DomainSelector
51b3877 feat: add auth rate limiting, external plugin system, docs overhaul, and settings UI
6304497 chore: merge remote-tracking branch 'origin/main' into feat/proxy-log-batch-writer
7d4c7e2 feat: implement job queue for route table updates and enhance deployment handling
6fc7d14 feat: add Google Indexing API plugin for Temps

📊 Changes

376 files changed (+58540 additions, -5858 deletions)

View changed files

📝 .gitignore (+2 -0)
📝 CHANGELOG.md (+10 -0)
📝 Cargo.lock (+100 -4)
📝 Cargo.toml (+4 -0)
📝 README.md (+18 -18)
📝 crates/temps-auth/src/auth_service.rs (+237 -5)
📝 crates/temps-auth/src/deployment_token_service.rs (+148 -2)
📝 crates/temps-auth/src/handlers.rs (+19 -6)
📝 crates/temps-auth/src/lib.rs (+1 -0)
📝 crates/temps-auth/src/middleware.rs (+5 -7)
➕ crates/temps-auth/src/rate_limit.rs (+235 -0)
📝 crates/temps-auth/src/temps_middleware.rs (+8 -9)
📝 crates/temps-auth/src/user_service.rs (+3 -0)
📝 crates/temps-cli/src/commands/proxy.rs (+6 -0)
📝 crates/temps-cli/src/commands/serve/console.rs (+45 -15)
📝 crates/temps-cli/src/commands/serve/mod.rs (+57 -75)
📝 crates/temps-cli/src/main.rs (+20 -0)
📝 crates/temps-core/src/encryption.rs (+69 -1)
➕ crates/temps-core/src/external_plugin/channel.rs (+260 -0)
📝 crates/temps-core/src/external_plugin/manifest.rs (+139 -0)

...and 80 more files

📄 Description

Summary

Fix proxy memory leak: Replace unbounded tokio::spawn fire-and-forget proxy log writes with a bounded channel (mpsc::channel(8192)) + batch INSERT writer that flushes up to 200 rows every 500ms with backpressure
Add server-side domain pagination: New list_domains_with_total() service method with search support (substring match), paginated ListDomainsResponse with total/page/page_size fields
Create reusable DomainSelector combobox: Searchable domain picker using Command+Popover pattern with server-side search, debounce, status badges, and "showing X of Y" hint
Fix all unpaginated domain queries: Every listDomainsOptions() call site now uses proper pagination or targeted search — no more fetching all domains into memory

Proxy Batch Writer

The LoadBalancer previously called tokio::spawn on every request to fire-and-forget an individual INSERT for proxy logs. Under high traffic this created unbounded task growth and memory pressure.

New approach:

ProxyLogBatchWriter runs on a dedicated OS thread with its own tokio runtime
Receives log entries via bounded mpsc::channel(8192) — producers use .send().await (backpressure) for HTML responses and .try_send() (drop if full) for static/error paths
Batches up to 200 rows per INSERT, flushing every 500ms or when batch is full
Graceful shutdown drains remaining entries on drop
5 unit tests covering batch accumulation, flush timing, channel capacity, and shutdown

Domain Pagination

Backend (temps-domains):

DomainService::list_domains_with_total() — returns (Vec<Model>, u64) with optional substring search on domain name
ListDomainsParams — custom query params struct with page, page_size, search
ListDomainsResponse — now includes total, page, page_size

Frontend (web/):

DomainsManagement.tsx — search bar with debounce, numbered pagination controls, mobile-responsive
DomainSelector.tsx — new reusable combobox component with server-side search, used in DomainForm, AddRoute
DomainForm.tsx — removed domains prop dependency; now self-contained (fetches wildcards internally when editing)
All 9 listDomainsOptions() call sites reviewed and fixed:

File	Fix
`Domains.tsx`	Already paginated (page, page_size, search)
`DomainSelector.tsx`	search + page_size: 50, fetches only when open
`AddRoute.tsx`	`page_size: 1`, checks `total > 0` for existence
`DomainChallengeStep.tsx`	`search: wildcardDomain, page_size: 1`
`useExternalConnectivity.ts`	`search: '*.', page_size: 100`
`GitProviderFlow.tsx`	`search: '*.', page_size: 10`
`ImprovedOnboardingDashboard.tsx`	`page_size: 100`, uses `total` for existence
`AddDomainDialog.tsx`	Removed fetch entirely (DomainSelector handles it)
`EditDomainDialog.tsx`	Removed fetch entirely (DomainForm handles it)

Files Changed (21 files, +1201 / -410)

Rust (proxy)

crates/temps-proxy/src/service/proxy_log_batch_writer.rs — new batch writer + 5 tests
crates/temps-proxy/src/proxy.rs — removed sync log call, removed request_logger field, use batch handle
crates/temps-proxy/src/server.rs — create batch writer, spawn on dedicated thread
crates/temps-proxy/src/proxy_test.rs — updated LoadBalancer::new call sites

Rust (domains)

crates/temps-domains/src/domain_service.rs — list_domains_with_total()
crates/temps-domains/src/handlers/domain_handler.rs — ListDomainsParams, updated handler
crates/temps-domains/src/handlers/types.rs — response fields

Frontend

web/src/components/domains/DomainSelector.tsx — new reusable combobox
web/src/components/domains/DomainsManagement.tsx — search + pagination UI
web/src/pages/Domains.tsx — pagination state management
web/src/pages/AddRoute.tsx — paginated existence check + DomainSelector
web/src/components/project/settings/DomainForm.tsx — self-contained wildcard matching
web/src/components/project/settings/AddDomainDialog.tsx — removed domains fetch
web/src/components/project/settings/EditDomainDialog.tsx — removed domains fetch
web/src/components/onboarding/ImprovedOnboardingDashboard.tsx — paginated query
web/src/components/onboarding/DomainChallengeStep.tsx — targeted search
web/src/components/git-providers/GitProviderFlow.tsx — wildcard search
web/src/hooks/useExternalConnectivity.ts — wildcard search
web/src/api/client/types.gen.ts — updated types

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/gotempsh/temps/pull/24 **Author:** [@dviejokfs](https://github.com/dviejokfs) **Created:** 2/28/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/proxy-log-batch-writer` --- ### 📝 Commits (8) - [`1beb0d0`](https://github.com/gotempsh/temps/commit/1beb0d05de9a2f9e253385069e6521fcaa9fc50b) feat(domains): implement paginated domain listing with search functionality - [`07e9b6e`](https://github.com/gotempsh/temps/commit/07e9b6e7d29644f4a97eba760a685d12a863958e) chore: merge remote-tracking branch 'origin/main' into feat/proxy-log-batch-writer - [`40d0587`](https://github.com/gotempsh/temps/commit/40d0587fc71fe064648de408dd1d3aa7faaec659) feat(domains): enhance domain management with wildcard support and pagination - [`48c8744`](https://github.com/gotempsh/temps/commit/48c874458bd0cb88ba3b48862a4908bcafffab07) docs: update CHANGELOG with proxy batch writer, domain pagination, and DomainSelector - [`51b3877`](https://github.com/gotempsh/temps/commit/51b3877a98c42f24ea5f1f2645c94ff99bde2722) feat: add auth rate limiting, external plugin system, docs overhaul, and settings UI - [`6304497`](https://github.com/gotempsh/temps/commit/6304497e1faea81fe4e09b3eaaea8feeea58829a) chore: merge remote-tracking branch 'origin/main' into feat/proxy-log-batch-writer - [`7d4c7e2`](https://github.com/gotempsh/temps/commit/7d4c7e2e4d25e1de38ff56b698f74bed7d0d7ad6) feat: implement job queue for route table updates and enhance deployment handling - [`6fc7d14`](https://github.com/gotempsh/temps/commit/6fc7d14d570ad78a57a7dfa735a36b7dd7e6fb90) feat: add Google Indexing API plugin for Temps ### 📊 Changes **376 files changed** (+58540 additions, -5858 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+2 -0) 📝 `CHANGELOG.md` (+10 -0) 📝 `Cargo.lock` (+100 -4) 📝 `Cargo.toml` (+4 -0) 📝 `README.md` (+18 -18) 📝 `crates/temps-auth/src/auth_service.rs` (+237 -5) 📝 `crates/temps-auth/src/deployment_token_service.rs` (+148 -2) 📝 `crates/temps-auth/src/handlers.rs` (+19 -6) 📝 `crates/temps-auth/src/lib.rs` (+1 -0) 📝 `crates/temps-auth/src/middleware.rs` (+5 -7) ➕ `crates/temps-auth/src/rate_limit.rs` (+235 -0) 📝 `crates/temps-auth/src/temps_middleware.rs` (+8 -9) 📝 `crates/temps-auth/src/user_service.rs` (+3 -0) 📝 `crates/temps-cli/src/commands/proxy.rs` (+6 -0) 📝 `crates/temps-cli/src/commands/serve/console.rs` (+45 -15) 📝 `crates/temps-cli/src/commands/serve/mod.rs` (+57 -75) 📝 `crates/temps-cli/src/main.rs` (+20 -0) 📝 `crates/temps-core/src/encryption.rs` (+69 -1) ➕ `crates/temps-core/src/external_plugin/channel.rs` (+260 -0) 📝 `crates/temps-core/src/external_plugin/manifest.rs` (+139 -0) _...and 80 more files_ </details> ### 📄 Description ## Summary - **Fix proxy memory leak**: Replace unbounded `tokio::spawn` fire-and-forget proxy log writes with a bounded channel (`mpsc::channel(8192)`) + batch INSERT writer that flushes up to 200 rows every 500ms with backpressure - **Add server-side domain pagination**: New `list_domains_with_total()` service method with search support (substring match), paginated `ListDomainsResponse` with `total`/`page`/`page_size` fields - **Create reusable `DomainSelector` combobox**: Searchable domain picker using Command+Popover pattern with server-side search, debounce, status badges, and "showing X of Y" hint - **Fix all unpaginated domain queries**: Every `listDomainsOptions()` call site now uses proper pagination or targeted search — no more fetching all domains into memory ## Proxy Batch Writer The `LoadBalancer` previously called `tokio::spawn` on every request to fire-and-forget an individual INSERT for proxy logs. Under high traffic this created unbounded task growth and memory pressure. New approach: - `ProxyLogBatchWriter` runs on a dedicated OS thread with its own tokio runtime - Receives log entries via bounded `mpsc::channel(8192)` — producers use `.send().await` (backpressure) for HTML responses and `.try_send()` (drop if full) for static/error paths - Batches up to 200 rows per INSERT, flushing every 500ms or when batch is full - Graceful shutdown drains remaining entries on drop - 5 unit tests covering batch accumulation, flush timing, channel capacity, and shutdown ## Domain Pagination **Backend** (`temps-domains`): - `DomainService::list_domains_with_total()` — returns `(Vec<Model>, u64)` with optional substring search on domain name - `ListDomainsParams` — custom query params struct with `page`, `page_size`, `search` - `ListDomainsResponse` — now includes `total`, `page`, `page_size` **Frontend** (`web/`): - `DomainsManagement.tsx` — search bar with debounce, numbered pagination controls, mobile-responsive - `DomainSelector.tsx` — **new** reusable combobox component with server-side search, used in `DomainForm`, `AddRoute` - `DomainForm.tsx` — removed `domains` prop dependency; now self-contained (fetches wildcards internally when editing) - All 9 `listDomainsOptions()` call sites reviewed and fixed: | File | Fix | |------|-----| | `Domains.tsx` | Already paginated (page, page_size, search) | | `DomainSelector.tsx` | search + page_size: 50, fetches only when open | | `AddRoute.tsx` | `page_size: 1`, checks `total > 0` for existence | | `DomainChallengeStep.tsx` | `search: wildcardDomain, page_size: 1` | | `useExternalConnectivity.ts` | `search: '*.', page_size: 100` | | `GitProviderFlow.tsx` | `search: '*.', page_size: 10` | | `ImprovedOnboardingDashboard.tsx` | `page_size: 100`, uses `total` for existence | | `AddDomainDialog.tsx` | Removed fetch entirely (DomainSelector handles it) | | `EditDomainDialog.tsx` | Removed fetch entirely (DomainForm handles it) | ## Files Changed (21 files, +1201 / -410) ### Rust (proxy) - `crates/temps-proxy/src/service/proxy_log_batch_writer.rs` — **new** batch writer + 5 tests - `crates/temps-proxy/src/proxy.rs` — removed sync log call, removed `request_logger` field, use batch handle - `crates/temps-proxy/src/server.rs` — create batch writer, spawn on dedicated thread - `crates/temps-proxy/src/proxy_test.rs` — updated `LoadBalancer::new` call sites ### Rust (domains) - `crates/temps-domains/src/domain_service.rs` — `list_domains_with_total()` - `crates/temps-domains/src/handlers/domain_handler.rs` — `ListDomainsParams`, updated handler - `crates/temps-domains/src/handlers/types.rs` — response fields ### Frontend - `web/src/components/domains/DomainSelector.tsx` — **new** reusable combobox - `web/src/components/domains/DomainsManagement.tsx` — search + pagination UI - `web/src/pages/Domains.tsx` — pagination state management - `web/src/pages/AddRoute.tsx` — paginated existence check + DomainSelector - `web/src/components/project/settings/DomainForm.tsx` — self-contained wildcard matching - `web/src/components/project/settings/AddDomainDialog.tsx` — removed domains fetch - `web/src/components/project/settings/EditDomainDialog.tsx` — removed domains fetch - `web/src/components/onboarding/ImprovedOnboardingDashboard.tsx` — paginated query - `web/src/components/onboarding/DomainChallengeStep.tsx` — targeted search - `web/src/components/git-providers/GitProviderFlow.tsx` — wildcard search - `web/src/hooks/useExternalConnectivity.ts` — wildcard search - `web/src/api/client/types.gen.ts` — updated types --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>