starred/ALwrity

Fork 0

mirror of https://github.com/AJaySi/ALwrity.git synced 2026-04-25 08:55:58 +03:00

[GH-ISSUE #208] Implement SSE starlette & deprecate polling mechanism #484

New issue

Open

opened 2026-03-13 20:36:22 +03:00 by kerem · 0 comments

kerem commented

2026-03-13 20:36:22 +03:00

Owner

Originally created by @AJaySi on GitHub (Aug 18, 2025).
Original GitHub issue: https://github.com/AJaySi/ALwrity/issues/208

Originally assigned to: @AJaySi on GitHub.

SSE Migration Strategy & Implementation Plan

🚨 Current Implementation Problems

Backend Issues

Complex SSE Manager: The SSEAIServiceManager with lambda functions is overly complex
Async Generator Problems: The sse_yield function using __anext__() is fragile
Message Format Inconsistency: Backend sends different message formats that frontend struggles to parse
Tight Coupling: AI service manager is tightly coupled to SSE implementation
Error Propagation: Errors in one component cascade to others
Debugging Difficulty: Complex async flows make debugging hard

Frontend Issues

EventSource Limitations: No built-in reconnection, poor error handling
Message Parsing Complexity: Too many message types to handle
Timeout Handling: Frontend timeouts don't align with backend processing
Connection State Management: Poor handling of connection states
Progress Tracking: Inconsistent progress calculation and display

Architecture Problems

Tight Coupling: Frontend and backend are tightly coupled to specific message formats
No Reusability: SSE implementation is specific to strategy generation
Error Handling: Inconsistent error handling across components
Testing Difficulty: Complex async flows make testing challenging

🎯 Proposed Solution: Clean SSE with sse-starlette

Phase 1: MVP Polling Solution (1-2 hours)

Goal: Get strategy generation working immediately with simple polling

Implementation:

Replace complex SSE with simple polling mechanism
Poll strategy status every 10 seconds
Show progress modal with educational content
Handle timeouts gracefully
Remove all SSE-related complexity

Benefits:

✅ Immediate working solution
✅ Simple to implement and debug
✅ Reliable and predictable
✅ Easy to test

Phase 2: Proper SSE Implementation (1-2 days)

Goal: Implement clean, reusable SSE infrastructure

Implementation:

Use sse-starlette for backend SSE
Create reusable SSE client for frontend
Standardize message format
Add proper error handling and reconnection
Make SSE infrastructure reusable for other features

Benefits:

✅ Real-time updates
✅ Better user experience
✅ Reusable infrastructure
✅ Proper error handling

🏗️ Technical Architecture

Backend: sse-starlette Implementation

Core SSE Module (`backend/services/sse/`)

backend/services/sse/
├── __init__.py
├── sse_manager.py          # Core SSE management
├── message_formatter.py    # Standardized message formatting
├── connection_manager.py   # Connection lifecycle management
├── error_handler.py        # SSE error handling
└── types.py               # SSE message types and schemas

SSE Manager Features

Connection Management: Handle multiple SSE connections
Message Broadcasting: Send messages to specific clients
Error Handling: Graceful error handling and recovery
Message Formatting: Consistent message format across all features
Connection Monitoring: Track connection health and status

Message Format Standardization

# Standard SSE message format
{
    "event": "progress|complete|error|educational",
    "data": {
        "step": 1,
        "progress": 10,
        "message": "Processing...",
        "educational_content": {...},
        "timestamp": "2024-01-01T00:00:00Z"
    }
}

Frontend: Reusable SSE Client

Core SSE Module (`frontend/src/services/sse/`)

frontend/src/services/sse/
├── index.ts
├── SSEConnection.ts        # Core SSE connection management
├── SSEEventManager.ts      # Event handling and message parsing
├── SSEReconnection.ts      # Automatic reconnection logic
├── SSEMessageTypes.ts      # TypeScript types for messages
└── SSEUtils.ts            # Utility functions

SSE Client Features

Automatic Reconnection: Handle connection drops gracefully
Message Parsing: Parse standardized message format
Event Handling: Handle different event types
Error Recovery: Recover from errors automatically
Connection Monitoring: Monitor connection health

React Hook (`frontend/src/hooks/useSSE.ts`)

const useSSE = (url: string, options?: SSEOptions) => {
  // Returns: { data, error, isConnected, reconnect }
}

📋 Implementation Phases

Phase 1: MVP Polling (Immediate - 1-2 hours)

Backend Changes

Remove SSE complexity from ai_generation_endpoints.py
Simplify AI generation to return immediately after starting
Add status endpoint to check generation progress
Remove SSEAIServiceManager and related complexity

Frontend Changes

Replace SSE with polling in ContentStrategyBuilder.tsx
Implement simple progress modal with educational content
Add polling mechanism (every 10 seconds)
Handle timeouts gracefully (5-minute timeout)
Remove all SSE-related code

Files to Modify

backend/api/content_planning/api/content_strategy/endpoints/ai_generation_endpoints.py
frontend/src/components/ContentPlanningDashboard/components/ContentStrategyBuilder.tsx
frontend/src/services/contentPlanningApi.ts

Phase 2: Clean SSE Infrastructure (1-2 days)

Backend Implementation

Create SSE infrastructure (backend/services/sse/)
Implement sse-starlette endpoints for strategy generation
Standardize message format across all SSE endpoints
Add connection management and error handling
Create reusable SSE utilities

Frontend Implementation

Create SSE client infrastructure (frontend/src/services/sse/)
Implement React hook for SSE connections
Add automatic reconnection logic
Standardize message parsing and event handling
Create reusable SSE components

New Files to Create

Backend:
- backend/services/sse/__init__.py
- backend/services/sse/sse_manager.py
- backend/services/sse/message_formatter.py
- backend/services/sse/connection_manager.py
- backend/services/sse/error_handler.py
- backend/services/sse/types.py

Frontend:
- frontend/src/services/sse/index.ts
- frontend/src/services/sse/SSEConnection.ts
- frontend/src/services/sse/SSEEventManager.ts
- frontend/src/services/sse/SSEReconnection.ts
- frontend/src/services/sse/SSEMessageTypes.ts
- frontend/src/services/sse/SSEUtils.ts
- frontend/src/hooks/useSSE.ts

Phase 3: Migration & Testing (1 day)

Migration Steps

Migrate strategy generation to new SSE infrastructure
Test end-to-end functionality with new SSE
Add comprehensive error handling and recovery
Implement educational content streaming
Add monitoring and logging for SSE connections

Testing Strategy

Unit tests for SSE infrastructure
Integration tests for SSE endpoints
End-to-end tests for strategy generation
Error scenario testing (network drops, timeouts)
Performance testing (multiple concurrent connections)

🔧 Technical Specifications

Backend SSE Manager Interface

class SSEManager:
    async def create_connection(self, client_id: str) -> SSEConnection
    async def send_message(self, client_id: str, message: SSEMessage)
    async def broadcast_message(self, message: SSEMessage, filter_func=None)
    async def close_connection(self, client_id: str)
    async def get_connection_status(self, client_id: str) -> ConnectionStatus

Frontend SSE Client Interface

interface SSEConnection {
  connect(): Promise<void>
  disconnect(): void
  send(message: SSEMessage): void
  on(event: string, handler: EventHandler): void
  off(event: string, handler: EventHandler): void
  isConnected(): boolean
  reconnect(): Promise<void>
}

Message Format Specification

interface SSEMessage {
  event: 'progress' | 'complete' | 'error' | 'educational' | 'status'
  data: {
    step?: number
    progress?: number
    message?: string
    educational_content?: EducationalContent
    error?: string
    timestamp: string
    [key: string]: any
  }
}

🎯 Success Criteria

Phase 1 Success Criteria

✅ Strategy generation works reliably
✅ No more "Request timed out" errors
✅ Users can see progress and educational content
✅ Simple, debuggable implementation
✅ Strategy creation completes successfully

Phase 2 Success Criteria

✅ Real-time progress updates via SSE
✅ Automatic reconnection on network issues
✅ Standardized message format across features
✅ Reusable SSE infrastructure
✅ Proper error handling and recovery
✅ Educational content streaming

Phase 3 Success Criteria

✅ All features migrated to new SSE infrastructure
✅ Comprehensive testing coverage
✅ Performance meets requirements
✅ Error scenarios handled gracefully
✅ Monitoring and logging in place

🚀 Migration Benefits

Immediate Benefits (Phase 1)

Reliability: No more timeout errors
Simplicity: Easy to debug and maintain
User Experience: Clear progress feedback
Stability: Predictable behavior

Long-term Benefits (Phase 2+)

Reusability: SSE infrastructure for other features
Real-time Updates: Better user experience
Scalability: Handle multiple concurrent connections
Maintainability: Clean, modular architecture
Extensibility: Easy to add new SSE features

📝 Implementation Notes

Dependencies

Backend: sse-starlette package
Frontend: No additional dependencies (uses native EventSource)

Configuration

SSE Timeout: 5 minutes for long-running operations
Reconnection: Exponential backoff (1s, 2s, 4s, 8s, max 30s)
Message Format: JSON with standardized structure
Error Handling: Graceful degradation with fallback options

Monitoring & Logging

Connection Status: Track active connections
Message Flow: Log message types and frequencies
Error Tracking: Monitor and alert on SSE errors
Performance Metrics: Track response times and throughput

Security Considerations

Authentication: Validate client connections
Rate Limiting: Prevent abuse of SSE endpoints
Message Validation: Validate all incoming messages
Connection Limits: Limit concurrent connections per user

🔄 Rollback Plan

If Phase 1 Fails

Revert to current SSE implementation
Keep polling as fallback option
Document issues for future reference

If Phase 2 Fails

Keep Phase 1 polling implementation
Identify specific issues with sse-starlette
Consider alternative SSE libraries or WebSocket implementation

If Phase 3 Fails

Rollback to Phase 2 implementation
Fix specific issues identified during testing
Re-run migration with fixes

📚 References & Resources

Documentation

Best Practices

Examples & Templates

Next Steps:

Commit current code
Refresh session
Start Phase 1 implementation (MVP polling)
Test strategy generation works
Proceed to Phase 2 (clean SSE infrastructure)

Originally created by @AJaySi on GitHub (Aug 18, 2025). Original GitHub issue: https://github.com/AJaySi/ALwrity/issues/208 Originally assigned to: @AJaySi on GitHub. # SSE Migration Strategy & Implementation Plan ## 🚨 **Current Implementation Problems** ### **Backend Issues** - **Complex SSE Manager**: The `SSEAIServiceManager` with lambda functions is overly complex - **Async Generator Problems**: The `sse_yield` function using `__anext__()` is fragile - **Message Format Inconsistency**: Backend sends different message formats that frontend struggles to parse - **Tight Coupling**: AI service manager is tightly coupled to SSE implementation - **Error Propagation**: Errors in one component cascade to others - **Debugging Difficulty**: Complex async flows make debugging hard ### **Frontend Issues** - **EventSource Limitations**: No built-in reconnection, poor error handling - **Message Parsing Complexity**: Too many message types to handle - **Timeout Handling**: Frontend timeouts don't align with backend processing - **Connection State Management**: Poor handling of connection states - **Progress Tracking**: Inconsistent progress calculation and display ### **Architecture Problems** - **Tight Coupling**: Frontend and backend are tightly coupled to specific message formats - **No Reusability**: SSE implementation is specific to strategy generation - **Error Handling**: Inconsistent error handling across components - **Testing Difficulty**: Complex async flows make testing challenging ## 🎯 **Proposed Solution: Clean SSE with sse-starlette** ### **Phase 1: MVP Polling Solution (1-2 hours)** **Goal**: Get strategy generation working immediately with simple polling **Implementation**: - Replace complex SSE with simple polling mechanism - Poll strategy status every 10 seconds - Show progress modal with educational content - Handle timeouts gracefully - Remove all SSE-related complexity **Benefits**: - ✅ Immediate working solution - ✅ Simple to implement and debug - ✅ Reliable and predictable - ✅ Easy to test ### **Phase 2: Proper SSE Implementation (1-2 days)** **Goal**: Implement clean, reusable SSE infrastructure **Implementation**: - Use `sse-starlette` for backend SSE - Create reusable SSE client for frontend - Standardize message format - Add proper error handling and reconnection - Make SSE infrastructure reusable for other features **Benefits**: - ✅ Real-time updates - ✅ Better user experience - ✅ Reusable infrastructure - ✅ Proper error handling ## 🏗️ **Technical Architecture** ### **Backend: sse-starlette Implementation** #### **Core SSE Module** (`backend/services/sse/`) ``` backend/services/sse/ ├── __init__.py ├── sse_manager.py # Core SSE management ├── message_formatter.py # Standardized message formatting ├── connection_manager.py # Connection lifecycle management ├── error_handler.py # SSE error handling └── types.py # SSE message types and schemas ``` #### **SSE Manager Features** - **Connection Management**: Handle multiple SSE connections - **Message Broadcasting**: Send messages to specific clients - **Error Handling**: Graceful error handling and recovery - **Message Formatting**: Consistent message format across all features - **Connection Monitoring**: Track connection health and status #### **Message Format Standardization** ```python # Standard SSE message format { "event": "progress|complete|error|educational", "data": { "step": 1, "progress": 10, "message": "Processing...", "educational_content": {...}, "timestamp": "2024-01-01T00:00:00Z" } } ``` ### **Frontend: Reusable SSE Client** #### **Core SSE Module** (`frontend/src/services/sse/`) ``` frontend/src/services/sse/ ├── index.ts ├── SSEConnection.ts # Core SSE connection management ├── SSEEventManager.ts # Event handling and message parsing ├── SSEReconnection.ts # Automatic reconnection logic ├── SSEMessageTypes.ts # TypeScript types for messages └── SSEUtils.ts # Utility functions ``` #### **SSE Client Features** - **Automatic Reconnection**: Handle connection drops gracefully - **Message Parsing**: Parse standardized message format - **Event Handling**: Handle different event types - **Error Recovery**: Recover from errors automatically - **Connection Monitoring**: Monitor connection health #### **React Hook** (`frontend/src/hooks/useSSE.ts`) ```typescript const useSSE = (url: string, options?: SSEOptions) => { // Returns: { data, error, isConnected, reconnect } } ``` ## 📋 **Implementation Phases** ### **Phase 1: MVP Polling (Immediate - 1-2 hours)** #### **Backend Changes** 1. **Remove SSE complexity** from `ai_generation_endpoints.py` 2. **Simplify AI generation** to return immediately after starting 3. **Add status endpoint** to check generation progress 4. **Remove SSEAIServiceManager** and related complexity #### **Frontend Changes** 1. **Replace SSE with polling** in `ContentStrategyBuilder.tsx` 2. **Implement simple progress modal** with educational content 3. **Add polling mechanism** (every 10 seconds) 4. **Handle timeouts gracefully** (5-minute timeout) 5. **Remove all SSE-related code** #### **Files to Modify** - `backend/api/content_planning/api/content_strategy/endpoints/ai_generation_endpoints.py` - `frontend/src/components/ContentPlanningDashboard/components/ContentStrategyBuilder.tsx` - `frontend/src/services/contentPlanningApi.ts` ### **Phase 2: Clean SSE Infrastructure (1-2 days)** #### **Backend Implementation** 1. **Create SSE infrastructure** (`backend/services/sse/`) 2. **Implement sse-starlette endpoints** for strategy generation 3. **Standardize message format** across all SSE endpoints 4. **Add connection management** and error handling 5. **Create reusable SSE utilities** #### **Frontend Implementation** 1. **Create SSE client infrastructure** (`frontend/src/services/sse/`) 2. **Implement React hook** for SSE connections 3. **Add automatic reconnection** logic 4. **Standardize message parsing** and event handling 5. **Create reusable SSE components** #### **New Files to Create** ``` Backend: - backend/services/sse/__init__.py - backend/services/sse/sse_manager.py - backend/services/sse/message_formatter.py - backend/services/sse/connection_manager.py - backend/services/sse/error_handler.py - backend/services/sse/types.py Frontend: - frontend/src/services/sse/index.ts - frontend/src/services/sse/SSEConnection.ts - frontend/src/services/sse/SSEEventManager.ts - frontend/src/services/sse/SSEReconnection.ts - frontend/src/services/sse/SSEMessageTypes.ts - frontend/src/services/sse/SSEUtils.ts - frontend/src/hooks/useSSE.ts ``` ### **Phase 3: Migration & Testing (1 day)** #### **Migration Steps** 1. **Migrate strategy generation** to new SSE infrastructure 2. **Test end-to-end functionality** with new SSE 3. **Add comprehensive error handling** and recovery 4. **Implement educational content** streaming 5. **Add monitoring and logging** for SSE connections #### **Testing Strategy** 1. **Unit tests** for SSE infrastructure 2. **Integration tests** for SSE endpoints 3. **End-to-end tests** for strategy generation 4. **Error scenario testing** (network drops, timeouts) 5. **Performance testing** (multiple concurrent connections) ## 🔧 **Technical Specifications** ### **Backend SSE Manager Interface** ```python class SSEManager: async def create_connection(self, client_id: str) -> SSEConnection async def send_message(self, client_id: str, message: SSEMessage) async def broadcast_message(self, message: SSEMessage, filter_func=None) async def close_connection(self, client_id: str) async def get_connection_status(self, client_id: str) -> ConnectionStatus ``` ### **Frontend SSE Client Interface** ```typescript interface SSEConnection { connect(): Promise<void> disconnect(): void send(message: SSEMessage): void on(event: string, handler: EventHandler): void off(event: string, handler: EventHandler): void isConnected(): boolean reconnect(): Promise<void> } ``` ### **Message Format Specification** ```typescript interface SSEMessage { event: 'progress' | 'complete' | 'error' | 'educational' | 'status' data: { step?: number progress?: number message?: string educational_content?: EducationalContent error?: string timestamp: string [key: string]: any } } ``` ## 🎯 **Success Criteria** ### **Phase 1 Success Criteria** - ✅ Strategy generation works reliably - ✅ No more "Request timed out" errors - ✅ Users can see progress and educational content - ✅ Simple, debuggable implementation - ✅ Strategy creation completes successfully ### **Phase 2 Success Criteria** - ✅ Real-time progress updates via SSE - ✅ Automatic reconnection on network issues - ✅ Standardized message format across features - ✅ Reusable SSE infrastructure - ✅ Proper error handling and recovery - ✅ Educational content streaming ### **Phase 3 Success Criteria** - ✅ All features migrated to new SSE infrastructure - ✅ Comprehensive testing coverage - ✅ Performance meets requirements - ✅ Error scenarios handled gracefully - ✅ Monitoring and logging in place ## 🚀 **Migration Benefits** ### **Immediate Benefits (Phase 1)** - **Reliability**: No more timeout errors - **Simplicity**: Easy to debug and maintain - **User Experience**: Clear progress feedback - **Stability**: Predictable behavior ### **Long-term Benefits (Phase 2+)** - **Reusability**: SSE infrastructure for other features - **Real-time Updates**: Better user experience - **Scalability**: Handle multiple concurrent connections - **Maintainability**: Clean, modular architecture - **Extensibility**: Easy to add new SSE features ## 📝 **Implementation Notes** ### **Dependencies** - **Backend**: `sse-starlette` package - **Frontend**: No additional dependencies (uses native EventSource) ### **Configuration** - **SSE Timeout**: 5 minutes for long-running operations - **Reconnection**: Exponential backoff (1s, 2s, 4s, 8s, max 30s) - **Message Format**: JSON with standardized structure - **Error Handling**: Graceful degradation with fallback options ### **Monitoring & Logging** - **Connection Status**: Track active connections - **Message Flow**: Log message types and frequencies - **Error Tracking**: Monitor and alert on SSE errors - **Performance Metrics**: Track response times and throughput ### **Security Considerations** - **Authentication**: Validate client connections - **Rate Limiting**: Prevent abuse of SSE endpoints - **Message Validation**: Validate all incoming messages - **Connection Limits**: Limit concurrent connections per user ## 🔄 **Rollback Plan** ### **If Phase 1 Fails** - Revert to current SSE implementation - Keep polling as fallback option - Document issues for future reference ### **If Phase 2 Fails** - Keep Phase 1 polling implementation - Identify specific issues with sse-starlette - Consider alternative SSE libraries or WebSocket implementation ### **If Phase 3 Fails** - Rollback to Phase 2 implementation - Fix specific issues identified during testing - Re-run migration with fixes ## 📚 **References & Resources** ### **Documentation** - [sse-starlette Documentation](https://github.com/sysid/sse-starlette) - [Server-Sent Events MDN](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events) - [EventSource API](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) ### **Best Practices** - [SSE Best Practices](https://html.spec.whatwg.org/multipage/server-sent-events.html) - [Real-time Web Applications](https://web.dev/real-time-web-applications/) - [Error Handling in SSE](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#Error_handling) ### **Examples & Templates** - [sse-starlette Examples](https://github.com/sysid/sse-starlette/tree/main/examples) - [React SSE Hook Examples](https://github.com/facebook/react/tree/main/packages/react-dom/src/events) - [FastAPI SSE Examples](https://fastapi.tiangolo.com/advanced/websockets/) --- **Next Steps**: 1. Commit current code 2. Refresh session 3. Start Phase 1 implementation (MVP polling) 4. Test strategy generation works 5. Proceed to Phase 2 (clean SSE infrastructure)

kerem added the

enhancement

label