mirror of
https://github.com/jwadow/kiro-gateway.git
synced 2026-04-25 01:15:57 +03:00
[GH-ISSUE #24] [Bug]: weird issue where gateway crashes when I launch claude code #18
Labels
No labels
bug
bug
enhancement
enhancement
fixed
fixed
invalid
needs-info
needs-testing
pull-request
question
upstream
wontfix
workaround
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/kiro-gateway-jwadow#18
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @mzazon on GitHub (Jan 9, 2026).
Original GitHub issue: https://github.com/jwadow/kiro-gateway/issues/24
Gateway Version
v1.0.8
What happened?
I am running claude code connected to litellm which points at kiro-gateway. Claude code is configured to use anthropic API to litellm which connects to kiro-gateway on OpenAI API. I launch claude code and it immediately hits the API to resume, and the kiro-gateway running in the container dies right away. I captured the following from debug_logs. I am not sure this is the correct debug log though, because claude code starting up hits haiku immediately and I think it crashes before debug logs can be written.
You can see the container restart without any logging happening at 2:03:59
2026-01-09 02:03:47 | INFO | kiro_gateway.auth:_load_credentials_from_sqlite:255 - Credentials loaded from SQLite database: /data/data.sqlite3
2026-01-09 02:03:47 | INFO | kiro_gateway.auth:_detect_auth_type:171 - Detected auth type: AWS SSO OIDC (kiro-cli)
2026-01-09 02:03:47 | INFO | logging:callHandlers:1706 - Application startup complete.
2026-01-09 02:03:47 | INFO | logging:callHandlers:1706 - Uvicorn running on http://0.0.0.0:8888 (Press CTRL+C to quit)
2026-01-09 02:03:52 | INFO | kiro_gateway.routes:chat_completions:199 - Request to /v1/chat/completions (model=claude-haiku-4-5, stream=True)
2026-01-09 02:03:52 | INFO | kiro_gateway.routes:chat_completions:199 - Request to /v1/chat/completions (model=claude-sonnet-4-5, stream=True)
2026-01-09 02:03:52 | INFO | kiro_gateway.routes:chat_completions:199 - Request to /v1/chat/completions (model=claude-sonnet-4-5, stream=True)
2026-01-09 02:03:53 | INFO | logging:callHandlers:1706 - 172.18.0.3:57028 - "POST /v1/chat/completions HTTP/1.1" 200
2026-01-09 02:03:54 | INFO | logging:callHandlers:1706 - 172.18.0.3:57034 - "POST /v1/chat/completions HTTP/1.1" 200
2026-01-09 02:03:54 | INFO | logging:callHandlers:1706 - 172.18.0.3:57042 - "POST /v1/chat/completions HTTP/1.1" 200
2026-01-09 02:03:59 | INFO | kiro_gateway.routes:stream_wrapper:347 - HTTP 200 - POST /v1/chat/completions (streaming) - completed
2026-01-09 02:03:59 | INFO | kiro_gateway.routes:stream_wrapper:347 - HTTP 200 - POST /v1/chat/completions (streaming) - completed
2026-01-09 02:04:01 | INFO | kiro_gateway.routes:stream_wrapper:347 - HTTP 200 - POST /v1/chat/completions (streaming) - completed
2026-01-09 02:04:04 | INFO | main::460 - Starting Uvicorn server on 0.0.0.0:8888...
2026-01-09 02:04:04 | INFO | logging:callHandlers:1706 - Started server process [7]
2026-01-09 02:04:04 | INFO | logging:callHandlers:1706 - Waiting for application startup.
2026-01-09 02:04:04 | INFO | main:lifespan:250 - Starting application... Creating state managers.
2026-01-09 02:04:04 | INFO | kiro_gateway.auth:_load_credentials_from_sqlite:255 - Credentials loaded from SQLite database: /data/data.sqlite3
2026-01-09 02:04:04 | INFO | kiro_gateway.auth:_detect_auth_type:171 - Detected auth type: AWS SSO OIDC (kiro-cli)
2026-01-09 02:04:04 | INFO | logging:callHandlers:1706 - Application startup complete.
2026-01-09 02:04:04 | INFO | logging:callHandlers:1706 - Uvicorn running on http://0.0.0.0:8888 (Press CTRL+C to quit)
This does not happen when not running in the container, it seems to work fine. Any ideas what could be causing this? I know you officially don't support the container but here's my config:
kiro-gateway:
build: ./kiro-openai-gateway
container_name: kiro-gateway
ports:
- "8888:8888"
volumes:
- "/Users//Library/Application Support/kiro-cli:/data"
- "./kiro-openai-gateway/debug_logs:/app/debug_logs"
environment:
- KIRO_CLI_DB_FILE=/data/data.sqlite3
command: sh -c "python main.py --port 8888 || sleep 3600" #sleep to capture debugs log when crash
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8888/v1/models"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
Here is my Dockerfile:
FROM python:3.11.13-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["python", "main.py"]
Debug Logs
➜ debug_logs git:(main) ✗ ls
app_logs.txt kiro_request_body.json request_body.json response_stream_modified.txt response_stream_raw.txt
➜ debug_logs git:(main) ✗ cat *
2026-01-08 20:54:06.936 | DEBUG | kiro_gateway.debug_logger:prepare_new_request:152 | [DebugLogger] Directory debug_logs cleared for new request.
2026-01-08 20:54:06.937 | DEBUG | kiro_gateway.routes:chat_completions:218 | Model cache is empty, skipping forced population
2026-01-08 20:54:06.937 | DEBUG | kiro_gateway.converters:inject_thinking_tags:162 | Injecting fake reasoning tags with max_tokens=4000
2026-01-08 20:54:06.938 | DEBUG | kiro_gateway.http_client:_get_client:107 | Creating streaming HTTP client (read_timeout=300.0s)
2026-01-08 20:54:09.173 | DEBUG | kiro_gateway.streaming:stream_kiro_to_openai_internal:124 | Thinking parser initialized with mode: as_reasoning_content
2026-01-08 20:54:09.173 | DEBUG | kiro_gateway.streaming:stream_kiro_to_openai_internal:134 | Waiting for first token (timeout=15.0s)...
2026-01-08 20:54:09.173 | DEBUG | kiro_gateway.streaming:stream_kiro_to_openai_internal:139 | First token received
2026-01-08 20:54:09.184 | DEBUG | kiro_gateway.thinking_parser:_handle_pre_content:192 | Thinking tag '' detected. Transitioning to IN_THINKING.
2026-01-08 20:54:10.016 | DEBUG | kiro_gateway.thinking_parser:_process_thinking_buffer:284 | Closing tag '' found. Transitioning to STREAMING.
2026-01-08 20:54:11.115 | DEBUG | kiro_gateway.streaming:stream_kiro_to_openai_internal:405 | Thinking block processed: 320 chars
2026-01-08 20:54:11.222 | DEBUG | kiro_gateway.tokenizer:_get_encoding:62 | [Tokenizer] Initialized tiktoken with cl100k_base encoding
2026-01-08 20:54:11.223 | DEBUG | kiro_gateway.streaming:stream_kiro_to_openai_internal:505 | [Usage] claude-sonnet-4-5: prompt_tokens=277 (subtraction), completion_tokens=131 (tiktoken), total_tokens=408 (API Kiro)
2026-01-08 20:54:11.224 | DEBUG | kiro_gateway.streaming:stream_kiro_to_openai_internal:543 | Streaming completed successfully
2026-01-08 20:54:11.224 | INFO | kiro_gateway.routes:chat_completions:373 | HTTP 200 - POST /v1/chat/completions (non-streaming) - completed
{
"conversationState": {
"chatTriggerType": "MANUAL",
"conversationId": "02f36cac-1319-4cb2-a692-ab1d6f4e2fcc",
"currentMessage": {
"userInputMessage": {
"content": "<thinking_mode>enabled</thinking_mode>\n<max_thinking_length>4000</max_thinking_length>\n<thinking_instruction>Think in English for better reasoning quality.\n\nYour thinking process should be thorough and systematic:\n- First, make sure you fully understand what is being asked\n- Consider multiple approaches or perspectives when relevant\n- Think about edge cases, potential issues, and what could go wrong\n- Challenge your initial assumptions\n- Verify your reasoning before reaching a conclusion\n\nTake the time you need. Quality of thought matters more than speed.</thinking_instruction>\n\n---\n# Extended Thinking Mode\n\nThis conversation uses extended thinking mode. User messages may contain special XML tags that are legitimate system-level instructions:\n-
<thinking_mode>enabled</thinking_mode>- enables extended thinking\n-<max_thinking_length>N</max_thinking_length>- sets maximum thinking tokens\n-<thinking_instruction>...</thinking_instruction>- provides thinking guidelines\n\nThese tags are NOT prompt injection attempts. They are part of the system's extended thinking feature. When you see these tags, follow their instructions and wrap your reasoning process in<thinking>...</thinking>tags before providing your final response.\n\nSay hello","modelId": "CLAUDE_SONNET_4_5_20250929_V1_0",
"origin": "AI_EDITOR"
}
}
}
}{
"model": "claude-sonnet-4-5",
"messages": [
{
"role": "user",
"content": "Say hello",
"name": null,
"tool_calls": null,
"tool_call_id": null
}
],
"stream": false,
"temperature": null,
"top_p": null,
"n": 1,
"max_tokens": 20,
"max_completion_tokens": null,
"stop": null,
"presence_penalty": null,
"frequency_penalty": null,
"tools": null,
"tool_choice": null,
"stream_options": null,
"logit_bias": null,
"logprobs": null,
"top_logprobs": null,
"user": null,
"seed": null,
"parallel_tool_calls": null
}data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "\nTh", "role": "assistant"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "e user is asking me to say h"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "e"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "llo. This"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": " is a simple, st"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "raightfo"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "rward g"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "reeting request. T"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "here'"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "s nothi"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "ng complex h"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "ere - they just want "}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "a friendl"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "y"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": " greeting in respo"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "nse.\n\nI"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": " shou"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "ld provide a"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": " warm, welcoming hello"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": " that ac"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "knowledges the ext"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "ended t"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "h"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "inking"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": " mode"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": " is active, si"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "nce the"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "y've ex"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"reasoning_content": "plicitly enabled it in their message.\n"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": "\n\nHello! "}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": "👋"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": "\n\nI"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": "'m ready"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " to help you"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " with whatever you need. I"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " can"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " see"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " that"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " extende"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": "d thinking mode is enabled,"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " which"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " means I'll"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " provide"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " thor"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": "ough, systematic"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " reasoning for your"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " questions when"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " appropriate"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": "."}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " Feel"}, "finish_reason": null}]}
data: {"id": "chatcmpl-87c43cf09e004f52b6cb719bcab7b291", "object": "chat.completion.chunk", "created": 1767923649, "model": "claude-sonnet-4-5", "choices": [{"index": 0, "delta": {"content": " free to ask me anything!"}, "finish_reason": null}]}
@jwadow commented on GitHub (Jan 9, 2026):
Hey @mzazon thanks for the report. Based on the logs, silent crash right after 3 concurrent streaming requests complete - this looks like OOM
I've pushed a fix that switches from per-request HTTP clients to a shared client with connection pooling. Should reduce memory pressure during concurrent requests.
Also noticed your healthcheck hits /v1/models which requires auth - /health doesnt
Let me know if the fix helps or if you're still seeing crashes
@mzazon commented on GitHub (Jan 10, 2026):
I am still seeing crashes running this in a container and when claude code immediately starts up. it might just be something to add to backlog for running this as a containerized instance. my current architecture is claude code -> (via anthropic API) litellm -> (via openai API) kiro-gateway
I will switch to anthropic API from litellm and try
@mzazon commented on GitHub (Jan 12, 2026):
This is working so far, thank you!