[GH-ISSUE #582] Redis errors are not being returned to the caller on server run #1295

Open
opened 2026-03-07 22:08:23 +03:00 by kerem · 4 comments
Owner

Originally created by @edalford11 on GitHub (Dec 1, 2022).
Original GitHub issue: https://github.com/hibiken/asynq/issues/582

Originally assigned to: @hibiken on GitHub.

Describe the bug
Errors are not bubbled up to the caller when starting server. For example:

ERROR: cannot subscribe to cancelation channel: UNKNOWN: redis pubsub receive error: dial tcp: address redis.dev.xyz: missing port in address

To Reproduce

srv := asynq.NewServer(
	asynq.RedisClientOpt{Addr: redisURL},
	asynq.Config{
		Queues: map[string]int{
			"high":    6,
			"default": 3,
			"low":     1,
		},
		ErrorHandler: asynq.ErrorHandlerFunc(HandleErrorFunc),
	},
)

mux := asynq.NewServeMux()
mux.HandleFunc(TypeFirebaseSyncUser, HandleFirebaseSyncUserTask)

if err := srv.Run(mux); err != nil {
  // No error is ever returned here
  log.Fatalf("could not run server: %v", err)
}

Expected behavior
If an error happens such as not being able to connect to redis, the caller should be able to do what they need to do with that error. This is a problem because when I run this server in a container, the container continues to report as running with no issues, even though there are definitely issues.

Originally created by @edalford11 on GitHub (Dec 1, 2022). Original GitHub issue: https://github.com/hibiken/asynq/issues/582 Originally assigned to: @hibiken on GitHub. **Describe the bug** Errors are not bubbled up to the caller when starting server. For example: ```ERROR: cannot subscribe to cancelation channel: UNKNOWN: redis pubsub receive error: dial tcp: address redis.dev.xyz: missing port in address``` **To Reproduce** ```go srv := asynq.NewServer( asynq.RedisClientOpt{Addr: redisURL}, asynq.Config{ Queues: map[string]int{ "high": 6, "default": 3, "low": 1, }, ErrorHandler: asynq.ErrorHandlerFunc(HandleErrorFunc), }, ) mux := asynq.NewServeMux() mux.HandleFunc(TypeFirebaseSyncUser, HandleFirebaseSyncUserTask) if err := srv.Run(mux); err != nil { // No error is ever returned here log.Fatalf("could not run server: %v", err) } ``` **Expected behavior** If an error happens such as not being able to connect to redis, the caller should be able to do what they need to do with that error. This is a problem because when I run this server in a container, the container continues to report as running with no issues, even though there are definitely issues.
Author
Owner

@hibiken commented on GitHub (Dec 21, 2022):

Thank you @edalford11 for reporting this.

I see, I think redis backend is a "start-up critical" dependency of asynq server, so if the server cannot connect to redis on start up, it should fail (as you said). However, losing connection to redis shouldn't fail the server, since connection may get lost intermittently.

Incidentally, the current implementation of Server provides this HealthCheckFunc option in the config, so you could provide a callback function to report server health.

<!-- gh-comment-id:1362154715 --> @hibiken commented on GitHub (Dec 21, 2022): Thank you @edalford11 for reporting this. I see, I think redis backend is a "start-up critical" dependency of asynq server, so if the server cannot connect to redis on start up, it should fail (as you said). However, losing connection to redis shouldn't fail the server, since connection may get lost intermittently. Incidentally, the current implementation of `Server` provides this [`HealthCheckFunc` option](https://pkg.go.dev/github.com/hibiken/asynq#Config.HealthCheckFunc) in the config, so you could provide a callback function to report server health.
Author
Owner

@edalford11 commented on GitHub (Jan 6, 2023):

I am using the HealthCheckFunc for running the periodic health checks, though its not sufficient for reporting the start up critical dependency errors such as a bad redis url.

<!-- gh-comment-id:1373677529 --> @edalford11 commented on GitHub (Jan 6, 2023): I am using the HealthCheckFunc for running the periodic health checks, though its not sufficient for reporting the start up critical dependency errors such as a bad redis url.
Author
Owner

@camillof commented on GitHub (May 12, 2023):

Hey @edalford11 were you able to implement a workaround?

I'm also interested in raising an error if the redis server is not ready when starting the asynq server.

<!-- gh-comment-id:1546272451 --> @camillof commented on GitHub (May 12, 2023): Hey @edalford11 were you able to implement a workaround? I'm also interested in raising an error if the redis server is not ready when starting the asynq server.
Author
Owner

@edalford11 commented on GitHub (May 17, 2023):

@camillof unfortunately I didn't. We are just hoping that no redis connection issues ever happen + check the logs every now and then to be sure. @hibiken would love to hear about if/when proper error handling for redis will be implemented so that users may properly handle these errors when they happen.

<!-- gh-comment-id:1551250675 --> @edalford11 commented on GitHub (May 17, 2023): @camillof unfortunately I didn't. We are just hoping that no redis connection issues ever happen + check the logs every now and then to be sure. @hibiken would love to hear about if/when proper error handling for redis will be implemented so that users may properly handle these errors when they happen.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/asynq#1295
No description provided.