[GH-ISSUE #635] [BUG] Use a custom redis client that will panic if redis close

kerem commented

2026-03-07 22:08:38 +03:00

Owner

Originally created by @shuqingzai on GitHub (Mar 31, 2023).
Original GitHub issue: https://github.com/hibiken/asynq/issues/635

Originally assigned to: @hibiken on GitHub.

Describe the bug
If you implement the asynq.RedisConnOpt interface and provide a custom redis client, Ctrl + C stops the asynq.Server and asynq.Subscriber panics

To Reproduce
Steps to reproduce the behavior (Code snippets if applicable):

Implement the asynq.RedisConnOpt interface and provide a custom redis client
Instantiate asynq.Server and set up a custom redis client
Start the Server
Stop custom redis client and asynq.RedisConnOpt return nil
Ctrl + C stops the Server, asynq.Subscriber panics

Expected behavior

Can support custom providing redis client instead of panic

Because asynq will start multiple redis clients, and redis will also be used in my project, I hope that only one redis client can be instantiated for unified management and reuse clients

Screenshots
If applicable, add screenshots to help explain your problem.

my redis client

Implement the asynq.RedisConnOpt interface

stop custom redis client will return nil

asynq.Subscriber panics

Because my redis client returns nil ??

Environment (please complete the following information):

OS: [e.g. MacOS, Linux]

MacOS

Version of asynq package [e.g. v1.0.0]

v0.24.0

Additional context
Add any other context about the problem here.

Originally created by @shuqingzai on GitHub (Mar 31, 2023). Original GitHub issue: https://github.com/hibiken/asynq/issues/635 Originally assigned to: @hibiken on GitHub. **Describe the bug** If you implement the `asynq.RedisConnOpt` interface and provide a custom redis client, `Ctrl + C` stops the `asynq.Server` and `asynq.Subscriber` panics **To Reproduce** Steps to reproduce the behavior (Code snippets if applicable): 1. Implement the `asynq.RedisConnOpt` interface and provide a custom redis client 2. Instantiate `asynq.Server` and set up a custom redis client 3. Start the Server 4. Stop custom redis client and `asynq.RedisConnOpt` return nil 5. Ctrl + C stops the Server, `asynq.Subscriber` panics **Expected behavior** **Can support custom providing redis client instead of panic** Because asynq will start multiple redis clients, and redis will also be used in my project, I hope that only one redis client can be instantiated for unified management and reuse clients **Screenshots** If applicable, add screenshots to help explain your problem. 1. my redis client > Implement the asynq.RedisConnOpt interface > ![image](https://user-images.githubusercontent.com/44406033/229015867-ebd7ca78-4f14-4e25-a636-96723f80edef.png) stop custom redis client will return nil 3. `asynq.Subscriber` panics ![image](https://user-images.githubusercontent.com/44406033/229016233-9ebc1786-9358-482d-a861-f1551ee53c82.png) ![image](https://user-images.githubusercontent.com/44406033/229017169-cfcc376e-b364-46cf-a530-98207472156b.png) Because `my redis client` returns nil ?? **Environment (please complete the following information):** - OS: [e.g. MacOS, Linux] ```shell MacOS ``` - Version of `asynq` package [e.g. v1.0.0] ```shell v0.24.0 ``` **Additional context** Add any other context about the problem here.

kerem

2026-03-07 22:08:38 +03:00

closed this issue
added the
bug
label

kerem commented

2026-03-07 22:08:39 +03:00

Author

Owner

@bfwbbrj commented on GitHub (Mar 31, 2023):

@shuqingzai My guess would be the msg is nil. And this is related to to the pubsub returned by your custom redis's Subscribe. Since I have no idea which lib you are using, you probably need to check your lib's implementation.

@bfwbbrj commented on GitHub (Mar 31, 2023): @shuqingzai My guess would be the [msg](https://github.com/hibiken/asynq/blob/cc777ebdaa62b69bd6e985fa97117b854e7d1cd6/subscriber.go#L83) is nil. And this is related to to the pubsub returned by your custom redis's [Subscribe](https://github.com/hibiken/asynq/blob/cc777ebdaa62b69bd6e985fa97117b854e7d1cd6/internal/rdb/rdb.go#L1451). Since I have no idea which lib you are using, you probably need to check your lib's implementation.

kerem commented

2026-03-07 22:08:39 +03:00

Author

Owner

@shuqingzai commented on GitHub (Mar 31, 2023):

@shuqingzai My guess would be the msg is nil. And this is related to to the pubsub returned by your custom redis's Subscribe. Since I have no idea which lib you are using, you probably need to check your lib's implementation.

@bfwbbrj First of all, thank you for answering my question

What do you mean by library?
I have not changed any configuration and the payload field has not been modified. I just switch my own Redis client back to asynq's built-in implementation and there will be no panic, but using my own implementation will cause panic

@shuqingzai commented on GitHub (Mar 31, 2023): > @shuqingzai My guess would be the [msg](https://github.com/hibiken/asynq/blob/cc777ebdaa62b69bd6e985fa97117b854e7d1cd6/subscriber.go#L83) is nil. And this is related to to the pubsub returned by your custom redis's [Subscribe](https://github.com/hibiken/asynq/blob/cc777ebdaa62b69bd6e985fa97117b854e7d1cd6/internal/rdb/rdb.go#L1451). Since I have no idea which lib you are using, you probably need to check your lib's implementation. @bfwbbrj First of all, thank you for answering my question 1. What do you mean by library? 2. I have not changed any configuration and the payload field has not been modified. I just switch my own Redis client back to asynq's built-in implementation and there will be no panic, but using my own implementation will cause panic

kerem commented

2026-03-07 22:08:39 +03:00

Author

Owner

@bfwbbrj commented on GitHub (Apr 1, 2023):

What do you mean by library?

The redis client implementation that you are using.

I have not changed any configuration and the payload field has not been modified. I just switch my own Redis client back to asynq's built-in implementation and there will be no panic, but using my own implementation will cause panic

It should be related to the Subscribe method of your redis client implementation. Can you share which redis client implementation you are using?

@bfwbbrj commented on GitHub (Apr 1, 2023): >What do you mean by library? The redis client implementation that you are using. >I have not changed any configuration and the payload field has not been modified. I just switch my own Redis client back to asynq's built-in implementation and there will be no panic, but using my own implementation will cause panic It should be related to the [Subscribe](https://github.com/hibiken/asynq/blob/cc777ebdaa62b69bd6e985fa97117b854e7d1cd6/internal/rdb/rdb.go#L1451) method of your redis client implementation. Can you share which redis client implementation you are using?

kerem commented

2026-03-07 22:08:39 +03:00

Author

Owner

@shuqingzai commented on GitHub (Apr 1, 2023):

What do you mean by library?

The redis client implementation that you are using.

I have not changed any configuration and the payload field has not been modified. I just switch my own Redis client back to asynq's built-in implementation and there will be no panic, but using my own implementation will cause panic

It should be related to the Subscribe method of your redis client implementation. Can you share which redis client implementation you are using?

Observe my screenshot, using go-redis's built-in interface analysis

See: https://github.com/redis/go-redis/blob/v8/universal.go#L208

@shuqingzai commented on GitHub (Apr 1, 2023): > > What do you mean by library? > > The redis client implementation that you are using. > > > I have not changed any configuration and the payload field has not been modified. I just switch my own Redis client back to asynq's built-in implementation and there will be no panic, but using my own implementation will cause panic > > It should be related to the [Subscribe](https://github.com/hibiken/asynq/blob/cc777ebdaa62b69bd6e985fa97117b854e7d1cd6/internal/rdb/rdb.go#L1451) method of your redis client implementation. Can you share which redis client implementation you are using? Observe my screenshot, using go-redis's built-in interface analysis See: https://github.com/redis/go-redis/blob/v8/universal.go#L208 <img width="818" alt="image" src="https://user-images.githubusercontent.com/44406033/229303612-afa5a458-2da1-46a1-96ba-5e591fc8f0f5.png">

kerem commented

2026-03-07 22:08:40 +03:00

Author

Owner

@bfwbbrj commented on GitHub (Apr 1, 2023):

Observe my screenshot, using go-redis's built-in interface analysis

See: https://github.com/redis/go-redis/blob/v8/universal.go#L208

所以你的m.redisClient也是用https://github.com/redis/go-redis/ 这个库的客户端吗？你现在的问题是你如果在MakeRedisClient里返回一个nil，在关服务器的时候会panic？

@bfwbbrj commented on GitHub (Apr 1, 2023): > Observe my screenshot, using go-redis's built-in interface analysis > > See: https://github.com/redis/go-redis/blob/v8/universal.go#L208 所以你的`m.redisClient`也是用`https://github.com/redis/go-redis/` 这个库的客户端吗？你现在的问题是你如果在`MakeRedisClient`里返回一个nil，在关服务器的时候会panic？

kerem commented

2026-03-07 22:08:40 +03:00

Author

Owner

@shuqingzai commented on GitHub (Apr 2, 2023):

Observe my screenshot, using go-redis's built-in interface analysis
See: https://github.com/redis/go-redis/blob/v8/universal.go#L208

所以你的m.redisClient也是用https://github.com/redis/go-redis/ 这个库的客户端吗？你现在的问题是你如果在MakeRedisClient里返回一个nil，在关服务器的时候会panic？
@bfwbbrj

Yes
Hmm, maybe I closed the Redis client first?

@shuqingzai commented on GitHub (Apr 2, 2023): > > Observe my screenshot, using go-redis's built-in interface analysis > > See: https://github.com/redis/go-redis/blob/v8/universal.go#L208 > > 所以你的`m.redisClient`也是用`https://github.com/redis/go-redis/` 这个库的客户端吗？你现在的问题是你如果在`MakeRedisClient`里返回一个nil，在关服务器的时候会panic？ @bfwbbrj 1. Yes 2. Hmm, maybe I closed the Redis client first?

kerem commented

2026-03-07 22:08:40 +03:00

Author

Owner

@bfwbbrj commented on GitHub (Apr 3, 2023):

Hmm, maybe I closed the Redis client first?

Is it possible for you not to close the Redis client and leave the work to asynq? Currently if you closed the redis client before you shutdown the asynq, msg will be nil since the channel is already closed. I will create a pr to fix this but since rn prs are not being reviewed and merge. I suggest that you let asynq do the client closing if possible.

@bfwbbrj commented on GitHub (Apr 3, 2023): > 2. Hmm, maybe I closed the Redis client first? Is it possible for you not to close the Redis client and leave the work to asynq? Currently if you closed the redis client before you shutdown the asynq, [msg](https://github.com/hibiken/asynq/blob/cc777ebdaa62b69bd6e985fa97117b854e7d1cd6/subscriber.go#L83) will be nil since the channel is already closed. I will create a pr to fix this but since rn prs are not being reviewed and merge. I suggest that you let asynq do the client closing if possible.

kerem commented

2026-03-07 22:08:40 +03:00

Author

Owner

@kamikazechaser commented on GitHub (Nov 13, 2024):

If the new API (NewClientFromRedisClient) is used, then attempting to close the client through asynq will return an error https://github.com/hibiken/asynq/blob/master/client.go#L321.

A warning has also been added to guide users on this change. I would recommend moving to the new API and handling the closing on your own.

For now, I will close this Issue.

@kamikazechaser commented on GitHub (Nov 13, 2024): If the new API (NewClientFromRedisClient) is used, then attempting to close the client through asynq will return an error https://github.com/hibiken/asynq/blob/master/client.go#L321. A warning has also been added to guide users on this change. I would recommend moving to the new API and handling the closing on your own. For now, I will close this Issue.

Rows
Columns

[GH-ISSUE #635] [BUG] Use a custom redis client that will panic if redis close #1327