[GH-ISSUE #321] Only 1 active worker from 800 #88

New issue

Closed

opened 2026-03-02 05:18:40 +03:00 by kerem · 1 comment

kerem commented

2026-03-02 05:18:40 +03:00

Owner

Originally created by @ludovit-ubrezi on GitHub (Feb 27, 2024).
Original GitHub issue: https://github.com/hibiken/asynqmon/issues/321

Hello, running worker with this configuration

func main() {
	concurrency := runtime.NumCPU() * 200

	srv := asynq.NewServer(
		asynq.RedisClientOpt{Addr: redisHost + ":" + redisPort, Password: redisPassword, DB: 1},
		asynq.Config{
			Concurrency:    concurrency,
			Queues:         map[string]int{tasks.QueueName: 9},
			StrictPriority: true,
		},
	)

	// close publisher connection functionality
	defer tasks.Publisher.Channel.Close()

	// Use asynq.HandlerFunc adapter for a handler function
	if err := srv.Run(asynq.HandlerFunc(tasks.HandleMonitorDeliveryTask)); err != nil {
		log.Fatal(err)
	}
}

Problem is that even Concurency is set to 800 workers there is always only 1 worker processing queue. This is why the Pending queue is not processing correctly and whole system has delays. (screenshots below)

Code of function tasks.HandleMonitorDeliveryTask is working correctly and fast because when i load all the data, it processes Task with group of 5000 items and it load into RabbitMQ -> 1 million items in e.g. 1 minute.

But when Task has 1 item it slows down due to only 1 worker processing this queue even when there is 800 workers setup for server it only run 1.

Is there something what i missing in configuration? How to setup it?

Originally created by @ludovit-ubrezi on GitHub (Feb 27, 2024). Original GitHub issue: https://github.com/hibiken/asynqmon/issues/321 Hello, running worker with this configuration ``` func main() { concurrency := runtime.NumCPU() * 200 srv := asynq.NewServer( asynq.RedisClientOpt{Addr: redisHost + ":" + redisPort, Password: redisPassword, DB: 1}, asynq.Config{ Concurrency: concurrency, Queues: map[string]int{tasks.QueueName: 9}, StrictPriority: true, }, ) // close publisher connection functionality defer tasks.Publisher.Channel.Close() // Use asynq.HandlerFunc adapter for a handler function if err := srv.Run(asynq.HandlerFunc(tasks.HandleMonitorDeliveryTask)); err != nil { log.Fatal(err) } } ``` Problem is that even **Concurency is set to 800 workers** there is **always only 1 worker processing queue**. This is why the Pending queue is not processing correctly and whole system has delays. (screenshots below) Code of function tasks.HandleMonitorDeliveryTask is working correctly and fast because when i load all the data, it processes Task with group of 5000 items and it load into RabbitMQ -> 1 million items in e.g. 1 minute. But when Task has 1 item it slows down due to only 1 worker processing this queue even when there is 800 workers setup for server it only run 1. **Is there something what i missing in configuration? How to setup it?** ![Screenshot 2024-02-27 at 12 30 38](https://github.com/hibiken/asynqmon/assets/29823609/0022d157-fdce-4839-a733-365b7d8f8ea0) ![Screenshot 2024-02-27 at 12 30 06](https://github.com/hibiken/asynqmon/assets/29823609/bb31cd09-e9ef-4f2a-a059-5b97d22a63d1)