[GH-ISSUE #296] WebUI not loading Queues #427

New issue

Open

opened 2026-03-07 22:15:20 +03:00 by kerem · 13 comments

kerem commented 2026-03-07 22:15:20 +03:00

Owner

Copy link

Originally created by @jirevwe on GitHub (Mar 22, 2023).
Original GitHub issue: https://github.com/hibiken/asynqmon/issues/296

Hi 👋🏿,

I'm on running asynq: 0.23.0 and asynqmon: 0.7.1 in my production environment and I have been unable to access the Web UI, when this happened in the past running flushall in redis fixed it (got the idea from #149), but I can't afford to do that now as I am storing other things in redis.

What do you think caused this (so I can prevent this from happening in the future) and how can I get it to work without clearing all the keys?

I attached a screenshot of the error. Thanks in advance.

Originally created by @jirevwe on GitHub (Mar 22, 2023). Original GitHub issue: https://github.com/hibiken/asynqmon/issues/296 Hi 👋🏿, I'm on running asynq: 0.23.0 and asynqmon: 0.7.1 in my production environment and I have been unable to access the Web UI, when this happened in the past running `flushall` in redis fixed it (got the idea from #149), but I can't afford to do that now as I am storing other things in redis. What do you think caused this (so I can prevent this from happening in the future) and how can I get it to work without clearing all the keys? I attached a screenshot of the error. Thanks in advance. 

kerem commented 2026-03-07 22:15:21 +03:00

Author

Owner

Copy link

@netixx commented on GitHub (Sep 13, 2023):

I have the same issue, I ran flushall, but it didn't solve my issue either.

<!-- gh-comment-id:1717910292 --> @netixx commented on GitHub (Sep 13, 2023): I have the same issue, I ran flushall, but it didn't solve my issue either.

kerem commented 2026-03-07 22:15:21 +03:00

Author

Owner

Copy link

@MrRobo-t commented on GitHub (Sep 14, 2023):

Facing the same issue.. It momentarily shows the active task.. That's all I get.. There is no archive task coming in..
@hibiken Can you please help on this? I have build using the docker image .. I am getting graph for task processed though.. The db is deployed on aws elasticache redis..

<!-- gh-comment-id:1719216109 --> @MrRobo-t commented on GitHub (Sep 14, 2023): Facing the same issue.. It momentarily shows the active task.. That's all I get.. There is no archive task coming in.. @hibiken Can you please help on this? I have build using the docker image .. I am getting graph for task processed though.. The db is deployed on aws elasticache redis..

kerem commented 2026-03-07 22:15:21 +03:00

Author

Owner

Copy link

@netixx commented on GitHub (Sep 14, 2023):

I looked at the call graph for the /queues route, and given the error is lua running in redis, my guess is that the issue is in this script:

var memoryUsageCmd = redis.NewScript(`
local sample_size = tonumber(ARGV[2])
if sample_size <= 0 then
    return redis.error_reply("sample size must be a positive number")
end
local memusg = 0
for i=1,2 do
    local ids = redis.call("LRANGE", KEYS[i], 0, sample_size - 1)
    local sample_total = 0
    if (table.getn(ids) > 0) then
        for _, id in ipairs(ids) do
            local bytes = redis.call("MEMORY", "USAGE", ARGV[1] .. id)
            sample_total = sample_total + bytes
        end
        local n = redis.call("LLEN", KEYS[i])
        local avg = sample_total / table.getn(ids)
        memusg = memusg + (avg * n)
    end
    local m = redis.call("MEMORY", "USAGE", KEYS[i])
    if (m) then
        memusg = memusg + m
    end
end
for i=3,6 do
    local ids = redis.call("ZRANGE", KEYS[i], 0, sample_size - 1)
    local sample_total = 0
    if (table.getn(ids) > 0) then
        for _, id in ipairs(ids) do
            local bytes = redis.call("MEMORY", "USAGE", ARGV[1] .. id)
            sample_total = sample_total + bytes
        end
        local n = redis.call("ZCARD", KEYS[i])
        local avg = sample_total / table.getn(ids)
        memusg = memusg + (avg * n)
    end
    local m = redis.call("MEMORY", "USAGE", KEYS[i])
    if (m) then
        memusg = memusg + m
    end
end
local groups = redis.call("SMEMBERS", KEYS[7])
if table.getn(groups) > 0 then
	local agg_task_count = 0
	local agg_task_sample_total = 0
	local agg_task_sample_size = 0
	for i, gname in ipairs(groups) do
		local group_key = ARGV[4] .. gname
		agg_task_count = agg_task_count + redis.call("ZCARD", group_key)
		if i <= tonumber(ARGV[3]) then
			local ids = redis.call("ZRANGE", group_key, 0, sample_size - 1)
			for _, id in ipairs(ids) do
				local bytes = redis.call("MEMORY", "USAGE", ARGV[1] .. id)
				agg_task_sample_total = agg_task_sample_total + bytes
				agg_task_sample_size = agg_task_sample_size + 1
			end
		end
	end
	local avg = agg_task_sample_total / agg_task_sample_size
	memusg = memusg + (avg * agg_task_count)
end
return memusg
`)

Since the issue is the "bytes", then I guess more specifically here:

local bytes = redis.call("MEMORY", "USAGE", ARGV[1] .. id)
            sample_total = sample_total + bytes

running asynqmon v0.7.2
and asynq v0.24.1

This part of the code is asynq code (internal/rdb/inspect.go). All calls to CurrentStats should be impacted as well.

<!-- gh-comment-id:1719456169 --> @netixx commented on GitHub (Sep 14, 2023): I looked at the call graph for the `/queues` route, and given the error is lua running in redis, my guess is that the issue is in this script: ``` var memoryUsageCmd = redis.NewScript(` local sample_size = tonumber(ARGV[2]) if sample_size <= 0 then return redis.error_reply("sample size must be a positive number") end local memusg = 0 for i=1,2 do local ids = redis.call("LRANGE", KEYS[i], 0, sample_size - 1) local sample_total = 0 if (table.getn(ids) > 0) then for _, id in ipairs(ids) do local bytes = redis.call("MEMORY", "USAGE", ARGV[1] .. id) sample_total = sample_total + bytes end local n = redis.call("LLEN", KEYS[i]) local avg = sample_total / table.getn(ids) memusg = memusg + (avg * n) end local m = redis.call("MEMORY", "USAGE", KEYS[i]) if (m) then memusg = memusg + m end end for i=3,6 do local ids = redis.call("ZRANGE", KEYS[i], 0, sample_size - 1) local sample_total = 0 if (table.getn(ids) > 0) then for _, id in ipairs(ids) do local bytes = redis.call("MEMORY", "USAGE", ARGV[1] .. id) sample_total = sample_total + bytes end local n = redis.call("ZCARD", KEYS[i]) local avg = sample_total / table.getn(ids) memusg = memusg + (avg * n) end local m = redis.call("MEMORY", "USAGE", KEYS[i]) if (m) then memusg = memusg + m end end local groups = redis.call("SMEMBERS", KEYS[7]) if table.getn(groups) > 0 then local agg_task_count = 0 local agg_task_sample_total = 0 local agg_task_sample_size = 0 for i, gname in ipairs(groups) do local group_key = ARGV[4] .. gname agg_task_count = agg_task_count + redis.call("ZCARD", group_key) if i <= tonumber(ARGV[3]) then local ids = redis.call("ZRANGE", group_key, 0, sample_size - 1) for _, id in ipairs(ids) do local bytes = redis.call("MEMORY", "USAGE", ARGV[1] .. id) agg_task_sample_total = agg_task_sample_total + bytes agg_task_sample_size = agg_task_sample_size + 1 end end end local avg = agg_task_sample_total / agg_task_sample_size memusg = memusg + (avg * agg_task_count) end return memusg `) ``` Since the issue is the "bytes", then I guess more specifically here: ``` local bytes = redis.call("MEMORY", "USAGE", ARGV[1] .. id) sample_total = sample_total + bytes ``` running asynqmon v0.7.2 and asynq v0.24.1 This part of the code is asynq code (internal/rdb/inspect.go). All calls to `CurrentStats` should be impacted as well.

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@Psuta90 commented on GitHub (Oct 12, 2023):

any fix for this issue

<!-- gh-comment-id:1759530418 --> @Psuta90 commented on GitHub (Oct 12, 2023): any fix for this issue

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@Psuta90 commented on GitHub (Oct 12, 2023):

i got the same issue

<!-- gh-comment-id:1759533312 --> @Psuta90 commented on GitHub (Oct 12, 2023): <img width="1552" alt="image" src="https://github.com/hibiken/asynqmon/assets/75711078/10c4a707-0b6d-475a-b81d-fdb83f068ec0"> i got the same issue

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@Psuta90 commented on GitHub (Oct 12, 2023):

the redis connected but cannot get the queues

<!-- gh-comment-id:1759534115 --> @Psuta90 commented on GitHub (Oct 12, 2023): the redis connected but cannot get the queues

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@ianmuhia commented on GitHub (Oct 12, 2023):

@Psuta90 share logs from the asynqmon server.

<!-- gh-comment-id:1760479897 --> @ianmuhia commented on GitHub (Oct 12, 2023): @Psuta90 share logs from the asynqmon server.

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@Psuta90 commented on GitHub (Oct 13, 2023):

@Psuta90 share logs from the asynqmon server.

i try to scheduled task is success in terminal manualjobs is succes but in asynqmon is not showing the queues

in the sidebar server is showing but i click is not working like previous picture

<!-- gh-comment-id:1761006973 --> @Psuta90 commented on GitHub (Oct 13, 2023): > @Psuta90 share logs from the asynqmon server. <img width="1440" alt="image" src="https://github.com/hibiken/asynqmon/assets/75711078/c63555ce-a6dd-4fb2-8e51-b070da4b5197"> i try to scheduled task is success in terminal manualjobs is succes but in asynqmon is not showing the queues in the sidebar server is showing but i click is not working like previous picture <img width="1552" alt="image" src="https://github.com/hibiken/asynqmon/assets/75711078/2bc821c8-9aa9-4dc2-97d8-0aa2ede1496d">

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@Psuta90 commented on GitHub (Oct 13, 2023):

<!-- gh-comment-id:1761019100 --> @Psuta90 commented on GitHub (Oct 13, 2023): <img width="1552" alt="image" src="https://github.com/hibiken/asynqmon/assets/75711078/66921750-d1af-44a0-8bf9-8d2e93f733d7">

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@Psuta90 commented on GitHub (Oct 13, 2023):

@Psuta90 share logs from the asynqmon server.

"time":"2023-10-13T07:13:46.906285637Z","id":"","remote_ip":"172.18.0.1","host":"localhost:3005","method":"GET","uri":"/monitoring/tasks/api/queues/default/active_tasks?page=1&size=100","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36","status":500,"error":"","latency":491845,"latency_human":"491.845µs","bytes_in":0,"bytes_out":23}

<!-- gh-comment-id:1761022807 --> @Psuta90 commented on GitHub (Oct 13, 2023): > @Psuta90 share logs from the asynqmon server. "time":"2023-10-13T07:13:46.906285637Z","id":"","remote_ip":"172.18.0.1","host":"localhost:3005","method":"GET","uri":"/monitoring/tasks/api/queues/default/active_tasks?page=1&size=100","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36","status":500,"error":"","latency":491845,"latency_human":"491.845µs","bytes_in":0,"bytes_out":23}

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@wahyudibo commented on GitHub (Oct 26, 2023):

I also encountered the same issue. The asynqmon can connect to redis but unable to load queue data. I'm using asynq v0.24.1 and asynqmon v0.7.2. Here's the log from asynqmon

2023/10/26 03:38:00 Failed to collect metrics data: failed to get queue info: UNKNOWN: UNKNOWN: redis eval error: ERR user_script:30: attempt to perform arithmetic on local 'bytes' (a boolean value) script: 0c80fc1868e3aae205155ae2017ce995a3a7cec0, on @user_script:30.

thanks

<!-- gh-comment-id:1780550702 --> @wahyudibo commented on GitHub (Oct 26, 2023): I also encountered the same issue. The asynqmon can connect to redis but unable to load queue data. I'm using asynq v0.24.1 and asynqmon v0.7.2. Here's the log from asynqmon ` 2023/10/26 03:38:00 Failed to collect metrics data: failed to get queue info: UNKNOWN: UNKNOWN: redis eval error: ERR user_script:30: attempt to perform arithmetic on local 'bytes' (a boolean value) script: 0c80fc1868e3aae205155ae2017ce995a3a7cec0, on @user_script:30. ` thanks

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@wahyudibo commented on GitHub (Oct 30, 2023):

Just want to give an update. my asynq and asynqmon setup works now after i run go mod tidy

<!-- gh-comment-id:1785036607 --> @wahyudibo commented on GitHub (Oct 30, 2023): Just want to give an update. my asynq and asynqmon setup works now after i run `go mod tidy`

kerem commented 2026-03-07 22:15:22 +03:00

Author

Owner

Copy link

@piavgh commented on GitHub (Nov 10, 2023):

Just want to give an update. my asynq and asynqmon setup works now after i run go mod tidy

does not work for me.
It looks like v0.24 of asynq is not compatible with the asyncmon web UI v0.7.2

<!-- gh-comment-id:1805084253 --> @piavgh commented on GitHub (Nov 10, 2023): > Just want to give an update. my asynq and asynqmon setup works now after i run `go mod tidy` does not work for me. It looks like v0.24 of asynq is not compatible with the asyncmon web UI v0.7.2

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

dependabot/npm_and_yarn/ui/react-redux-8.0.5

dependabot/npm_and_yarn/ui/typescript-4.9.5

dependabot/npm_and_yarn/ui/testing-library/user-event-14.4.3

dependabot/npm_and_yarn/ui/testing-library/jest-dom-5.16.5

dependabot/npm_and_yarn/ui/react-syntax-highlighter-15.5.0

refactor/generic-tasks-table

refactor/generic-tasks

v0.7.2

v0.7.1

v0.7.0

v0.6.1

v0.6.0

v0.5.0

v0.4.0

v0.3.2

v0.3.1

v0.3.0

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1-beta2

v0.1-beta1

v0.1-alpha2

v0.1-alpha

Labels

Clear labels   

documentation

  

enhancement

  

enhancement

  

good first issue

  

pull-request

Mirrored from GitHub Pull Request

No labels

documentation

enhancement

enhancement

good first issue

pull-request

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynqmon#427

No description provided.