mirror of
https://github.com/misiektoja/instagram_monitor.git
synced 2026-04-25 22:35:49 +03:00
[GH-ISSUE #17] Problem with followers json #15
Labels
No labels
Stale
Stale
bug
enhancement
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/instagram_monitor#15
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @tomballgithub on GitHub (Oct 17, 2025).
Original GitHub issue: https://github.com/misiektoja/instagram_monitor/issues/17
I noticed that sometimes a follower change happens, but the CSV, EMAIL, and console don't show the name(s) that changed.
While debugging, I noticed that within my current JSON includes a count of 178, but there are 180 names. I've seen it with some backup files also, but not all.
I am attempting to debug this with print statements to see what is going on.
Here's what I noticed so far:
@tomballgithub commented on GitHub (Oct 17, 2025):
I am seeing similar things with followings.
Here's an example from the log (4085 - 1 removed + 5 added = 4089 [not 4088])
One theory that fits this log, is that in the 36 mins the script was retrieving the 4000+ followings, another was added. So the count was 4088 at the start, but 4089 at the end?
Looking at the profile, the count is actually 4089
@tomballgithub commented on GitHub (Oct 17, 2025):
The CSV code writes the Followings before it does the fetch, so my theory makes sense. The same thing happens for the console printing, it uses the followings # before the fetching occurs (which takes a while in this case)
However, for the JSON, it reads all followers/followings, fetches the count a final time, and then writes to the JSON. So, the count should always match the current accurate #.
I am seeing this in the CSV, so there simply might be a bug in the code doing the compares, because there is no reason to believe these adds and drops are accurate
@misiektoja commented on GitHub (Oct 18, 2025):
Your observations are correct, but this behavior is intentional: the script relies on the follower/following counts reported on the profile page to detect changes, instead of fetching the full lists each time. Fetching full lists for large accounts (thousands of entries) is a very heavy operation and tends to trigger Instagram’s anti-bot protections, captchas or temporary blocks.
As a result, short-term mismatches (like when counts change mid-fetch) or cases where followers/followings swap, but the total count stays the same can appear in the logs or CSV. It's a tradeoff for stability and avoiding rate limits.
I don't currently have a better idea how to fix this or a cleaner way to reconcile the counts safely without causing heavier API usage or captchas. Any ideas or suggestions are definitely welcome. We could also introduce a flag to enforce an alternative mode that always fetches the full lists, but it would come with higher risk of triggering restrictions.
@tomballgithub commented on GitHub (Oct 18, 2025):
Understood. However, when there is a follower or following change, all of the names are refetched. So, comparing the old complete list to the new complete list should net out all the changes without discrepancy.
Still debugging, but last night my code caught an instance where instagram said the count was 4090 before the fetching started, it was 4090 after the fetching, but the actual count of names in the JSON was 4089. That shouldn't happen if Instagram is providing complete and accurate data when fetched.
@misiektoja commented on GitHub (Oct 18, 2025):
This is interesting, but look what I've found: https://stackoverflow.com/questions/75078174/instagram-follower-and-following-count-mismatch
It indicates this might be an issue on Instagram's side, most likely the follower/following count is served from a different cache/shard than the paginated edges and it can lag or lead the actual list by 1–2 (sometimes more) users for minutes or longer. So you can read 4090 twice while the edges you can enumerate add up to 4089.
We could implement some stabilization logic to reduce false positives, for example by reading the count two or three times a few seconds apart before deciding there's a change. However, even with that small mismatches can still happen due to caching delays, blocked users or accounts deactivated mid-fetch.
BTW, how long does it take to fetch 4k followers/followings ? I have never tried an account with such high number.
@tomballgithub commented on GitHub (Oct 19, 2025):
So today I captured the followers for an account where before the fetches the API returned 178 and after the fetches the API returned 178. The actual count of the # of accounts received was 181. I opened the users profile in the app and one-by-one verified that all 181 accounts were followers. But the app and web page still said 178, and both still show 178 over 8 hours later.
The previous article implied the difference could last up to 48 hours, but it very much seems like speculation. It's a theory but with no data to prove it.
@tomballgithub commented on GitHub (Oct 19, 2025):
Looks like ~36 minutes and exactly 360 requests to fetch 4080 followings.
@tomballgithub commented on GitHub (Oct 19, 2025):
So today, 24 hours later, the api, app, and website all now show 180 [was 178], but it still fetches 181
@misiektoja commented on GitHub (Oct 20, 2025):
I just checked with the Instaloader CLI client directly and confirmed that there are sometimes discrepancies between the profile counter and the actual list. It appears to be an Instagram inconsistency issue and the counter is no longer very accurate, so implementing a feature that uses the list lengths instead of the profile counter might be a good idea.
So instead of:
We use:
But that would require some slight changes in the logic to treat it as the only source for follower and following counters if the specific flag is used (some people might still prefer to use an older, less heavy and accurate approach).
@tomballgithub commented on GitHub (Oct 20, 2025):
At 5am this morning, the script noticed an update from 180 to 179 on followers. 181 were fetched. No changes through 8am when the script stopped due to the time constraints I have configured.
I just manually checked the app and web and they are both showing 181. I restarted the script and now it shows 181. All caught up
@tomballgithub commented on GitHub (Oct 20, 2025):
I've been mulling this over, and I'm not sure of a good answer, but here are some musings:
I'm thinking of adding a 'fetch X no more often than Y' type feature. In my case, I blocked off hours such that my 4000+ set should only get fetched 3-4 times a day maximum, but if that happened I think it would trigger protections. But I could just leave the window wide open, looking for changes, but fetching them at the next allowed time versus doing it immediately. I'd know immediately if something changed, but not exactly what. I've definitely seen a slowdown in getting accounts flagged proportional to how often an account's followings changes (in my 4000+ case). More updates = faster flagging. I have some debug code running to see exactly what specific command happens immediately before the problems start.
More later...
@tomballgithub commented on GitHub (Oct 22, 2025):
I tried adding a check of profile.get_followers().count every hour, but it reports the same things as API, even when the JSON has a greater number of accounts. Saw follower count drop from 181 to 180 last night, everything reports 180, but JSON has 181
@misiektoja commented on GitHub (Oct 23, 2025):
Can you compare your raw dump list with the JSON contents? Maybe it will give you a clue about what this additional entry is and whether it is really on the followers list or not.
@tomballgithub commented on GitHub (Oct 23, 2025):
The raw dump matches the JSON. The count provided by Instagram (regardless of method) does not
@misiektoja commented on GitHub (Oct 23, 2025):
Ok, thanks, this clarifies the topic. So in fact, we cannot really rely on the profile counter anymore ...
@tomballgithub commented on GitHub (Oct 23, 2025):
Right, it's not a code problem. The reason the follower and following JSON have a count that is different than the number of accounts actually in the JSON is due what Instagram is returning.
So now it's more of a "what, if anything, could be done about it"?
@tomballgithub commented on GitHub (Oct 23, 2025):
I am monitoring with debug information to see what happens over time (is there a pattern to the madness?). Unfortunately, the account I am monitoring hasn't had many changes lately.
One thing I've noticed is that all of the 'unfollows' I see are from accounts that 'go away' (deleted). They are not from an account actually 'unfollowing'. Could those make the count off versus a true unfollow might not?
@misiektoja commented on GitHub (Oct 23, 2025):
From an implementation standpoint, the easiest thing would be to fully switch to using the list length and just forget about the counter on the profile.
But the sexy part of our previous approach was that we only fetched counters with each loop and only grabbed the list when there was an actual change in the number. The downside was we missed any changes in the actual list, like if a follower/following changed their username or the monitored user unfollowed someone and then quickly followed someone else. It was pretty basic, but we managed to dodge those heavy list-fetching requests. Instagram really doesn't like doing that too often.
I need to run a few tests on my end, maybe I'll come up with some idea.
What you described about those deleted accounts might be the root of all that craziness, but only the Instagram engineers would know for sure. They definitely have an issue on their end and maybe they'll eventually fix it?
@tomballgithub commented on GitHub (Oct 23, 2025):
I will do some experiments with accounts over the weekend to see if I can induce the behavior. I was hopeful that using profile.get_followers().count would work, because it actually does a single fetch of accounts (but not all) before returning the count. But it actually matches the API value as opposed to len(profile.get_followers()).
For clarity:
profile.get_followers().count != len(profile.get_followers())@misiektoja commented on GitHub (Oct 23, 2025):
Yes, it is because profile.get_followers().count returns the follower count reported by Instagram’s API (a single value - the same you actually see on the profile), while len(profile.get_followers()) actually iterates through the generator of follower objects returned by get_followers() and counts how many were fetched.
Let me know how your tests go. I'll play with it too and let's see if we come up with an idea.
@tomballgithub commented on GitHub (Oct 23, 2025):
It's not clear in the Instaloader docs that profile.get_followers().count = API count. It is part of the iterator, so I expected it to match the iterator size.
Profile.followers is definitely supposed to be the count you see on the profile and doesn't require additional fetching.
@misiektoja commented on GitHub (Oct 23, 2025):
Yeah, I fully agree. I couldn't find a clear explanation in the Instaloader docs either. It's kinda vague, but based on what you wrote it is what I suspect.
@tomballgithub commented on GitHub (Oct 23, 2025):
Well, look what happened overnight. I'll look at the logs later. API says 181/4090 (oddly the app said 173 until I refreshed it)
These should have matched based on what we just discussed...
Before this, they did match (180/4089)
@tomballgithub commented on GitHub (Oct 23, 2025):
Strange...
At (Thu, 23 Oct 2025, 11:39:56):
Restarted the script...
At (Thu, 23 Oct 2025, 11:56:57):
So either the underlying data from Instagram updated in that 15 minute span (after being that way for hours), or something about restarting the script caused updated data to be provided by Instagram
@tomballgithub commented on GitHub (Oct 23, 2025):
I just realized that a situation that can generate a -1 dropped account and a +1 added account is if an account changes its name.
@tomballgithub commented on GitHub (Oct 23, 2025):
And this:
Also turned into this after the restart:
BUT the actual count -> 4089
Although there was a +6 and -6 on the accounts names.
@tomballgithub commented on GitHub (Oct 24, 2025):
I've noticed it twice now. There's a mismatch in counts and once I restart, profile.followers has the correct #. Is there something unique occurring at startup versus during profile refreshes while looping?
after restart becomes
@misiektoja commented on GitHub (Oct 25, 2025):
It seems to be a caching issue. Could you set no-cache headers by inserting the following after
session = ctx._session:Let me know if it helped out in any way.
@tomballgithub commented on GitHub (Oct 25, 2025):
Testing now
@tomballgithub commented on GitHub (Oct 25, 2025):
The no-cache change didn't fix it. After 1.5 hours, got this.
There's a fundamental difference between this tool and instaloader. This runs continuously, while instaloader is meant to be run periodically via a script. I wonder why profile.followees gets stuck?
after > 2 hrs, still the same, restarted, then got
Here's all I am doing after every sleep (1.x hours) while ignoring any time periods chosen:
@tomballgithub commented on GitHub (Oct 25, 2025):
I did notice that the very first account fetched was new, so I just threw it into the JSON and restarted to avoid refetching all 4093.
That's not foolproof, as there could have been two added and one removed, BUT how often do people really unfollow? So, if we set that aside as a don't care, then if you are looking for X new accounts, and see them right at the start, you can stop fetching the rest. Any other +/- can be attributed to accounts changing names, accounts disappearing, or unfollows, which I am willing to postpone finding out about, as opposed to having my account get flagged as suspicious.
@tomballgithub commented on GitHub (Oct 26, 2025):
With respect to the count mismatch happening, I think the issue is that I wasn't doing the following before each of my various debug statement paths. There's still a count problem between instagram and JSON, but I think this explains why I saw differences with the two count variations I was printing for debug.
profile = instaloader.Profile.from_username(bot.context, user)@tomballgithub commented on GitHub (Nov 1, 2025):
Over a 10-day period, I saw 9 changes to the API count, but the actual count was '181', and there was never an add/drop found within that 181. Then the actual count went up to 182 then 183, but API count is at 182. The app and web page always seem to match the API count.
There seems to be no pattern to why the actual count stays consistent while the reported count changes.
@misiektoja commented on GitHub (Nov 11, 2025):
There is a new Instaloader package released a few days ago. I have not looked into the release notes, but perhaps they made some changes related to this.
@github-actions[bot] commented on GitHub (Dec 12, 2025):
This issue has been marked as stale because it has been inactive for 30 days. It will be closed in 7 days if no further activity occurs.
@tomballgithub commented on GitHub (Dec 18, 2025):
The new instaloader release didn't have any documented fixes for this.
Keeping this open. At a minimum, I will create a PR with some proposed messaging over the holidays
@tomballgithub commented on GitHub (Jan 1, 2026):
I added PR #27 to provide clarity as to the count of names that Instagram returns AND the actual number in the JSON
@tomballgithub commented on GitHub (Jan 1, 2026):
I have been running this script for months, with everything off except initial login, updating profile, fetching changes to followers/followings within certain hour windows, and I haven't seen a ban.
So, my initial concern about taking 45 mins to download 4000+ followings doesn't seem to be founded in reality. I'll add that while adjusting code, I have restarted the script in the middle of fetching followers/followings, and in general, at least 100 times.
It looks like Instagram is not detecting patterns for logging in, profile updates, and getting followers/followings. Or at least I am not triggering it. It does make sense that downloading media would be a priority for pattern detection.
@tomballgithub commented on GitHub (Jan 2, 2026):
here is data from a month of scans, for posterity:
@tomballgithub commented on GitHub (Jan 2, 2026):
@misiektoja
I don't see that there is anything else to do on this. There simply isn't a foolproof scheme.
However, I think you should add a quick check to not send emails for change in # of followers/followings if the list of names is blank.
Example email, which made me start looking at this topic in the first place:
@misiektoja commented on GitHub (Jan 3, 2026):
I haven't implemented it yet because the tool can report changes in the number of followers or followings without actually obtaining the list (for example in anonymous mode, even for private profiles). I don't have a clear idea of how to handle this at the moment.
@tomballgithub commented on GitHub (Jan 3, 2026):
Well how about for the modes that do fetch the list, mask these emails or have an option to mask them. For the others, since there is no list, it's a don't care, because the emails always don't have any names.
@tomballgithub commented on GitHub (Jan 6, 2026):
PR #36 has an interesting solution for this, which is to fetch all followers/followees every time, ignoring the # from Instagram. I never considered that as a fix, to avoid getting accounts flagged, but given my recent comment above, that I never saw any type of issue over multiple months of testing leads me to believe that this approach might be OK.
@misiektoja commented on GitHub (Jan 6, 2026):
Yep, just getting comfy and ready to dive into this massive PR... wish me luck! ;-)
@misiektoja commented on GitHub (Jan 16, 2026):
I tested the new
--detailed-followers/DETAILED_FOLLOWER_LOGGINGfeature and I think after todays fixes (5aaac6d) it should solve the issue. Please check and let me know (everything loaded into dev branch). I also added warning if there are more than 5k followers/followings to be tracked (0a8fabf).@tomballgithub commented on GitHub (Jan 17, 2026):
I created new issues with my feedback
@misiektoja commented on GitHub (Jan 18, 2026):
Thanks a bunch for the valuable feedback! There are quite a few remarks in there and I really appreciate it!
I'm on a business trip until Wednesday, but once I return I'll jump right on it.
@tomballgithub commented on GitHub (Jan 20, 2026):
Well, it could simply be coincidence, but my instagram account has been running since 10/26 and was just flagged today 1/19 after running the new DETAILED_FOLLOWER_LOGGING and other updates for < for just a few days.
I logged back into Firefox, re-saved the session, and it's working again
- loading own profile... * Error: ConnectionException: JSON Query to graphql/query: 401 Unauthorized - "fail" status, message "Please wait a few minutes before you try again." when accessing https://www.instagram.com/graphql/query?query_hash=d6f4427fbe92d846298cf93df0b937d3&variables=%7B%7D@tomballgithub commented on GitHub (Jan 21, 2026):
Account keeps getting flagged. I restarted with a new account with DETAILED_FOLLOWER_LOGGING disabled.
If is shows longevity, I'll reenable DETAILED_FOLLOWER_LOGGING and see what happens
@misiektoja commented on GitHub (Jan 22, 2026):
Check the new code, I think I fixed all the issues you reported.
I also improved the logic for detailed follower logging and renamed it to
FOLLOWERS_CHURN_DETECTION/--followers-churnas its main purpose is to fetch the full list of followers and followings every check (not just when the count changes) and to compare usernames to detect who followed/unfollowed even when counts remain the same (churn detection).Let me know your feedback!