[GH-ISSUE #668] [FBref] requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url #133

New issue

Closed

opened 2026-03-02 15:56:04 +03:00 by kerem · 3 comments

kerem commented

2026-03-02 15:56:04 +03:00

Owner

Originally created by @URRealHero on GitHub (Aug 5, 2024).
Original GitHub issue: https://github.com/probberechts/soccerdata/issues/668

requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url
This error occurs when I'm trying to scrape team_match_stats and player_match_stats.
What should I do to avoid this

Originally created by @URRealHero on GitHub (Aug 5, 2024). Original GitHub issue: https://github.com/probberechts/soccerdata/issues/668 `requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url` This error occurs when I'm trying to scrape team_match_stats and player_match_stats. What should I do to avoid this

kerem

2026-03-02 15:56:04 +03:00

closed this issue
added the
bug

FBref
labels

kerem commented

2026-03-02 15:56:05 +03:00

Author

Owner

@probberechts commented on GitHub (Aug 15, 2024):

It looks like FBref has increased rate limiting from twenty to ten requests per minute recently (see https://www.sports-reference.com/bot-traffic.html).

You can increase the wait between requests as such:

import soccerdata as sd

fbref = sd.FBref("ESP-La Liga", "23-24")
fbref.rate_limit = 6   # max 1 request per 6 seconds

I will update the default.

@probberechts commented on GitHub (Aug 15, 2024): It looks like FBref has increased rate limiting from twenty to ten requests per minute recently (see https://www.sports-reference.com/bot-traffic.html). You can increase the wait between requests as such: ```python import soccerdata as sd fbref = sd.FBref("ESP-La Liga", "23-24") fbref.rate_limit = 6 # max 1 request per 6 seconds ``` I will update the default.

kerem commented

2026-03-02 15:56:06 +03:00

Author

Owner

@mhd0528 commented on GitHub (Aug 15, 2024):

The fix works! And just a suggestion when using the fbref class, if you need to scrape from multiple pages/leagues, try to initialize or check league schedule as less as you can. All the calls will first go to comps, which is usually the cause of the rate limit issue.

@mhd0528 commented on GitHub (Aug 15, 2024): The fix works! And just a suggestion when using the fbref class, if you need to scrape from multiple pages/leagues, try to initialize or check league schedule as less as you can. All the calls will first go to [comps](https://fbref.com/en/comps/), which is usually the cause of the rate limit issue.

kerem commented

2026-03-02 15:56:06 +03:00

Author

Owner

@URRealHero commented on GitHub (Aug 18, 2024):

THXXXXXX A Lot~

@URRealHero commented on GitHub (Aug 18, 2024): THXXXXXX A Lot~

kerem referenced this issue

2026-03-02 15:57:20 +03:00

[PR #133] [MERGED] Update actions/cache action to v3.2.2 #306