[PR #85] [MERGED] Faster scraping of player seasons stats - fbref. #269

New issue

Closed

opened 2026-03-02 15:57:09 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 15:57:09 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/probberechts/soccerdata/pull/85
Author: @andrewRowlinson
Created: 9/29/2022
Status: ✅ Merged
Merged: 10/23/2022
Merged by: @probberechts

Base: master ← Head: master

📝 Commits (10+)

75b717f Amended the FBRef scraper so it uses the Big 5 pages if all five leagues are requested.
b7d26a0 style changes black/flake8.
0b4a09c Add error raises to docstrings.
b525ebe Update actions/cache action to v3.0.9
fd297fb Update actions/cache action to v3.0.10
9dd5403 Update dependency furo to v2022.9.29
2c202f6 Update actions/checkout action to v3.1.0
07cadd7 Update dependency mypy to ^0.982
72e8cb7 Update actions/setup-python action to v4.3.0
63b57c3 Merge branch 'master' of https://github.com/andrewRowlinson/soccerdata into andrewRowlinson-big5

📊 Changes

8 files changed (+316 additions, -110 deletions)

View changed files

📝 .github/workflows/ci.yml (+5 -5)
📝 .github/workflows/release.yml (+2 -2)
📝 docs/requirements.txt (+1 -1)
📝 poetry.lock (+26 -26)
📝 pyproject.toml (+1 -1)
📝 soccerdata/fbref.py (+208 -69)
📝 tests/conftest.py (+6 -6)
📝 tests/test_FBref.py (+67 -0)

📄 Description

I am having another go at this [previous attempt #69] because you have updated the FBRef class to use the league pages. I have tried this against all stat_types for 2020-2021 and it seems to work

Amended the FBRef scraper so it uses the Big 5 pages if all five leagues are requested.
Added some type checks for the stats_type argument.

I am not able to run the tests locally, but I'll try to fix anything that doesn't work after.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/probberechts/soccerdata/pull/85 **Author:** [@andrewRowlinson](https://github.com/andrewRowlinson) **Created:** 9/29/2022 **Status:** ✅ Merged **Merged:** 10/23/2022 **Merged by:** [@probberechts](https://github.com/probberechts) **Base:** `master` ← **Head:** `master` --- ### 📝 Commits (10+) - [`75b717f`](https://github.com/probberechts/soccerdata/commit/75b717ff3dcb15c00e3b5fe89d57b54dfbd49009) Amended the FBRef scraper so it uses the Big 5 pages if all five leagues are requested. - [`b7d26a0`](https://github.com/probberechts/soccerdata/commit/b7d26a04196c8e48ff538c40f3eb23bccd60c8ac) style changes black/flake8. - [`0b4a09c`](https://github.com/probberechts/soccerdata/commit/0b4a09c51e32ec379d5e25c596b59b8590450106) Add error raises to docstrings. - [`b525ebe`](https://github.com/probberechts/soccerdata/commit/b525ebe7e499d090536da1c3642b653a92e5ef2c) Update actions/cache action to v3.0.9 - [`fd297fb`](https://github.com/probberechts/soccerdata/commit/fd297fbb36370fa4fa80bcaf8a18600eefcec5db) Update actions/cache action to v3.0.10 - [`9dd5403`](https://github.com/probberechts/soccerdata/commit/9dd540310a16f486e7678f45316ee78fb83f757f) Update dependency furo to v2022.9.29 - [`2c202f6`](https://github.com/probberechts/soccerdata/commit/2c202f696eb3a1cf9de743fa7d2cab20114d621e) Update actions/checkout action to v3.1.0 - [`07cadd7`](https://github.com/probberechts/soccerdata/commit/07cadd749ec01a8bad1527a14ca3cd0a8d4b29a1) Update dependency mypy to ^0.982 - [`72e8cb7`](https://github.com/probberechts/soccerdata/commit/72e8cb7603dcf33463116866409290d13eec285b) Update actions/setup-python action to v4.3.0 - [`63b57c3`](https://github.com/probberechts/soccerdata/commit/63b57c3eef94803e25a76435c285f13f6d809f08) Merge branch 'master' of https://github.com/andrewRowlinson/soccerdata into andrewRowlinson-big5 ### 📊 Changes **8 files changed** (+316 additions, -110 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/ci.yml` (+5 -5) 📝 `.github/workflows/release.yml` (+2 -2) 📝 `docs/requirements.txt` (+1 -1) 📝 `poetry.lock` (+26 -26) 📝 `pyproject.toml` (+1 -1) 📝 `soccerdata/fbref.py` (+208 -69) 📝 `tests/conftest.py` (+6 -6) 📝 `tests/test_FBref.py` (+67 -0) </details> ### 📄 Description I am having another go at this [previous attempt #69] because you have updated the FBRef class to use the league pages. I have tried this against all stat_types for 2020-2021 and it seems to work - Amended the FBRef scraper so it uses the Big 5 pages if all five leagues are requested. - Added some type checks for the stats_type argument. I am not able to run the tests locally, but I'll try to fix anything that doesn't work after. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>