mirror of
https://github.com/ArchiveBox/ArchiveBox.git
synced 2026-04-25 17:16:00 +03:00
[GH-ISSUE #1349] Support: Multiple binaries present from brew and pip installs + difficulty running on macOS #3845
Labels
No labels
expected: maybe someday
expected: next release
expected: release after next
expected: unlikely unless contributed
good first ticket
help wanted
pull-request
scope: all users
scope: windows users
size: easy
size: hard
size: medium
size: medium
status: backlog
status: blocked
status: done
status: idea-phase
status: needs followup
status: wip
status: wontfix
touches: API/CLI/Spec
touches: configuration
touches: data/schema/architecture
touches: dependencies/packaging
touches: docs
touches: js
touches: views/replayers/html/css
why: correctness
why: functionality
why: performance
why: security
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/ArchiveBox#3845
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @porkchoppie on GitHub (Feb 18, 2024).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1349
$ sudo find / -name config.py -print | grep archivebox
/usr/local/lib/python3.11/site-packages/archivebox/config.py
/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/IPython/core/magics/config.py
/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/matplotlib_inline/config.py
/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/django/apps/config.py
/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/archivebox/config.py
/System/Volumes/Data/usr/local/lib/python3.11/site-packages/archivebox/config.py
/System/Volumes/Data/Users/submerger/Library/Python/3.9/lib/python/site-packages/archivebox/config.py
$ which archivebox
/usr/local/bin/archivebox
I am on a mac, and used brew to install. What to I remove (if anything) and which config file do I update?
@pirate commented on GitHub (Feb 18, 2024):
Why are you trying to find
config.py? You should never need to directly access that file.ArchiveBox's exposed config file lives in your data folder (wherever you create it) e.g.
~/archivebox/dataand is calledArchiveBox.conf.No need to remove anything just makes sure you have run
pip install --ignore-installed archivebox. All the setup instructions are in the README & Wiki too.@porkchoppie commented on GitHub (Feb 18, 2024):
Simply because it’s not where it is supposed to be. when I did the install, I created directories as indicated in the instructions, but there’s no sign of a configuration file except for configuration.conf or something like that with some environment variables indicated, but nothing else. If I should be creating a new configuration file from scratch, that’s fine if you should be using this one which is no mention of this file away with extension in the documentation.
As such I looked for where was it installed of my machine and I found three different binaries and three different locations and found their Python Configuration files I wasn’t looking to edit the python file mind you but if I looked at it, it should tell me where I can find it’s getting its settings from.. Either they’re all pointing to the location that they’re supposed to or they’re pointing to different locations.
I’m at the point now. I might uninstall everything and start from scratch because it’s a mess. Never a good sign when you have it installed in several different locations. But I was hoping to save time and energy and salvage what I could get it up and running.
if you also think it’s a good idea to remove some of the binaries and their directories. Let me know and I will take a look.
On Sun, Feb 18, 2024 at 2:54 AM, Nick Sweeting @.***(mailto:On Sun, Feb 18, 2024 at 2:54 AM, Nick Sweeting < wrote:
@pirate commented on GitHub (Feb 19, 2024):
Ok, here's a breakdown of the paths you're seeing above:
/usr/local/lib/python3.11/site-packages/archivebox/config.pyindicates you have
pip-installed archivebox (its ok but not required to have bothpipandbrew-installedarchiveboxversions aspip's ArchiveBox version is often newer, butbrewprovides auto installation of all the system dependencies thatpipcant install automatically)/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/IPython/core/magics/config.pyignore this, it's a sub-dependency of archivebox and is not archivebox's own
config.py/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/matplotlib_inline/config.pyignore this, it's a sub-dependency of archivebox and is not archivebox's own
config.py/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/django/apps/config.pyignore this, it's a sub-dependency of archivebox and is not archivebox's own
config.py/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/archivebox/config.pythis is the correct original source location of
config.pyinstalled by homebrew/System/Volumes/Data/usr/local/lib/python3.11/site-packages/archivebox/config.pythis is an artifact of how macOS creates a Data volume for user data causing paths to show up twice, it's not a real duplicate
/System/Volumes/Data/Users/submerger/Library/Python/3.9/lib/python/site-packages/archivebox/config.pysame deal, this is a macOS artifact and not a real duplicate
In summary, your
pip-installedconfig.pyis here:/usr/local/lib/python3.11/site-packages/archivebox/config.py(you can always find pip's source paths with
pip show -f archiveboxbtw)and your
brew-installedconfig.pyis here:/usr/local/Cellar/archivebox/0.7.1-1/libexec/lib/python3.11/site-packages/archivebox/config.py(it's fine for this to shadow the
pip-installed version,pip's will take precedence if you have both and they don't conflict even if both are in your$PATH)I don't think you need to salvage anything, this looks like a normal install on macOS with
brew+pip(as the Wiki/README docs used to recommend).However if you like, I've improved the
brewrelease +brewinstall docs somewhat since you installed this, you can reset and get the newer version with:To configure archivebox, edit
ArchiveBox.confinside your data folder, for example:/usr/local/var/archivebox/data~/archivebox/data/Volumes/USB-DRIVE/archivebox/dataThe
data/ArchiveBox.conffile starts empty by default, only settings you actually configure get added.The full list of options you can set is in the Configuration wiki.
More info on how to configure archivebox is here: https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#run-archivebox-with-configuration-options
Both those pages document how you can edit ArchiveBox.conf, use the interactive
archivebox configCLI, or use environment variables, and show that all 3 are equivalent.The docs have changes here too since you probably last checked: https://github.com/ArchiveBox/ArchiveBox/wiki/Install#macos
@porkchoppie commented on GitHub (Mar 1, 2024):
Apologies as i went in to work through your feedback a week or so ago. Before I did anything, i went ahead and updated brew and for some reason is wacky. Likely a path, env issue when using zsh, but of course mac os, has different ways to set up paths, and its different then say ubuntu, or debian. Anyhoo, at the end of the update brew completely vomited.
Error: Thebrew link` step did not complete successfullyThe formula built, but is not symlinked into /usr/local
Could not symlink bin/archivebox
Target /usr/local/bin/archivebox
already exists. You may want to remove it:
rm '/usr/local/bin/archivebox'
To force the link and overwrite all conflicting files:
brew link --overwrite archivebox
To list all files that would be deleted:
brew link --overwrite archivebox --dry-run
Possible conflicting files are:
/usr/local/bin/archivebox
/usr/local/bin/yt-dlp
==> Downloading https://formulae.brew.sh/api/formula.jws.json
#=#=-# #
==> rm -Rf /usr/local/var/archivebox/data/node_modules
==> ln -sf /usr/local/Cellar/archivebox/0.7.2-1/libexec/lib/node_modules/archivebox/node_modules /usr/local/var/archivebox/data/
==> /usr/local/bin/archivebox init
==> /usr/local/bin/archivebox setup
==> Caveats
To start archivebox/archivebox/archivebox now and restart at login:
brew services start archivebox/archivebox/archivebox
Or, if you don't want/need a background service you can just run:
/usr/local/opt/archivebox/bin/archivebox server --quick-init 0.0.0.0:8000
==> Summary
🍺 /usr/local/Cellar/archivebox/0.7.2-1: 20,661 files, 254.2MB, built in 8 minutes 8 seconds
==> Running
brew cleanup archivebox...Removing: /usr/local/Cellar/archivebox/0.7.1-1... (10,231 files, 65.2MB)
Removing: /Users/submerger/Library/Caches/Homebrew/archivebox--0.7.1-1.tar.gz... (436.3KB)
Removing: /Users/submerger/Library/Caches/Homebrew/archivebox--yt-dlp--2023.10.13.tar.gz... (2.7MB)
==> Caveats
==> archivebox
To start archivebox/archivebox/archivebox now and restart at login:
brew services start archivebox/archivebox/archivebox
Or, if you don't want/need a background service you can just run:
/usr/local/opt/archivebox/bin/archivebox server --quick-init 0.0.0.0:8000`
I have no idea which rabbit hole to go down, and decided to pause until I had some downtime. Any more feedback you have would be great, and I could use it.
Best.
@pirate commented on GitHub (Mar 2, 2024):
Don't worry this error is a very common one with
brew&pipin general, and the fix is simple. Brew is basically complaining that you've already installedyt-dlpandarchiveboxwithpip, and it doesn't want to overwritepip's version. You can fix it like so:The reason we install
pipagain afterbrewis that thebrewchannel is unfortunately outdated sometimes because it's difficult to package forhomebrewand I don't always update it as fast.However, brew still provides a lot of the base system packages that
pipcan't install on its own: likepython3,ripgrep,curl,wget,sonicetc.Installing both is good way to make sure you have all the system packages needed for your OS, but also have the latest
archiveboxcodebase version frompip. You just need to make sure thepipversion is always the one installed last (so it overwritesbrew's).They don't conflict once they're installed like the instructions above show, the newer
piparchiveboxversion takes precedence, and thebrewarchiveboxversion stays installed but unlinked, still providing the needed system packages.Hope that helps clear things up.
@porkchoppie commented on GitHub (Mar 3, 2024):
Cool. So I went through all the recommended steps, and I have comments/questions "## inline".
@pirate commented on GitHub (Mar 7, 2024):
archivebox/archivebox/archiveboxis not the folder structure, it's the<brew tapname>/<brew packagename>. The tap isarchivebox/archiveboxand the package name isarchiveboxso you see it three times, this is normal for brew packages distributed via taps.Packages are installed into Cellar, and binaries are symlinked out into your
/usr/local/bindirectory after install is complete in a separate linking step. Because the instructions you're looking at execute during the final install step, the binary is not guaranteed to be linked yet (e.g. you could decline linking or it or linking could've been blocked by a conflicting file). So it shows the absolute path to the original binary/usr/local/opt/archivebox/bin/archiveboxin the those particular instructions in case you don't have it linked.It's not in the same dir twice, you just have the same dir twice in your
$PATH. The binary is only present in that dir once. It's harmless you can ignore the duplicated output, or remove the duplicate$PATHentry in your~/.zshrcor~/.bash_profileif it bothers you.No it's not required, it's only needed if you're archiving w/ a login session to preserve some private content that needs cookies to view.
Up to you, you don't necessarily need it but it's a nice backup if
readabilityfails or produces bad output on some URLs.It'll run fine without it, or you can try to force install it globally with: