[GH-ISSUE #58] Improve model download speed, progress display, etc

kerem commented

2026-03-03 13:52:22 +03:00

Owner

Originally created by @0xdevalias on GitHub (Aug 26, 2024).
Original GitHub issue: https://github.com/jehna/humanify/issues/58

I was reading the node-llama-cpp docs, and they mention that the ipull package can be useful for improved model download speeds:

https://withcatai.github.io/node-llama-cpp/guide/#getting-a-model-file
- For improved download speeds, you can use ipull to download the model
https://github.com/ido-pluto/ipull
- iPull
  Super fast file downloader with multiple connections
- Features
  - Download using parallels connections
  - Pausing and resuming downloads
  - Node.js and browser support
  - Smart retry on fail
  - CLI Progress bar
  - Download statistics (speed, time left, etc.)

I can see that the current download command calls downloadModel:

github.com/jehna/humanify@14e4ae5365/src/commands/download.ts (L1-L6)

Which is defined here, and seems to just use fetch currently, as well as implementing its own download progress tracking in showProgress:

github.com/jehna/humanify@14e4ae5365/src/local-models.ts (L38-L65)

github.com/jehna/humanify@14e4ae5365/src/progress.ts (L4-L13)

I wonder if using iPull might make sense, both in increased download speed, as well as better download progress visibility/etc.

Originally created by @0xdevalias on GitHub (Aug 26, 2024). Original GitHub issue: https://github.com/jehna/humanify/issues/58 I was reading the `node-llama-cpp` docs, and they mention that the `ipull` package can be useful for improved model download speeds: - https://withcatai.github.io/node-llama-cpp/guide/#getting-a-model-file - > For improved download speeds, you can use [ipull](https://www.npmjs.com/package/ipull) to download the model - https://github.com/ido-pluto/ipull - > iPull > Super fast file downloader with multiple connections - > Features > - Download using parallels connections > - Pausing and resuming downloads > - Node.js and browser support > - Smart retry on fail > - CLI Progress bar > - Download statistics (speed, time left, etc.) --- I can see that the current `download` command calls `downloadModel`: https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/commands/download.ts#L1-L6 Which is defined here, and seems to just use `fetch` currently, as well as implementing its own download progress tracking in `showProgress`: https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/local-models.ts#L38-L65 https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/progress.ts#L4-L13 --- I wonder if using iPull might make sense, both in increased download speed, as well as better download progress visibility/etc.

kerem closed this issue

2026-03-03 13:52:22 +03:00

kerem commented

2026-03-03 13:52:22 +03:00

Author

Owner

@0xdevalias commented on GitHub (Aug 26, 2024):

Partially related context:

I can see that the instructions are in the README here:

https://github.com/jehna/humanify#local-mode

Which suggests I need to run humanify download 2b first.

I wonder if it might make more sense to have the local model download as a sub-command of humanify local, as that's where I was first looking for help for how to download the models, and it didn't even occur to me to check the root level command, since things local things seemed to be 'scoped' under the local command:
⇒ npx humanifyjs local -h
(node:97623) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Usage: humanify local [options] <input>

Use a local LLM to unminify code

Arguments:
  input                     The input minified Javascript file

Options:
  -m, --model <model>       The model to use (default: "2b")
  -o, --outputDir <output>  The output directory (default: "output")
  -s, --seed <seed>         Seed for the model to get reproduceable results (leave out for random seed)
  --disableGpu              Disable GPU acceleration
  --verbose                 Show verbose output
  -h, --help                display help for command
There also seems to be very minimal information output during the download. It might be nice to know a bit more about which model is being downloaded, from where, where it's being saved, how large it is, etc:
 ⇒ npx humanifyjs download 2b
(node:97932) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Downloaded 1.63 GB
I guess it does provide slightly more info when the download is completed:
⇒ npx humanifyjs download 2b
(node:97932) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
                  Model "2b" downloaded to /Users/devalias/.humanifyjs/models/Phi-3.1-mini-4k-instruct-Q4_K_M.gguf
I can see it's downloaded here:
⇒ ls ~/.humanifyjs/models
Phi-3.1-mini-4k-instruct-Q4_K_M.gguf
And the code for that is here:

github.com/jehna/humanify@85d17e73d6/src/local-models.ts (L13-L25)

I also notice that MODEL_DIRECTORY is hardcoded currently. I wonder if that would be something useful to be able to specify/customize via a CLI arg/env variable/etc.

It seems the humanify local command uses getModelPath:

github.com/jehna/humanify@85d17e73d6/src/plugins/local-llm-rename/llama.ts (L19-L22)

Which only seems to work for model aliases defined in MODELS:

github.com/jehna/humanify@85d17e73d6/src/local-models.ts (L69-L75)

Even though the error text for humanify download sounds as though it would be capable of downloading any named model:

github.com/jehna/humanify@85d17e73d6/src/local-models.ts (L77-L85)

And usually for LLM apps, the --model param would let us specify arbitrary models from huggingface or similar.

Originally posted by @0xdevalias in https://github.com/jehna/humanify/issues/53#issuecomment-2306107630

@0xdevalias commented on GitHub (Aug 26, 2024): Partially related context: > I can see that the instructions are in the README here: > > - https://github.com/jehna/humanify#local-mode > > Which suggests I need to run `humanify download 2b` first. > > I wonder if it might make more sense to have the local model download as a sub-command of `humanify local`, as that's where I was first looking for help for how to download the models, and it didn't even occur to me to check the root level command, since things local things seemed to be 'scoped' under the `local` command: > > ``` > ⇒ npx humanifyjs local -h > (node:97623) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. > (Use `node --trace-deprecation ...` to show where the warning was created) > Usage: humanify local [options] <input> > > Use a local LLM to unminify code > > Arguments: > input The input minified Javascript file > > Options: > -m, --model <model> The model to use (default: "2b") > -o, --outputDir <output> The output directory (default: "output") > -s, --seed <seed> Seed for the model to get reproduceable results (leave out for random seed) > --disableGpu Disable GPU acceleration > --verbose Show verbose output > -h, --help display help for command > ``` > > There also seems to be very minimal information output during the download. It might be nice to know a bit more about which model is being downloaded, from where, where it's being saved, how large it is, etc: > > ```shell > ⇒ npx humanifyjs download 2b > (node:97932) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. > (Use `node --trace-deprecation ...` to show where the warning was created) > Downloaded 1.63 GB > ``` > > I guess it does provide slightly more info when the download is completed: > > ```shell > ⇒ npx humanifyjs download 2b > (node:97932) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. > (Use `node --trace-deprecation ...` to show where the warning was created) > Model "2b" downloaded to /Users/devalias/.humanifyjs/models/Phi-3.1-mini-4k-instruct-Q4_K_M.gguf > ``` > > I can see it's downloaded here: > > ``` > ⇒ ls ~/.humanifyjs/models > Phi-3.1-mini-4k-instruct-Q4_K_M.gguf > ``` > > And the code for that is here: > > https://github.com/jehna/humanify/blob/85d17e73d6b760bf896d01dd99fa3a5f98ea2848/src/local-models.ts#L13-L25 > > I also notice that `MODEL_DIRECTORY` is hardcoded currently. I wonder if that would be something useful to be able to specify/customize via a CLI arg/env variable/etc. > > It seems the `humanify local` command uses `getModelPath`: > > https://github.com/jehna/humanify/blob/85d17e73d6b760bf896d01dd99fa3a5f98ea2848/src/plugins/local-llm-rename/llama.ts#L19-L22 > > Which only seems to work for model aliases defined in `MODELS`: > > https://github.com/jehna/humanify/blob/85d17e73d6b760bf896d01dd99fa3a5f98ea2848/src/local-models.ts#L69-L75 > > Even though the error text for `humanify download` sounds as though it would be capable of downloading any named model: > > https://github.com/jehna/humanify/blob/85d17e73d6b760bf896d01dd99fa3a5f98ea2848/src/local-models.ts#L77-L85 > > And usually for LLM apps, the `--model` param would let us specify arbitrary models from huggingface or similar. > > _Originally posted by @0xdevalias in https://github.com/jehna/humanify/issues/53#issuecomment-2306107630_

kerem commented

2026-03-03 13:52:23 +03:00

Author

Owner

@neoOpus commented on GitHub (Sep 4, 2024):

I ve been downloading the 8b for almost an hour now... So yeah any improvement in this regard will be great. Or maybe allow to just download the models then place them (I don't know where the model will be located as I didn't analyse the source code yet) But having them shared with other softwares like Ollama would be great (Symbolic link)

@neoOpus commented on GitHub (Sep 4, 2024): I ve been downloading the 8b for almost an hour now... So yeah any improvement in this regard will be great. Or maybe allow to just download the models then place them (I don't know where the model will be located as I didn't analyse the source code yet) But having them shared with other softwares like Ollama would be great (Symbolic link)

kerem commented

2026-03-03 13:52:23 +03:00

Author

Owner

@0xdevalias commented on GitHub (Sep 12, 2024):

I don't know where the model will be located as I didn't analyse the source code yet

~/.humanifyjs/models

github.com/jehna/humanify@14e4ae5365/src/local-models.ts (L13-L25)

@0xdevalias commented on GitHub (Sep 12, 2024): > I don't know where the model will be located as I didn't analyse the source code yet `~/.humanifyjs/models` https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/local-models.ts#L13-L25

kerem commented

2026-03-03 13:52:23 +03:00

Author

Owner

@neoOpus commented on GitHub (Sep 12, 2024):

I don't know where the model will be located as I didn't analyse the source code yet

~/.humanifyjs/models

github.com/jehna/humanify@14e4ae5365/src/local-models.ts (L13-L25)

Yeah I figured that out and updated to Phi 3.5 but the PR shows an error (unrelated I guess as many dependabot PRs are rejected as well)

My mistake (I am extremely exhausted lately and I cannot focus)... Still I want to make some progress with this and create a workflow that would allow me to puruse some new venues in the near future by learning from some extensions how they operate internally... alter some and analyse some other for malwares.

This is the error of the PR, I thought it would be a drop from 3.1 to 3.5 but I guess I have to learn more about the difference between the tokenization of both.

# [2024-09-12 04:24:09]  Loading model with options {
#   modelPath: '/Users/runner/.humanifyjs/models/Phi-3.5-mini-instruct-Q4_K_M.gguf',
#   gpuLayers: 0
# }
# [node-llama-cpp] Using this model ("~/.humanifyjs/models/Phi-3.5-mini-instruct-Q4_K_M.gguf") to tokenize text with special tokens and then detokenize it resulted in a different text. There might be an issue with the model or the tokenizer implementation. Using this model may not work as intended
# Subtest: /Users/runner/work/humanify/humanify/src/test/e2e.geminitest.ts
not ok 1 - /Users/runner/work/humanify/humanify/src/test/e2e.geminitest.ts
  ---

@neoOpus commented on GitHub (Sep 12, 2024): > > I don't know where the model will be located as I didn't analyse the source code yet > > `~/.humanifyjs/models` > > https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/local-models.ts#L13-L25 Yeah I figured that out and updated to Phi 3.5 but the PR shows an error (unrelated I guess as many dependabot PRs are rejected as well) My mistake (I am extremely exhausted lately and I cannot focus)... Still I want to make some progress with this and create a workflow that would allow me to puruse some new venues in the near future by learning from some extensions how they operate internally... alter some and analyse some other for malwares. This is the error of the PR, I thought it would be a drop from 3.1 to 3.5 but I guess I have to learn more about the difference between the tokenization of both. ``` # [2024-09-12 04:24:09] Loading model with options { # modelPath: '/Users/runner/.humanifyjs/models/Phi-3.5-mini-instruct-Q4_K_M.gguf', # gpuLayers: 0 # } # [node-llama-cpp] Using this model ("~/.humanifyjs/models/Phi-3.5-mini-instruct-Q4_K_M.gguf") to tokenize text with special tokens and then detokenize it resulted in a different text. There might be an issue with the model or the tokenizer implementation. Using this model may not work as intended # Subtest: /Users/runner/work/humanify/humanify/src/test/e2e.geminitest.ts not ok 1 - /Users/runner/work/humanify/humanify/src/test/e2e.geminitest.ts --- ```

kerem commented

2026-03-03 13:52:23 +03:00

Author

Owner

@0xdevalias commented on GitHub (Sep 30, 2024):

Some more relevant links/functions/etc that could be used here:

https://node-llama-cpp.withcat.ai/guide/downloading-models
- Downloading Models
  node-llama-cpp is equipped with solutions to download models to use them in your project.
- https://node-llama-cpp.withcat.ai/guide/downloading-models#cli
  - Using the CLI
    node-llama-cpp is equipped with a model downloader you can use to download models and their related files easily and at high speed (using ipull).
    
    It's recommended to add a models:pull script to your package.json to download all the models used by your project to a local models folder.
    
    It's also recommended to ensure all the models are automatically downloaded after running npm install by setting up a postinstall script
- https://node-llama-cpp.withcat.ai/guide/downloading-models#programmatic
  - Programmatically Downloading Models
    You can also download models programmatically using the createModelDownloader method, and combineModelDownloaders to combine multiple model downloaders.
    
    This option is recommended for more advanced use cases, such as downloading models based on user input.
  - https://node-llama-cpp.withcat.ai/api/functions/createModelDownloader
    - Function: createModelDownloader()
      Create a model downloader to download a model from a URL. Uses ipull to download a model file as fast as possible with parallel connections and other optimizations.
  - https://node-llama-cpp.withcat.ai/api/functions/combineModelDownloaders
    - Function: combineModelDownloaders()
      Combine multiple models downloaders to a single downloader to download everything using as much parallelism as possible.
      
      You can check each individual model downloader for its download progress, but only the onProgress passed to the combined downloader will be called during the download.
- https://node-llama-cpp.withcat.ai/guide/downloading-models#inspecting-remote-models
  - Inspecting Remote Models
    You can inspect the metadata of a remote model without downloading it by either using the inspect gguf command with a URL, or using the readGgufFileInfo method with a URL
- https://node-llama-cpp.withcat.ai/guide/downloading-models#detecting-the-compatibility-of-remote-models
  - Detecting the Compatibility of Remote Models
    It's handy to check the compatibility of a remote model with your current machine hardware before downloading it, so you won't waste time downloading a model that won't work on your machine.
    
    You can do so using the inspect estimate command with a URL

@0xdevalias commented on GitHub (Sep 30, 2024): Some more relevant links/functions/etc that could be used here: - https://node-llama-cpp.withcat.ai/guide/downloading-models - > **Downloading Models** > `node-llama-cpp` is equipped with solutions to download models to use them in your project. - https://node-llama-cpp.withcat.ai/guide/downloading-models#cli - > **Using the CLI** > `node-llama-cpp` is equipped with a [model downloader](https://node-llama-cpp.withcat.ai/cli/pull) you can use to download models and [their related files](https://node-llama-cpp.withcat.ai/api/functions/createModelDownloader) easily and at high speed (using [ipull](https://www.npmjs.com/package/ipull)). > > It's recommended to add a `models:pull` script to your `package.json` to download all the models used by your project to a local `models` folder. > > It's also recommended to ensure all the models are automatically downloaded after running `npm install` by setting up a `postinstall` script - https://node-llama-cpp.withcat.ai/guide/downloading-models#programmatic - > **Programmatically Downloading Models** > You can also download models programmatically using the `createModelDownloader` method, and `combineModelDownloaders` to combine multiple model downloaders. > > This option is recommended for more advanced use cases, such as downloading models based on user input. - https://node-llama-cpp.withcat.ai/api/functions/createModelDownloader - > **Function: `createModelDownloader()`** > Create a model downloader to download a model from a URL. Uses [`ipull`](https://github.com/ido-pluto/ipull) to download a model file as fast as possible with parallel connections and other optimizations. - https://node-llama-cpp.withcat.ai/api/functions/combineModelDownloaders - > **Function: `combineModelDownloaders()`** > Combine multiple models downloaders to a single downloader to download everything using as much parallelism as possible. > > You can check each individual model downloader for its download progress, but only the onProgress passed to the combined downloader will be called during the download. - https://node-llama-cpp.withcat.ai/guide/downloading-models#inspecting-remote-models - > **Inspecting Remote Models** > You can inspect the metadata of a remote model without downloading it by either using the [`inspect gguf` command](https://node-llama-cpp.withcat.ai/cli/inspect/gguf) with a URL, or using the [`readGgufFileInfo`](https://node-llama-cpp.withcat.ai/api/functions/readGgufFileInfo) method with a URL - https://node-llama-cpp.withcat.ai/guide/downloading-models#detecting-the-compatibility-of-remote-models - > **Detecting the Compatibility of Remote Models** > It's handy to check the compatibility of a remote model with your current machine hardware before downloading it, so you won't waste time downloading a model that won't work on your machine. > > You can do so using the [`inspect estimate` command](https://node-llama-cpp.withcat.ai/cli/inspect/estimate) with a URL

kerem commented

2026-03-03 13:52:23 +03:00

Author

Owner

@0xdevalias commented on GitHub (Oct 20, 2024):

FYI: It looks like @jehna opted for directly using ipull in this PR:

https://github.com/jehna/humanify/pull/166

Rather than the createModelDownloader / combineModelDownloaders abstractions from node-llama-cpp mentioned above in https://github.com/jehna/humanify/issues/58#issuecomment-2381885999

@0xdevalias commented on GitHub (Oct 20, 2024): FYI: It looks like @jehna opted for directly using `ipull` in this PR: - https://github.com/jehna/humanify/pull/166 Rather than the `createModelDownloader` / `combineModelDownloaders` abstractions from `node-llama-cpp` mentioned above in https://github.com/jehna/humanify/issues/58#issuecomment-2381885999

kerem referenced this issue

2026-03-03 13:53:10 +03:00

[PR #42] [MERGED] Add automated release notes #121

Rows
Columns

[GH-ISSUE #58] Improve model download speed, progress display, etc #31