starred/humanify
1
0
Fork 0
mirror of https://github.com/jehna/humanify.git synced 2026-04-27 01:26:00 +03:00
Code Issues 88 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #167] Parallel renames #51

New issue
Open
opened 2026-03-03 13:52:35 +03:00 by kerem · 14 comments
kerem commented 2026-03-03 13:52:35 +03:00
Owner
Copy link

Originally created by @jehna on GitHub (Oct 18, 2024).
Original GitHub issue: https://github.com/jehna/humanify/issues/167

My thought is, that it should speed up the process a lot if the renames were done in parallel. Especially if the user has enough OpenAI quota, it could be much faster to process large files by parallelising the work.

Local inference should also be able to be run in parallel, if the user has good enough GPU at hand.

One big problems is, that I've gotten the best results when applying renames from the bottom up – so say we have:

function a() {
  const b = (c) => {
    return c * c
  }
}

It seems that running the rename in order a -> b -> c yields much better results than running c -> b -> a.

But if we'd have multiple same-level identifiers like:

function a() {
  function b() {}
  function c() {}
  function d() {}
}

At least in theory it would be possible to run a first and [b, c, d] in parallel to get feasible results.

In the best case scenario there would be a second LLM step to check that all variables still make sense after the parallel run has finished.

Originally created by @jehna on GitHub (Oct 18, 2024). Original GitHub issue: https://github.com/jehna/humanify/issues/167 My thought is, that it should speed up the process a lot if the renames were done in parallel. Especially if the user has enough OpenAI quota, it could be much faster to process large files by parallelising the work. Local inference should also be able to be run in parallel, if the user has good enough GPU at hand. One big problems is, that I've gotten the best results when applying renames from the bottom up – so say we have: ``` function a() { const b = (c) => { return c * c } } ``` It seems that running the rename in order `a -> b -> c` yields much better results than running `c -> b -> a`. But if we'd have multiple same-level identifiers like: ``` function a() { function b() {} function c() {} function d() {} } ``` At least in theory it would be possible to run `a` first and `[b, c, d]` in parallel to get feasible results. In the best case scenario there would be a second LLM step to check that all variables still make sense after the parallel run has finished.
kerem commented 2026-03-03 13:52:35 +03:00
Author
Owner
Copy link

@jehna commented on GitHub (Oct 18, 2024):

Need to implement proper request throttling and retry logic when doing this

<!-- gh-comment-id:2423260327 --> @jehna commented on GitHub (Oct 18, 2024): Need to implement proper request throttling and retry logic when doing this
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@0xdevalias commented on GitHub (Oct 20, 2024):

Need to implement proper request throttling and retry logic when doing this

Related:

This seems to be the section of code for implementing better throttling/retry logic (at least for the openai plugin):

<!-- gh-comment-id:2424388620 --> @0xdevalias commented on GitHub (Oct 20, 2024): > Need to implement proper request throttling and retry logic when doing this Related: - https://github.com/jehna/humanify/issues/2 - https://github.com/jehna/humanify/issues/139 This seems to be the section of code for implementing better throttling/retry logic (at least for the openai plugin): - https://github.com/jehna/humanify/issues/2#issuecomment-2381078484
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@brianjenkins94 commented on GitHub (Oct 21, 2024):

Resume-ability would also be a good thing to consider.

<!-- gh-comment-id:2425451248 --> @brianjenkins94 commented on GitHub (Oct 21, 2024): Resume-ability would also be a good thing to consider.
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@0xdevalias commented on GitHub (Oct 21, 2024):

Resume-ability would also be a good thing to consider.

Some of the discussion in the following issue could tangentially relate to resumability (specifically if a consistent 'map' of renames was created, perhaps that could also show which sections of the code hadn't yet been processed):

<!-- gh-comment-id:2425538385 --> @0xdevalias commented on GitHub (Oct 21, 2024): > Resume-ability would also be a good thing to consider. Some of the discussion in the following issue could tangentially relate to resumability (specifically if a consistent 'map' of renames was created, perhaps that could also show which sections of the code hadn't yet been processed): - https://github.com/jehna/humanify/issues/97
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@brianjenkins94 commented on GitHub (Oct 23, 2024):

I'm trying to process a pretty huge file and just ran into this:

RateLimitError: 429 Rate limit reached for gpt-4o-mini in organization org-abcdefghijklmnopqrstuvwx on requests per day (RPD): Limit 10000, Used 10000

I'm going to see about improving the rate limiting here:

// /src/plugins/openai/openai-rename.ts
+import Bottleneck from "bottleneck/light";

+// Math.floor(10_000 / 24) requests/hour
+const limiter = new Bottleneck({
+	"reservoir": Math.floor(10_000 / 24),
+	"reservoirRefreshAmount": Math.floor(10_000 / 24),
+	"reservoirRefreshInterval": 3_600_000
+});

export function openaiRename({
  apiKey,
  baseURL,
  model,
  contextWindowSize
}: {
  apiKey: string;
  baseURL: string;
  model: string;
  contextWindowSize: number;
}) {
  const client = new OpenAI({ apiKey, baseURL });

+  const wrapped = limiter.wrap(async (code: string): Promise<string> => {
    return await visitAllIdentifiers(
      code,
      async (name, surroundingCode) => {
        verbose.log(`Renaming ${name}`);
        verbose.log("Context: ", surroundingCode);

        const response = await client.chat.completions.create(
          toRenamePrompt(name, surroundingCode, model)
        );
        const result = response.choices[0].message?.content;
        if (!result) {
          throw new Error("Failed to rename", { cause: response });
        }
        const renamed = JSON.parse(result).newName;

        verbose.log(`Renamed to ${renamed}`);

        return renamed;
      },
      contextWindowSize,
      showPercentage
    );
+  });

+  return wrapped();
}
<!-- gh-comment-id:2430584145 --> @brianjenkins94 commented on GitHub (Oct 23, 2024): I'm trying to process a pretty huge file and just ran into this: ``` RateLimitError: 429 Rate limit reached for gpt-4o-mini in organization org-abcdefghijklmnopqrstuvwx on requests per day (RPD): Limit 10000, Used 10000 ``` I'm going to see about improving the rate limiting here: ```diff // /src/plugins/openai/openai-rename.ts +import Bottleneck from "bottleneck/light"; +// Math.floor(10_000 / 24) requests/hour +const limiter = new Bottleneck({ + "reservoir": Math.floor(10_000 / 24), + "reservoirRefreshAmount": Math.floor(10_000 / 24), + "reservoirRefreshInterval": 3_600_000 +}); export function openaiRename({ apiKey, baseURL, model, contextWindowSize }: { apiKey: string; baseURL: string; model: string; contextWindowSize: number; }) { const client = new OpenAI({ apiKey, baseURL }); + const wrapped = limiter.wrap(async (code: string): Promise<string> => { return await visitAllIdentifiers( code, async (name, surroundingCode) => { verbose.log(`Renaming ${name}`); verbose.log("Context: ", surroundingCode); const response = await client.chat.completions.create( toRenamePrompt(name, surroundingCode, model) ); const result = response.choices[0].message?.content; if (!result) { throw new Error("Failed to rename", { cause: response }); } const renamed = JSON.parse(result).newName; verbose.log(`Renamed to ${renamed}`); return renamed; }, contextWindowSize, showPercentage ); + }); + return wrapped(); } ```
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@0xdevalias commented on GitHub (Oct 23, 2024):

Context from other thread:

Aside: Can I run humanify against multiple files simultaneously? Or would that run the risk of making requests too fast?

In trying to produce benchmark results for #172, I have determined that simultaneous runs cause 429s and cause the application to crash.

Just closing the loop.

Originally posted by @brianjenkins94 in https://github.com/jehna/humanify/issues/67#issuecomment-2427685842

<!-- gh-comment-id:2430847229 --> @0xdevalias commented on GitHub (Oct 23, 2024): Context from other thread: > > Aside: Can I run humanify against multiple files simultaneously? Or would that run the risk of making requests too fast? > > In trying to produce benchmark results for #172, I have determined that simultaneous runs cause 429s and cause the application to crash. > > Just closing the loop. > > _Originally posted by @brianjenkins94 in https://github.com/jehna/humanify/issues/67#issuecomment-2427685842_
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@neoOpus commented on GitHub (Oct 24, 2024):

Context from other thread:

Aside: Can I run humanify against multiple files simultaneously? Or would that run the risk of making requests too fast?

In trying to produce benchmark results for #172, I have determined that simultaneous runs cause 429s and cause the application to crash.
Just closing the loop.
Originally posted by @brianjenkins94 in #67 (comment)

Could we have a PR with the majority of the fixes, even if it's not production ready? I paused my work as I lost track of the tasks and became discouraged by the errors, compounded by a sluggish machine. I still want to deobfuscate some Chrome extensions to modify them or understand their functions better.

<!-- gh-comment-id:2436276228 --> @neoOpus commented on GitHub (Oct 24, 2024): > Context from other thread: > > > > Aside: Can I run humanify against multiple files simultaneously? Or would that run the risk of making requests too fast? > > > > > > In trying to produce benchmark results for #172, I have determined that simultaneous runs cause 429s and cause the application to crash. > > Just closing the loop. > > _Originally posted by @brianjenkins94 in [#67 (comment)](https://github.com/jehna/humanify/issues/67#issuecomment-2427685842)_ Could we have a PR with the majority of the fixes, even if it's not production ready? I paused my work as I lost track of the tasks and became discouraged by the errors, compounded by a sluggish machine. I still want to deobfuscate some Chrome extensions to modify them or understand their functions better.
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@neoOpus commented on GitHub (Mar 12, 2025):

Maybe allowing to use multiple API keys from different accounts can achieve a better result too

<!-- gh-comment-id:2716523216 --> @neoOpus commented on GitHub (Mar 12, 2025): Maybe allowing to use multiple API keys from different accounts can achieve a better result too
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@0xdevalias commented on GitHub (Mar 12, 2025):

use multiple API keys from different accounts can achieve a better result too

@neoOpus Curious, would you see this as:

<!-- gh-comment-id:2716987159 --> @0xdevalias commented on GitHub (Mar 12, 2025): > use multiple API keys from different accounts can achieve a better result too @neoOpus Curious, would you see this as: - Multiple API keys from a single model provider (eg. OpenAI)? - Or would you see this more like being able to 'load balance' across multiple different model providers (eg. OpenAI, Gemini, Anthropic (see https://github.com/jehna/humanify/issues/213 / https://github.com/jehna/humanify/pull/272), etc) at the same time/within a 'single run'? - Or maybe both?
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@neoOpus commented on GitHub (Mar 12, 2025):

use multiple API keys from different accounts can achieve a better result too

@neoOpus Curious, would you see this as:

I would say both to bypass rate limitations, speedup the processing and make it more resilient as well, so it can have some switching algorithms associated to hop between them if we decide to or have them work in tandem, sequence... Of course having this implanted for all the models would be the best as well.

So yeah it can be all keys from OpenAI using different accounts or any other provider... And mixing should be allowed too.

Having some proxying implanted would make things even better so each request using an API key can be routed via a configured proxy. (I still need to know if there are some who work that available for a small fee or for free 😜)

<!-- gh-comment-id:2719153515 --> @neoOpus commented on GitHub (Mar 12, 2025): > > use multiple API keys from different accounts can achieve a better result too > > [@neoOpus](https://github.com/neoOpus) Curious, would you see this as: > > * Multiple API keys from a single model provider (eg. OpenAI)? > * Or would you see this more like being able to 'load balance' across multiple different model providers (eg. OpenAI, Gemini, Anthropic (see [[Idea] Adding Anthropic as a provider #213](https://github.com/jehna/humanify/issues/213) / [Add anthropic plugin #272](https://github.com/jehna/humanify/pull/272)), etc) at the same time/within a 'single run'? > * Or maybe both? I would say both to bypass rate limitations, speedup the processing and make it more resilient as well, so it can have some switching algorithms associated to hop between them if we decide to or have them work in tandem, sequence... Of course having this implanted for all the models would be the best as well. So yeah it can be all keys from OpenAI using different accounts or any other provider... And mixing should be allowed too. Having some proxying implanted would make things even better so each request using an API key can be routed via a configured proxy. (I still need to know if there are some who work that available for a small fee or for free 😜)
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@0xdevalias commented on GitHub (Apr 17, 2025):

See also:

<!-- gh-comment-id:2811363677 --> @0xdevalias commented on GitHub (Apr 17, 2025): See also: - https://github.com/jehna/humanify/issues/405 - https://github.com/jehna/humanify/issues/406 - https://github.com/jehna/humanify/issues/416
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@0xdevalias commented on GitHub (Apr 24, 2025):

For one potential solution to 'load balancing' / using different models/providers when hitting an error like a rate limit/etc:

Specifically parts like:

  • https://openrouter.ai/docs/features/model-routing

    • Model Routing

    • Dynamically route requests to models

    • The models parameter lets you automatically try other models if the primary model’s providers are down, rate-limited, or refuse to reply due to content moderation.

    • If the model you selected returns an error, OpenRouter will try to use the fallback model instead. If the fallback model is down or returns an error, OpenRouter will return that error.

  • https://openrouter.ai/docs/features/provider-routing

    • Provider Routing

    • Route requests to the best provider

    • OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers to maximize uptime.

Originally posted by @0xdevalias in https://github.com/jehna/humanify/issues/416

<!-- gh-comment-id:2826694406 --> @0xdevalias commented on GitHub (Apr 24, 2025): For one potential solution to 'load balancing' / using different models/providers when hitting an error like a rate limit/etc: - https://github.com/jehna/humanify/issues/416 Specifically parts like: > - https://openrouter.ai/docs/features/model-routing > - > Model Routing > - > Dynamically route requests to models > - > The `models` parameter lets you automatically try other models if the primary model’s providers are down, rate-limited, or refuse to reply due to content moderation. > - > If the model you selected returns an error, OpenRouter will try to use the fallback model instead. If the fallback model is down or returns an error, OpenRouter will return that error. > - https://openrouter.ai/docs/features/provider-routing > - > Provider Routing > - > Route requests to the best provider > - > OpenRouter routes requests to the best available providers for your model. By default, [requests are load balanced](https://openrouter.ai/docs/features/provider-routing#load-balancing-default-strategy) across the top providers to maximize uptime. > > _Originally posted by @0xdevalias in https://github.com/jehna/humanify/issues/416_
kerem commented 2026-03-03 13:52:36 +03:00
Author
Owner
Copy link

@0xdevalias commented on GitHub (May 30, 2025):

Having some proxying implanted would make things even better so each request using an API key can be routed via a configured proxy.

@neoOpus For proxy support, the following issues may be of interest:

But more specifically, at least for openai, this most recent update I posted:

Which is mirroring what I originally shared here:

And comes from this version bump PR (or any that replace it):

<!-- gh-comment-id:2921157154 --> @0xdevalias commented on GitHub (May 30, 2025): > Having some proxying implanted would make things even better so each request using an API key can be routed via a configured proxy. @neoOpus For proxy support, the following issues may be of interest: - https://github.com/jehna/humanify/issues/68 - https://github.com/jehna/humanify/issues/265 - And tangentially: - https://github.com/jehna/humanify/issues/52 But more specifically, at least for `openai`, this most recent update I posted: - https://github.com/jehna/humanify/issues/265#issuecomment-2921150326 - > Looks like `openai` v5 is out now, which uses the native fetch, and has better support for proxies: > > ..snip.. Which is mirroring what I originally shared here: - https://github.com/jehna/humanify/issues/52#issuecomment-2921148059 And comes from this version bump PR (or any that replace it): - https://github.com/jehna/humanify/pull/459
kerem commented 2026-03-03 13:52:37 +03:00
Author
Owner
Copy link

@neoOpus commented on GitHub (Jun 9, 2025):

@0xdevalias I stumbled upon this in HN

https://llmgateway.io/

<!-- gh-comment-id:2954841525 --> @neoOpus commented on GitHub (Jun 9, 2025): @0xdevalias I stumbled upon this in HN https://llmgateway.io/
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
claude/rust-ast-rewrite-SjZI6
dependabot/npm_and_yarn/babel/types-7.29.0
dependabot/npm_and_yarn/globals-17.3.0
dependabot/npm_and_yarn/babel/core-7.29.0
dependabot/npm_and_yarn/commander-14.0.3
dependabot/npm_and_yarn/types/node-25.2.0
dependabot/npm_and_yarn/pkgroll-2.23.0
fix/tests
dependabot/npm_and_yarn/openai-6.17.0
dependabot/npm_and_yarn/typescript-eslint-8.54.0
dependabot/npm_and_yarn/node-llama-cpp-3.15.1
dependabot/npm_and_yarn/prettier-3.8.1
dependabot/npm_and_yarn/eslint-9.39.2
dependabot/npm_and_yarn/eslint/js-9.39.2
dependabot/npm_and_yarn/tsx-4.21.0
dependabot/npm_and_yarn/eslint-plugin-unused-imports-4.3.0
dependabot/npm_and_yarn/typescript-5.9.3
dependabot/npm_and_yarn/dotenv-17.2.3
dependabot/npm_and_yarn/ipull-4.0.3
dependabot/npm_and_yarn/google/generative-ai-0.24.1
ollama-support
add/github-azure-integration
v1
v2.2.2
v2.2.1
v2.2.0
v2.1.3
v2.1.2
v2.1.1
v2.1.0
v2.0.3
v2.0.2
v2.0.1
v2.0.0
v1.0
Labels
Clear labels   
bug

   
enhancement

   
pull-request

Mirrored from GitHub Pull Request

  
wontfix
No labels
bug
enhancement
pull-request
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/humanify#51
No description provided.