[GH-ISSUE #3611] Bad Bot Blocker #2392

Open
opened 2026-02-26 07:35:23 +03:00 by kerem · 10 comments
Owner

Originally created by @Ahmadshoh on GitHub (Mar 7, 2024).
Original GitHub issue: https://github.com/NginxProxyManager/nginx-proxy-manager/issues/3611

Is your feature request related to a problem? Please describe.

The current setup of nginx-proxy-manager lacks built-in support for integrating the nginx-ultimate-bad-bot-blocker module, resulting in a gap in security measures against malicious bots. Without this integration, administrators may face challenges in effectively mitigating bot-related threats and protecting their systems and applications from potential vulnerabilities and performance issues.

Describe the solution you'd like

I propose adding native support for enabling the nginx-ultimate-bad-bot-blocker within nginx-proxy-manager. This would involve integrating the functionality of the bot blocker module directly into nginx-proxy-manager's interface, allowing administrators to easily activate and configure bot blocking settings without the need for manual configuration or separate installations. This enhancement would streamline the process of fortifying web servers against malicious bot activities, enhancing security and performance for users of nginx-proxy-manager.

Describe alternatives you've considered

One alternative approach would be for administrators to manually configure the nginx-ultimate-bad-bot-blocker module alongside nginx-proxy-manager. However, this approach requires additional technical expertise and may introduce complexities in managing the integration between the two components. Additionally, manual configuration increases the risk of misconfigurations and potential conflicts between settings, which could impact system stability and security.

Additional context

Integrating the nginx-ultimate-bad-bot-blocker module into nginx-proxy-manager would provide a comprehensive solution for protecting web servers against a wide range of bot-based threats, including web scraping, DDoS attacks, and vulnerability scanning. By incorporating this functionality directly into nginx-proxy-manager's interface, administrators can easily enable and customize bot blocking settings, enhancing the overall security posture of their infrastructure. Additionally, the integration would align with the goal of nginx-proxy-manager to provide a user-friendly interface for managing NGINX configurations, simplifying the implementation of advanced security measures for both novice and experienced users.

Originally created by @Ahmadshoh on GitHub (Mar 7, 2024). Original GitHub issue: https://github.com/NginxProxyManager/nginx-proxy-manager/issues/3611 **Is your feature request related to a problem? Please describe.** The current setup of nginx-proxy-manager lacks built-in support for integrating the nginx-ultimate-bad-bot-blocker module, resulting in a gap in security measures against malicious bots. Without this integration, administrators may face challenges in effectively mitigating bot-related threats and protecting their systems and applications from potential vulnerabilities and performance issues. **Describe the solution you'd like** I propose adding native support for enabling the nginx-ultimate-bad-bot-blocker within nginx-proxy-manager. This would involve integrating the functionality of the bot blocker module directly into nginx-proxy-manager's interface, allowing administrators to easily activate and configure bot blocking settings without the need for manual configuration or separate installations. This enhancement would streamline the process of fortifying web servers against malicious bot activities, enhancing security and performance for users of nginx-proxy-manager. **Describe alternatives you've considered** One alternative approach would be for administrators to manually configure the nginx-ultimate-bad-bot-blocker module alongside nginx-proxy-manager. However, this approach requires additional technical expertise and may introduce complexities in managing the integration between the two components. Additionally, manual configuration increases the risk of misconfigurations and potential conflicts between settings, which could impact system stability and security. **Additional context** Integrating the nginx-ultimate-bad-bot-blocker module into nginx-proxy-manager would provide a comprehensive solution for protecting web servers against a wide range of bot-based threats, including web scraping, DDoS attacks, and vulnerability scanning. By incorporating this functionality directly into nginx-proxy-manager's interface, administrators can easily enable and customize bot blocking settings, enhancing the overall security posture of their infrastructure. Additionally, the integration would align with the goal of nginx-proxy-manager to provide a user-friendly interface for managing NGINX configurations, simplifying the implementation of advanced security measures for both novice and experienced users.
Author
Owner

@Ahmadshoh commented on GitHub (Mar 7, 2024):

More info you can find here:

https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker

<!-- gh-comment-id:1983443554 --> @Ahmadshoh commented on GitHub (Mar 7, 2024): More info you can find here: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker
Author
Owner

@github-actions[bot] commented on GitHub (Oct 28, 2024):

Issue is now considered stale. If you want to keep it open, please comment 👍

<!-- gh-comment-id:2440359484 --> @github-actions[bot] commented on GitHub (Oct 28, 2024): Issue is now considered stale. If you want to keep it open, please comment :+1:
Author
Owner

@gizmocuz commented on GitHub (Nov 10, 2024):

Would still be great if this would be supported

<!-- gh-comment-id:2466728432 --> @gizmocuz commented on GitHub (Nov 10, 2024): Would still be great if this would be supported
Author
Owner

@eikaramba commented on GitHub (Nov 16, 2024):

yes please, metas new crawler are extremly annoying and do not respect the robots.txt - i am flooded with multiple requests per second and various domains

in the meantime i added this to the custom nginx configuration under advanced:

# Block various bots
if ($http_user_agent ~* (meta-externalagent|facebookexternalhit|facebookbot|facebook|crawler|bot|spider|crawl)) {
    return 403;
}
<!-- gh-comment-id:2480731968 --> @eikaramba commented on GitHub (Nov 16, 2024): yes please, metas new crawler are extremly annoying and do not respect the robots.txt - i am flooded with multiple requests per second and various domains in the meantime i added this to the custom nginx configuration under advanced: ``` # Block various bots if ($http_user_agent ~* (meta-externalagent|facebookexternalhit|facebookbot|facebook|crawler|bot|spider|crawl)) { return 403; } ```
Author
Owner

@ichilver commented on GitHub (Dec 3, 2024):

This would be useful. I found this https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker but couldn't see how this would work with an existing NPM docker image.

<!-- gh-comment-id:2514509254 --> @ichilver commented on GitHub (Dec 3, 2024): This would be useful. I found this https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker but couldn't see how this would work with an existing NPM docker image.
Author
Owner

@Kisaragi-ng commented on GitHub (Dec 17, 2024):

would love to have this feature, has anyone tried to use the manual configuration method?

<!-- gh-comment-id:2547397396 --> @Kisaragi-ng commented on GitHub (Dec 17, 2024): would love to have this feature, has anyone tried to use the manual configuration method?
Author
Owner

@ErroneousBosch commented on GitHub (Apr 1, 2025):

Bumping that there should be a plugin or something for this. Even an ability to put in custom configurations. I threw together a quick and dirty shell script that should work for bots and IP addresses, I may revisit it later this week.

it generates files that can be placed in /data/nginx/custom/ that *should * get loaded in.

#! /bin/bash
# this script generates bad bot blocking configurations for nginx proxy manager
# grab latest version of known bad bots from https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list and map to an array
# Download the bad user agents list
bad_bots_list=$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)

# Map the list to an array
IFS=$'\n' read -r -d '' -a bad_bots_array <<< "$bad_bots_list"

# Generate the config file
# Create the config file
CONFIGFILE="http_top.conf"
echo "# This file is generated by generate_configs.sh" > $CONFIGFILE
 echo "map \$http_user_agent \$bad_bot {" >> $CONFIGFILE
	echo "  default		0;" >> $CONFIGFILE
# Loop through the array and generate the config
for bot in "${bad_bots_array[@]}"; do
    # Remove any leading or trailing whitespace
    bot=$(echo "$bot" | xargs)
    # Skip empty lines
    if [ -z "$bot" ]; then
        continue
    fi
    # Generate the config for the bot
    echo "  \"~*(?:\b)$bot(?:\b)\"		3;" >> $CONFIGFILE
done
# Close the map block
{
  echo "}" 
  echo "geo \$validate_client {" 
  echo "  default		0;" 
}>> $CONFIGFILE
bad_ip_list=$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-ip-addresses.list)
bad_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
bad_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/nibbler-seo.list)
bad_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/seo-analysis-tools.list)

# Map the list to an array
IFS=$'\n' read -r -d '' -a bad_ip_array <<< "$bad_ip_list"
# Loop through the array and generate the config
for ip in "${bad_ip_array[@]}"; do
    # Remove any leading or trailing whitespace
    ip=$(echo "$ip" | xargs)
    # Skip empty lines
    if [ -z "$ip" ]; then
        continue
    fi
    # Generate the config for the bot
    echo "  $ip		1;" >> $CONFIGFILE
done
good_ip_list=$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/google-ip-ranges.list)
good_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/cloudflare-ip-ranges.list)
good_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bunnycdn-net.list)
good_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bing-ip-ranges.list)

# Map the list to an array
IFS=$'\n' read -r -d '' -a good_ip_array <<< "$good_ip_list"
# Loop through the array and generate the config
for ip in "${good_ip_array[@]}"; do
    # Remove any leading or trailing whitespace
    ip=$(echo "$ip" | xargs)
    # Skip empty lines
    if [ -z "$ip" ]; then
        continue
    fi
    # Generate the config for the bot
    echo "$ip		0;" >> $CONFIGFILE
done
# Close the geo block
echo "}" >> $CONFIGFILE

CONFIGFILE="server_proxy.conf"
echo "# This file is generated by generate_configs.sh" > $CONFIGFILE
{
  echo "if (\$bad_bot = '3') {" 
  echo "  return 444;" 
  echo "}" 
  echo "if (\$validate_client) {"
  echo "  return 444; "
  echo "}"

} >> $CONFIGFILE
<!-- gh-comment-id:2770626712 --> @ErroneousBosch commented on GitHub (Apr 1, 2025): Bumping that there should be a plugin or something for this. Even an ability to put in custom configurations. I threw together a quick and dirty shell script that *should* work for bots and IP addresses, I may revisit it later this week. it generates files that can be placed in ` /data/nginx/custom/` that *should * get loaded in. ``` #! /bin/bash # this script generates bad bot blocking configurations for nginx proxy manager # grab latest version of known bad bots from https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list and map to an array # Download the bad user agents list bad_bots_list=$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) # Map the list to an array IFS=$'\n' read -r -d '' -a bad_bots_array <<< "$bad_bots_list" # Generate the config file # Create the config file CONFIGFILE="http_top.conf" echo "# This file is generated by generate_configs.sh" > $CONFIGFILE echo "map \$http_user_agent \$bad_bot {" >> $CONFIGFILE echo " default 0;" >> $CONFIGFILE # Loop through the array and generate the config for bot in "${bad_bots_array[@]}"; do # Remove any leading or trailing whitespace bot=$(echo "$bot" | xargs) # Skip empty lines if [ -z "$bot" ]; then continue fi # Generate the config for the bot echo " \"~*(?:\b)$bot(?:\b)\" 3;" >> $CONFIGFILE done # Close the map block { echo "}" echo "geo \$validate_client {" echo " default 0;" }>> $CONFIGFILE bad_ip_list=$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-ip-addresses.list) bad_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) bad_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/nibbler-seo.list) bad_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/seo-analysis-tools.list) # Map the list to an array IFS=$'\n' read -r -d '' -a bad_ip_array <<< "$bad_ip_list" # Loop through the array and generate the config for ip in "${bad_ip_array[@]}"; do # Remove any leading or trailing whitespace ip=$(echo "$ip" | xargs) # Skip empty lines if [ -z "$ip" ]; then continue fi # Generate the config for the bot echo " $ip 1;" >> $CONFIGFILE done good_ip_list=$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/google-ip-ranges.list) good_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/cloudflare-ip-ranges.list) good_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bunnycdn-net.list) good_ip_list+=$'\n'$(curl -s https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bing-ip-ranges.list) # Map the list to an array IFS=$'\n' read -r -d '' -a good_ip_array <<< "$good_ip_list" # Loop through the array and generate the config for ip in "${good_ip_array[@]}"; do # Remove any leading or trailing whitespace ip=$(echo "$ip" | xargs) # Skip empty lines if [ -z "$ip" ]; then continue fi # Generate the config for the bot echo "$ip 0;" >> $CONFIGFILE done # Close the geo block echo "}" >> $CONFIGFILE CONFIGFILE="server_proxy.conf" echo "# This file is generated by generate_configs.sh" > $CONFIGFILE { echo "if (\$bad_bot = '3') {" echo " return 444;" echo "}" echo "if (\$validate_client) {" echo " return 444; " echo "}" } >> $CONFIGFILE ```
Author
Owner

@github-actions[bot] commented on GitHub (Oct 5, 2025):

Issue is now considered stale. If you want to keep it open, please comment 👍

<!-- gh-comment-id:3368683090 --> @github-actions[bot] commented on GitHub (Oct 5, 2025): Issue is now considered stale. If you want to keep it open, please comment :+1:
Author
Owner

@gizmocuz commented on GitHub (Oct 5, 2025):

I know you're very busy converting the project to React... But just commenting to keep this open...

<!-- gh-comment-id:3368793569 --> @gizmocuz commented on GitHub (Oct 5, 2025): I know you're very busy converting the project to React... But just commenting to keep this open...
Author
Owner

@njfsilva commented on GitHub (Jan 1, 2026):

This is an incredibly important feature, please add it.

<!-- gh-comment-id:3703604450 --> @njfsilva commented on GitHub (Jan 1, 2026): This is an incredibly important feature, please add it.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/nginx-proxy-manager-NginxProxyManager#2392
No description provided.