Skip to content

Custom Rules

Custom Rules are different to BotGuard's Core Rules. Whilst our default Core Rules greatly decrease bot traffic for your website(s), some visitor behavior might still present a cause for concern. This is where our Custom Rules engine can help you to eliminate those visitors from accessing your website. The Custom Rules engine may be used for bespoke purposes, but the main two purposes are:

  • Whitelisting (unblocking) of currently blocked services and clients.
  • Blocking of services and clients that currently have access to your site.

Access Custom Rules editor

  1. Ensure that you are logged into your Botguard account.
  2. From the navigation bar, select Websites. rule-editor-1.png
  3. In the secondary menu, select Settings.
  4. From the rules menu, select Custom Rules.

Add a Custom Rule

You can override default rules, as Custom Rules always have priority over Core Rules and Rulesets. You can add a new Custom Rule at any time, by perform the following actions:

  1. Name your rule, by typing a unique name into the RULE NAME field. rule-editor-2.png { width=50% }

  2. Set the trigger conditions for your rule:

    1. In the FIELD dropdown menu, select from the listed criteria.
      Click to review the criteria options
      • Autonomous System Number - type the AS number using numbers only, but do not include a prefix with AS/ASN abbreviations.
      • User-Agent - the corresponding HTTP header, which is sent by any software that makes a request to your server. You could use this condition to block/allow requests from various browsers, like Google Chrome, Mozilla Firefox (or their mobile versions), CLI tools like wget or curl, programming languages libraries, like Urllib or requests in Python, Go Http Client, libwww-perl, etc. You can also try exploring the world of various User-Agents to get more examples via special databases. We recommend checking out whatismybrowser.com or useragentstring.com.
      • Country Code - used to restrict access based on visitor geography.
      • IP Address - our advice is to whitelist administrator/developer/QA IP addresses (IPv4 or IPv6 - BotGuard supports both). These people often use custom software and send unusual requests to the site, which might be blocked otherwise. Please be sure to whitelist your site’s uptime monitor IP address. This condition supports CIDR notation to list IP subnets, like 123.123.123.123/24. You can also easily list different subnets in a single rule using the ‘Is any of’ match type (separating each subnet with a comma followed by a space, for example: first, second, third, ).
      • Referrer URL - a browser sends the location of where a link to your site was clicked in a special "Referer HTTP Header. One common Custom Rule, which might be useful for you, is to block visitors with an empty ‘Referer’ header. When the ‘Referer’ header is empty, it means that the client made a direct request to the site, and didn’t follow any link or perform any search request prior to the visit. Most of the time it’s fine, because it means the user typed your site address from memory, but sometimes it’s an indicator of bot traffic.
      • Internet Service Provider - firstly, type the ISP name and secondly, type the AS number using numbers only. Do not include a prefix with AS/ASN abbreviations.
      • HTTP Header - condition does the same as the two headers above, but you could use any HTTP header here, even non-standard ones.
      • URI Path - this condition allows you to grant or deny requests that were sent to certain URLs on your site. You should omit the scheme and domain name to use it. For example, a bot is sending requests to URL https://www.example.net/some/path/resource.html, in which case you would use /some/path/resource.html for the rule. Commonly used with the Match to expression match type to cover multiple files and directories via Regular Expressions.
      • Domain - our service automatically protects the subdomains for your main domain name. But sometimes you might need to apply a Custom Rule to some specific subdomains, this is where you should use this condition.
    2. In the LOGICAL CONDITION dropdown menu, choose an operator. The operator options available in this dropdown are pre-defined based upon the criterion selected in the previous step.
      Click to review the operator options
      • Is / Is not - this means a strict match
      • Contains / Not contains - a partial match to the rule string
      • Greater than / Less than - applicable to number values only
      • Is any of - a strict match to stated possible values
      • Matches to expression - matches a regular expression
    3. Type a value into the FIELD VALUE text field. This is the value for the corresonding operator selected in the previous step.
    4. Optionally, click Add Condition and repeat steps a-c to add more than one condition to a rule. You can add multiple conditions to a rule, but note that when adding more than one condition, all conditions are treated with the logical operator AND. This means that all listed conditions must be triggered before the associated action is performed.
    5. Optionally, to remove a condition that you do not want associated with your rule, click Remove Condition beside the condition to be deleted.
  3. Now choose an action that is to be triggered by your condition(s). Choose from the following actions:
    Click to review the actions available
    • Deny access - access is denied to your website.
    • Grant access - access is granted to your website.
    • Use CAPTCHA - access is granted to your website, but only after successful completion of a CAPTCHA challenge.
    • Redirect to: - regular access is denied and the traffic is redirected to either a particular page in your website, or another website page entirely. After selecting this action, you must provide the URL of the redirection target page.
    • Limit request rate by: - access requests from the same source are restricted within the bounds set in the additional fields for this action. After selecting this action, you must state how many requests that you will allow within a stated time period.
  4. Click ADD RULE to add your new rule.
  5. Enure that your new rule has been activated.
    1. In the Custom Rules list, locate your new rule. rule-editor-3.png
    2. Make sure that the rule activation toggle switch is set to ON. In the future, you can also choose to set the toggle switch to OFF to deactivate the rule.

Once created, rules can not be edited. If you need to change a rule, add a new rule with the new configuration desired and then remove the old rule.

Note

For FIELD VALUE entry for step 2c and the Redirect option for available actions in step 3, you can add URLs in two ways. You can:

  • Add the URL as an absolute URL including the transfer protocol; e.g. http://anywebsite.com/any-page
  • Alternatively, you can add the URL as a relative URL, so long as the URL is in the same website as the one for which you are creating a custom rule; e.g. /any-page. In effect, this is the same as using http://your-protected-website.com/any-page.

Delete Custom Rule

To delete a Custom Rule, perform the following actions:

  1. In the Custom Rules list, locate a rule to be deleted. rule-editor-4.png
  2. Click the bin/trash button that corresponds with the rule that you wish to delete.