Bot Management Beta

This guide walks you through RudderStack’s bot management feature in detail and explains how to use it for detecting and filtering bot traffic for web sources.

Overview

RudderStack’s Bot Management feature lets you detect, flag, and drop bot traffic on web sources before it enters your data pipelines. By identifying and managing bot events early in your data flow, you can significantly improve data quality and reduce unnecessary costs.

Note that:

• The bot management feature is available only in RudderStack’s Enterprise plan.

• You must have Org Admin privileges to configure bot management for your workspace.

• This feature is only applicable for the following web sources:

  • JavaScript
  • Flutter
  • AMP Analytics

• The current implementation focuses on user agent-based detection and does not include behavioral detection.

• Some sophisticated bots may evade detection if they use standard user agents.

What is bot management?

Bot management is a feature that identifies web events generated by bots, then gives you the ability to flag or drop those events.

With this feature, you can:

• Detect bot events on web sources.
• Flag bot events for downstream analysis.
• Block bot events entirely.
• Create global bot management settings for all web sources.
• Apply custom bot management settings for individual web sources.
• Monitor bot event volume patterns through the source Events view.

Which pain points does bot management address?

• Inconsistent bot detection: Previously, each analytics platform in your stack implemented its own approach to bot detection, creating inconsistent visitor counts and engagement metrics across platforms. Bot management provides a single, reliable filtering layer that standardizes bot detection across your web sources.
• Data quality degradation: Bot events mixed with legitimate customer events corrupts datasets, leading to unreliable conversion metrics and misleading signals about feature usage. Bot management ensures your teams can work with clean, accurate data.
• Unnecessary costs: Bot events generate unwanted costs across the stack, from piplines to data stores to analytics tools. Bot management gives you the tools to control cost at every stage.
• Marketing inefficiency: Marketing and advertising budgets are often consumed by non-human bot events, inflating customer acquisition costs and making campaign performance metrics unreliable. Bot management helps ensure your marketing efforts target real users.

How bot management works

This section explains the bot detection, flagging, and dropping mechanisms in detail.

Bot detection

RudderStack analyzes user agent strings in the context of each event payload to detect bots. (See our Event Spec for more information.)

Two detection options are available:

1. Default detection: Uses a predefined list of known bot patterns in the user agent string to identify common bots. Note that this list includes common AI crawlers.
2. IAB list integration: Uses the International Advertising Bureau (IAB) bot list to identify bots in the user agent string. This is available only for customers who are members of the IAB or have purchased the IAB bot list.

Bot event management actions

When bot events are detected, RudderStack offers two options for taking action:

1. Forward events with a flag

When enabled, RudderStack forwards the event with bot details in the context object of the payload. Flagging payloads is useful when you want to perform deeper bot analysis in a downstream system.

In this method, RudderStack:

• Adds an isBot: true flag in the root level context object
• Includes a bot object with helpful details like name and url (when available)

Flagged events are ingested and not dropped, so they count towards your paid event volume.

Example of a flagged event:

{
  "userId": "",
  "anonymousId": "anon-2dfb-4ba389c577037c5fdd4a",
  "event": "Product Added",
  "properties": {
    "product_id": "SKU123",
    "product_name": "Wireless Headphones"
  },
  "context": {
    "ip": "11.22.33.44",
    "userAgent": "Mozilla/5.0 (Macintosh; Crawler)",
    "isBot": true,
    "bot": {
      "name": "OAISearchBot",
      "url": "https://platform.openai.com/docs/bots"
    },
    "app": {
      "name": "Store Web App",
      "version": "2.1.0"
    }
  },
  "timestamp": "2025-02-17T10:30:45.123Z",
  "type": "track"
}

2. Drop events

When enabled, RudderStack drops bot events at the gateway and they are not ingested into the pipeline. Note that these events:

• Are not included your billable event volume
• Are not sent to any destinations

Dropped events cannot be replayed or recovered.

Configure bot management settings

Only Org Admins can configure the bot management settings.

You can configure bot management at two levels:

Workspace-level configuration

1. Go to Govern > Bot Management to configure bot detection, flagging, and dropping for your workspace.
2. Turn on the Bot Detection toggle to enable the bot detection feature in your workspace. This will enable bot detection across all web sources. You can then configure the bot management settings globally, or for specific sources in your workspace.

Workspace bot event management

This section lets you flag or drop bot events across all web sources in the workspace.

To configure bot management for a specific web source, see Source-level configuration.

Source-level configuration

You can use the Source-level bot event management section to override the global bot management settings for specific web sources and specify the configuration per source based on your requirements.

See the below use cases for more information on how to use the settings in this section:

1. Disable bot management for a specific source

1. Select the source from the dropdown. You will see the Bot event management setting toggled on and the global setting selected by default.
2. Turn off the Bot event management toggle.

In this case, bots will still be detected, but not flagged or dropped.

2. Configure different bot management actions a specific source

1. Select the source from the dropdown. You will see the Bot event management setting toggled on and the global setting selected by default.
2. Choose a different management action setting (flag/drop).

In this case, a different bot management action will apply to the configured source. The global bot management settings will continue to apply to all the other sources.

Manage bot management in source settings

1. Go to the Settings tab of the source page.
2. Scroll down to the Bot event management.
3. If you have already configured the bot management settings in the Source-level bot event management section of the Govern > Bot Blocking page, then they will be reflected here.
4. Update the settings as required.

Any updates made to the bot management settings in this section will automatically reflect in the Source-level bot event management section.

View bot event metrics

You can view the bot event metrics for a particular source in the Events tab of that source page.

Depending on the volume of bot events, there can be a small variation in the accuracy of the bot event count. This variation is generally <1%.

Note that the ingested event count depends on whether the bot management settings are configured to flag or drop bot events. See the below example for more information:

Considerations for implementing bot management

Keep the following points in mind before you implement the bot management feature:

Good bots vs. bad bots

When enabling the bot detection and management feature, consider the different types of bots that may interact with your site.

Good bots

• Search engine crawlers like Google or Bing
• Price aggregators
• Helpful for SEO and product discovery
• Generate events that don’t represent customer behavior

Bad bots

• Competitive scrapers (monitoring prices, inventory)
• Automated threats (account takeover attempts, credit card testing)
• Particularly common in competitive ecommerce sectors

Impact on data analysis

• Historical data comparison: Data patterns may change after enabling bot dropping and it is likely that downstream teams will see meaningful changes in event volume.
• Marketing attribution: Metrics like conversion rates and customer acquisition costs may shift.
• A/B testing: Results may differ with cleaner data.

Recommendations

• Start with flagging: Begin by forwarding bot events with a flag to understand your bot traffic patterns before dropping events.
• Monitor bot metrics: Use the provided analytics to understand how much of your traffic is from bots.
• Review flagged events: Periodically check flagged bot events to ensure legitimate traffic isn’t being misclassified.

Questions? Contact us by email or on Slack