# Bot Management


{{< announcement >}}
This feature is in **Public Beta**, where we work with early users and customers to test new features and get feedback before making them generally available. 

[Contact the Product team](mailto:product@rudderstack.com) if you have any questions.
{{< /announcement >}}

This guide walks you through RudderStack's bot management feature in detail and explains how to use it for detecting and filtering bot traffic for web sources.

## Overview

RudderStack's **Bot Management** feature lets you detect, flag, and drop bot traffic on web sources before it enters your data pipelines. By identifying and managing bot events early in your data flow, you can significantly improve data quality and reduce unnecessary costs.

Note that:

- Bot management is available in the [Growth](https://www.rudderstack.com/pricing/) and [Enterprise](https://www.rudderstack.com/enterprise-quote/) plans only.
- Only [Admins]({{< ref "access-management/member-management.md#member-roles" >}}) can configure bot management for the workspace
- This feature is only applicable for the following web sources:

  - [JavaScript]({{< ref "sources/event-streams/sdks/rudderstack-javascript-sdk/" >}}) (v3.0.1 and above)
  - [Flutter]({{< ref "sources/event-streams/sdks/rudderstack-flutter-sdk/" >}}) 
  - [AMP Analytics]({{< ref "sources/event-streams/sdks/rudderstack-amp-analytics.md" >}})

- The current implementation focuses on user agent-based detection and **does not** include behavioral detection
- Some sophisticated bots may evade detection if they use standard user agents

## Required permissions

Only [Admins]({{< ref "access-management/member-management.md#member-roles" >}}) and [Members]({{< ref "access-management/member-management.md#member-roles" >}}) with the [**Bot Management**]({{< ref "access-management/policies-overview.md#resource-permissions" >}}) permission can configure bot management at a [workspace](#workspace-level-configuration) and [source](#source-level-configuration) level.

{{< image src="images/access-management/bot-management.webp" alt="Bot Management permissions in new Access Management system" >}}

#### Permissions for legacy RBAC system

In the [legacy Permissions Management (RBAC) system]({{< ref "archive/dashboard-guides/user-management.md" >}}), only [Org Admins]({{< ref "archive/dashboard-guides/user-management.md#organization-roles" >}}) and members with the **Connections Admin** role in their workspace policy can use this feature.

{{< image src="images/access-management/tracking-plan-permissions-legacy-framework.webp" alt="Data Catalog permissions in the legacy framework" >}}

## What is bot management?

Bot management is a feature that identifies web events generated by bots, then gives you the ability to flag or drop those events. 

With this feature, you can:

- Detect bot events on web sources.
- Flag bot events for downstream analysis.
- Block bot events entirely.
- Create global bot management settings for all web sources.
- Apply custom bot management settings for individual web sources.
- Monitor bot event volume patterns through the source Events view. 

#### Which pain points does bot management address?

- **Inconsistent bot detection**: Previously, each analytics platform in your stack implemented its own approach to bot detection, creating inconsistent visitor counts and engagement metrics across platforms. Bot management provides a single, reliable filtering layer that standardizes bot detection across your web sources.
- **Data quality degradation**: Bot events mixed with legitimate customer events corrupts datasets, leading to unreliable conversion metrics and misleading signals about feature usage. Bot management ensures your teams can work with clean, accurate data.
- **Unnecessary costs**: Bot events generate unwanted costs across the stack, from piplines to data stores to analytics tools. Bot management gives you the tools to control cost at every stage. 
- **Marketing inefficiency**: Marketing and advertising budgets are often consumed by non-human bot events, inflating customer acquisition costs and making campaign performance metrics unreliable. Bot management helps ensure your marketing efforts target real users.

## How bot management works

This section explains the bot detection, flagging, and dropping mechanisms in detail.

### Bot detection

To detect bots, RudderStack analyzes user agent strings in the [context]({{< ref "event-spec/standard-events/common-fields.md#contextual-fields" >}}) of each event payload.

RudderStack provides two detection options:

- **Default detection**: Uses a [predefined list](https://github.com/monperrus/crawler-user-agents/blob/master/crawler-user-agents.json) of known bot patterns in the user agent string to identify common bots. This list includes common AI crawlers.
- **IAB list integration**: Uses the [International Advertising Bureau (IAB) bot list](https://www.iab.com/guidelines/iab-abc-international-spiders-bots-list/) to identify bots in the user agent string. This option is available only if you have purchased the IAB bot list or have an active subscription for list updates.

{{< info >}}
To use the IAB list integration, you must have purchased the IAB bot list or have a subscription for updates. Contact RudderStack to set up a RudderStack-assisted integration using your IAB credentials.
{{< /info >}}

### Bot event management actions

When bot events are detected, RudderStack offers two options for taking action:

#### 1. Forward events with a flag

When enabled, RudderStack forwards the event with bot details in the `context` object of the payload. Flagging payloads is useful when you want to perform deeper bot analysis in a downstream system. 

In this method, RudderStack:

- Adds an `isBot: true` flag in the root level `context` object
- Includes a `bot` object with helpful details like `name` and `url` (when available)

{{< info >}}
Flagged events are ingested and not dropped, so they count towards your paid event volume. 
{{< /info >}}

Example of a flagged event:

```json
{
  "userId": "",
  "anonymousId": "anon-2dfb-4ba389c577037c5fdd4a",
  "event": "Product Added",
  "properties": {
    "product_id": "SKU123",
    "product_name": "Wireless Headphones"
  },
  "context": {
    "ip": "11.22.33.44",
    "userAgent": "Mozilla/5.0 (Macintosh; Crawler)",
    "isBot": true,
    "bot": {
      "name": "OAISearchBot",
      "url": "https://platform.openai.com/docs/bots"
    },
    "app": {
      "name": "Store Web App",
      "version": "2.1.0"
    }
  },
  "timestamp": "2025-02-17T10:30:45.123Z",
  "type": "track"
}
```

#### 2. Drop events

When enabled, RudderStack drops bot events at the gateway and they are not ingested into the pipeline. Note that these events:

- Are not included your billable event volume
- Are not sent to any destinations

{{< warning >}}
Dropped events cannot be [replayed]({{< ref "user-guides/administrators-guide/event-replay.md" >}}) or recovered.
{{< /warning >}}

## Configure bot management settings

You can configure bot management at two levels:

### Workspace-level configuration

1. Go to **Govern** > **Bot Management** to configure bot detection, flagging, and dropping for your workspace.
2. Turn on the **Bot Detection** toggle to enable the bot detection feature in your workspace. This will enable bot detection across all web sources. You can then configure the bot management settings globally, or for specific sources in your workspace.

{{< image src="images/data-governance/bot-management/bot-management-view.webp" alt="Bot management setting in RudderStack dashboard" >}}

#### Workspace bot event management

This section lets you [flag](#2-forward-event-with-a-flag) or [drop](#1-drop-events) bot events across **all** web sources in the workspace.

To configure bot management for a specific web source, see [Source-level configuration](#source-level-configuration).

{{< image src="images/data-governance/bot-management/workspace-settings.webp" alt="Bot management global setting in RudderStack dashboard" >}}

### Source-level configuration

You can use the **Source-level bot event management** section to override the global bot management settings for specific web sources and specify the configuration per source based on your requirements.

{{< image src="images/data-governance/bot-management/source-level-settings.webp" alt="Bot management source level settings in RudderStack dashboard" >}}

See the below use cases for more information on how to use the settings in this section:

#### 1. Disable bot management for a specific source

1. Select the source from the dropdown. You will see the **Bot event management** setting toggled on and the [global setting](#global-bot-event-management) selected by default.
2. Turn off the **Bot event management** toggle.

In this case, bots will still be detected, but not flagged or dropped.

{{< image src="images/data-governance/bot-management/disable-setting.webp" alt="Disable bot detection and filtering feature for a particular source" >}}

#### 2. Configure different bot management actions a specific source

1. Select the source from the dropdown. You will see the **Bot event management** setting toggled on and the [global setting](#global-bot-event-management) selected by default.
2. Choose a different management action setting (flag/drop). 

In this case, a different bot management action will apply to the configured source. The global bot management settings will continue to apply to all the other sources.

{{< image src="images/data-governance/bot-management/custom-filtering-setting.webp" alt="Specify a different bot filtering setting for a particular source" >}}

#### Manage bot management in source settings

1. Go to the **Settings** tab of the source page.
2. Scroll down to the **Bot event management**.
3. If you have already configured the bot management settings in the [**Source-level bot event management** section](#source-level-configuration) of the **Govern** > **Bot Blocking** page, then they will be reflected here.
4. Update the settings as required.

{{< info >}}
Any updates made to the bot management settings in this section will automatically reflect in the [Source-level bot event management](#source-level-configuration) section.
{{< /info >}}

{{< image src="images/data-governance/bot-management/bot-management-source-setting.webp" alt="Bot filtering setting in source settings" >}}

## View bot event metrics

You can view the bot event metrics for a particular source in the **Events** tab of that source page.

{{< image src="images/data-governance/bot-management/bot-events-metrics-new.webp" alt="Bot event metrics in source page" >}}

### Total bot events

The **Total bot events** metric corresponds to the total number of non-[deduplicated](#effect-of-deduplication-on-ingested-event-metrics) events generated by bots for that source.

### Ingested bot events 

The **Ingested bot events** metric corresponds to the number of [deduplicated](#effect-of-deduplication-on-ingested-event-metrics) bot events ingested by the source.

Note that this event count also depends on whether the bot management settings are configured to [flag](#1-forward-events-with-a-flag) or [drop](#2-drop-events) bot events. See the [end-to-end example](#end-to-end-example) below for more information on how this metric is calculated.

### Ingested events

The **Ingested events** metric corresponds to the total number of events ingested by the source. Note that this number:

- Includes [deduplicated](#effect-of-deduplication-on-ingested-event-metrics) bot events
- Excludes dropped bot events if [bot dropping](#2-drop-events) is enabled

### Effect of deduplication on ingested bot events

RudderStack's [Event Deduplication]({{< ref "user-guides/administrators-guide/deduplication.md" >}}) feature ensures that the same event is not processed multiple times, with the goal of achieving exactly-once delivery — this is important because events can sometimes be sent more than once due to retries.

{{< info >}}
**How event deduplication works**

Every event sent through RudderStack SDKs is assigned a unique`messageId`. This ID remains the same even if the event is retried due to delivery failures (caused by network issues or various other factors).

When RudderStack receives an event, it checks if the `messageId` has already been processed. If it has, the event is dropped; otherwise, the event is processed and the `messageId` is stored for future reference.
{{< /info >}}

Deduplication has more effect on bot events than non-bot events, as bots generate a lot of events with the same `messageId` which end up getting deduplicated.

As a result, while you might see a higher number of bot events in the **Total bot events** metric, whereas the **Ingested bot events** count can be lower.

### End-to-end example

This section illustrates how deduplication affects bot event metrics.

Consider a JavaScript SDK that sends 1000 events, where:

- 450 events are from bots, with only 300 unique bot events
- 550 events are unique, non-bot events
- Event deduplication is enabled by default

#### Case 1: Bot events are forwarded with a flag

The following table shows how the metrics look like when bot management is configured to [forward events with flag](#1-forward-events-with-a-flag):

| Metric | Value | Breakdown |
| ------ | ----- | --------- |
| **Total bot events** | 450 | <ul><li>300 unique bot events</li><li>150 duplicate bot events</li></ul> |
| **Ingested bot events** | 300 | 300 unique bot events |
| **Ingested events** | 850 | <ul><li>550 non-bot events</li><li>300 deduplicated bot events (included because of the bot management setting)</li></ul> |

#### Case 2: Bot events are dropped

The following table shows how the metrics look like when bot management is configured to [drop](#2-drop-events) bot events:

| Metric | Value | Breakdown |
| ------ | ----- | --------- |
| **Total bot events** | 450 | <ul><li>300 unique bot events</li><li>150 duplicate bot events</li></ul> |
| **Ingested bot events** | 0 | Excluded because of the bot management setting |
| **Ingested events** | 550 | <ul><li>550 non-bot events</li><li>0 bot events (excluded because of the bot management setting)</li></ul> |

## Considerations for implementing bot management

Keep the following points in mind before you implement the bot management feature:

### Good bots vs. bad bots

When enabling the bot detection and management feature, consider the different types of bots that may interact with your site.

#### Good bots

- Search engine crawlers like Google or Bing
- Price aggregators
- Helpful for SEO and product discovery
- Generate events that don't represent customer behavior

#### Bad bots

- Competitive scrapers (monitoring prices, inventory)
- Automated threats (account takeover attempts, credit card testing)
- Particularly common in competitive ecommerce sectors

### Impact on data analysis

- **Historical data comparison**: Data patterns may change after enabling bot dropping and it is likely that downstream teams will see meaningful changes in event volume.
- **Marketing attribution**: Metrics like conversion rates and customer acquisition costs may shift.
- **A/B testing**: Results may differ with cleaner data.

### Recommendations

- **Start with flagging**: Begin by forwarding bot events with a flag to understand your bot traffic patterns before dropping events.
- **Monitor bot metrics**: Use the provided analytics to understand how much of your traffic is from bots.
- **Review flagged events**: Periodically check flagged bot events to ensure legitimate traffic isn't being misclassified.

## FAQ

#### Does bot management work with the older JavaScript SDK versions?

The bot management feature is supported in [JavaScript SDK](https://github.com/rudderlabs/rudder-sdk-js/releases) v3.0.1 and above. RudderStack recommends using the latest version of the SDK for best results.

<br />