Convert Data to AVRO format: Enable this setting to serialize your data using Avro and send it to the specified Kafka topic.
Schema ID: Enter the schema ID for the schema list.
Schema: Enter the schema corresponding to the above-mentioned ID.
Enable multiple topic support for event types and event names: Enable this setting to deliver your events to multiple Kafka topics.
Map event type to topic: You can use this setting to send a particular event type (identify, group, page, screen, alias) to a specific Kafka topic.
Map Track events to topic: You can use this setting to map a particular track event to a specific Kafka topic.
If this setting is disabled, RudderStack sends all events to the Kafka topic specified in the Topic Name setting. Also, if you configure only a few event types or specific track events with the above setting, RudderStack sends the rest of the events to the topic defined in the Topic Name setting.
Sending events to Kafka topics
You can send your events to a specific Kafka topic by configuring the Topic Name dashboard setting.
Alternatively, you can send your events to a Kafka topic by specifying the topic name in the integrations object:
Step 6: Add the RudderStack CA certificate to your truststore
Add the above RudderStack CA certificate to your truststore by using the certificate’s location in the following command:
keytool -keystore kafka.server.truststore.jks -alias CARootRudder -import -file ca-cert-rudder
// here ca-cert-rudder is the RudderStack CA certificate
How can I connect to RudderStack if my Kafka server is running in a Kubernetes cluster?
You will need to expose one public address to which RudderStack can connect. It is recommended to use SSL for that.
You should allow only the authenticated clients to this exposed address.
If you use PLAINTEXT for your internal services within your cluster, you might already have the address. Open this address with SSL. You need to update advertised.listeners in your server.properties, as shown in the below example:
# Hostname and port the broker will advertise to producers and consumers.
# here the INTERNAL listener is your cluster kafka service host for kafka server
# and the EXTERNAL listener is the public loadbalancer for kafka server
Does RudderStack support SASL_PLAINTEXT authentication?
RudderStack does not support SASL_PLAINTEXT authentication. You can use SASL_SSL instead. The official Kafka documentation recommends using SASL with SSL in production.
Why is the connection between Kafka and Zookeeper is failing for SASL?
You can try configuring your Zookeeper with SASL_SSL to resolve this issue.
cookies, the cookies that are categorized as necessary are stored on your browser as they are as
for the working of basic functionalities of the website. We also use third-party cookies that
analyze and understand how you use this website. These cookies will be stored in your browser
consent. You also have the option to opt-out of these cookies. But opting out of some of these
have an effect on your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This
category only includes cookies that ensures basic functionalities and security
features of the website. These cookies do not store any personal information.
learn more about cookies and why we use them, visit our cookie
policy. We'll assume you're ok with this, but you can opt-out if you wish Cookie Settings.