Understanding Kafka Client Using Node

I was trying to use node-rdkafka npm module to interact with my Kafka cluster and I got confused with the parameters of some APIs. So I thought of sharing here in this blog.

producer.produce(topic, partition, msg, key, timestamp, opaque)

topic: It’s the topic name against which we are going to send the message

partition:
Partitions are the main method of concurrency for topics. A topic, a dedicated location for events or messages, will be broken into multiple partitions among one or more Kafka brokers.

Important Note:
While one consumer can work with many different partitions, but partitions can work with a single consumer at a time.

In Kafka, the number of partitions for a topic is typically set when the topic is created. When you create a topic, you can specify the number of partitions it should have. The number of partitions is an important parameter that influences parallelism and scalability in Kafka.

Here I had a question:
How do I send a message to a particular Partition? How can I know the ID of a partition among 3 partitions?
In Kafka, when you produce a message to a topic, the partition to which the message is sent is typically determined by Kafka’s partitioning strategy. Kafka uses a partitioning algorithm to distribute messages across partitions, and you don’t explicitly specify the partition ID when producing a message.
However, you can indirectly influence the partition to which a message is sent by providing a key when producing the message. Kafka’s default partitioning strategy is to hash the key and use the result to determine the target partition. If the key is null or not provided, Kafka may use a round-robin approach.

producer.on('ready', () => {
  const messageValue = 'Hello, Kafka!';
  const messageKey = 'my-key';  // Specify a key to influence the partition

  // The partition will be determined based on the hashing of the key
  producer.produce('my-topic', -1, Buffer.from(messageValue), messageKey);

  producer.disconnect();
});

const Kafka = require("node-rdkafka");
const fs = require('fs');
const config = readConfigFile("client.properties");
config["group.id"] = "node-group";

const consumer = new Kafka.KafkaConsumer(config, {"auto.offset.reset": "earliest" });
consumer.connect();
consumer.on("ready", () => {
    consumer.subscribe(["topic_0_noms"]);
    consumer.consume();
}).on("data", (message) => {
    // const messageJson = JSON.parse(message.value);
    // console.log("Consumed message", messageJson);
    console.log(message);
});