I'm having issues where my php producer script is consuming large amounts of memory and eventually causing swap issues. I'm using the high-level producer. Any ideas as to what I'm doing wrong? Code is below.
'''
$kafka_producer_obj = new RdKafka\Producer();
if(!is_object($kafka_producer_obj))
{
syslog(LOG_CRIT, $this_file.": CRITICAL: Could not create a Kafka Producer object. Exiting now.");
exit(1);
}
if($kafka_producer_obj->addBrokers(gethostname()) <= 0)
{
syslog(LOG_CRIT, $this_file.": CRITICAL: Could not add a broker to the Kafa producer object. Exiting now.");
exit(1);
}
$kafka_producer_topic_obj = $kafka_producer_obj->newTopic(SonusDbAbstract::getRegistryValue("kafka_topic"));
if(!is_object($kafka_producer_topic_obj))
{
syslog(LOG_CRIT, $this_file.": CRITICAL: Could not create a Kafka producer topic object. Exiting now.");
exit(1);
}
... for each line of data...
$kafka_producer_topic_obj->produce(RD_KAFKA_PARTITION_UA, 0, $this_data_line);
'''