Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 17:27
    Naros review_requested #3526
  • 10:45
    elgca synchronize #3526
  • 10:24
    elgca synchronize #3526
  • 09:57
    elgca opened #3526
  • May 21 11:19
    harveyyue synchronize #3525
  • May 21 11:17
    harveyyue synchronize #3525
  • May 21 11:00
    harveyyue opened #3525
  • May 20 22:07
    dependabot[bot] labeled #3524
  • May 20 22:07
    dependabot[bot] opened #3524
  • May 20 18:13
    Naros synchronize #3520
  • May 20 16:18
    Naros synchronize #3520
  • May 20 16:07
    Chrisss93 synchronize #3493
  • May 20 16:00
    Naros labeled #3523
  • May 20 16:00
    Naros review_requested #3523
  • May 20 16:00
    Naros labeled #3523
  • May 20 16:00
    Naros opened #3523
  • May 20 13:46
    Naros labeled #3522
  • May 20 13:46
    Naros opened #3522
  • May 20 08:41
    ismailsimsek closed #3122
  • May 20 02:51
    Naros labeled #3521
Sungho Hwang
@sgc109
@gunnarmorling I created a Jira issue for the first time. Does it make sense? https://issues.redhat.com/projects/DBZ/issues/DBZ-4191
1 reply
Jiri Pechanec
@jpechane
@sgc109 Debezium Community migrated to https://debezium.zulipchat.com/
Jiri Pechanec
@jpechane
@Naros Debezium Community migrated to https://debezium.zulipchat.com/
akshay kushal
@akshay2305_twitter
Hi Team,
Jiri Pechanec
@jpechane
@akshay2305_twitter Debezium Community migrated to https://debezium.zulipchat.com/
Rotem Adhoh
@rotem-ad
Hey guys, I’m trying to open a Jira issue and nothing happens when I click the “Create” button. Not sure what to do..
Jiri Pechanec
@jpechane
@rotem-ad Debezium Community migrated to https://debezium.zulipchat.com/
Davit Beradze
@dberadze_twitter

Hello, the 1.7 and 1.8 version gets :

Error: Could not find or load main class org.apache.kafka.connect.cli.ConnectDistributed

I run with the default config
docker run -it --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my-connect-configs -e OFFSET_STORAGE_TOPIC=my-connect-offsets -e ADVERTISED_HOST_NAME=192.100.100.100 debezium/connect

what is the wrong?

Jiri Pechanec
@jpechane
@dberadze_twitter Debezium Community migrated to https://debezium.zulipchat.com/
Waqas Dilawar
@waqasdilawar
@gunnarmorling Hi dear, I'm trying to expose metrics of Kafka Connector using this docker-compose file (https://github.com/debezium/debezium-examples/blob/main/monitoring/docker-compose.yml), but I'm confusing myself with this line (KAFKA_OPTS=-javaagent:/kafka/etc/jmx_prometheus_javaagent.jar=8080:/kafka/etc/config.yml). Can you please tell me what I'm supposed to replace here? Btw I can access logs in JConsole but it's neither working in the browser nor in Prometheus without this line.
Jiri Pechanec
@jpechane
@waqasdilawar Debezium Community migrated to https://debezium.zulipchat.com/
Jiri Pechanec
@jpechane
@blcksrx Debezium Community migrated to https://debezium.zulipchat.com/
yashpreetmsft
@yashpreetmsft

@jpechane

Hi Jiri, I wanted to use kafka connect in different machines( with different IP's) , I started zookeeper, server and kafka connect with debezium and realised the connectors I connect in 1st machine is syncing with the other machine, I can't figure out why ( I am using same DB from SQL Server in org), is it because of CDC of same DB or there could be some other reason?

I am using advertised.host.name as localhost should I change it too... I am using distributed mode configuration... Is there any article that I can follow to use different machines in Debezium and sync data accordingly.. ( I want to use more than 1 machines for fault tolerence and scaling.)

Any help will be really appreciated, thanks in advance :)

Jiri Pechanec
@jpechane
@yashpreetmsft Debezium Community migrated to https://debezium.zulipchat.com/
Jiri Pechanec
@jpechane
@zxxz Debezium Community migrated to https://debezium.zulipchat.com/
1 reply
@zxxz Debezium Community migrated to https://debezium.zulipchat.com/
Phil
@philipsamuel
Hey @jpechane - is there a way to pass ‐Dlog4j2.formatMsgNoLookups=True in kafka connect to prevent zero-day vulnerability for log4j
Jiri Pechanec
@jpechane
@philipsamuel Debezium Community migrated to https://debezium.zulipchat.com/
Gunnar Morling
@gunnarmorling
Jiri Pechanec
@jpechane
@gunnarmorling Debezium Community migrated to https://debezium.zulipchat.com/
Gunnar Morling
@gunnarmorling
Kafka Connect does not use log4j 2.x
Jiri Pechanec
@jpechane
@gunnarmorling Debezium Community migrated to https://debezium.zulipchat.com/
RadioGuy
@RadioGuy
Is debezium-postgres 1.4.x impacted by CVE-2021 security vulnerability with log4j ?
Also does it use jms appender by any chance ?
Jiri Pechanec
@jpechane
@RadioGuy Debezium Community migrated to https://debezium.zulipchat.com/
RadioGuy
@RadioGuy
Is debezium-postgres 1.4.x impacted by CVE-2021 security vulnerability with log4j ?
Also does it use jms appender by any chance ?
Jiri Pechanec
@jpechane
@RadioGuy Debezium Community migrated to https://debezium.zulipchat.com/
hkokay
@h_kokay_twitter
@jpechane I'm trying to execute an ad-hoc snapshot. I inserted a record into the signal table but the snapshot does not get triggered. I'm using Postgres RDS as the source.
GitHubZhangH
@GitHubZhangH
image.png
cuongtl1992
@cuongtl1992
Hi Everyone
I got problem when use debezium 1.6.1.Final. some time a get status task running but it not working, after i try restart kafka-connect it working again, then i check log file but i not get any error log
i use debezium with EventOutbox routing ?. some one get same problem? .
Vladislav Borisov
@Sherybedrock_twitter
@Sherybedrock_twitter
Hello :) how you are dealing with parsing exception on ddl
database.history.skip.unparsable.dd : true
is hot fix
vaibhav pandey
@vaibhavpandey2000
@jpechane could you help me to setup mysql debezium connector i have try 2 dayes but i am unable to do this.
vaibhav pandey
@vaibhavpandey2000
when i start my connector it takes snapshot of the table but after snapshot it through error like Error during binlog processing. Last offset stored = null, binlog reader near position = mysql-bin-changelog.040423/44985833 how can i resolve it?
jsraoghub
@jsraoghub

.. debezium mysql connector losing the GTID binlog position every-time after restart. Wondering is there fix available for this ?

"MySQL current GTID set 03e063ff-0fd2-11ec-b44a-42010a65001c:1-172385422,a40da53c-bcfb-11ea-8866-42010a650002:1-246188644 does contain the GTID set required by the connector 03e063ff-0fd2-11ec-b44a-42010a65001c:168185238-171001552
Server has already purged 03e063ff-0fd2-11ec-b44a-42010a65001c:1-163307835,a40da53c-bcfb-11ea-8866-42010a650002:1-246188644 GTIDs
GTIDs known by the server but not processed yet 03e063ff-0fd2-11ec-b44a-42010a65001c:1-168185237:171001553-172385422,a40da53c-bcfb-11ea-8866-42010a650002:1-246188644, for replication are available only 03e063ff-0fd2-11ec-b44a-42010a65001c:163307836-168185237:171001553-172385422
Some of the GTIDs needed to replicate have been already purged
Stopping down connector"

Going through the chat, the same issue is described in below post which is archived. I can not see the replies. So any help would be appreciated.
https://gitter.im/debezium/user/archives/2021/03/30

WangMinChao
@minchowang
image.png
Phuong Hai Nguyen
@nhp.0712_gitlab

Screen Shot 2022-04-25 at 2.04.46 PM.png
Target: Manually sending 7th offset with same key/content as 2nd offset to offset-topic in order to read messages from postgres as from offset 2 to 6
Reality: Connector only reads the 6th offset (the last one), then keeps going.

Can anyone tell me what I did wrong here? Here is my connector config:
{
"name": "rev_msa_mylgdb_local_1",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"database.hostname": "localhost",
"database.port": "5432",
"database.user": "postgres",
"database.password": "postgres",
"database.dbname": "msa_mylgdb",
"database.server.name": "REV_msa_mylgdb",
"table.include.list": "smartux.pt_ux_pairing",
"plugin.name": "pgoutput",
"snapshot.mode": "never",
"decimal.handling.mode": "double",
"time.precision.mode": "connect",
"binary.handling.mode": "hex",
"datatype.propagate.source.type": ".+\.BYTEA",
"slot.name": "rev_msa_mylgdb_1",
"tombstones.on.delete":"false"
}
}

Diwakar1997
@Diwakar1997

Diwakar Mishra1:17 AM
Hi All
I'm using debezium mysql connector using Amazon MSK connect, it works fine but when the connector restarts it starts giving error, "Skipping invalid database history record", ... "This is often not an issue, but if it happens repeatedly please check the 'mysql-database.db_name.table_name' topic. (io.debezium.relational.history.KafkaDatabaseHistory:306)".

And it continuously gives that error, and I need to create new connector every time after restart
Could anyone please help me find solution for this?
Thanks

HugoMRAmaro
@HugoMRAmaro

Hello,
I am using debezium-connector-postgres with Azure Postgres. However, when I try to perform the put to create the connector I always get the following error:
JdbcConnectionException: ERROR: replication slot "debezium" already exists
The problem is that before performng the POST I performed the following query: select * from pg_replication_slots;
And it was empty before the POST. So I don't understand why it created the replication slot and then complains that it already exists. Maybe has something to do with the fact that I'm using Azure Postgres. Any help?

Thank you

Jun Wan
@jwan3
Hi We just upgrade Debezium from 1.4 to 1.8. With Debezium 1.4, mongodb snapshot works fine. But with 1.8, we found that it only ingest 1-2 MB's data and then it stops ingesting data, maybe the snapshot stops? Do you guys have an idea what happens? Thank you!
Artsiom Yudovin
@ayudovin
Hi, I have such an issue when I try to registry MySQL connector javax.management.InstanceAlreadyExistsException: debezium
The version of kafka connect 6.2.4 and the versions of debezium 1.9.2, Could anyone help to find the cause?
vaibhav pandey
@vaibhavpandey2000
how we can generate ts_usec in place of ts_ms in source object anyone has any idea?