Please join the Debezium community on Zulip (https://debezium.zulipchat.com). This room is not used any longer.
1 reply
Hello, the 1.7 and 1.8 version gets :
Error: Could not find or load main class org.apache.kafka.connect.cli.ConnectDistributed
I run with the default config
docker run -it --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my-connect-configs -e OFFSET_STORAGE_TOPIC=my-connect-offsets -e ADVERTISED_HOST_NAME=192.100.100.100 debezium/connect
what is the wrong?
@jpechane
Hi Jiri, I wanted to use kafka connect in different machines( with different IP's) , I started zookeeper, server and kafka connect with debezium and realised the connectors I connect in 1st machine is syncing with the other machine, I can't figure out why ( I am using same DB from SQL Server in org), is it because of CDC of same DB or there could be some other reason?
I am using advertised.host.name as localhost should I change it too... I am using distributed mode configuration... Is there any article that I can follow to use different machines in Debezium and sync data accordingly.. ( I want to use more than 1 machines for fault tolerence and scaling.)
Any help will be really appreciated, thanks in advance :)
.. debezium mysql connector losing the GTID binlog position every-time after restart. Wondering is there fix available for this ?
"MySQL current GTID set 03e063ff-0fd2-11ec-b44a-42010a65001c:1-172385422,a40da53c-bcfb-11ea-8866-42010a650002:1-246188644 does contain the GTID set required by the connector 03e063ff-0fd2-11ec-b44a-42010a65001c:168185238-171001552
Server has already purged 03e063ff-0fd2-11ec-b44a-42010a65001c:1-163307835,a40da53c-bcfb-11ea-8866-42010a650002:1-246188644 GTIDs
GTIDs known by the server but not processed yet 03e063ff-0fd2-11ec-b44a-42010a65001c:1-168185237:171001553-172385422,a40da53c-bcfb-11ea-8866-42010a650002:1-246188644, for replication are available only 03e063ff-0fd2-11ec-b44a-42010a65001c:163307836-168185237:171001553-172385422
Some of the GTIDs needed to replicate have been already purged
Stopping down connector"
Going through the chat, the same issue is described in below post which is archived. I can not see the replies. So any help would be appreciated.
https://gitter.im/debezium/user/archives/2021/03/30
Target: Manually sending 7th offset with same key/content as 2nd offset to offset-topic in order to read messages from postgres as from offset 2 to 6
Reality: Connector only reads the 6th offset (the last one), then keeps going.
Can anyone tell me what I did wrong here? Here is my connector config:
{
"name": "rev_msa_mylgdb_local_1",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"database.hostname": "localhost",
"database.port": "5432",
"database.user": "postgres",
"database.password": "postgres",
"database.dbname": "msa_mylgdb",
"database.server.name": "REV_msa_mylgdb",
"table.include.list": "smartux.pt_ux_pairing",
"plugin.name": "pgoutput",
"snapshot.mode": "never",
"decimal.handling.mode": "double",
"time.precision.mode": "connect",
"binary.handling.mode": "hex",
"datatype.propagate.source.type": ".+\.BYTEA",
"slot.name": "rev_msa_mylgdb_1",
"tombstones.on.delete":"false"
}
}
Diwakar Mishra1:17 AM
Hi All
I'm using debezium mysql connector using Amazon MSK connect, it works fine but when the connector restarts it starts giving error, "Skipping invalid database history record", ... "This is often not an issue, but if it happens repeatedly please check the 'mysql-database.db_name.table_name' topic. (io.debezium.relational.history.KafkaDatabaseHistory:306)".
And it continuously gives that error, and I need to create new connector every time after restart
Could anyone please help me find solution for this?
Thanks
Hello,
I am using debezium-connector-postgres with Azure Postgres. However, when I try to perform the put to create the connector I always get the following error:
JdbcConnectionException: ERROR: replication slot "debezium" already exists
The problem is that before performng the POST I performed the following query: select * from pg_replication_slots;
And it was empty before the POST. So I don't understand why it created the replication slot and then complains that it already exists. Maybe has something to do with the fact that I'm using Azure Postgres. Any help?
Thank you
