Please join the Debezium community on Zulip (https://debezium.zulipchat.com). This room is not used any longer.
1763 [debezium-oracleconnector-orclserver1-change-event-source-coordinator] INFO io.debezium.relational.RelationalSnapshotChangeEventSource - Snapshot step 8 - Finalizing
[debezium-oracleconnector-orclserver1-change-event-source-coordinator] ERROR io.debezium.pipeline.ErrorHandler - Producer failure
java.lang.RuntimeException: oracle.streams.StreamsException: ORA-21560: argument last_position is null, invalid, or out of range
at io.debezium.connector.oracle.OracleStreamingChangeEventSource.execute(OracleStreamingChangeEventSource.java:75)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:91)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: oracle.streams.StreamsException: ORA-21560: argument last_position is null, invalid, or out of range
at oracle.streams.XStreamOut.XStreamOutAttachNative(Native Method)
at oracle.streams.XStreamOut.attachInternal(XStreamOut.java:373)
at oracle.streams.XStreamOut.attach(XStreamOut.java:343)
at io.debezium.connector.oracle.OracleStreamingChangeEventSource.execute(OracleStreamingChangeEventSource.java:63)
... 6 more
Hi @Naros, Thanks for the info.
In my setup, I have Oracle 12cR1 (12.1.0.2) in a Standalone mode. So for that, the compliant instant client is 12.1 (ojdbc7/jdk8). When I run with that I got the following error just after the snapshot is done.
Caused by: java.lang.NoSuchMethodError: oracle.streams.XStreamUtility.convertSCNToPosition(Loracle/sql/NUMBER;I)[B
at io.debezium.connector.oracle.OracleStreamingChangeEventSource.convertScnToPosition(OracleStreamingChangeEventSource.java:131)
When I change the instant client to version 12.2 or 19.3 (ojdbc8/jdk8) I am getting the following error instead.
Caused by: oracle.streams.StreamsException: ORA-21560: argument last_position is null, invalid, or out of range
at oracle.streams.XStreamOut.XStreamOutAttachNative(Native Method)
As this is an ORA-* error, could this be due to an incompatible version? Any idea on what is wrong here?
@Naros - Before I open up an issue, I’d like to confirm something with you. It doesn’t appear that the connector is explicitly acquiring locks. Rather, an AccessShareLock
is being acquired as a result of the connector running a SELECT * FROM table
query during the snapshot. Based on the documentation, I would have expected that this query would not be run given that the documentation says
The fifth snapshot mode, exported, will perform a database snapshot based on the point in time when the replication slot was created. This mode is an excellent way to perform a snapshot in a lock-free way.
Is this expected?
Hello everyone! I'm trying to deploy Debezium to do CDC on AWS RDS MySQL (5.7) server and today it stopped working with an exception: org.apache.kafka.connect.errors.ConnectException: Received DML 'DELETE FROM mysql.rds_sysinfo where name = 'innodb_txn_key'' for processing, binlog probably contains events generated with statement or mixed based replication format
This is weird, because our DB is configured with correct binlog format:
+--------------------------------------------+----------------------+
| Variable_name | Value |
+--------------------------------------------+----------------------+
| binlog_format | ROW |
| binlog_row_image | FULL |
+--------------------------------------------+----------------------+
I'm searching around for a solution for some time now and didn't find anything useful yet. Did I do something wrong when configuring the connector, or should I create a ticket for the problem? Maybe the fact that we have replication in place could cause the DML to appear in the binlog?
LOCK TABLE <table> IN ACCESS SHARE UPDATE EXCLUSIVE MODE
(at the time this was the lock we used).
Hello,
I am using EmbeddedEngine to capture data from MySQL.
After a few weeks of running, I have changed schemas of tables while debezium running and I found that some of the value mismatched to the key.
For example: expected key/value is {"col1":"value1", "col2":"value2", "col3":"value3"}
But debezium exports something like {"col1":"value2", "col2": null, "col3":"value1"}
I try to use snapshot.mode=schema_only_recovery
and restart it to fix this problem but it doesn't work.
Here are some logs when starting:
2019-11-16 03:56:28.201 [WARN] [taskExecuteManager-2] i.d.c.mysql.MySqlValueConverters - Using UTF-8 charset by default for column without charset: player_country VARCHAR(40) NOT NULL DEFAULT VALUE
2019-11-16 03:56:28.201 [WARN] [taskExecuteManager-2] i.d.c.mysql.MySqlValueConverters - Column is missing a character set: player_nickname VARCHAR(30) NOT NULL DEFAULT VALUE
2019-11-16 03:56:28.201 [WARN] [taskExecuteManager-2] i.d.c.mysql.MySqlValueConverters - Using UTF-8 charset by default for column without charset: player_nickname VARCHAR(30) NOT NULL DEFAULT VALUE
2019-11-16 03:56:28.205 [WARN] [taskExecuteManager-2] i.d.c.mysql.MySqlValueConverters - Column uses MySQL character set ''utf8mb4'', which has no mapping to a Java character set
2019-11-16 03:56:28.205 [WARN] [taskExecuteManager-2] i.d.c.mysql.MySqlValueConverters - Using UTF-8 charset by default for column without charset: last_login_ip VARCHAR(40) CHARSET 'utf8mb4' NOT NULL DEFAULT VALUE
2019-11-16 03:56:28.228 [WARN] [taskExecuteManager-4] i.d.c.mysql.MySqlValueConverters - Column is missing a character set: banker_id VARCHAR(16) NOT NULL DEFAULT VALUE
And when row has been updated, it comes with logs like this which I think is relevant to below logs:
2019-11-16 04:13:05.241 [ERROR] [blc-DB_MEMBER:3306] i.d.relational.TableSchemaBuilder - Failed to properly convert data value for 'otg_member.player.last_logout_time' of type BIGINT for row [[97, 97, 97, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49, 52, 54], [97, 111, 116, 103, 48, 49, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49], [97, 97, 97], 98, [107, 97, 105, 54, 54, 54], [75, 97, 105], [], [50, 101, 53, 53, 54, 100, 101, 55, 100, 97, 97, 55, 53, 56, 51, 99, 55, 97, 52, 52, 97, 51, 49, 51, 56, 55, 55, 49, 97, 101, 57, 100], [52, 102, 51, 51, 101, 54, 57, 50, 56, 57, 55, 57, 53, 57, 53, 56, 48, 55, 99, 97, 99, 54, 102, 54, 102, 101, 52, 102, 50, 55, 98, 55], [], [67, 78, 89], [97, 111, 116, 103, 48, 49, 48, 48, 48, 48, 48, 48, 48, 48, 48, 51], [68, 95, 97, 97, 97, 100, 97, 67, 78, 89], [107, 97, 105, 64, 111, 50, 111, 103, 116, 46, 99, 111, 109], [49, 50, 51, 52], [], [49, 50, 51, 52], [50, 48, 49, 57, 45, 49, 48, 45, 50, 52], [49, 50, 51, 52], [], [], [], 1573725030137, 1573802504922, 1573877541737, [49, 57, 50, 46, 49, 54, 56, 46, 54, 51, 46, 49, 49], [49, 57, 50, 46, 49, 54, 56, 46, 54, 51, 46, 49, 49], [79, 49], 6, 3, 0, 0, 0, 1573722463033, [50, 48, 49, 57, 45, 49, 49, 45, 49, 52, 32, 48, 57, 58, 48, 55, 58, 52, 51], 1573802504922, [50, 48, 49, 57, 45, 49, 49, 45, 49, 53, 32, 48, 55, 58, 50, 49, 58, 52, 52], 1]:
java.lang.IllegalArgumentException: Unexpected value for JDBC type -5 and column last_logout_time BIGINT(20) NOT NULL DEFAULT VALUE 0: class=class [B
at io.debezium.jdbc.JdbcValueConverters.handleUnknownData(JdbcValueConverters.java:1171)
at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1214)
at io.debezium.jdbc.JdbcValueConverters.convertBigInt(JdbcValueConverters.java:816)
at io.debezium.jdbc.JdbcValueConverters.lambda$converter$6(JdbcValueConverters.java:287)
at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$4(TableSchemaBuilder.java:257)
at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:143)
at io.debezium.connector.mysql.RecordMakers$1.update(RecordMakers.java:272)
at io.debezium.connector.mysql.RecordMakers$RecordsForTable.update(RecordMakers.java:499)
at io.debezium.connector.mysql.Binlo