I've flushed both buckets (master DC and replicated) and reuploading 120M of records from scratch right now. I have no issues with replication at the moment, but every time issues come a few days after when I'm pulling updates from Kafka and pushing to Couchbase.
Primary DC has more records than replicated.
thinking out loud here - more documents in the primary DC would indicate that it is inserts that are not being replicated (at least more inserts than removes). [ the document count discrepancy being due to more removes being processed on the replicated DC than on the primary is not a possibility]. Some of the "skipped mutations" may also be replaces. Theoretically, a remove cannot be a "skipped mutation" because once a document is removed, there cannot be any subsequent operations on it to replicate. Even when there are "skipped mutations" detected, it doesn't mean that there are "lost permutations" - a "skipped permutation" just means that a later permutation arrived before an earlier one arrived/was processed - when the replication DC "caught up" they would be consistent. I can't really think of anything that would cause your situation.
From the XDCR Mutations Skipped graph - did something happen around 8pm when the mutations skipped started to climb? A rebalance, maybe? Or was that just when the kafka intake started?
Is there a possibility that the difference is document counts is temporary, if the replicated DC was allowed to "catch up" they would be the same? I can't think of anything else.
I’ve just pushed https://github.com/couchbase/gocb-opentelemetry which currently uses HEAD of gocb, it will be updated to 2.3.0 when released in July
Thanks for the helpful answers! @chvck
http://localhost:8091/pools/default/buckets/bucketName/docs?skip=0&include_docs=false&limit=100, but it's probably not the most performant option if you're hitting it often. My understanding is the 8091 HTTP server isn't designed for high load.
Hi, I am getting a segmentation fault with the NodeJS client (v 3.1.0) on MacOS 11.4:
PID 26285 received SIGSEGV for address: 0x0 0 segfault-handler.node 0x0000000104bbbf90 _ZL16segfault_handleriP9__siginfoPv + 304 1 libsystem_platform.dylib 0x00007fff20684d7d _sigtramp + 29 2 ??? 0x000000010a1ab6c0 0x0 + 4464490176 3 couchbase_impl.node 0x00000001070281a3 _ZN3lcb4http7Request6finishE10lcb_STATUS + 275 4 couchbase_impl.node 0x000000010702c71b _ZL12on_connectedP12lcbio_SOCKETPv10lcb_STATUSi + 123 5 couchbase_impl.node 0x0000000107036bdb _ZN3lcb2io11PoolRequest6invokeEv + 251 6 couchbase_impl.node 0x0000000107038013 timer_callback + 595 7 node 0x0000000100a000a7 uv__run_timers + 103 8 node 0x0000000100a048dd uv_run + 205 9 node 0x00000001000f2075 _ZN4node16NodeMainInstance3RunEv + 309 10 node 0x00000001000851b6 _ZN4node5StartEiPPc + 294 11 libdyld.dylib 0x00007fff2065af5d start + 1 sh: line 1: 26285 Segmentation fault: 11 ts-node test.ts
Does that sound familiar to anyone? I followed all of Google's suggestions like
rm -rf node_modules && npm install,
ulimit unlimited and
npm rebuildbut to no avail...