Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Aug 20 19:18

    erilong on 3.10

    0004072: Potential bottleneck f… Merge branch '3.10' of https://… (compare)

  • Aug 19 19:14

    joshahicks on 3.10

    0004071: Added method to retrie… Merge branch '3.10' of https://… (compare)

  • Aug 16 15:00

    philipmarzullo64 on 3.10

    0004069: SQL Server tests faili… (compare)

  • Aug 16 14:39

    philipmarzullo64 on 3.10

    0004068: Row.getDate(String val… (compare)

  • Aug 15 12:31
    nikidimi synchronize #106
  • Aug 14 14:55
    els0r commented #95
  • Aug 13 17:48

    philipmarzullo64 on 3.10

    0004067: Install using open jdk… (compare)

  • Aug 09 13:00

    erilong on 3.10

    0004066: Potential bottleneck f… (compare)

  • Aug 08 17:34

    philipmarzullo64 on 3.10

    0003560: Unable to use Symmetri… (compare)

  • Aug 08 12:48

    erilong on 3.10

    0004065: Allow zero date on MyS… (compare)

  • Aug 08 12:34

    erilong on 3.9

    0004064: Allow zero date on MyS… Merge branch '3.9' of https://g… (compare)

  • Aug 07 16:58

    joshahicks on 3.10

    0004063: SQL Server table creat… Merge branch '3.10' of https://… (compare)

  • Aug 06 14:36

    joshahicks on 3.11

    DbFill fixes and extract only e… (compare)

  • Aug 05 18:05

    philipmarzullo64 on 3.10

    0004060: Mac: Support Mac servi… (compare)

  • Jul 30 20:20

    joshahicks on 3.10

    0004058: LookupTableRouter shou… Merge branch '3.10' of https://… (compare)

  • Jul 25 19:40

    JJStarrett on 3.10

    0004043: Load Data screen does … Merge branch '3.10' of https://… (compare)

  • Jul 25 18:38

    philipmarzullo64 on 3.10

    0004056: WrapperService needs t… (compare)

  • Jul 25 18:34

    philipmarzullo64 on 3.9

    0004055: WrapperService needs t… (compare)

  • Jul 23 20:07

    JJStarrett on 3.10

    0003726: Delete or truncate sho… Merge branch '3.10' of https://… Merge branch '3.10' of https://… (compare)

  • Jul 23 18:40

    joshahicks on 3.10

    0004054: During extraction of a… Merge branch '3.10' of https://… (compare)

Pradyumna
@Prady5543_gitlab

image.png

But still getting this UI error

zhouz
@zhouz_twitter
@Prady5543_gitlab UI error? What are you trying to get in the browser? There isnt a GUI in the open source symmetricDS if thats what youre trying to do
Pradyumna
@Prady5543_gitlab
can you suggest any front end GUI (Open Source) for monitoring purpose.
Jared
@jaredfrees
@Prady5543_gitlab there is no open source front end GUI for monitoring purposes. You can monitor it with the command line
bhartendu007
@bhartendu007
I am trying to setup 2 way replication between oracle and postgres
in oracle schema name is OPENVOICEED1 where as postgres it is public
this is how i have configured symmetricds tables

-- Channels

insert into sym_channel
(channel_id, processing_order, max_batch_size, enabled, description)

values('announcement', 1, 100000, 1, 'announcement info');

-- Node Groups

insert into sym_node_group (node_group_id) values ('ORA');

insert into sym_node_group (node_group_id) values ('POST');

-- Node Group Links

-- oracle sends changes to postgres when postgres pulls from oracle
insert into sym_node_group_link (source_node_group_id, target_node_group_id, data_event_action) values ('ORA', 'POST', 'W');
-- postgres sends changes to oracle when postgres pushes to oracle

insert into sym_node_group_link (source_node_group_id, target_node_group_id, data_event_action) values ('POST', 'ORA', 'P');

-- Triggers

-- Triggers for tables on channel_id
insert into sym_trigger
(trigger_id,source_schema_name,source_table_name,channel_id,last_update_time,create_time)

values('announcement','OPENVOICEED1','ANNOUNCEMENT','announcement',current_timestamp,current_timestamp);

-- Routers

-- Default router sends all data from oracle to postgres
insert into sym_router
(router_id,source_node_group_id,target_node_group_id,router_type,create_time,last_update_time)
values('oracle_2_postgres','ORA', 'POST', 'default',current_timestamp, current_timestamp);
-- Default router sends all data from postgres to oracle
insert into sym_router
(router_id,source_node_group_id,target_node_group_id,router_type,create_time,last_update_time)
values('postgres_2_oracle','POST', 'ORA', 'default',current_timestamp, current_timestamp);


-- Trigger Routers

insert into sym_trigger_router
(trigger_id,router_id,initial_load_order,last_update_time,create_time)
values('announcement','oracle_2_postgres', 100, current_timestamp, current_timestamp);
insert into sym_trigger_router
(trigger_id,router_id,initial_load_order,last_update_time,create_time)
values('announcement','postgres_2_oracle', 100, current_timestamp, current_timestamp);

i am getting error as : [POST-001] - TriggerRouterService - Could not find any database tables matching 'OPENVOICEED1.ANNOUNCEMENT' in the datasource that is configured
why it is not looking for public.announcement in the postgres side?
let me know if anyone can support here
Mazino Ukah
@mazeeblanke
Hi everyone
Please i have two instances of symmetricds running on my vm using the commands 'bin/sym --port 9000' and 'bin/sym --port 4000' but i want to be able to run both both instances through the linux service. I have tried installing the service using 'bin/sym_service install' in the respective directories. but its not working and when i try to start the server i get an error stating that the port is in use
Mazino Ukah
@mazeeblanke
how can i set up the linux service to use different ports ?
Eric Long
@erilong
@bhartendu007 try this: update sym_router set target_schema_name = 'announcement' where router_id = 'oracle_2_postgres'
@mazeeblanke Instead of setting the port on the command line, set it in conf/symmetric-server.properties. In the conf/sym_service.conf file, you can give the service a unique name with the wrapper.name property.
Mazino Ukah
@mazeeblanke
@erilong thanks
yuphing-ong
@yuphing-ong
Is there a setting in symmetricds to ignore transactions on records older than a certain age/date? e.g. let's say I want to build a data warehouse, so I replicate from source nodes, but the source nodes delete records older than 30 days, so I want to ignore those deletes?
I can't just ignore deletes, since data is still "live" until a certain elapsed time (let's say 7 days)
joshahicks
@joshahicks
You could use the sync on delete condition for this table trigger. This would allow you to only capture values where a date column is within last 7 days.
yuphing-ong
@yuphing-ong
@joshahicks OK let me have a read, thanks
gancity
@gancity1_twitter

Hi, I had google and double checking engine file and registration. i still facing the following problem. Basically i duplicate the H00002 and rename and change all the setting become H00004. Both engine files sit in a same server while the db is pointing to RDS. Before I register H00004, i had double check sym_node, sym_node_host and sym_node_security and there is no H00004 record. After i register H00004 and start my client symmetric with H00004 engine file ONLY. It return me the following error.

~~~Error log~~~~~
2019-07-18 23:19:31,988 ERROR [corp-H00004] [AbstractSymmetricEngine] [symmetric-engine-startup-1] An error occurred while starting SymmetricDS StackTraceKey.init [SymmetricException:2675662801]
org.jumpmind.symmetric.SymmetricException: The configured state does not match recorded database state. The recorded external id is 'H00002' while the configured external id is 'H00004'. The recorded node group id is 'corp' while the configured node group id is 'corp'
at org.jumpmind.symmetric.AbstractSymmetricEngine.start(AbstractSymmetricEngine.java:658)
at org.jumpmind.symmetric.AbstractSymmetricEngine.start(AbstractSymmetricEngine.java:634)
at org.jumpmind.symmetric.ClientSymmetricEngine.start(ClientSymmetricEngine.java:271)
at org.jumpmind.symmetric.web.SymmetricEngineHolder$EngineStarter.run(SymmetricEngineHolder.java:598)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

I even compare the engine file using beyong compare.

Need your advise where i need to check?

image.png
joshahicks
@joshahicks
Changing node groups is difficult and not recommended as all the configuration is built from the node group. In addition to that all the runtime changes like batches etc are routed on node group. The simplest way from here would be stop SDS and drop your SYM_* tables. Restart and it will rebuild based purely on the engine property file
robvis
@robvis
Hi, I have a question on sync_on_incoming batch and syncing within the same database.
I like to sync tables within the same database on event trigger, from incoming SymmetricDs batch or even a column change from another application. I read this is not possible, because the source and targe nodes are the same. Is there way/setting to workaround this? Thanks in advance.
joshahicks
@joshahicks
To sync tables in the same database you would need to user accounts that each default to a different database/schema. This would allow you to setup two SymmetricDS nodes with runtime tables (SYM_*) in different databases/schemas so that they do not conflict with each other. Then you can setup your replication as any other two node setup.
robvis
@robvis
Hi all,
I have an issue on a table reload requests.
I use this sql: insert into sym_table_reload_request (TARGET_NODE_ID, SOURCE_NODE_ID, TRIGGER_ID, ROUTER_ID, CREATE_TIME, LAST_UPDATE_TIME, CREATE_TABLE, DELETE_FIRST) VALUES ('erikvis_webdb_local', 'erikvis_webdb', 'artikel_web',' webdb2webdb_local', current_timestamp, current_timestamp, 1, 1);
When I use 'ALL', 'ALL' for "trigger_id" and "Router_ID", I have the same issue.
Additon: In my Outgoing_Batch record I can see the batch, but the NODE_ID has a value of "-1".
robvis
@robvis
Target Db is FireBird, Source Db is MySQL.
joshahicks
@joshahicks
Do you have group links setup between the source and target nodes
robvis
@robvis
Hi Josh,
Yes I have vice versa.
insert into sym_node_group_link (SOURCE_NODE_GROUP_ID, TARGET_NODE_GROUP_ID, DATA_EVENT_ACTION, SYNC_CONFIG_ENABLED, IS_REVERSIBLE, CREATE_TIME, LAST_UPDATE_BY, LAST_UPDATE_TIME) VALUES ('erikvis_webdb', 'erikvis_webdb_local', 'P', 1, 0, current_timestamp, 'CWS', current_timestamp);
insert into sym_node_group_link (SOURCE_NODE_GROUP_ID, TARGET_NODE_GROUP_ID, DATA_EVENT_ACTION, SYNC_CONFIG_ENABLED, IS_REVERSIBLE, CREATE_TIME, LAST_UPDATE_BY, LAST_UPDATE_TIME) VALUES ('erikvis_webdb_local', 'erikvis_webdb', 'P', 1, 0, current_timestamp, 'CWS', current_timestamp);
zhouz
@zhouz_twitter
hate to bring this one up again but i'm stumped @erilong
now that i'm synching data to both a mysql and an mssql host i cant find a way to get initial.load.delete.first=true to work, the previous workaround of setting initial.load.delete.first.sql=delete from \"%s\" breaks the mysql side where it seems that escaping is already being done?
KashchenkoP
@KashchenkoP
Good day. I have the task of replicating data from MSSQL to Greenplum. But MSSQL supports only active directory authentication . is it possible to solve this problem with SymmetryiDB? And how can i do that? Sorry for my english. ))
zhouz
@zhouz_twitter
@KashchenkoP the jtds driver supports windows authentication, you'll have to include useNTLMv2=true and domain=yourdomain in your connection string, if you encounter issues you'll have to search for issues specific to the jtds jdbc driver as symmetric does all the DB talking via jdbc.
Todd Hayden
@toddjhayden
Hi, I am setting up my first node, but cannot connect to SQL Server. I keep seeing "Failed to connect then my server name and instance. There is not error.
joshahicks
@joshahicks
You may want to try using a tool like Squirrel SQL that will just allow you test a pure jdbc sql connection to your database. Once you have the proper url and credentials you can use the same to provide in the engine file for SymmetricDS. Did you look in the logs/symmetric.log for any additional information
John
@Pizzeria3_twitter
hi everyone. has anyone here tried to use symmetricDS inside a docker machine?
Todd Hayden
@toddjhayden
Anyone know if you can use ODBC?
zhouz
@zhouz_twitter
@toddjhayden you mean like to use an existing ODBC configuration? You would need something to translate or bridge the odbc to a jdbc compatible connection, much easier to just specify a jdbc driver in symmetric with the appropriate jdbc connection string than to set up a bridge
joshahicks
@joshahicks
Todd Hayden
@toddjhayden
@zhouz_twitter I have jdbc-odbc-bridge-jre7.jar. DBeaver uses this to connect to my ODBC connections. I loaded this driver to a local node, but cannot connect to any databases via ODBC. The URL is not complicated jdbc:odbc:{database} so I'm a little lost.
zhouz
@zhouz_twitter
what the exact error message?
Todd Hayden
@toddjhayden
failed to connect null. I'll go look at the log
2019-08-08 16:27:05,270 WARN [gui] [ds] [qtp1861781750-20] Unsuccessful attempt to connect to a database using url: jdbc:odbc:MYDATABASENAME with driver: sun.jdbc.odbc.JdbcOdbcDriver as user: Designer StackTraceKey [NullPointerException:997867951]
John
@Pizzeria3_twitter
@joshahicks thanks!
gancity
@gancity1_twitter

Hi, I have a production environment with up 3 layers of communication store -> region (aws) -> HQ (aws). Our DB is Ms SQL and install at RDS.
I also running symmetric clustering on region layer with 2 region server (each region server have 5 engine file to handle different region request).
Both region server also have network load balance based on AWS setting. At first, the clustering is running fine until last 2 week ago, the data somehow duplicated.
The actual table PK is just an ID identity. And we are using sym_trigger sync_key_names to determine the actual PK for the table. The same table we also have
insert and update action. When the incident happen, a same line of record it had duplicate 10 to 20 times. May I know how the symmetric clustering work? or any
advise for us to take note? We are using symmetric version 3.9.4.

We also have another RDS performance issue when we start all the symmetric. RDS CPU, High IOPS usage when we start up the symmetric.
Currently, we had upgrade the RDS spec to a very high performance server but the CPU usage still remain very high. To minimize the problem,
we had econfigure our symmetric engine
http.concurrent.workers.max=12 (for the region engine),14 (for HQ engine)
purge.retention.minutes=4320
FYI, we will end up have 8 region which going to manage 150 stores data sync up.
However, lesser threads it will cause the data flow in from store to region become very slow.
For this case, is there anything we can further fine-tune?

image.png
ChaoYang
@nuoyaneasy
Hi, for clustering, how do cluster nodes communicate? I started two nodes in different servers with same group, id, sync-url behind an Nginx server by proxy-pass requests to them. Is this supposed to work this way?
When I got my client node registered, one server is working fine. The other prompted error with client wrong password. I mean, should cluster nodes work like this way, not communicate with each other? Or am I wrong with cluster configuration?