@sirinath I get it now, thanks. Initial version was the way you suggested. There can only be single instance of writer for same journal at any given point in time. This is the case even cross processes. Attempt to create second writer instance will result in exception. At the same time there could be multiple simultaneous readers against the same journal. If both reader and writer function is wrapped by a same interface single writer enforcement will be deferred and less clear as some methods will work some will not. Also having single interface hides intent of passing around instance of Journal class.
You need multiple streams hence multiple readers.
It isn't a problem having multiple readers. For SQL implementation and any other concurrent access there is class JournalPool (which should be renamed to JournalFactoryPool), which gives out JournalReaderFactory via get/release methods. It caches factories and readers to avoid opening/closing readers often. You can of course use normal JournalFactory to do the same if performance is not a concern.
This pooling is what I was thinking
The pooling can be done to abstract also
A collection of tables
Which is also a table
You can have views fined as queries
And a DB which has many schemas
Under the hood they are a collection of pools, pools and journals
Actually you can have Calccite as a frondend
This looks cool to have as the front end
May be it can be one front end
This is a very useful idea, in fact my friend is doing very similar project for a bank. It is very useful to integrate legacy data sources under single query system. That said what i'm doing is slightly different. Calcite query system simply would not do for my project for three reasons: its query system does not offer functionality beyond what you get from individual databases, it looks more of an overlap between functionality of data sources it supports (check what kind of query functionality splunk provides vs. calcite). Pick a source file on calcite github and search for "new " operator usage, it is far too many for what i'm building. Third: name sounds strange (https://en.wikipedia.org/wiki/Calcite) what does it have to do with either querying or integration? ;)
may be one day somebody would honour my project by writing an adaptor for calcite? :smile:
Following article might be of interest to you: http://preshing.com/20130107/this-hash-table-is-faster-than-a-judy-array/
Some project I stumbled upon: http://impala.io/, http://getkudu.io/