Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    David Bouyssié
    @david-bouyssie
    I'm missing some implementation/specification details regarding the ChronicleQueue data format. I hope I will soon fill this gap.
    @dpisklov talk to you soon
    Dmitry Pisklov
    @dpisklov
    @david-bouyssie what platform are you targeting? We currently support ARM and x64, and as you mentioned we have C++ implementation - so I'm not entirely sure at what you are trying to achieve?
    David Bouyssié
    @david-bouyssie
    @dpisklov sorry, my message was not clear. I was speaking about native programming languages (C like) vs JVM, not in terms of hardware architecture.
    Scala Native is a very interesting technology. It is natively compiled to binaries thanks to the LLVM toolchain, but the high-level language is Scala. So it's very easy to use. And Scala can also run on the JVM.
    My goal is to be able to use the Chronicle-Queue technology on both systems (native and JVM) and to provide IPC features for high throughput streaming without backpressure
    David Bouyssié
    @david-bouyssie
    I plan to deliver the code under the Apache 2 license, since it is the current license of the libchronicle project. Does this makes sense to you?
    Dmitry Pisklov
    @dpisklov
    @david-bouyssie fair enough
    so I take it it's your pet project?
    David Bouyssié
    @david-bouyssie
    let's say it will be the foundation of something that requires this technology ;-)
    @dpisklov but yes for now it's one of my main projects
    Dmitry Pisklov
    @dpisklov
    well the thing here is - if it's required for some commercial development - it might be easier to contact our sales team and talk about our C++ offering (and we soon will have C/Python, too). If it's pet project then it's different story. The main difference is a level of support you will receive
    David Bouyssié
    @david-bouyssie
    @dpisklov I'm working in the public life science field. So the goal is to have an open-source product in the end. But I guess the highest the number of users you have and the better it is for your sales team, since in a long-term perspective it should increase interest and thus support needs.
    Dmitry Pisklov
    @dpisklov
    @david-bouyssie except, we won't be able to support people using your library, and essentially they won't be our users per se. We are a very small company, and our resources are limited, so always paid clients take precedence. Although that doesn't mean we won't help - but you can count on as much help from us as for any other open source user. Hope that makes sense
    David Bouyssié
    @david-bouyssie
    @dpisklov if they use both the Scala Native and your JVM implementation in an IPC architecture, then they will be definitely your users ;-)
    but yes I got your point, if they have issues for the Scala Native part they may ask to you while it's not your business
    Although that doesn't mean we won't help - but you can count on as much help from us as for any other open source user. Hope that makes sense
    Yes it does
    Dmitry Pisklov
    @dpisklov

    but yes I got your point, if they have issues for the Scala Native part they may ask to you while it's not your business

    exactly

    David Bouyssié
    @david-bouyssie
    I will be clear on this, and state that the two products are really independant
    I hope it will be enough
    Dmitry Pisklov
    @dpisklov
    ok cool
    maheswar4b4
    @maheswar4b4
    Hi, I want to try Chronicle-Threads, but the documentation is not clear. How do I get started with this. Please share any link with examples
    Dmitry Pisklov
    @dpisklov
    @maheswar4b4 Depending what you want to use Chronicle-Threads for. It's not a general purpose library
    maheswar4b4
    @maheswar4b4
    Can i use this as a replacement for ExecutorService in java ?
    Dmitry Pisklov
    @dpisklov
    @maheswar4b4 not as a drop-in replacement. Chronicle-Threads library provides an implementation of Event Loop concept https://en.wikipedia.org/wiki/Event_loop
    Abhijith V Mohan
    @avmohan
    In chronicle queue, is there a way to rewind the tailer by k documents?
    The use case is read k documents, process it, but if the processing fails, rewind the tailer so that it can be retried
    Peter K Lawrey
    @peter-lawrey
    You can change the direction() of a tailer is read backward, you can also tailer.moveToIndex(index), though you need to consider what you want to happen if there is less than k messages in the current cycle.
    Dmitry Pisklov
    @dpisklov
    @avmohan you can always store the index from which you started (tailer.index()), and then rewind tailer to that index
    Abhijith V Mohan
    @avmohan
    That's what I've been doing. I was thinking since moveToIndex is a very general method - it can move to any arbitrary index instead of rewinding going back a short distance - that it might be less efficient, and there might be a better way to move back a short distance.
    Abhijith V Mohan
    @avmohan

    @dpisklov I am currently following the approach you suggested. Storing startIndex=tailer.index() and then if the processing fails doing tailer.moveToIndex(startIndex). But for the very first batch, startIndex is 0, which doesn't seem to be a valid index, and moveToIndex fails (returns false). For now, what I'm doing in case of failure is calling tailer.toStart() in case startIndex is 0. That is....

    startIndex = tailer.index();
    // Read maximum k documents, and process them
    if(processingFailed) {
        if (startIndex == 0) {
                readTailer.toStart();
        } else {
                readTailer.moveToIndex(startIndex);
        }
    }

    This works correctly, but it would be nice to know why this special handling is required for the start index, and if i'm doing anything wrong.

    Abhijith V Mohan
    @avmohan
    tailer.index() returns 0 instead of the correct index for a newly created tailer, until tailer.toStart() has been called.
    Madeline Grogg
    @madelinegrogg-rp
    good morning. I'm having issues resolving the new 5.17.33 chronicle-queue version from maven central. 5.17.30 downloads just fine. help please :(
    Dmitry Pisklov
    @dpisklov
    @madelinegrogg-rp I can see 5.17.33 is in OSS repo which means it will get propagated to central eventually. it was only released 1.5hrs ago so give it a few hours and it should be available
    Madeline Grogg
    @madelinegrogg-rp
    ok thank you! will try then
    well it is weird because I can see it here https://repo1.maven.org/maven2/net/openhft/chronicle-queue/5.17.33/
    Madeline Grogg
    @madelinegrogg-rp
    @dpisklov thoughts? ^
    Madeline Grogg
    @madelinegrogg-rp
    @dpisklov appears that the chronicle-queue 5.17.33 pom https://repo1.maven.org/maven2/net/openhft/chronicle-queue/5.17.33/chronicle-queue-5.17.33.pom references a dependency that does not exist
    <groupId>net.openhft</groupId>
    <artifactId>chronicle-bom</artifactId>
    <version>2.17.506</version>
    Dmitry Pisklov
    @dpisklov
    @madelinegrogg-rp seems the bom is stuck in staging repo in nexus, I'll try make sure it's released
    Andreas Mack
    @andrm
    I'm having the same issue as @madelinegrogg-rp
    Dmitry Pisklov
    @dpisklov
    @madelinegrogg-rp @andrm guys please try to use one of previous versions - latest version has been screwed up by nexus not releasing bunch of artifacts (they are stuck in staging repos and it won't let me release, so will have to re-release but got no time right now)
    Dmitry Pisklov
    @dpisklov
    @madelinegrogg-rp @andrm I have re-released the bom in question, should be all available now (within a few hours I recon)
    Jan Nielsen
    @JanStureNielsen
    Should OpenHFT/checkout.sh and OpenHFT/checkout.bat clone the same repos? They are quite different at the moment...and the .bat file likely won't allow a build to start on Windows. I submitted OpenHFT/OpenHFT#32 to get the build working on Linux.
    Dmitry Pisklov
    @dpisklov
    @JanStureNielsen I'm not aware of anyone using those scripts, certainly none of us does
    Jan Nielsen
    @JanStureNielsen
    Yeah -- they look abandoned...I'll submit a PR to remove.
    Brian Rothermich
    @BrianRothermich
    Hi! Is there a recommended way to check for a corrupted .cq4 file? The following discussion references checking if moveToIndex() returns false on a particular index, indicating that index is corrupt, but I am not sure if that is a guarantee: OpenHFT/Chronicle-Queue#628. Any help would be much appreciated.
    Ayub
    @ayubsayyad
    @peter-lawrey in enterprise chronicle, java process writes and c++ reads. I see some padding added in java to make length+4 multiple of 16.
    While c++ application when writing does not add it. How can we write or consume it correctly
    @dpisklov
    blondcrossfit
    @blondcrossfit:matrix.org
    [m]
    I wanted to know how you could figure out how many elements are still present in the Chronicle queue? Can you give me an example or some feedback?