Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Tomasz Muszynski
    @herki
    Ok I will check.
    Tomasz Muszynski
    @herki
    It's working perfect, I have csv file with separator ';' and every thing working now, thank you.
    Rodney Lopes Gomes
    @rlgomes
    Great!
    Kirill
    @infloop
    Hi! I am researching Juttle. Looks awesome. I have only one question: Can I process every point and do smth externally? I need to increment some counters in Redis and use the increment result? Something like: | put redis_write_command = 'HINCRBY juttle_hash value 1' | put value = write redis | view table
    Mark Stemm
    @mstemm
    Hi, thanks! Does ‘smth’ mean something?
    Kirill
    @infloop
    Edited the question.
    I know that I can't use put after sinks.
    But is it possible somehow?
    Mark Stemm
    @mstemm
    In order to do that, you’d want to fork the program into 2 branches. Something like the following:
    ... | (put redis_write_command = 'HINCRBY juttle_hash value 1' | write redis; view table)
    in one branch, you’re preparing the points to be written to redis. in the other branch, you’re viewing the output in a table
    Kirill
    @infloop
    Will check now.
    Mark Stemm
    @mstemm
    oh if you want the increment to be reflected in the table output, you’d have to do the increment in juttle instead of in the HINCRBY
    something like read redis | put value = value + 1 | (write redis; view table)
    Kirill
    @infloop
    Hmm... That's the problem that I need to increment in Redis...
    And somehow get the value from it back in the middle of the stream.
    Like native Node.js stream Transform.
    Michael Demmer
    @demmer

    Hi @infloop — unfortunately there really isn’t a good way to do this now.

    One idea would be to potentially extend the language so that in addition to read and write there would be a way for an adapter to expose an update implementation that would work as you suggest.

    But this isn’t something we can do right now.
    Kirill
    @infloop
    Oh, it is a pity :( I really love the idea of Juttle. Is it hard to extend the language for an adapter update? I mean I can try to do that.
    Mark Stemm
    @mstemm
    I suppose you could repeat the work of what the HINCRBY does. One thing that wouldn’t be covered is the side-effect stuff of creating the hash if doesn’t exist and initializing the field if it doesn’t exist. And I’m guessing you’re relying on that.
    Kirill
    @infloop
    The value I store in HINCRBY can be updated from different apps, even from different machines.
    So, I have to do that in Redis... I think @demmer offered a good solution.
    Mark Stemm
    @mstemm
    yeah so as @demmer said that kind of operation isn’t a good match for the capabilities of the adapters right now
    Kirill
    @infloop
    Yeah.
    Mark Stemm
    @mstemm
    regardless feel free to file a github issue for the feature request so we can keep track of it when looking for things to add
    Kirill
    @infloop
    I wonder is it hard for me to extend these capabilities?
    Hmm... I think I got the idea in sources. Will try to.
    Thanks.
    Michael Demmer
    @demmer
    Cool — feel free to shoot questions our way. I’m not 100% sure this is the best way to accomplish what you’d want, but there have been other cases where we’ve wanted to do similar things.
    饶琛琳
    @chenryn
    Is @rlgomes here?
    Kirill
    @infloop
    Hi! Is Juttle using Node.js streams under the hood?
    I have been looking into the source for a while and seems not. Am I missing something?
    饶琛琳
    @chenryn
    can't juttle-adopter-elastic support multi-field like path.raw in logstash schema?
    Daria Mehra
    @dmehra
    Hi @chenryn we are just adding full support for nested fields throughout juttle, give it another couple of weeks. Specifically for elastic adapter @davidvgalbraith can comment, maybe there's something we can do straight away for your use case. We do support accessing analyzed fields via field.raw. Are you able to give more detail of what you're trying to do?
    David Majda
    @dmajda
    @infloop No, we don't use Node.js streams, flowgraph nodes just pass around points as objects using regular method calls. In fact, I wasn't aware that it's possible to have non-textual Node.js streams -- I only found out about the object mode now when I looked at the docs.
    Kirill
    @infloop
    @dmajda Ok. I am a bit surprised after a journey into sources (I was absolutely sure that streams are somewhere in the base classes) :)
    @dmajda Do you have any thoughts/plans to use them?
    @dmajda I really want to use Juttle for some heavy stream processing ))
    David Majda
    @dmajda
    Oh, right, they are used in limited way in reading from files, HTTP streams, etc. This is pretty much given by Node.js APIs which use them. But they are not used for sending points around the flowgraph.
    TBH, I'm not sure what using Node.js streams there would get us (beside additional dependency), but @bkutil may want to have a look more deeply.
    Kirill
    @infloop
    In fact is is possible to manually emit points at any given time in Node.js streams. Right, there are some issues with native streams like buffering, but to be honest these things are very rare.
    Ok, I will continue digging...
    饶琛琳
    @chenryn
    @dmajda I just try read elastic -index 'logstash-*' -from :1 year ago: -to :now: | reduce -every :week: count() by 'path.raw', I got the result, but there are 'no known property "path.raw"' warning...
    Rodney Lopes Gomes
    @rlgomes
    @chenryn are you still experiencing the error in the browser ? "a.default.values(...).find is not a function" ?
    饶琛琳
    @chenryn
    @rlgomes yes!
    Rodney Lopes Gomes
    @rlgomes
    ok well looking at the various versions of your os + node + browser those seem perfectly fine so it must be something else
    can you see anything in the browser console logs ?
    饶琛琳
    @chenryn
    TypeError: Object.assign is not a function {stack: (...), message: "Object.assign is not a function"}
    (anonymous function) @ index.js:25
    Rodney Lopes Gomes
    @rlgomes
    Give me a sec I saw something like this but to do with Array methods that wer enot a function and it was chrome specific (ie some bug in an older version of chrome). Give me 2 minutes to figure out if I can repro in an older version