Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Tjad Clark
    @tjad
    numo_ractor.png
    In order to make mxnet.rb ractor compatible, it seems that we'll require numo/narray to be ractor compatible too.
    nevermind, this doesn't extend into numo/narray
    Tjad Clark
    @tjad
    mxnet.rb: mrkn/mxnet.rb#51
    I've looked through the set of C code, and there is no obvious code that looks like interference to cause errors. -fPIC is enabled during compilation (which is normal for ruby extensions), this is the only thing that comes to mind that could possibly cause the error I am receiving for the code change, as the symptoms of the error are similar.
    PIC flag would only be a problem if there is memory dependent code - and I believe there is not memory dependent code (hence this flag should not cause any problems). I only mention it for the similar symptoms
    Kenta Murata
    @mrkn
    @tjad What the version of MXNet are you using currently for mxnet.rb?
    Please tell me also the version of ruby. I want to check the behavior on the same environment as yours.
    dsisnero
    @dsisnero
    I am trying to figure out how to use the compute functions from red-arrow but don't know how to even start. Can anyone get me started? thanks
    dsisnero
    @dsisnero
    I am trying to run unique function on an array
    arr = Arrow::Array.new([0,1,1,1,1,2,3,4,2,6,7])
    fun = Arrow::Function.find('unique') returns a function but I am not sure how to run it. I tried fun.execute(arr) but get TypeError: no implicit conversion of Arrow::UInt8Array into Array
    fun.arity
    Sutou Kouhei
    @kou
    fun.execute([Arrow::ArrayDatum.new(arr)]).value
    I hope that we improve the API until the next release!
    dsisnero
    @dsisnero
    thank you. I will try it out. AlsoIs it possible to use the compute functions on a Arrow::ChunkedArray as well as an array? I noticed that Arrow::Array has instance methods with some of the compute functions but ChunkedArray doesn't.
    Sutou Kouhei
    @kou
    Some of compute functions don't support ChunkedArray yet.
    We can use them by concatenating Arrow::Arrays in ChunkedArray as workaround. (This causes overhead.)
    Right way is implementing ChunkedArray support in Apache Arrow C++.
    We need to implement Arrow::ChunkedArray#combine or something to concatenate Arrow::Arrays.
    I'll do it until the next release.
    dsisnero
    @dsisnero
    I already had submitted a JIRA issue for ChunkedArray combine_chunks. I was trying to see if I could do a faster groupby in ruby using the compute kernels but it would be better if this done in Arrow C++. Thanks for all your work
    Sutou Kouhei
    @kou
    It'll be included in the next release: apache/arrow#9621
    dsisnero
    @dsisnero
    Thank you for adding combine_chunks and array.concatenate
    Henry Tseng
    @henrytseng
    has any one had success installing the gem through a docker image based on apk?
    dsisnero
    @dsisnero
    added some memoryview requests https://bugs.ruby-lang.org/issues/17832, https://bugs.ruby-lang.org/issues/17831, https://bugs.ruby-lang.org/issues/17851 among others - I am not too comfortable with C yet
    Sutou Kouhei
    @kou
    Could you file 17831 and 17832 to https://github.com/ruby/fiddle/ ?
    We can resolve them in Fiddle.
    dsisnero
    @dsisnero
    Sutou Kouhei
    @kou
    Thanks.
    dsisnero
    @dsisnero
    I see red-arrow-flight in the arrow/ruby repository but there is no gem for this.
    dsisnero
    @dsisnero
    Also - looking at the Glib classes there doesn't seem to be any bindings for Arrow::Scalar so cannot use compute function 'add' Arrow::Function.find('add')
    dsisnero
    @dsisnero

    I got the unique compute to work as you showed but can't get sum to work
    fun = Arrow::Function.find('sum')

    sum = fun.execute([Arrow::ArrayDatum.new(ar)]).value

    I get nil as the result where ar= Arrow::Array.new([0,1,1,1,1,2,3,4,2,6,7])
    Sutou Kouhei
    @kou
    We'll release red-arrow-flight in the next release.
    Sutou Kouhei
    @kou
    We should improve scalar support.
    sum should work with the scalar support improvement.
    dsisnero
    @dsisnero
    @kou Thank you for all your great work
    Sutou Kouhei
    @kou
    No problem. Thanks for your report.
    jaspreetsd902
    @jaspreetsd902
    Is there a way I can change the compression type? I'm assuming it uses Snappy by default, i'd like to test with other compression methods as well (e.g. gzip2, LZO, etc)
    Sutou Kouhei
    @kou
    Are you saying about Apache Parquet format? Or other format?