Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Kenta Murata
    @mrkn
    If so, we can simply enclose the MXImperativeInvoke call by rb_thread_call_without_gvl.
    Tjad Clark
    @tjad
    @mrkn Correct. That's some great insight - I've been wondering where rb_thread_call_without_gvl would fit in.
    1 reply
    @mrkn as an aside: I also just caught up with some Ruby 3 features which may change the game for concurrency a bit in the future.
    https://rubykaigi.org/2019/presentations/yukihiro_matz.html
    Kenta Murata
    @mrkn
    @tjad It is better if we could support Ractor.
    Tjad Clark
    @tjad
    Sounds good. Probably a major version change for mxnet.rb ha ha
    Tjad Clark
    @tjad
    In Ruby 2.x, it is not at all possible to truly "free" mxnet's engine from the GV as I imagined, not without creating a ruby mxnet engine implementation. Not desirable.
    This said, giving mxnet engine true parallel capability through Ruby 3.x Ractors as per your suggestion is the only route forward.
    Is it possible to permanently disable a Ractor's Lock mechanism ?
    Tjad Clark
    @tjad
    I've ugraded my mxnet.rb / eventmachine application to use ruby 3.x . It's actually running! Thanks @kou for keeping numo-narray updated!
    This aligns my short term path with Ractor implementation. Moving ahead with mrkn/mxnet.rb#53
    Erik Fonselius
    @Fonsan
    Having trouble installing head, apache-arrow works fine but apache-arrow-glib fails
    20:20 fonsan@Eriks-MBP:~/code/arrow/ruby/red-arrow d1091bbe3(ruby-refactor-table-initialize✗)(ruby-2.7.2)
     $ brew install apache-arrow-glib --head
    Updating Homebrew...
    ==> Auto-updated Homebrew!
    Updated Homebrew from 12495bc80 to 6d850a97a.
    No changes to formulae.
    
    ==> Cloning https://github.com/apache/arrow.git
    Updating /Users/fonsan/Library/Caches/Homebrew/apache-arrow-glib--git
    ==> Checking out branch master
    Already on 'master'
    Your branch is up to date with 'origin/master'.
    HEAD is now at a7e02c4 ARROW-10639: [Rust] Added examples to is_null kernel and simplified signature.
    Entering 'cpp/submodules/parquet-testing'
    Entering 'testing'
    /Users/fonsan/Library/Caches/Homebrew/apache-arrow-glib--git/cpp/submodules/parquet-testing
    /Users/fonsan/Library/Caches/Homebrew/apache-arrow-glib--git/testing
    ==> ./configure --prefix=/usr/local/Cellar/apache-arrow-glib/HEAD-a7e02c4
    Last 15 lines from /Users/fonsan/Library/Logs/Homebrew/apache-arrow-glib/01.configure:
    2020-11-19 20:20:42 +0100
    
    ./configure
    --prefix=/usr/local/Cellar/apache-arrow-glib/HEAD-a7e02c4
    
    
    READ THIS: https://docs.brew.sh/Troubleshooting
    
    Please create pull requests instead of asking for help on Homebrew's GitHub,
    Twitter or any other official channels.
    brew install apache-arrow-glib --head  5.27s user 7.80s system 79% cpu 16.497 total
    20:20 fonsan@Eriks-MBP:~/code/arrow/ruby/red-arrow d1091bbe3(ruby-refactor-table-initialize✗)(ruby-2.7.2)
    This is on Big Sur
    Erik Fonselius
    @Fonsan
    I am not sure where I should be looking for the error logs
    Sutou Kouhei
    @kou
    Ah, we can't use configurehttps://github.com/Homebrew/homebrew-core/blob/master/Formula/apache-arrow-glib.rb#L27 for --head.
    configure isn't included in repository. It's an auto generated file.
    We should change to use meson + ninja in Homebrew like https://github.com/Homebrew/homebrew-core/blob/master/Formula/glib.rb#L55-L57 .
    Could you send a pull request for it to Homebrew?
    Erik Fonselius
    @Fonsan
    I will have a go at it
    Erik Fonselius
    @Fonsan

    @kou Homebrew/homebrew-core#65245

    When I run the tests in red-arrow I get Error: test_uint8(RawRecordsTableStructArrayTest): TypeError: uninitialize GLib::Object for several of them.

    Erik Fonselius
    @Fonsan
    I am not sure this is due to faulty configuration in my homebrew pull or if it is due to another issue
    Sutou Kouhei
    @kou
    Thanks!
    Could you show full error log?
    Erik Fonselius
    @Fonsan
    from the test?
    Sutou Kouhei
    @kou
    Yes.
    Could you also show the command line you used to run the tests?
    Erik Fonselius
    @Fonsan
    rake test
    so everything is passing now
    the error was probably due to trying on a dirty tree
    I reset before running
    my bad
    Sutou Kouhei
    @kou
    No problem. :-)
    Tjad Clark
    @tjad
    numo_ractor.png
    In order to make mxnet.rb ractor compatible, it seems that we'll require numo/narray to be ractor compatible too.
    nevermind, this doesn't extend into numo/narray
    Tjad Clark
    @tjad
    mxnet.rb: mrkn/mxnet.rb#51
    I've looked through the set of C code, and there is no obvious code that looks like interference to cause errors. -fPIC is enabled during compilation (which is normal for ruby extensions), this is the only thing that comes to mind that could possibly cause the error I am receiving for the code change, as the symptoms of the error are similar.
    PIC flag would only be a problem if there is memory dependent code - and I believe there is not memory dependent code (hence this flag should not cause any problems). I only mention it for the similar symptoms
    Kenta Murata
    @mrkn
    @tjad What the version of MXNet are you using currently for mxnet.rb?
    Please tell me also the version of ruby. I want to check the behavior on the same environment as yours.
    dsisnero
    @dsisnero
    I am trying to figure out how to use the compute functions from red-arrow but don't know how to even start. Can anyone get me started? thanks
    dsisnero
    @dsisnero
    I am trying to run unique function on an array
    arr = Arrow::Array.new([0,1,1,1,1,2,3,4,2,6,7])
    fun = Arrow::Function.find('unique') returns a function but I am not sure how to run it. I tried fun.execute(arr) but get TypeError: no implicit conversion of Arrow::UInt8Array into Array
    fun.arity
    Sutou Kouhei
    @kou
    fun.execute([Arrow::ArrayDatum.new(arr)]).value
    I hope that we improve the API until the next release!
    dsisnero
    @dsisnero
    thank you. I will try it out. AlsoIs it possible to use the compute functions on a Arrow::ChunkedArray as well as an array? I noticed that Arrow::Array has instance methods with some of the compute functions but ChunkedArray doesn't.
    Sutou Kouhei
    @kou
    Some of compute functions don't support ChunkedArray yet.
    We can use them by concatenating Arrow::Arrays in ChunkedArray as workaround. (This causes overhead.)
    Right way is implementing ChunkedArray support in Apache Arrow C++.
    We need to implement Arrow::ChunkedArray#combine or something to concatenate Arrow::Arrays.