Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Sutou Kouhei
    @kou
    I hope that we improve the API until the next release!
    dsisnero
    @dsisnero
    thank you. I will try it out. AlsoIs it possible to use the compute functions on a Arrow::ChunkedArray as well as an array? I noticed that Arrow::Array has instance methods with some of the compute functions but ChunkedArray doesn't.
    Sutou Kouhei
    @kou
    Some of compute functions don't support ChunkedArray yet.
    We can use them by concatenating Arrow::Arrays in ChunkedArray as workaround. (This causes overhead.)
    Right way is implementing ChunkedArray support in Apache Arrow C++.
    We need to implement Arrow::ChunkedArray#combine or something to concatenate Arrow::Arrays.
    I'll do it until the next release.
    dsisnero
    @dsisnero
    I already had submitted a JIRA issue for ChunkedArray combine_chunks. I was trying to see if I could do a faster groupby in ruby using the compute kernels but it would be better if this done in Arrow C++. Thanks for all your work
    Sutou Kouhei
    @kou
    It'll be included in the next release: apache/arrow#9621
    dsisnero
    @dsisnero
    Thank you for adding combine_chunks and array.concatenate
    Henry Tseng
    @henrytseng
    has any one had success installing the gem through a docker image based on apk?
    dsisnero
    @dsisnero
    added some memoryview requests https://bugs.ruby-lang.org/issues/17832, https://bugs.ruby-lang.org/issues/17831, https://bugs.ruby-lang.org/issues/17851 among others - I am not too comfortable with C yet
    Sutou Kouhei
    @kou
    Could you file 17831 and 17832 to https://github.com/ruby/fiddle/ ?
    We can resolve them in Fiddle.
    dsisnero
    @dsisnero
    Sutou Kouhei
    @kou
    Thanks.
    dsisnero
    @dsisnero
    I see red-arrow-flight in the arrow/ruby repository but there is no gem for this.
    dsisnero
    @dsisnero
    Also - looking at the Glib classes there doesn't seem to be any bindings for Arrow::Scalar so cannot use compute function 'add' Arrow::Function.find('add')
    dsisnero
    @dsisnero

    I got the unique compute to work as you showed but can't get sum to work
    fun = Arrow::Function.find('sum')

    sum = fun.execute([Arrow::ArrayDatum.new(ar)]).value

    I get nil as the result where ar= Arrow::Array.new([0,1,1,1,1,2,3,4,2,6,7])
    Sutou Kouhei
    @kou
    We'll release red-arrow-flight in the next release.
    Sutou Kouhei
    @kou
    We should improve scalar support.
    sum should work with the scalar support improvement.
    dsisnero
    @dsisnero
    @kou Thank you for all your great work
    Sutou Kouhei
    @kou
    No problem. Thanks for your report.
    jaspreetsd902
    @jaspreetsd902
    Is there a way I can change the compression type? I'm assuming it uses Snappy by default, i'd like to test with other compression methods as well (e.g. gzip2, LZO, etc)
    Sutou Kouhei
    @kou
    Are you saying about Apache Parquet format? Or other format?
    zodiac
    @owzodiac_twitter

    Hey, running into some problems with my Docker container when trying to add the gem:

    installing 'libglib2.0-dev' native package... failed
    11-Aug-2021 17:16:10 Failed to run 'apt install -V -y libglib2.0-dev'
    11-Aug-2021 17:16:10 extconf.rb failed
    11-Aug-2021 17:16:10 Could not create Makefile due to some reason, probably lack of necessary
    11-Aug-2021 17:16:10 libraries and/or headers. Check the mkmf.log file for more details. You may
    11-Aug-2021 17:16:10 need configuration options.
    11-Aug-2021 17:16:10 An error occurred while installing glib2 (3.4.9), and Bundler cannot continue.
    11-Aug-2021 17:16:10 Make sure that gem install glib2 -v '3.4.9' succeeds before bundling.
    11-Aug-2021 17:16:10 In Gemfile:
    11-Aug-2021 17:16:10 red-parquet was resolved to 5.0.0, which depends on
    11-Aug-2021 17:16:10 red-arrow was resolved to 5.0.0, which depends on
    11-Aug-2021 17:16:10 gio2 was resolved to 3.4.9, which depends on
    11-Aug-2021 17:16:10 gobject-introspection was resolved to 3.4.9, which depends on
    11-Aug-2021 17:16:10 glib2

    Sutou Kouhei
    @kou
    Could you open an issue at https://github.com/ruby-gnome/ruby-gnome/issues and provide a Dockerfile that reproduces this case?