Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 15 2018 17:51
    maartenbreddels commented #108
  • Dec 15 2018 16:44
    rpfk commented #155
  • Dec 15 2018 09:48
    stevesimmons commented #108
  • Dec 15 2018 09:47
    stevesimmons commented #108
  • Dec 15 2018 09:12
    DougRzz closed #100
  • Dec 15 2018 09:12
    DougRzz commented #100
  • Dec 15 2018 08:30
    maartenbreddels commented #100
  • Dec 15 2018 08:28
    maartenbreddels opened #155
  • Dec 15 2018 08:27
    maartenbreddels commented #154
  • Dec 15 2018 08:19

    maartenbreddels on master

    Move import fcntl inside if sta… Merge pull request #154 from rp… (compare)

  • Dec 15 2018 08:19
    maartenbreddels closed #154
  • Dec 15 2018 08:19
    maartenbreddels closed #93
  • Dec 15 2018 08:19
    maartenbreddels commented #154
  • Dec 14 2018 21:52
    rpfk opened #154
  • Dec 14 2018 21:42
    DougRzz synchronize #100
  • Dec 14 2018 21:15
    DougRzz synchronize #100
  • Dec 14 2018 20:36
    DougRzz commented #146
  • Dec 14 2018 19:27
    cclauss opened #153
  • Dec 14 2018 13:14
    nbecker closed #152
  • Dec 14 2018 13:14
    nbecker commented #152
mrogojanu
@mrogojanu
Hi, I am trying to join two data frames by datetime column without success, is this future available in Vaex? Ex: df2=df.join(df1, left_on='Date', right_on='Date') --> KeyError: 'index_hash_datetime64[ns]'
Maarten Breddels
@maartenbreddels
Hi @mrogojanu that sounds like a good, would you mind opening an issue on github?
should not be too difficult to fix
mrogojanu
@mrogojanu
Hi @maartenbreddels, thanks for quick feedback. I will proceed as proposed.
Andrew DiLosa
@adilosa
Hi, I am trying to use Vaex on an EC2 instance running Amazon Linux 2 using Pipenv, and am getting the import error described here vaexio/vaex#464. I've tried the steps mentioned there without any luck (although I am not using Conda). The same Pipenv environment installs Vaex just fine on my Mac, but not on EC2. Does anyone know what needs to be done for Vaex on EC2?
Weirdly, I just tried it in a Docker container on the same EC2 box (base image is python 3.7.6) and it imports Vaex fine in the container
Maarten Breddels
@maartenbreddels
hi @adilosa which OS did you install?
Andrew DiLosa
@adilosa
Hi Maarten, I installed Amazon Linux 2 AMI 2.0.20200520
Andrew DiLosa
@adilosa
Hey all, is there a Vaex pattern that mimics Pandas' explode()method? Or otherwise for dealing with nested records?
MikePfaffenberger
@MikePfaffenberger
Hi all, I have a question about Vaex row slicing...
Is this something that I should assume works?
In [51]: vdf.col.member_id[0:1000]                                                                                                                                                                         
Out[51]: 
Expression = member_id
Length: 1,000 dtype: str (column)
---------------------------------
Error evaluating: ValueError('array is of length 1001, while the length of the DataFrame is 1000')
Jovan Veljanoski
@JovanVeljanoski

Hi @MikePfaffenberger

slicing works like
df['some_column_name'][0:1000]
or
df.some_column_name[0:1000]

alexrabe91
@alexrabe91
hey guys, does anyone know why iam getting "ValueError: buffer is smaller than requested size
" when trying the following approach described in this github issue? https://github.com/vaexio/vaex/issues/812#issuecomment-647447267
Maarten Breddels
@maartenbreddels
arg, my reply is gone after the github outage yesterday
@alexrabe91 you should have a reply now :)
@MikePfaffenberger that should work.. that's odd, that looks like a bug
alexrabe91
@alexrabe91
@maartenbreddels do you have any how-tos for the vaex source installation? always getting into errors that only vaex-meta has been installed, i would like to test pr 517
Maarten Breddels
@maartenbreddels
Thomas Boch
@tboch
hi, when I try to re-read an arrow table exported by vaex, I have the following error:
AttributeError: 'str' object has no attribute 'num_rows'
df = vaex.from_csv('dustpedia.csv')
df.export_arrow('test.arrow')

df = vaex.from_arrow_table('test.arrow')
am I missing something obvious?
BLaurent
@BLaurent
Hi, how many osx do you generate when releasing vaex package. I just got an error
Symbol not found: ____chkstk_darwin
As superstrings as been built for 10.15 and I am on 10.13.6
Maarten Breddels
@maartenbreddels
@tboch use vaex.open, from_table expects an Arrow table object
@BLaurent good question, i just changed how we do the releases, would you mind opening an issue?
Maarten Breddels
@maartenbreddels
@BLaurent it should be 10.9 if I look at https://pypi.org/project/vaex-core/#files
@BLaurent is this installed from conda or pip
BLaurent
@BLaurent
@maartenbreddels pip
BLaurent
@BLaurent
@maartenbreddels vaexio/vaex#892
Maarten Breddels
@maartenbreddels
thanks!
Thomas Boch
@tboch
Is there a way to use external libraries (eg healpy) to compute virtual columns for a vaex df?
I would like to add the HEALPix index from the sky coordinatesin my df
Maarten Breddels
@maartenbreddels
Thomas Boch
@tboch
thanks! I also found add_column_healpix which is exactly what I was looking for
Maarten Breddels
@maartenbreddels
yeah, you're lucky we didn't remove that :)
Thomas Boch
@tboch
;)
alexkim-avant
@alexkim-avant
Hi @maartenbreddels, I am also seeing the error @alexrabe91 ran into for the turbodbc+vaex approach. What was the solution?
Maarten Breddels
@maartenbreddels
@alexkim-avant what exactly is the issue?
alexkim-avant
@alexkim-avant
Encountering the error ValueError: buffer is smaller than requested size when trying to read an arrow table into Vaex using df = vaex.from_arrow_table(batch)
Batches are retrieved from turbodbc's fetcharrowbatches() function
alexkim-avant
@alexkim-avant
@maartenbreddels
Maarten Breddels
@maartenbreddels
did you try the latest alpha (pip install vaex-core --pre) ?
alexkim-avant
@alexkim-avant
Still getting the same error
Maarten Breddels
@maartenbreddels
great, thanks for checking
where do you connect to, which database?
and, can you open an issue, and give me the full stacktrace?
alexkim-avant
@alexkim-avant
Actually I think I found the issue
alexkim-avant
@alexkim-avant
Not an issue with Vaex but rather numpy.frombuffer having a problem with dtype datetime64
Any suggestions to work around this issue?
Maarten Breddels
@maartenbreddels
that might be sth that vaex needs to solve though
would be great if you can open an issue that reproduces it
alexkim-avant
@alexkim-avant
Will do
Maarten Breddels
@maartenbreddels
@alexkim-avant awesome, that would be great, please reference that issue above to keep track of things