Where communities thrive

Join over 1.5M+ people
Join over 100K+ communities
Free without limits
Create your own community

Explore more communities

ibis/Lobby

People

Repo info

Activity

Christopher Ostrouchov

@costrouc

At the moment just trying to get the PR to pass. I was overly ambitious adding more and more operations to a new backend :smile:

Ivan Ogasawara

@xmnlab

not sure if it is just for me .. but it seems ibis-dev/lobby gitter channel is using the new pandas logo ...

Scott Hajek

@scottcode

@xmnlab Yep, looks that way to me too

Ivan Ogasawara

@xmnlab

OT: hey everyone I hope everyone here is safe! #staysafe

Tim Swast

@tswast

I noticed that the existing Hash ValueOp doesn't match the behavior of other databases like BigQuery (ibis-project/ibis#2310) or Postgres (https://ibis-project.org/docs/user_guide/extending_elementwise_expr.html)

Hash takes any type and outputs an int64, whereas BigQuery and Postgres take string/bytes and output string/bytes

I'm wondering if these should try to be reconciled? Or maybe we could add a HashBytes operator?

Estíbaliz Parcero

@esparig

Hello everyone! I wanted to take a look to the project, and I'm trying to set up the dev environment, but apparently the "contribute webpage" is not updated (ci/requirements-dev-3.7-main.yml file doesn't exist) please can anyone give me some insight? thanks

Tim Swast

@tswast

Requirements are now split by backend in https://github.com/ibis-project/ibis/tree/master/ci/deps

Estíbaliz Parcero

@esparig

Thank you Tim

Saurabh Jain

@saurabhsjain

Hi!

Jesús Corrius

@jcorrius

hi there

Marc Garcia

@datapythonista

Hi everybody, we're starting a remote Ibis sprint now, for the ones joining, this is the link to the call: https://meet.jit.si/ibis-sprint

Marc Garcia

@datapythonista

PR with the new tutorials: ibis-project/ibis#2266

Marc Garcia

@datapythonista

Slides on how to do a PR: https://docs.google.com/presentation/d/1rOSYXZPyMe9KXnbVK_xbJzw_-ijxd6bIxndmvPU6L2o/edit?usp=sharing

Ariel Silvio Norberto RAMOS

@asnramos

Hello ..!! Im installing IBIS in my notebook

Thanks @datapythonista ..!!!

Hello @xmnlab ..!!!

Matthew Turner

@matthewmturner

hi all, im working on ibis-project/ibis#2238 (configuring pep8speaks for line length). its unclear to me if its a config issue though since it looks like pep8speaks incorrectly labeled a short line (<79 chars) as long. i also see in pyproject.toml that black is configured for line length of 79, which aligns to what pep8speaks is already checking for. any thoughts on whether this is an actual config issue or something else?

Ariel Silvio Norberto RAMOS

@asnramos

pip install ibis-framework...!!! Its Ok..!!!

@datapythonista , @xmnlab .-

Marc Garcia

@datapythonista

Seems like we're using <79 chars everywhere, so I think that issue was opened by mistake

9 replies

Marc Garcia

@datapythonista

How are people at the sprint doing? We've got a problem in the CI caused by a dependency, there will be a build failing, even if your PR is correct

Two builds actually

Matthew Turner

@matthewmturner

working on the impala backend move, going ok so far. will have a couple questions shortly

CameronHenshaw

@CameronHenshaw

My job called with some work for me, so I might have to wait to this weekend to do more :'(

Marc Garcia

@datapythonista

No worries, you're always welcome

Thanks for joining

Matthew Turner

@matthewmturner

is it expected that the isort precommit hook would change setup.cfg (removing impala from isort known third parties)

Marc Garcia

@datapythonista

Not sure, I don't use the precommit hook, but doesn't sound reasonable

Marc Garcia

@datapythonista

@matthewmturner I reviewed your PR, I commented there why of this behaviour

Matthew Turner

@matthewmturner

@datapythonista thanks! adding responses there

Marc Garcia

@datapythonista

Thanks to everybody who participated in the sprint. Lots of great contributions, we have plenty of PRs ready to merge once the problem in the CI is fixed (we're almost there)

Matthew Turner

@matthewmturner

thanks for your time and guidance! great experience.

Edoardo Abati

@EdAbati_gitlab

Thank you @datapythonista for organising!

Manik Kaushal

@manikkaushal_gitlab

Hi guyzz... Can anyone please help me in answering few questions on ibis performance??

Daniel Kim

@pybokeh

Is there a way to access a table cell value? I am trying to parse the last name from a full name column containing string of the form: "last_name,first name" and so I have this code: associates['NAME'].mutate(LAST_NAME=associates['NAME'].split(',')[0]) But get no results since LAST_NAME column is of type ArrayColumn and so indexing does not apply. If I remove the [0] from the end of my split, I get a ArrayColumn, but there doesn't appear to be a way to access the scalar value or the cell value for each row in that column.

Scott Hajek

@scottcode

@pybokeh I’m not sure about indexing into an ArrayColumn, but for your specific example sounds like re_extract might work:
https://ibis-project.org/docs/generated/ibis.expr.api.StringValue.re_extract.html#ibis.expr.api.StringValue.re_extract

Daniel Kim

@pybokeh

@scottcode That'll do even though I usually try to avoid regex since I often have to google or look-up the regex pattern. Thanks!

jdb

@jdb

Hello, do you know what is the truth for the BigQuery SQL dialect understood by Ibis? Google exposes the parser for the "BigQuery Standard SQL" available under the project ZetaSQL (https://github.com/google/zetasql). Does Ibis uses this? Do you see benefits for Ibis to use this parser?

For more context, I am trying figure how hard is it to add the BigQuery geospatial functions or the ML/AI functions to Ibis.

https://cloud.google.com/bigquery/docs/reference/standard-sql/geography_functions
https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-evaluate (timeseries forecasting)

Tim Swast

@tswast

The BigQuery connector does use Standard SQL syntax

It is not currently relying on the ZetaSQL project itself (as far as I know there's not a Python wrapper for it yet)

Adding functions is pretty straightforward. There is a guide here: https://ibis-project.org/docs/user_guide/extending/extending_elementwise_expr.html

Daniel Kim

@pybokeh

Hello, I've searched in the API docs, but can't seem to find an equivalent cumcount() function that is available in pandas. In pandas, I would use cumcount() to create a ROW_NUM column, an example pandas code:
df['ROW_NUM'] = df.sort_values(by=['PART_NUM']).groupby(['CLAIM_NUM']).cumcount() + 1 that is equivalent to performing SQL's ROW_NUMBER PARTION BY syntax, where I can then filter to the first record in each grouping (ROW_NUMBER ==1).

Daniel Kim

@pybokeh

I did an inelegant work around which was to create a dummy column containing just 1s, then doing a cumsum() on that dummy column using ibis window function. I see there is an ibis.row_number, but not sure how to use that in my example.

Daniel Kim

@pybokeh

I just realized that I can't user ibis.row_number() with a pandas backend. Using a Postgres backend, the ibis.row_number().over(w) does what I need or negates the need for cumcount(). That is pretty neat!

Arsen Kairkanov

@karsen_gitlab

Hi all! I can't access the documentation at https://ibis-project.org/docs/ - always get 404. Is it a known issue? (Can't access since yesterday)

Jeff Reback

@jreback

http://ibis-project.org is working

where do u see the other link?

Arsen Kairkanov

@karsen_gitlab

http://ibis-project.org is working

this page works for me too. At the top of this page you can see the links to Getting started, Documentation, Community...
So the link to Documentation is not working for me.
And my bookmarks of ibis documentation also not accessible with 404 error (e.g. https://ibis-project.org/docs/backends/impala.html)