Is there a way to access a table cell value? I am trying to parse the last name from a full name column containing string of the form: "last_name,first name" and so I have this code: associates['NAME'].mutate(LAST_NAME=associates['NAME'].split(',')[0]) But get no results since LAST_NAME column is of type ArrayColumn and so indexing does not apply. If I remove the [0] from the end of my split, I get a ArrayColumn, but there doesn't appear to be a way to access the scalar value or the cell value for each row in that column.
@pybokeh I’m not sure about indexing into an ArrayColumn, but for your specific example sounds like re_extract might work:
https://ibis-project.org/docs/generated/ibis.expr.api.StringValue.re_extract.html#ibis.expr.api.StringValue.re_extract
https://ibis-project.org/docs/generated/ibis.expr.api.StringValue.re_extract.html#ibis.expr.api.StringValue.re_extract
Hello, do you know what is the truth for the BigQuery SQL dialect understood by Ibis? Google exposes the parser for the "BigQuery Standard SQL" available under the project ZetaSQL (https://github.com/google/zetasql). Does Ibis uses this? Do you see benefits for Ibis to use this parser?
For more context, I am trying figure how hard is it to add the BigQuery geospatial functions or the ML/AI functions to Ibis.
It is not currently relying on the ZetaSQL project itself (as far as I know there's not a Python wrapper for it yet)
Adding functions is pretty straightforward. There is a guide here: https://ibis-project.org/docs/user_guide/extending/extending_elementwise_expr.html
Hello, I've searched in the API docs, but can't seem to find an equivalent cumcount() function that is available in pandas. In pandas, I would use cumcount() to create a ROW_NUM column, an example pandas code:
df['ROW_NUM'] = df.sort_values(by=['PART_NUM']).groupby(['CLAIM_NUM']).cumcount() + 1 that is equivalent to performing SQL's ROW_NUMBER PARTION BY syntax, where I can then filter to the first record in each grouping (ROW_NUMBER ==1).
Hi all! I can't access the documentation at https://ibis-project.org/docs/ - always get 404. Is it a known issue? (Can't access since yesterday)
where do u see the other link?
http://ibis-project.org is working
this page works for me too. At the top of this page you can see the links to Getting started, Documentation, Community...
So the link to Documentation is not working for me.
And my bookmarks of ibis documentation also not accessible with 404 error (e.g. https://ibis-project.org/docs/backends/impala.html)
also I've noticed that the mysql impl is marked as
experimental in the github page, I guess some features are missing , but is there a list of features that it should implement in order to be considered as "complete"?
maybe having a table with the features each backend implements/lacks would help to know what is in place and what needs to be worked on
experimental is really about having folks support it if there are issues - though that label can prob come off mysql
1 reply
@tilo_buschmann_aok_gitlab I think you would have to create a backend for that. It would be nice if there were a way to have a generic sqlalchemy backend that would cover the basic functionality for any available sqlalchemy dialect. Then it would only be necessary to create new specific backends if you want to leverage any of the functionality that is peculiar to that DB. But last I checked that hasn’t been done
the Binary module (interface?) has been removed from sqlalchemy, I'm pinning sqlalchemy to < 1.4.0 in my conda installation
Alternatively, I could probably make one in http://github.com/googleapis, but will need to go through some red tape to make that happen.
actually looks like marc already did
Hi everybody
I am quite new to ibis.
I am investigating whether I could use ibis with a bigquery backend. The goal is to replace pyspark with bigquery but keeping the possibility to build queries programmatically.
However I am facing some difficulties in understanding some parts of the ibis api.
Is this chat a good place to ask technical questions?
I posted my question on stackoverflow: https://stackoverflow.com/questions/67819129/adding-a-time-interval-to-a-date-column
This is one of my issues, others are about date/datetime operations. It seems that date diff is not supported by the bigquery backend yet.