Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
vrajat
@vrajat
So quark can't do anything. That's okay in the test
Connect should succeed. If it's throwing an exception then it's worth an investigation
adeshr
@adeshr
but connection wont get created, since there is no data in db
server would be up, but DriverManager.connect will fail
vrajat
@vrajat
Because there are no rows in the data source table ?
I still don't understand why connection should fail just because there is no data in the catalog
adeshr
@adeshr
Because there is no data in data source table
Amogh Margoor
@amoghmargoor
I think DriverManager.connect should not fail even if there is no data in catalog ... might be something we missed ... we need to fix it
after DDL support ...most people will start empty itself and are expected to fill data later
so we will have to fix it eventually
vrajat
@vrajat
Yes
It's valid to start with no data and even no tables
if there are no tables , migration should take care
if there is no data DDL statements will be used to add it
vrajat
@vrajat
@adeshr still confused by the expected behaviour ?
adeshr
@adeshr
no, understood it...but thinking about if I can do it together with server or we need to create a new issue?
vrajat
@vrajat
write the test. lets see
adeshr
@adeshr
wrote the test already, connection is failing, since we dont have data. Maybe we can use @ignore on test, and will remove it once db catalog have the required functionality?
vrajat
@vrajat
add ignore and push it.
I’ll check why
then we can decide if we want to take it up in a follow up jira or not.
adeshr
@adeshr
sure
vrajat
@vrajat
though I think it has to go in this sprint.
or with DDL statements.
adeshr
@adeshr
@vrajat pushed the test.
vrajat
@vrajat
ok
adeshr
@adeshr
@vrajat json catalog is also supported with server
vrajat
@vrajat
oh right
remove that sentence
adeshr
@adeshr
sure
adeshr
@adeshr
#32 is ready for merge.
adeshr
@adeshr
@vrajat @amoghmargoor need to understand how should I start #36 ?
vrajat
@vrajat
@amoghmargoor is your best bet @adeshr :)
adeshr
@adeshr
sure, will ask him
vrajat
@vrajat
@adeshr @amoghmargoor feed back on the current git flow of rebase and “merge” from PR ?
should we go back to squash merge ?
vrajat
@vrajat
@amoghmargoor #43 is finally ready for a merge
adeshr
@adeshr
:+1: for rebase, no need for command line
Amogh Margoor
@amoghmargoor
@vrajat Will merge #43
+1 for merge using PR... That is easier
vrajat
@vrajat
ok. thx
vrajat
@vrajat
@adeshr are these files required ? https://github.com/qubole/quark/tree/master/server/src/main/resources
for e.g. dbCatalog.json is not maintained
adeshr
@adeshr
dbCatalog.json is not required. must have missed removing it.
Logback.xml can also be removed, the server will pick it from fatjdbc/main/resources
vrajat
@vrajat
ok. I’ll clean them up
adeshr
@adeshr
sure
adeshr
@adeshr

Context:
1) Created two tables warehouse (in defaultDataSource) and warehouse_partition (views) using tpcds.sql files.
2) Added different set of data in both tables.
3) started testing (with data-sources tables having 2 rows, one for default data-source and other for views, without empty views table)

Error:
Now I created a warehouse_partition view using DDL statement and then tried 'select count() from warehouse w_warehouse_sq_ft > 200'
but this failed with error:
java.sql.SQLException: Error while executing SQL "select count() from canonical.public.warehouse as wr where wr.w_warehouse_sq_ft > 200": null
at com.qubole.quark.planner.MetadataSchema$1.createTable(MetadataSchema.java:99)
at org.apache.calcite.materialize.MaterializationService.defineMaterialization(MaterializationService.java:138)
at com.qubole.quark.planner.MetadataSchema.initialize(MetadataSchema.java:122)
at com.qubole.quark.sql.QueryContext.<init>(QueryContext.java:111)

But the same query passes if the warehouse_partition view was added before starting the test (instead of using the DDL).
Not sure if I am setting up things wrong, or missing something, any suggestions?

Amogh Margoor
@amoghmargoor
@adeshr will need to look at your changes ... what branch are you working on ??
adeshr
@adeshr
viewsupport (https://github.com/adeshr/quark/tree/viewsupport)
Filename: DDLViewTest.java
Amogh Margoor
@amoghmargoor
@adeshr looking at it now .... was away yesterday