by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 18:44
    scala-steward opened #1902
  • Jul 02 17:16
  • Jul 02 16:43
    gabfssilva edited #1901
  • Jul 02 16:21
    kag0 closed #1763
  • Jul 02 16:21
    kag0 commented #1763
  • Jul 01 22:23
    gabfssilva opened #1901
  • Jul 01 17:48
    jgillich commented #1763
  • Jul 01 16:58
    kag0 commented #1763
  • Jul 01 13:40
    yesky12 starred getquill/quill
  • Jul 01 12:40
    juliano labeled #1516
  • Jul 01 12:40
    juliano labeled #1522
  • Jul 01 12:34
    juliano commented #1666
  • Jul 01 12:27
    juliano commented #1750
  • Jul 01 10:40
    tschuchortdev commented #1750
  • Jul 01 09:38
    juliano commented #1879
  • Jun 30 22:22
    rtar starred getquill/quill
  • Jun 30 12:32

    juliano on master

    Update sbt to 1.3.13 (#1900) (compare)

  • Jun 30 12:32
    juliano closed #1900
  • Jun 29 22:09
  • Jun 29 17:53
    jnewman starred getquill/quill
Alexander Ioffe
@deusaquilus
To provide some context on what I have said earlier, I realized at some point that unless we introduce some notion of types into AST, we will never be able to expand subqueries correctly. This problem most notably manifests itself in ExpandNestedQueries and RenameProperties where we have built very, very complex machinery to unfold parent-queries into sub-queries in order to understand what is actually going on. As I piled more and more complexity into this machinery in order to fix existing bugs, more and more bugs appeared on the horizon and at some point, I realized that I was proverbially chasing my own tail. Trying to expand sub-queries based on things selected from super-queries is a fundamentally flawed proposition.
Also, I realized other problems like our boolean handling for non-postgres dialects was fundementally broken and unless we introduce a notion of BooleanValue/BooleanExpression into the AST as types, issues like #1685 can never, ever be properly addressed.
Additionally, after struggling to get quill-spark expansions to work with complex subqueries (having already rewritten the whole mechanism several times), I realized that this machinery is highly dependent on knowing the field-names of select values in super-queries. Things like #1843 are symptoms of this problem but it goes far, far deeper.
Finally, because of the brittle nature of RenameProperties, doing infix operations on renamed columns (and tables) can break query-schema renaming and yield bad results. #1625 is just one symptom of this.
Based on all of these things, I have realized that Quill's AST needs to be substantially changed in order to accommodate some level of typing of root-level elements such as Idents, Entities, and Infixes. This is a substantial change to the core mechanisms operating inside of Quill and doing anything to change them has massive ripple effects throughout the system. Due to this, I have attempted not to make this level of change for the past two years but I recent months I've realized it absolutely has to be done.
Additionally, because inferring Ident-level AST has consequences on Quill-Dotty as well, I decided to make this change now before "that" system evolves to the point where changes of this magnitude require substantial rewrites.
I hope that this explains some of my motivations and the reasons I have been out of the daily loop for so long. I know we have to push out new context with better async handling and I will work on this as soon as I have a free moment.
1 reply
Anton
@outkine
I'm getting an incredibly strange extract is already defined as value extract compilation error when I run queries (but not insert/delete). If anyone knows anything about this, please let me know.
Anton
@outkine
This appears to only happen in Scala 2.12.0
Alexander Ioffe
@deusaquilus
@outkine Can you post an example query?
Anton
@outkine

This happens for all queries, so something as simple as:

case class User {
   username: String
}
ctx.run(query[User])

I've found that upgrading to Scala 2.12.11 fixes this, so it's no longer a problem for me.

Anton
@outkine

I'm also struggling to create Postgresql enum encoders/decoders for an async context (Finagle, in my case). The guide in the README that involves

    encoder(java.sql.Types.OTHER, (index, value, row) =>
        row.setObject(index, value, java.sql.Types.OTHER))

only works with a jdbc context. Has anyone had this problem, or am I misunderstanding the api?

Kenner Stross
@kennerstross
Hello. I'm struggling to utilize postgresql array operators via quill, as in the following example:
select * from cycles where ARRAY[1,9] <@ features;
"features" as an ARRAY[Int] field in the cycles table. The query should select only those records where 1 and 9 are both present in features (although additional features may also be present). I found an old thread from 2017 about adding these type of features, but I can't find them. I've also tried to do this with an infix extension, but so far no luck. Any help is much appreciated. Thx
Kenner Stross
@kennerstross
Actually, the infix is working now (see below), but I'm wondering if this is still the best approach.
implicit class ArrayOps[T](array: Seq[T]) {
    def containsValue(value: T) = quote(infix"ANY($array) = $value".as[Boolean])
    def containsValues(values: Seq[T]) = quote(infix"$values <@ $array".as[Boolean])
  }
Raphael Mäder
@randm-ch
hi all. is there any way to make quill ignore a field completely? looks like insertmeta and updatemeta isn't enough, i need a way to exclude a field from querying too
danslapman
@danslapman
Something went wrong with 3.5.2, there are no artifacts in maven central :(
danslapman
@danslapman
And yeah, website is broken too
Jonathan Ostrander
@j-ostrich
@randm-ch there's a queryMeta.
Alexander Ioffe
@deusaquilus
Fixed the website. Doing a re-release now
Travis CI seems to have changed their permissions for stuff. I might need to regenerate keys
Anton
@outkine

Nontrivial array aggregations throw:

val m = ctx.run(query[Person].groupBy(_.name).map {
  case (name, person) => (name, infix"array_agg(${person.map(p => (p.age, p.name))})".as[Seq[(Int, String)]])
})

throws

trait Seq is abstract; cannot be instantiated
danslapman
@danslapman
2.11 build hanged and publish failed once again :(
Alexander Ioffe
@deusaquilus
@all I have just released 3.5.2. Thanks so much to @danslapman, @juliano, @jilen, @j-ostrich, @senia-psm, @majk-p and everyone else who contributed! I've been out for a while putting together Quill's longer-term strategy so please pardon my absence. Hopefully, in the coming weeks and months, these things will start to materialize and the waiting will have been worth it. Also, I know that there are other PRs in the pipeline so please bear with me in these unusual times.
Now it's 4:30 AM and I need to sleep
danslapman
@danslapman
Thanks @deusaquilus!
Raphael Mäder
@randm-ch
@j-ostrich Thanks. So infix"null".as[Option[String]] on the field that shouldn't be present in the SELECT ... part of the query should work?
ocordsen
@ocordsen
Hi, I'm really happy with the new release and the new scala 2.13 modules. How about quill-codegen-jdbc? It seems like some work has already been done, maybe it's not far away? Maybe I could try to help, if it makes sense, but I think, I would need some guidance.
Alexander Ioffe
@deusaquilus
@ocordsen It should have been published with the other 2.13 modules. Looks like another build mistake. I’m looking into it.
vonchav
@voonchav_gitlab
@deusaquilus How about 2.13 update for the Cassandra modules too? While you're at it, could you see if those C* modules can be updated too? Thank you.
Jonathan Ostrander
@j-ostrich
@randm-ch Yes, that should work, but I could've sworn there was a simpler way to ignore fields in queries.
Raphael Mäder
@randm-ch
I had hoped so as well, but didn't find anything so far. If you have an epiphany, pls let me know :) Cheers!
Alexander Ioffe
@deusaquilus
@ocordsen Quill Codegen 2.13 modules are published
They should be available in maven-central next time it reindexes
ocordsen
@ocordsen
@deusaquilus Great, thank you, it works.
Damian Reeves
@DamianReeves
I'm wondering if quill will work with DB2? I know certain DB provider specific things probably won't work, but should basic queries work? Or is a custom provider needed?
e-Evolution
@e-Evolution

Hello, dear Quill community!

I am using quill as my application's query engine by migrating from an old SQL based persistence engine.
I have the need to access the mapping of the attributes of a case class to the definition of the columns of the database.
I will create a Wrapper for my old classes to the new syntax, how could I access the tuple to know which class case attribute corresponds to the column of the database?

implicit val AgreementClauseCategorySchemaMeta : context.SchemaMeta[AgreementClauseCategory] =
schemaMeta[AgreementClauseCategory] ("SAgreementClauseCategory", .clientId -> "AD_ClientID", .orgId -> "AD_OrgID", .created -> "Created",
.createdBy -> "CreatedBy", .description -> "Description",
.help -> "Help", .isActive -> "IsActive",
.name -> "Name", .agreementClauseCategoryId -> "S_AgreementClauseCategoryID", .uuid -> "UUID",
.updated -> "Updated", .updatedBy -> "UpdatedBy",
_.value -> "Value")
I need a map containing the attribute -> column name , is posible ?
e-Evolution
@e-Evolution
this only a soudo code as that I need :
AgreementClauseCategorySchemaMeta.entity.getDBColumns.foreach { (columntName, value) => 
        //Legacy Persistence code  
        val contract  = new MSContract(Env.getCtx, 0 , null)
        contract.set_ValueOfColumn(columntName, value)
}
Thanks in advance for any tip, regards
jodersky
@jodersky
Is there a convenient way to import only what is needed to run a query? I'd like to avoid wildcard imports if possible
Andreas Våvang Solbrå
@andreavs

Hi, Sorry if this is a basic question, i have a table defined by

case class Person(id: Long, age: Int)

and i would like to run a query where the result is for each age, a list of all the ids with that age (so the return type should eventually be Seq[(Int, Seq[Long])]). I've tried

quote(query[Person].groupBy(p => p.age).map(x => (x._1, x._2.foreach(_.id))))

but i get an error saying No implicit view available from Long => A. Any insights into what i am doing wrong?

Victor Paraschiv
@vicpara
Hi. Are there any example of integrating playframework with quil-async-* (I'm currently using quill-async-postgres). I'm struggling the build the context and read its config from file.
André Gonçalves
@andrerigon_twitter
Hi. I'm trying to filter a table that has an Option[java.util.Date]. How can I do that? Thanks!
Victor Paraschiv
@vicpara
Is there support for jsonb columns in quill-async-postgres ? If yes, are there any examples. If no, are there any workarounds ?
Victor Paraschiv
@vicpara

Is there a way to specify a particular namespace in Postgresql?

As an example, I have a namespace auth that contains a table role defined like below:

CREATE TABLE auth.role
(
    id integer NOT NULL,
    name character varying,
    CONSTRAINT role_pkey PRIMARY KEY (id)
)

ctx.run(quote{query[Role]}) produces the following script that doesn't run when executed directly on postgres: SELECT x.id, x.name FROM role x.

mjsmith707
@mjsmith707

Hi everyone. I'm working on a project using quill with postgres and during some profiling I've seen a huge amount of time spent in quill (not postgres). We have some helper functions like:

def queryVisibleToTenant[T <: FilterableEntity,
                           U <: FilterableTenant](tenantId: UUID)
                                                 (implicit t: SchemaMeta[T], u: SchemaMeta[U]) = {

    quote {
      query[T]
        .join(query[U])
        .on((_, te) => te.id == lift(tenantId))
        .filter(eTe => infix"${eTe._1.lft} BETWEEN ${eTe._2.lft} AND ${eTe._2.rgt}".pure.as[Boolean])
        .map(_._1)
    }
  }

Which we then use like so:

val q = quote {
      queryVisibleToTenant[MyFilterableEntity, MyFilterableTenant](tenantid)
        .join(query[MyFilterableEntity])
        .on((fe, ft) => fe.id == ft.id)
        .map(_._2)
        .filter(_.id == lift(primaryKey))
    }
run(q)

This and others seems to be very very expensive for quill to process. I can post jprofiler screenshots if anyone is interested (and may be bug-worthy in it's own right). I'm wondering if there's some way of preserving the code-reuse of queryVisibleToTenant while speeding this up. If I inline the code from queryVisibleToTenant the query presumably returns to being non-dynamic and generated at compile time and there is a massive speedup.

Alexander Ioffe
@deusaquilus
@mjsmith707 Any profiling information would be highly welcome!
mjsmith707
@mjsmith707
@deusaquilus https://i.imgur.com/JKYQtas.png let me know if you want to see more. I disabled profiling of my code above this (work project, proprietary etc.) but we basically have a collection of model objects with .get functions on them and they all lead to these three places in quill being hotspots. Not all of them are using the above snippet but I've manually inlined that function, reprofiled and observed that it went away.
This is 3.5.2 but I also saw it on 3.5.0**
Here's another view as well https://i.imgur.com/lDLofD5.png
Janghwan Lee
@janghwan

I’m suddenly getting these compile errors which has been compiled fine before.

[error] ……. : not found: value Ord
[error]         .sortBy(_._3)(Ord.desc)

I think DSLs such as Ord or Query should be imported by import ctx._.
I’m not sure what triggered this issue. I was just messing around sbt and scala version.