Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    bledari
    @bledari:matrix.org
    [m]
    Normally I can generate with the default, this is the customization scenario
    nafg
    @nafg
    @akozumpl would that be fixed by slick/slick#2412 ?
    nafg
    @nafg
    @bledari:matrix.org no idea, can you put together a reproducer?
    bledari
    @bledari:matrix.org
    [m]
    @nafg:
    package generate.database
    
    import slick.jdbc.MySQLProfile
    
    import scala.concurrent.Await
    import scala.concurrent.ExecutionContext.Implicits.global
    import scala.concurrent.duration.Duration
    import scala.util.{Failure, Success}
    
    object ReproduceSlickMySqlCodeGenerator extends App {
      if (true) {
        import slick.codegen.SourceCodeGenerator
        // fetch data model
        val modelAction = MySQLProfile.createModel(Some(MySQLProfile.defaultTables)) // you can filter specific tables here
        val modelFuture = wiDB.run(modelAction)
        // customize code generator
        val codegenFuture = modelFuture.map(model => new SourceCodeGenerator(model) {
          // override mapped table and class name
          override def entityName =
            dbTableName => dbTableName.dropRight(1).toLowerCase.toCamelCase
    
          override def tableName =
            dbTableName => dbTableName.toLowerCase.toCamelCase
    
          // add some custom import
          override def code = "import foo.{MyCustomType,MyCustomTypeMapper}" + "\n" + super.code
    
          // override table generator
          override def Table = new Table(_) {
            // disable entity class generation for tables with more than 22 columns
            override def hugeClassEnabled = false
    
            // override contained column generator
            override def Column = new Column(_) {
              // use the data model member of this column to change the Scala type,
              // e.g. to a custom enum or anything else
              override def rawType =
                if (model.name == "SOME_SPECIAL_COLUMN_NAME") "MyCustomType" else super.rawType
            }
          }
        })
    
        codegenFuture.onComplete {
          case Success(codegen) =>
            codegen.writeToFile(
              "slick.jdbc.MySQLProfile", "/", "generate.database", "TablesCustom", "TablesCustom.scala"
            )
          case Failure(_) =>
        }
        //#customization
    
        Await.result(modelFuture, Duration.Inf)
        Await.result(codegenFuture, Duration.Inf)
      }
    }
    nafg
    @nafg
    @bledari:matrix.org I mean something that's easy for me to run, like a github repo, a zip file, or a scastie
    bledari
    @bledari:matrix.org
    [m]
    wiDB is a normal MySQL connection Database.forURL(db_url, driver = jdbcDriver, user = user, password = password, keepAliveConnection = keepAlive)
    nafg
    @nafg
    If there's a mysql or other non-embedded database needed to reproduce then I'll need that too, like a docker-compose setup, or github actions or some other way that doesn't require me to spend a lot of time setting it up for free
    bledari
    @bledari:matrix.org
    [m]
    I'll look at scastie, but really is just this one file with the additional MySQL connection info
    Actually I would like someone to just test generation on any MySQL DB they have on Windows. Even that would help
    nafg
    @nafg
    @bledari:matrix.org do you have reason to believe it's windows-specific?
    Ales Kozumplik
    @akozumpl

    @akozumpl would that be fixed by slick/slick#2412 ?

    yeah exactly!

    bledari
    @bledari:matrix.org
    [m]
    @nafg: nope, linux/os testing sheds light too
    nafg
    @nafg
    @bledari:matrix.org sorry I'm not so available right now, but again if you can prepare something that's very straightforward for me to run I can do it
    Also you could try the code generator I wrote: https://github.com/nafg/slick-additions#slick-additions-codegen
    Bartek Tonia
    @Gryfit
    Hey guys, I haven't really found a complete list of possible logging options for slick, so I have a question:
    Is it possible to log all statements without selects (so only create update delete)?
    I know about "slick.jdbc.JdbcBackend.statement" but it logs all statements .
    Ales Kozumplik
    @akozumpl
    Hi Bartek, currently I don't think the Slick logging machinery has the concept of various kinds of SQL statements. It's all just statements to it.
    1 reply
    Ales Kozumplik
    @akozumpl
    @nafg re: releasing 3.4.0. Should we branch out the 3.4 in git so we leave main open to land other things?
    Ales Kozumplik
    @akozumpl
    We could do stuff like merge the dotty set then, make an early 4.0 build and collect feedback from those who want to use Slick on Dotty projects.
    nafg
    @naftoligug:matrix.org
    [m]
    I'd rather have 4.0 be a branch if anything
    nafg
    @naftoligug:matrix.org
    [m]
    But mostly I think we should get 3.4.0 out the door ASAP. What's needed is
    nafg
    @naftoligug:matrix.org
    [m]
    ... is documenting the changes and splitting out the samples repo. But I think someone is taking care of that
    Gerard Hynes
    @gerhynes

    Hi folks,
    I'm currently writing unit tests for some Scala Data Access Objects that use Slick (3.3.3).

    I don't want to write integration tests at this stage and so I'm attempting to mock the database using ScalaMock.

    However when I try to mock a query hitting the database, I run into this issue:

    overloaded method value expects with alternatives:
      (matcher: org.scalamock.function.FunctionAdapter1[slick.dbio.DBIOAction[Nothing,slick.dbio.NoStream,Nothing],Boolean])org.scalamock.handlers.CallHandler1[slick.dbio.DBIOAction[Nothing,slick.dbio.NoStream,Nothing],scala.concurrent.Future[Nothing]] <and>
      (v1: org.scalamock.matchers.MockParameter[slick.dbio.DBIOAction[Nothing,slick.dbio.NoStream,Nothing]])org.scalamock.handlers.CallHandler1[slick.dbio.DBIOAction[Nothing,slick.dbio.NoStream,Nothing],scala.concurrent.Future[Nothing]]
     cannot be applied to (slick.jdbc.PostgresProfile.StreamingProfileAction[Seq[com.foram.models.UsersTable#TableElementType],com.foram.models.UsersTable#TableElementType,slick.dbio.Effect.Read])
          (mockDB.run _).expects(usersQuery).returning(Future(Seq(sampleUser)))

    I suspect I need some implicit conversion to convert the query into a DBIOAction but can't figure out how to.

    This is what the DAO looks like:

    package com.foram.dao
    
    import com.foram.auth.Auth
    import com.foram.models.{User, UsersTable}
    import slick.jdbc.PostgresProfile
    import slick.jdbc.PostgresProfile.api._
    import slick.lifted.TableQuery
    
    import java.util.UUID
    import scala.concurrent.Future
    
    class UsersDao(db: PostgresProfile.backend.Database) extends AbstractUsersDao {
      def findAll: Future[Seq[User]] = db.run(users.sortBy(_.createdAt.asc).result)
    
      def findById(id: UUID): Future[User] = db.run(users.filter(_.id === id).result.head)
    
      def findByUsername(username: String): Future[User] = db.run(users.filter(_.username === username).result.head)
    
      def create(user: User): Future[UUID] = db.run(users.returning(users.map(_.id)) += Auth.hashUserPassword(user))
    
      def update(id: UUID, user: User): Future[Int] = db.run(users.filter(_.id === user.id).update(user))
    
      def delete(id: UUID): Future[Int] = db.run(users.filter(_.id === id).delete)
    }
    
    // Trait for mocking purposes
    trait AbstractUsersDao {
      val users = TableQuery[UsersTable]
    
      def findAll: Future[Seq[User]]
    
      def findById(id: UUID): Future[User]
    
      def findByUsername(username: String): Future[User]
    
      def create(user: User): Future[UUID]
    
      def update(id: UUID, user: User): Future[Int]
    
      def delete(id: UUID): Future[Int]
    }

    And this is the current state of the unit tests:

    package com.foram.dao
    
    import com.foram.models.User
    import org.scalamock.scalatest.MockFactory
    import org.scalatest.concurrent.ScalaFutures
    import org.scalatest.matchers.should.Matchers
    import org.scalatest.wordspec.AnyWordSpec
    import slick.jdbc.PostgresProfile
    import slick.jdbc.PostgresProfile.api._
    import slick.lifted.{Query, TableQuery}
    
    import java.time.OffsetDateTime
    import java.util.UUID.randomUUID
    import scala.concurrent.ExecutionContext.Implicits.global
    import scala.concurrent.Future
    
    class UsersDaoSpec extends AnyWordSpec with Matchers with ScalaFutures with MockFactory {
      def mockDB = mock[PostgresProfile.backend.Database]
    
      val sampleUser: User = User(randomUUID(), "Quincy Lars", "quincy", "qlars@example.com", "password123", "admin", OffsetDateTime.now(), OffsetDateTime.now())
    
      "UsersDao" should {
        "return a Seq of Users from findAll" in {
          val usersDao = new UsersDao(mockDB)
    
          val usersQuery = usersDao.users.sortBy(_.createdAt.asc).result
    
          (mockDB.run _).expects(usersQuery).returning(Future(Seq(sampleUser)))
    
          val usersFuture = usersDao.findAll
    
          usersFuture.futureValue shouldBe Seq(sampleUser)
        }
      }
    }

    The full code for the project is in this GitHub repo.

    I'd be very grateful if anyone could point me in the right direction.

    jitu121786
    @jitu121786:matrix.org
    [m]
    i have a doubt
    nafg
    @nafg
    @jitu121786:matrix.org You mean a question? Then ask it please
    Sandeep Nayak
    @osgigeek
    Hello newbie here. I just started using Slick with Postgresdb. Could someone point me to an example which shows how to handle nullable foreign key?
    Lucas Della Bella
    @lucasdellabella
    Hi there! I'm fairly new to the JVM ecosystem, and would like to use slick to query an apache pinot instance - they have their own query language which is just a subset of SQL. I was wondering if I have to write my own driver in this case, or if (because pinot query lang is a subset of generic sql) there is something off the shelf that I could use. Maybe @nafg?
    nafg
    @nafg
    This message was deleted
    I'm not sure how much Slick is buying you if it can't do joins etc. Slick is good for composition but what can you compose?
    Sagar Gupta
    @sgrG24
    I am using slick with postgresdb. My question is performance related. I am running a simple query for getting a row based on name (select * from table_name where name = 'some-name'). The table contains 2 rows only. When I ran the query only one time, it rough took 50 ms to return the output. When I ran the same query for 5000 times using a loop, it took around 3000 ms to return for each query. Although preparing the query took only ~1 ms and executing the query took 30 ms roughly(which is same as when I ran the query only one time). So I want to understand from where the extra time in 3000ms is coming? I am using connection pool and running this query on local postgres instance
    nafg
    @nafg
    @sgrG24 it might be from compiling it
    Sagar Gupta
    @sgrG24
    @nafg
    '''
    DEBUG s.compiler.QueryCompilerBenchmark - ------------------- Phase: Time ---------
    DEBUG s.compiler.QueryCompilerBenchmark - assignUniqueSymbols: 0.023818 ms
    DEBUG s.compiler.QueryCompilerBenchmark - unrollTailBinds: 0.004980 ms
    DEBUG s.compiler.QueryCompilerBenchmark - inferTypes: 0.010956 ms
    DEBUG s.compiler.QueryCompilerBenchmark - expandTables: 0.045446 ms
    DEBUG s.compiler.QueryCompilerBenchmark - forceOuterBinds: 0.011063 ms
    DEBUG s.compiler.QueryCompilerBenchmark - removeMappedTypes: 0.010334 ms
    DEBUG s.compiler.QueryCompilerBenchmark - expandSums: 0.000574 ms
    DEBUG s.compiler.QueryCompilerBenchmark - expandRecords: 0.007662 ms
    DEBUG s.compiler.QueryCompilerBenchmark - flattenProjections: 0.017521 ms
    DEBUG s.compiler.QueryCompilerBenchmark - rewriteJoins: 0.003304 ms
    DEBUG s.compiler.QueryCompilerBenchmark - verifySymbols: 0.005054 ms
    DEBUG s.compiler.QueryCompilerBenchmark - relabelUnions: 0.002576 ms
    DEBUG s.compiler.QueryCompilerBenchmark - createAggregates: 0.000468 ms
    DEBUG s.compiler.QueryCompilerBenchmark - resolveZipJoins: 0.002218 ms
    DEBUG s.compiler.QueryCompilerBenchmark - pruneProjections: 0.012427 ms
    DEBUG s.compiler.QueryCompilerBenchmark - createResultSetMapping: 0.005653 ms
    DEBUG s.compiler.QueryCompilerBenchmark - hoistClientOps: 0.021788 ms
    DEBUG s.compiler.QueryCompilerBenchmark - reorderOperations: 0.004274 ms
    DEBUG s.compiler.QueryCompilerBenchmark - mergeToComprehensions: 0.058560 ms
    DEBUG s.compiler.QueryCompilerBenchmark - optimizeScalar: 0.003857 ms
    DEBUG s.compiler.QueryCompilerBenchmark - fixRowNumberOrdering: 0.000493 ms
    DEBUG s.compiler.QueryCompilerBenchmark - removeFieldNames: 0.028186 ms
    DEBUG s.compiler.QueryCompilerBenchmark - codeGen: 0.032082 ms
    DEBUG s.compiler.QueryCompilerBenchmark - TOTAL: 0.313294 ms
    [postgres-18] DEBUG slick.jdbc.JdbcBackend.benchmark - Execution of prepared statement took 18ms
    '''
    The future returned by db.run() method took 5000ms to complete. The preparation time and execution time is very less. Any idea here why it is taking 5000ms
    Andy Czerwonka
    @andyczerwonka
    What is the idiomatic way to handle failures on insert where the new record fails a constraint? E.g. duplicate name on a unique constraint, check constraint, etc?
    nafg
    @nafg
    @andyczerwonka is that a slick question or a database programming question?
    @sgrG24 no idea, can you create a self contained reproducer?
    akozumpl
    @akozumpl:matrix.org
    [m]
    Andy: personally I haven't found anything better than catching the driver-specific exception (they have error codes) and responding accordingly. Perhaps there is a slick extension out there that unifies this for various drivers.
    Oliver Schrenk
    @oschrenk

    I'm switching the test database from h2 to an embedded postgres. It is harder than I thought as it is uncovering some (circular?) table dependency issues:

    When I run

    db.run((Tables.Address.schema ++ Tables.Profile.schema).createIfNotExists).futureValue

    I get

    [info]   The future returned an exception of type: org.postgresql.util.PSQLException, with message: ERROR: relation "profile" does not exist. (PersistenceTest.scala:56)

    and when I switch the order

    db.run((Tables.Profile.schema ++ Tables.Address.schema).createIfNotExists).futureValue

    I get

    [info]   The future returned an exception of type: org.postgresql.util.PSQLException, with message: ERROR: relation "address" does not exist. (PersistenceTest.scala:56)

    Any advice?

    (I'm running 3.4.0-M1 since I ran into similar issues with dropIfExists which were solved by upgrading from 3.3.3 to 3.4.0)

    Arnaud Tanguy
    @ArnaudT

    Hello,

    I have trouble using slick with distinct and sort by a distance (postgis in postgresSQL)
    to run this query :
    It generate a SELECT DISTINCT ON expressions must match initial ORDER BY expressions
    but I can not see how to create a subquery for this

    ```
    val queryCenterLimit = (for {
      (((((((center, _), _), _), _), _), _), _) <- joinQueries
    } yield (center.id, center.location))
      .distinctOn(_._1)
      .sortBy(r => location.bind.distance(r._2))
      .drop(pagination.after.map(_.toInt).getOrElse(0))
      .take(pagination.limit) 
    ```

    Do you see how I can solve this ?
    Thanks for your help

    Carlo Romo
    @caromo

    What is the idiomatic way to handle failures on insert where the new record fails a constraint? E.g. duplicate name on a unique constraint, check constraint, etc?

    Hopping on this question as well, is there an example of error handling with slick? I want to be able to distinguish general db errors with more specific errors (e.g. failed constraints)

    James Phillips
    @jdrphillips
    Hello I have a query of a tuple. I would like to map this, and add a fake None column to it
    IE query.map { case (a, b) => (a, b, None) }
    however None is not a Rep of course, so I can't do this
    how do I input a rep of None into this? I have tried many things and none work
    I am trying to unify this query's type with a separate one that does an extra joinLeft. Note that the None is not a simple type, it is an Option[EntireTableClass] , rather than int or something
    Thus Rep.None does not seem to work, because it is a table rather than a TypedType
    nafg
    @nafg
    @oschrenk can you provide a reproducer please?
    @ArnaudT not sure I understand the question
    @jdrphillips can you wrap it in LiteralColumn
    水山清风
    @djx314
    @jdrphillips Have a try in LiteralColumn(Option.empty[Your Based Type])
    coding4cc
    @coding4cc

    @ArnaudT I have same question in postgresql;

    tables.articleMaterials
        .join(tables.articles).on(_.materialId === _.materialId)
        .filter(_._1.ownerDomain === cmd.domainId)
        .filterIf(cmd.beginTime > 0)(_._1.createTime > cmd.beginTime)
        .filterIf(cmd.endTime < Timestamps.MaxValue)(_._1.createTime <= cmd.endTime)
        .filterIf(cmd.ownerIds.nonEmpty)(_._1.ownerId.inSet(cmd.ownerIds))
        .filterOpt(cmd.kw.filter(Strings.nonEmpty).map(it => s"%$it%"))((it, q) => it._1.remarks.like(q) || it._2.title.like(q))
        .map(it => it._1)
        .distinctOn(_.materialId)
        .sortBy(_.modifyTime.desc)
        .drop(cmd.skip)
        .take(cmd.limit)
        .result

    this query works fine in mysql; in postgresql will cause a exception[SELECT DISTINCT ON expressions must match initial ORDER BY expressions] @nafg

    nafg
    @nafg
    @coding4cc a PR with a test case or even a fix would be appreciated ;)