Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Michal Lacko
    @visox
    Howdy, it there a nice typesafe, injection safe way to have a query with multiple where conditions ? so basically a collection of conditions/fragments ?
    Rob Norris
    @tpolecat
    Yes, you might look at the applied fragments example.
    Michal Lacko
    @visox
    k thx
    Michal Lacko
    @visox
    somehow managed to make it work, intellij isn't really helping :/ ;D
    Swoorup Joshi
    @Swoorup
    is there a way to batch commands in fs2 sink?
    using Session
    Rob Norris
    @tpolecat
    @Swoorup hi, sorry, just back from vacation. Can you say a little bit more about what you're trying to do?
    Session#pipe turns a Command[F, A] into a Pipe[F, A, Completion] which executes the command for each A in the originating stream. If A is a chunk of some fixed length then you can prepare your command to accept such a chunk, but if you can't guarantee this then you'll need to prepare different pipes for each length via PreparedCommand#pipe and then flatMap the originating stream into to the one you want.
    Rob Norris
    @tpolecat
    The pattern where you have N chunks of size M and a terminating chunk of size ≀M might be worth an example.
    Swoorup Joshi
    @Swoorup
    something like this
    val sink: Pipe[F, MarketData, Unit] = s => 
          for 
            chunk <- s.groupWithin(1000, 5.seconds)
            list = chunk.toList.map(_.convert[List[RawMarketDataRecord]]).flatten
            pc <- Stream
                    .resource(pool)
                    .flatMap(session => Stream.resource(session.prepare(insertMarketData(list.length))))
            result <- Stream.eval { 
                        pc
                          .execute(list)
                          .map(_ => ())
                          .recoverWith { case e => println(e); Applicative[F].pure(()) }
                      }
          yield ()
    ?
    it appears to work not sure if this can be simplified
    Rob Norris
    @tpolecat
    If you want to execute each chunk in its own transaction then you can do s.groupWithin(1000, 5.seconds).evalMap { chunk => ... } where ... is chunk => F[Whatever].
    Since this looks like a long-running thing that's how I would do it.
    Swoorup Joshi
    @Swoorup
    yep
    Swoorup Joshi
    @Swoorup
    woot
    TooManyParametersException
    [info] πŸ”₯
    [info] πŸ”₯ Problem: Statement has more than 32767 parameters.
    [info] πŸ”₯ Hint: Postgres can't handle this many parameters. Execute multiple
    [info] πŸ”₯ statements instead.
    [info] πŸ”₯
    why can’t it handle more parameters?
    Swoorup Joshi
    @Swoorup
    its an array of elements consiting 6 parameters
    about 1000 elements
    Rob Norris
    @tpolecat
    It expands to a statement with more than 32k parameters. That’s the most PG can handle.
    Swoorup Joshi
    @Swoorup
    i see
    Swoorup Joshi
    @Swoorup
    so the library is pretty barebones :smile:
    Swoorup Joshi
    @Swoorup
    still getting the exception
    val sink: Pipe[F, MarketData, Unit] = s => 
          s.groupWithin(PostgresConstant.MARKET_DATA_INSERT_CHUNK_SIZE, 5.seconds)
            .map(_.toList.map(_.convert[List[RawMarketDataRecord]]).flatten)
            .covary[F].evalMap {chunk => 
              pool.use { 
                _.prepare(insertMarketData(chunk.length))
                  .use{_.execute(chunk)}
                  .void
              }.adaptError{ 
                case ex =>
                  println(s"Exception: ${ex}")
                  ex
              }
            }
    Rob Norris
    @tpolecat
    How big is the chunk?
    And how many fields does each element in the chunk have? This product can’t be more than 32k
    Swoorup Joshi
    @Swoorup
    max of 256
    yeah it does seem weird
    val marketDataRecord: SkunkCodec[RawMarketDataRecord] =
        (varchar(32)
        *: varchar(32)
        *: timestamptz 
        *: varchar(64)
        *: bool 
        *: bool 
        *: bool.opt 
        *: numeric(30,6)
        *: numeric(18,6))
        .pimap[RawMarketDataRecord]
    256 * 9 = 2304 max
    never mind. uuhhhj
    the groupWithin should be after the flatten thing
    Rob Norris
    @tpolecat
    Print out the length of the chunk just to be sure.
    Ok.
    Aditya
    @aditya-K93
    Pardon me for being a total beginner, after going through the example docs and I'm able to make my service work with skunk(service based tagless final approach passing sessionpool resource). However I'm not able to get queries to work in seq like getIDQuery then use that to doAnotherQuery in sequence. How do I run queries in seq inside context F. My output type is F[Option[Item]]. I can't get it to compile. I'm using a Query type
    Fabio Epifani
    @epifab

    hello! I'm apparently having an issue decoding a field. in particular I have a query like this:
    SELECT 2::float8 + 3::numeric
    I'd expect the result to be numeric, but I get a

    [info] πŸ”₯  The actual and asserted output columns are
    [info] πŸ”₯  
    [info] πŸ”₯    ?column?  float8  ->  numeric  ── type mismatch

    I take this is how postgres works, and nothing to do with Skunk specifically?

    Fabio Epifani
    @epifab

    hi, me again. I've got this:

    val codec = varchar(16)
    sql"SELECT $codec".query(codec)

    if I run it I get a ?column? varchar -> varchar(16) type mismatch. is this expected?

    Swapnil S.
    @Iamswapnil619
    Hi , Is there any module present for testing the skunk queries like we do it for doobie using doobie.scalatest.IOChecker ?
    Swapnil S.
    @Iamswapnil619

    Hi ,

    i have below case class :

    case class StepRun(job_run_id: String, step_name: String, properties: Json )

    where properties is json type . How do i write a codec for this.

    I have written codec as below but it is failing on jsonb type. Error : Not found: jsonb

    def stepRunCodec: Codec[StepRunDB1] = (varchar ~ varchar ~ jsonb).gimap[StepRun]

    Any help ?

    Fabio Epifani
    @epifab
    if you're using circe you can import "org.tpolecat" %% "skunk-circe" % "0.2.0"
    then you have jsonb in skunk.circe.codec.all
    I think you can decode it directly into the final type rather than a generic json
    Swapnil S.
    @Iamswapnil619
    @epifab Got the issue.. Thanks for the help
    Swapnil S.
    @Iamswapnil619

    Hi , Is there a way we can process the dynamic queries in skunk. I have seen all the examples where we can define the queries at compile time and there codecs but is there any provision in skunk where we can execute the dynamic queries directly without knowing the codecs.

    Like in doobie we can do this :

     def executeQuery(query: String): IO[DBException, Unit] = {
            Fragment.const(query)
              .update
              .run
              .transact(transactor).unit
          }

    Any Help ?

    Rob Norris
    @tpolecat
    I'm no longer monitoring this channel. Please switch to Typelevel Discord at https://sca.la/typeleveldiscord
    Michal Lacko
    @visox
    Hi, i dont quite get how to deal with postgres arrays, i see Arr but i dont see how do i get a List out of Arr ??
    basicbang0x
    @basicbang0x

    Hi everyone, I have a session pool, poolResource : Resource[IO, Resource[IO, Session[IO]]]. I use Session.pooled(...).use to get the actual pool, pool: Resource[IO, Session[IO]] and all my postgres services use this pool. However, I get an error that says the following:

    πŸ”₯  
    πŸ”₯  EofException
    πŸ”₯  
    πŸ”₯    Problem: EOF was reached on the network socket.
    πŸ”₯     Detail: Attempt to read 5 byte(s) failed after 0 bytes(s) were read, because
    πŸ”₯             the connection had closed.
    πŸ”₯       Hint: Discard this session and retry with a new one.
    πŸ”₯

    When I use pool.use wouldn't that make a new session for me?

    How can I resolve this issue. Does it involve a certain config?

    Alexandr Oshlakov
    @shadowsmind
    Hi everyone. Who know how work with serial id in skunk, when need insert object without id and return with id?
    Alexandr Oshlakov
    @shadowsmind
    example:
    CREATE TABLE users(
      id        SERIAL NOT NULL,
      email     VARCHAR(255) NOT NULL,
      age       SMALLINT DEFAULT NULL,
      is_banned BOOLEAN NOT NULL DEFAULT FALSE
    );
    
    case class UserInsert(email: String, age: Option[Int])
    case class UserEntity(id: Long, email: String, age: Option[Int], isBanned: Boolean)
    
    trait UserRepository[F[_]] {
      def save(data: UserInsert): F[UserEntity]
    }