Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Abhishek Srivastava
@abhishes_twitter
humm... so how will they log to the same destination? I will end up duplicating the config for LogHandler (doobie) and log4cats?
I want to log to a file and not console.
Ideally there should be one cats effect based logging library which plugs into doobie
so that I configure it once for a file, and then all libraries like doobie just use that.
Fabio Labella
@SystemFw
ok, wait a second
there is essentially one cats-effect based logging library, log4cats
the problem is that the doobie hook for logging expects a side-effecting logger
however, that doesn't mean you will end up duplicating the config
have you worked with JVM based logging libraries, based on slf4j?
e.g. logbook or log4j2
Abhishek Srivastava
@abhishes_twitter
very very little
May be i'm asking a wrong question. So in an application where there is business logic and there is intermingled doobie code which is being called in multiple places from business logic. How do I create a single log (in a file) which contains all the logs (both doobie and log from business logic).
is there an example or a tutorial which touches upon this topic. Will be happy to read up rather than make you type everything here
Fabio Labella
@SystemFw

very very little

ah I see

you need to read about slf4j basically
from a coding point of view, there is no config
Abhishek Srivastava
@abhishes_twitter
ok. let me do that first
Fabio Labella
@SystemFw
all these framework use an xml file for config
Abhishek Srivastava
@abhishes_twitter
heading over to youtube :)
Fabio Labella
@SystemFw
the thing you need to do for doobie is write a LogHandler for whatever implementation of slf4j you want to use to back both log4cats and doobie
well, which tbh is going to look the same for both logback and log4j2, the xml file is going to have a different name, but that's it
but it will be a lot clearer once you are more familiar with how that system works
which btw is horrendous :) but we're pretty much stuck with it on the JVM
Rob Norris
@tpolecat
Yeah none of this is great.
jatcwang
@jatcwang:matrix.org
[m]
Hi everyone. I raised a ticket with some ideas on per-database module. lmk what you think before I start :)
tpolecat/doobie#1352
Timofey
@GusevTimofey
Hi there! I found the next in the official documentation: ' Note that arrays of advanced types and structs are not supported by the drive'. So, there is no way at all to fetch data from a database like List[List[(Int, Int)]] for example?
Rob Norris
@tpolecat
You may be able to do it with the low level api or with raw jdbc, depending on the driver.
LeRiton
@LeRiton

Hi everyone

I am new to doobie and this is probably an obvious question, but I'm struggling with some basic SELECT query where I wan't to retrieve nested case classes.
Say we have this very simplistic example (ids, fkeys and stuff ommited for simplicity):

case class Container(name: String, blocks: List[Block])
case class Block(name: String, zone: String)

What's the recommended (read: simple but efficient, as I will have loads of records) way to retrieve all Containers?
I have seen some propositions using PG arrays through array_agg() + custom Get implem (see https://gitter.im/tpolecat/doobie?at=5d78c4dcb9abfa4bd38b1e28) but this seems very unintuitive.

Can you point me on the recommended way of doing this and associated documentation, if any?
Thanks in advance!

Gavin Bisesi
@Daenyth
How is blocks represented in the db? A postgres array?
or are you trying to collect multiple rows from a join into one object
LeRiton
@LeRiton
Sorry for the missing details, yes it's #2, i'm JOINing containers and blocks tables. That's why I mention array_agg().
Gavin Bisesi
@Daenyth
The general advice is to fetch the rows as-is and group them scala side
so assuming post-join you have something like SELECT c.name, b.name, b.zone FROM containers c JOIN blocks b... then you select it as (String, String, String) and groupBy(_._1) or so
Or (my preference) you don't nest objects like that, just deal with the row form - often you go to the effort of grouping, and then the caller just unzips them all back down again, IME
eg Block(containerName, name, zone)
LeRiton
@LeRiton

The general advice is to fetch the rows as-is and group them scala side

Correct me if I misunderstand, but that means either:

  1. collecting way more data on the code side than necessary OR
  2. SELECTing many times for each nested ressource OR
  3. using something like WHERE fk_id = ANY(ARRAY[1, 2, 3...])

all of this seems a bit weird in a world where JOIN is an option.

Rob Norris
@tpolecat
Yeah use a join and select List[(Parent, Child)] and then use groupBy or something.
Or if you don't want the duplicated parent values in memory you can use a stream and do it on the fly.
I guess in some cases (if you're selecting a lot of data for the parent) it may be faster to select the parents first, then select the children with a big IN clause with the parent keys, but unless you're moving a lot of data I wouldn't worry about it.
jatcwang
@jatcwang:matrix.org
[m]
@LeRiton: Shameless plug but I wrote a library for doing this sort of work https://jatcwang.github.io/doobieroll/docs/assembler
It's helpful when you feel like you're writing a lot of these groupBys. For very simple joins like your example I'd just use groupBy to start :)
Gavin Bisesi
@Daenyth
@LeRiton To be clear, I was saying to still do the JOIN in sql. Just that you aggregate the rows by join id after reading the rows, rather than having the db do it and then mapping huge rows with jdbc
LeRiton
@LeRiton
Thanks @tpolecat @Daenyth @jatcwang:matrix.org , it really helps.
I will have to dig a little further to understand how it works, the doobie book is fine but a bit opaque for those who doesn't already masters FP.
Rob Norris
@tpolecat
Yeah, sorry. It's not great.
LeRiton
@LeRiton

Or if you don't want the duplicated parent values in memory you can use a stream and do it on the fly.

Do you know where I can find an example of that? I'm asking existing ressource, don't write it up.

Yeah, sorry. It's not great.

Don't be! The journey is the reward, plus existing documentation is very clear. Required background is not your responsibility.

Rob Norris
@tpolecat
Look at the combinators for fs2.Stream
There's one of them that lets you roll up parent/child stuff, kind of like groupBy.
I forget what it's called.
Gavin Bisesi
@Daenyth

I will have to dig a little further to understand how it works, the doobie book is fine but a bit opaque for those who doesn't already masters FP.

@LeRiton if you have specific things you'd like to see improved, please please feel free to open a github issue/discussion about it. The docs are all in github as well if you have an idea to PR in

the fs2 combinator is groupAdjacentBy it seems