tpolecat/doobie

Abhishek Srivastava

@abhishes_twitter

humm... so how will they log to the same destination? I will end up duplicating the config for LogHandler (doobie) and log4cats?

I want to log to a file and not console.

Ideally there should be one cats effect based logging library which plugs into doobie

so that I configure it once for a file, and then all libraries like doobie just use that.

Fabio Labella

@SystemFw

ok, wait a second

there is essentially one cats-effect based logging library, log4cats

the problem is that the doobie hook for logging expects a side-effecting logger

however, that doesn't mean you will end up duplicating the config

have you worked with JVM based logging libraries, based on slf4j?

e.g. logbook or log4j2

Abhishek Srivastava

@abhishes_twitter

very very little

May be i'm asking a wrong question. So in an application where there is business logic and there is intermingled doobie code which is being called in multiple places from business logic. How do I create a single log (in a file) which contains all the logs (both doobie and log from business logic).

is there an example or a tutorial which touches upon this topic. Will be happy to read up rather than make you type everything here

Fabio Labella

@SystemFw

very very little

ah I see

you need to read about slf4j basically

from a coding point of view, there is no config

Abhishek Srivastava

@abhishes_twitter

ok. let me do that first

Fabio Labella

@SystemFw

all these framework use an xml file for config

Abhishek Srivastava

@abhishes_twitter

heading over to youtube :)

Fabio Labella

@SystemFw

the thing you need to do for doobie is write a LogHandler for whatever implementation of slf4j you want to use to back both log4cats and doobie

well, which tbh is going to look the same for both logback and log4j2, the xml file is going to have a different name, but that's it

but it will be a lot clearer once you are more familiar with how that system works

which btw is horrendous :) but we're pretty much stuck with it on the JVM

Rob Norris

@tpolecat

Yeah none of this is great.

jatcwang

@jatcwang:matrix.org

[m]

Hi everyone. I raised a ticket with some ideas on per-database module. lmk what you think before I start :)
tpolecat/doobie#1352

Timofey

@GusevTimofey

Hi there! I found the next in the official documentation: ' Note that arrays of advanced types and structs are not supported by the drive'. So, there is no way at all to fetch data from a database like List[List[(Int, Int)]] for example?

Rob Norris

@tpolecat

You may be able to do it with the low level api or with raw jdbc, depending on the driver.

LeRiton

@LeRiton

Hi everyone

I am new to doobie and this is probably an obvious question, but I'm struggling with some basic SELECT query where I wan't to retrieve nested case classes.
Say we have this very simplistic example (ids, fkeys and stuff ommited for simplicity):

case class Container(name: String, blocks: List[Block])
case class Block(name: String, zone: String)

What's the recommended (read: simple but efficient, as I will have loads of records) way to retrieve all Containers?
I have seen some propositions using PG arrays through array_agg() + custom Get implem (see https://gitter.im/tpolecat/doobie?at=5d78c4dcb9abfa4bd38b1e28) but this seems very unintuitive.

Can you point me on the recommended way of doing this and associated documentation, if any?
Thanks in advance!

Gavin Bisesi

@Daenyth

How is blocks represented in the db? A postgres array?

or are you trying to collect multiple rows from a join into one object

LeRiton

@LeRiton

Sorry for the missing details, yes it's #2, i'm JOINing containers and blocks tables. That's why I mention array_agg().

Gavin Bisesi

@Daenyth

The general advice is to fetch the rows as-is and group them scala side

so assuming post-join you have something like SELECT c.name, b.name, b.zone FROM containers c JOIN blocks b... then you select it as (String, String, String) and groupBy(_._1) or so

Or (my preference) you don't nest objects like that, just deal with the row form - often you go to the effort of grouping, and then the caller just unzips them all back down again, IME

eg Block(containerName, name, zone)

LeRiton

@LeRiton

The general advice is to fetch the rows as-is and group them scala side

Correct me if I misunderstand, but that means either:

collecting way more data on the code side than necessary OR
SELECTing many times for each nested ressource OR
using something like WHERE fk_id = ANY(ARRAY[1, 2, 3...])

all of this seems a bit weird in a world where JOIN is an option.

Rob Norris

@tpolecat

Yeah use a join and select List[(Parent, Child)] and then use groupBy or something.

Or if you don't want the duplicated parent values in memory you can use a stream and do it on the fly.

I guess in some cases (if you're selecting a lot of data for the parent) it may be faster to select the parents first, then select the children with a big IN clause with the parent keys, but unless you're moving a lot of data I wouldn't worry about it.

jatcwang

@jatcwang:matrix.org

[m]

@LeRiton: Shameless plug but I wrote a library for doing this sort of work https://jatcwang.github.io/doobieroll/docs/assembler
It's helpful when you feel like you're writing a lot of these groupBys. For very simple joins like your example I'd just use groupBy to start :)

Gavin Bisesi

@Daenyth

@LeRiton To be clear, I was saying to still do the JOIN in sql. Just that you aggregate the rows by join id after reading the rows, rather than having the db do it and then mapping huge rows with jdbc

LeRiton

@LeRiton

Thanks @tpolecat @Daenyth @jatcwang:matrix.org , it really helps.
I will have to dig a little further to understand how it works, the doobie book is fine but a bit opaque for those who doesn't already masters FP.

Rob Norris

@tpolecat

Yeah, sorry. It's not great.

LeRiton

@LeRiton

Or if you don't want the duplicated parent values in memory you can use a stream and do it on the fly.

Do you know where I can find an example of that? I'm asking existing ressource, don't write it up.

Yeah, sorry. It's not great.

Don't be! The journey is the reward, plus existing documentation is very clear. Required background is not your responsibility.

Rob Norris

@tpolecat

Look at the combinators for fs2.Stream

There's one of them that lets you roll up parent/child stuff, kind of like groupBy.

I forget what it's called.

Gavin Bisesi

@Daenyth

I will have to dig a little further to understand how it works, the doobie book is fine but a bit opaque for those who doesn't already masters FP.

@LeRiton if you have specific things you'd like to see improved, please please feel free to open a github issue/discussion about it. The docs are all in github as well if you have an idea to PR in

jatcwang

@jatcwang:matrix.org

[m]

@LeRiton: https://github.com/jatcwang/doobieroll/blob/3d5428c9b635278573121ad8b773919959819adb/modules/bench/src/main/scala/doobierolltest/SQLComparisonBench.scala#L229

the fs2 combinator is groupAdjacentBy it seems

Where communities thrive

People

Repo info

Activity