Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Fabio Labella
@SystemFw
also, this would be easier with ListT (and therefore fs2.Stream, monix.Iterant, monix.Observable)
but the trick there is to imagine what you would do with F[List[Client]] and String => F[List(DataSpec, DataRef)]
so, first .flatMap to get to List[Client]
then traverse(c => listDataSets(c.id))
actually, flatTraverse
I'll write a snippet down
SemanticBeeng
@SemanticBeeng
thanks. parsing...
Fabio Labella
@SystemFw
yeah I'll just write the code
SemanticBeeng
@SemanticBeeng
This "rings" better than mine, the intuition being to not interrupt the flow with separate, disjoint, mapcalls. Right? Feels like Task create intermediate instances that need special action to run.
Fabio Labella
@SystemFw
yeah once you get there it means (in many but not all cases) that you've strayed off the right path
SemanticBeeng
@SemanticBeeng
oki. That is the reason I reached to Nested.
Fabio Labella
@SystemFw
that's a separate thing
you still need Nested/EitherT
(and ideally, if you could do it with Stream/Observable rather than List, it would be easier still)
but I'll write a solution with List
you actually need EitherT here, not just Nested , because you have some flatMaps
SemanticBeeng
@SemanticBeeng
Refactored apis to List.
Fabio Labella
@SystemFw
object Ex {
  import cats._, data._, implicits._
  import cats.effect._

  type Task[A] = IO[A]
  type DataLakeRepositoryError
  type DataSpecification
  type DataSetRef

  case class Client(id: String)

  def listClients: Task[Either[DataLakeRepositoryError, List[Client]]] = ???
  def listDatasets(clientId: String): Task[Either[DataLakeRepositoryError, List[(DataSpecification, DataSetRef)]]] = ???

  def result: Task[Either[DataLakeRepositoryError, List[DataSetRef]]] =
    EitherT(listClients).flatMap { clients =>
      clients.flatTraverse(c => EitherT(listDatasets(c.id)).map(_.map(_._2)))
    }.value
}
that .map(_.map(_._2)) is a bit ugly, but it is what it is
(sorry for using IO in this channel :P )
SemanticBeeng
@SemanticBeeng
oki, thanks. will try to use that.
Fabio Labella
@SystemFw
you can still use Task ofc, just to be clear
the code doesn't change
SemanticBeeng
@SemanticBeeng
so, no use of Nested, then?
Fabio Labella
@SystemFw
EitherT is similar to Nested
the difference is that Nested works for any F and G, but it can't have flatMap (it's an Applicative but not a Monad)
if you want a Monad you lose the ability to have one type for any F and G
and you need to use a transformer , where one of the types is known (in this case Either) and the other can vary
so you need EitherT in this case since you need flatMap to get to the List of IDs
SemanticBeeng
@SemanticBeeng
Yes, I felt the need for flatMap but liked Nested. Oki.
Fabio Labella
@SystemFw

I felt the need for flatMap but liked Nested

heh, this is the story of 10 years of FP research :joy:

but unfortunately you have encountered a fundamental limitations of effects
SemanticBeeng
@SemanticBeeng
Worked. Intellij cannot parse EitherT(dlRegistry.listDatasets(c.id)).map(_.map(_._2)) but, I will. ;-)
Fabio Labella
@SystemFw
cool, there are a few tricks you can use to learn how to work with types this way
you can get to a solution (in this case) by barely thinking about what the code should do, and only about types and shapes
SemanticBeeng
@SemanticBeeng
So, curious. Is there another way to stitch the same result together from pieces like I was trying? This is elegant but feel the need to cauterize that way of thinking about would help to see what it would take.
Fabio Labella
@SystemFw
wdym by another way? another method or another result?
I think that's how you would do it
you can write functions that basically do what traverse does
e.g. fold with mapN
SemanticBeeng
@SemanticBeeng
"only about types and shapes" - yes, I know. This is why I asked. Other than at this level one cannot really get it.
Fabio Labella
@SystemFw
well, I can give you a few hints
also, Haskell is nicer to learn this specific thing, but it works well in Scala as well
SemanticBeeng
@SemanticBeeng
Same result but somehow obtain the desired Task from the pieces my code creates. Those Task.Map or Task.Suspend
Fabio Labella
@SystemFw

Those Task.Map or Task.Suspend

No, those are bugs

those classes are internals of Task
SemanticBeeng
@SemanticBeeng
Ah. Was right to ask.
Fabio Labella
@SystemFw
it's like saying that you want to work with types of Some and None
you want to work with Option
same thing