@dispalt I have a rather strong opinion on the subject now. I would not use effects to organise a code base in the sense of having a DatabaseEffect, S3Effect, AuthenticationEffect and so on. I think it is better to use regular case classes, traits (for interfaces - no implementation) and a “wiring” library for this. My latest take on the subject is in Haskell: https://github.com/etorreborre/registry but I think the same idea can be applied to Scala (my previous idea on this: https://github.com/zalando/grafter)
Then the signatures of those components can be parametrized by a monad M which is specified at wiring time. Here you have different choices:
IOfor maximum interoperability between the components (because it means “anything can happen”)ZIOwhere you can have some guarantees about the kind of errors which can occur- a monad transformer or some MTL constraints. This is unavoidable if you need to pass something like a
RequestIdacross the calls of your whole application and then you need something like aMonadReader RequestId - an
Eff rstack which you can think of as an “extensible" monad transformer stack
In any case the monad you use needs to be more or less isomorphic to ReaderT r (IO (Either e)) but nothing more. If you add more “effects” to your monad Mit will be very difficult to use concurrency primitives or bracketing operations (at least in Haskell, in Scala the Eff library has some machinery to support this but at the expense of lots of internal complexity).
In my own work, the only large Scala application I created with Eff is using grafter for wiring and has effects like Task, FlowId, Error, Log and that’s all (Log actually should be a component not an effect).
I think that with this kind of set-up we get the best:
- modularity
- encapsulation
- mocking
- interoperability of components
- ease of creation of a subset of the application
- application lifecycle, with a clean assemble/start-up/run/shutdown separation
@etorreborre What do you mean by "Log actually should be a component not an effect", can you expand upon this?
Re the number of useful effects, I agree it seems to plateau; Error, Reader, Task are the usual suspects.
I use State effects myself, but I guess that depends if you're updating stateful memory-based system, or its a stateless system that sends all state to an external datastore. Also, I've externalized random RNG via a second orthogonal state effect at times and been happy with the outcome.
Logging via a Writer still seems to make conceptual sense to me.
Ive always used a monolithic IO effect to date, but Im sympathetic to the argument "if you believe in type-safety, shouldn't that apply to IO effect types too?"
I’ve been playing with tagless final and eff at the same time. I’m still investigating when I should just use tagless final or just eff. I also found eff is much more powerful than tagless when dealing with different effects, because I can use eff to handle multiple effects and I can even wrap an effect with another effect automatically. Some sample code I have tried to use both tagless and eff is as below:
def program[
F1[_]: FlatMap, //Tagless effect 1
F2[_]: FlatMap, //Tagless effect 2
F3[_]: FlatMap, //Tagless effect 3
R: MemberIn[LogOp, ?]: MemberIn[Option, ?]: MemberIn[F1, ?]: MemberIn[F2, ?]: MemberIn[F3, ?]
](
gdpr: GdprOp[F1],
user: UserOp[F2],
console: ConsoleOp[F3]
): Eff[R, Unit]But I’m still not sure if I should just choose one instead of mixing eff and tagless final. any thoughts?
I dont use tagless final myself but from a casual glance it looks like double abstraction/indirection to me.
My heuristic is to use the simplest solution that works, only take on extra complexity when you know you need it. I did FP without monad stacks for several years, only moved to MTL/Eff when I noticed the complexity ended up higher without monads in a large codebase, eg symptoms: implicit context passing, nested for expressions.
@benhutchison you can leave Log as an effect but you don’t have to. Advantages: the Log effect can manage its own environment and be similar to a ReaderT RequestId m ()to log requests with their id. Disadvantage: you cannot wire some logging behaviour for one part of your app and another behaviour for another part (not sure if that’s really needed anyway). Note that if you have Log as a component that does not make your whole stack infected with IO. You can defined the Log component as
data Logger m = Logger {
info :: Text -> m ()
, error :: Text -> m ()
}And have your app instantiated in IO while your test code is instantiated with WriterT IO and you use the Writer instance to collect the logs.
Hi! First of all, thank you for this project - I'm very excited to start using it!
I'm considering migrating from cats-mtl transformers to eff, because abstracting over the 5-level stack I have is really hard with mtl typeclasses. My question would be: how does eff compare in terms of performance to 1. mtl transformers, 2. a fused mtl stack used via typeclasses (assuming for a while it's feasible to remain sane after seeing the resulting signatures)?
final case class StreamT[F[_], A](value: F[Stream[A]]) {
def ap[B](f: F[A => B])(implicit F: Monad[F]): StreamT[F, B] =
StreamT(F.flatMap(f)(ff => F.map(value)((stream: Stream[A]) => stream.map(ff))))
def map[B](f: A => B)(implicit F: Functor[F]): StreamT[F, B] =
StreamT(F.map(value)(_.map(f)))
def flatMap[B](f: A => StreamT[F, B])(implicit F: Monad[F]): StreamT[F, B] =
flatMapF(a => f(a).value)
def flatMapF[B](f: A => F[Stream[B]])(implicit F: Monad[F]): StreamT[F, B] =
StreamT(Monad[F].flatMap(value)(astream => astream.flatTraverse(a => f(a))))
}Eff is not really optimised so I don’t think it will compared favourably in tight loops especially in case 2.I don’t know the answer to your second question. I think that we have to consider that we have a
Streaming effect which can be interleaved with other effects. In that case a “Streaming” effect could be supported by a simple Yield a primitive to return the next element in the stream. However this is likely to be super slow. Most streaming libraries support the idea of “chunking” the data in memory to operate on whole “blocks” at the time. So I would probably turn to one of those libraries
Option[A] or ThrowableEither[A], and some eventually rely on a Clock effect. So when I use Eff[R, ?] as the service's Monad, I then have to include the superset stack of effects, which pollutes the methods when calling them. I was thinking that I can have an all-encompassing IO effect as the base stack, and then each method would have its own if necessary... What's the ideal way to go about handling this? X)
@kusamakura I think having to acknowledge all effects used in that call-tree, in the type signature, is a feature not a bug ;)
Note you can extend the stack in particular parts of your program, see eg http://atnos-org.github.io/eff/org.atnos.site.Cookbook.html
@etorreborre Sorry for the late reply and for the unclear description, but @benhutchison read my intent correctly. It's about choosing between 1) putting all effects of all methods in the interface itself, or 2) putting only necessary effects in the methods themselves. I suppose what I wanted to say was that for choice 1 (which I'm currently using), I have to deal with effects that aren't even in use in some methods. To put it concretely:
import org.atnos.eff._
import org.atnos.eff.all._
import org.atnos.eff.syntax.all._
trait GreetingsRepo[M[_]] {
def get(id: Long): M[String]
def create(id: Long, greeting: String): M[Unit]
}
class EffGreetingsRepo[R: _Eval: _Option: _ThrowableEither] extends GreetingsRepo[Eff[R, ?]] {
private[this] val map = collection.mutable.Map.empty[Long, String]
def get(id: Long): Eff[R, String] =
delay(map.get(id)).flatMap(fromOption(_))
def create(id: Long, greeting: String): Eff[R, Unit] =
map.get(id) match {
case Some(_) => left[R, Throwable, Unit](new Throwable("already exists"))
case _ => delay(map.update(id, greeting)).flatMap(right(_))
}
}
val repo = new EffGreetingsRepo[Fx.fx3[Eval, Option, ThrowableEither]]
// Option[Either[Throwable, String]], ideally just Option[String]
// I realize that I can also take away the Option effect completely,
// in favor of using `either.optionEither` instead of `option.fromOption`
val doGet = repo.get(1L).runEither[Throwable].runOption.runEval.runI think I just need to understand more of Eff, but I guess this isn't so bad since dealing with this only happens at the end of the world anyway. In any case, thank you for your continued effort in this project/space. :)
Eff[R, A] where ConnectionIO |= R is a program having some ConnectionIO effects which means having a ConnectionIO “program” which itself is a set of instructions requiring a database connection to be executed. This is why when you run that effect with runConnectionIO you need to pass a Transactor which contains that connection to the database (actually a connection from a pool of connections)
freer where the order is constrained and that leads to refactoring issues because an interpreter for a given effect imposes a constraint on where the effect shoud live in the stack of effects. But you say you don’t use interpreters! Doesn’t that defeats the point of effects where we want to have an “abstract language” and then provide different implementations? The other thing that Eff supports is “extensible effects” meaning that a given expression using some effects can be embedded in another expression using more effects. And this is quite important to compose programs without having to know the whole context of the application. This leads to quite a lot of complexity in Eff, I agree, and this is why I have been stepping away from effects recently. I still find effects interesting though :-). I am in the process of translating Sandy Mac Guire’s example with simple records-of-functions and it is interesting to see that this cannot be done with simple modules! The reason is that he is using a “generator” effect which in a way can only be interpreted globally, once you have the full program. However in that case I would rather use a streaming library in my code because this is effectively what we want to do in his example.In a way we lose the ideal program he describes at the beginning, in another way his ideal program is effectively a series of stream transformations where we want to abstract some of the steps (where the inputs come from, how to process them, how to output them). Once we can use a stream data structure, we can still abstract over all these components without using effects. Anyway that’s probably not very understandable at this stage but I hope to put this in a blog post soon :-)
Hi @etorreborre Im wondering if you ever tried compiling Eff code with the Dotty compiler?
Dotty is supposed to support the semantics of implicits in Scala 2, but Im finding that Eff code that compiles OK in Scala 2 doesn't compile under dotty.
Perhaps predictably, the problem seems in the interpretation, when the Eff machinery has to strip off effects from the stack, eg runReader. That's powered by that giant implicit "rube-goldberg machine" and perhaps some minor deviation in semantics between the two compilers has thrown it out..
Thinking to make a pair of Scasties, one scala 2.12 & working, same code in dotty failing...
Eff[ProgramStack, Double], I get the following error:No instance found for Member[org.atnos.eff.ErrorEffect.ErrorOrOk, ProgramStack].
The effect org.atnos.eff.ErrorEffect.ErrorOrOk is not part of the stack DealSizeTransformer.this.DealSizeStack
or it was not possible to determine the stack that would result from removing org.atnos.eff.ErrorEffect.ErrorOrOkEff anymore and not even programming much in Scala these days so it might make more sense for someone else to take the lead on maintaining it. Note that @xuwei-k has been super nice in sending lots of small PRs to keep the project up to date but maybe someone wants to take it further in terms of updates and integrations. I will still be around of course to do my best to explain the various corner cases. A migration to Dotty would be super interesting by the way. The various effects of a stack are currently kept in a type-level tree because a type-level list was too slow in terms of compilation times after 6 or seven effects. But that makes the implicits for the Member typeclass a lot more complex than necessary. So it would be cool to see if this has improved with Dotty and the new implicit rules + implementation.
Hi @etorreborre ,
Im still using Eff and would be willing to take the lead to maintain it. I envisage a basic level of maintenance, keeping releases flowing and eventually migrating to dotty, rather than any major new initiatives. Some extensions might get deprecated if they dont have active user communities
I continue to see value in Effectful FP, and Eff is a very nice solution for that. To me, the challenge to Eff is whether other approaches (eg Cats MTL ) are a simpler way to achieve comparable benefits.
(apologies for the slow reply, I havent been checking Gitter frequently and when I do it usually to seek help with a question.)
I knew you would step up @benhutchison! I realize that you already are an admin on the project so the only thing left to me is to make an official announcement and whatever else you need from me (for publishing you will need to create an account on oss.sonatype.org and I can create an issue to give you access to the org.atnos group at issues.sonatype.org).
To me, the challenge to Eff is whether other approaches (eg Cats MTL ) are a simpler way to achieve comparable benefits.
This is an excellent remark and I am leaning towards that conclusion: https://medium.com/barely-functional/do-we-need-effects-to-get-abstraction-7d5dc0edfbef. In Haskell-land I see people doubling up on effect libraries (https://github.com/fused-effects/fused-effects, https://github.com/polysemy-research/polysemy). While those libraries are impressive I think the cases where they would bring a competitive advantage are very few. In the end I managed to recover one cool feature of effect libraries which is the ability to compose / recompose interpreters by using my own DI library (https://github.com/etorreborre/registry).
(apologies for the slow reply, I havent been checking Gitter frequently and when I do it usually to seek help with a question.)
You can set email notifications, I just realized that recently :-)
Hi Eric, thanks for your confidence in me, much appreciated :)
If you could give me a day or so to review the repo, release process and collect any questions I have around routine maintenance tasks, I'll get back to you.
As a maintainer, my goals will be primarily to issue releases against scala and library releases, review PRs and to investigate bug reports, so existing users of Eff (like REA) can proceed with confidence. Probably little new features from here on.
As mentioned in atnos-org/eff#184, I'll probably look to limit the set of supported integrations (eg Doobie, ZIO etc), and encourage future integrators to self-publish, offering a list of links in README.
Yes, I'll probably tap your knowledge and skills from time to time if I get stuck.
If you use Eff AddOn integrations with Twitter, Scalaz, Monix or Doobie, please register your use case at atnos-org/eff#186
Im reviewing which AddOns are supported, and understanding the user base for each is key.
