Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Ryan Peters
@sloshy
Observable[Unit] would actually be able to emit elements - ()
So if you have an Observable[Nothing] that means effectively it is empty.
@crakjie here's a simple example using List you can try yourself:
scala> val a = List.empty[Nothing]
a: List[Nothing] = List()

scala> val b = List(1, 2, 3)
b: List[Int] = List(1, 2, 3)

scala> a ++ b
res0: List[Int] = List(1, 2, 3)

scala> val c = List.empty[Unit]
c: List[Unit] = List()

scala> a ++ b ++ c
res1: List[AnyVal] = List(1, 2, 3)
Note how the last list has basically lost its type information, whereas the common supertype for a ++ b is actually still List[Int]
Ryan Peters
@sloshy
You'll note that no other type extends Nothing, so if you want a List[Nothing] or an Observable[Nothing] you cannot actually have any elements in it because they cannot exist in the scala type system.
Whereas there is exactly one possible Unit element, and Unit cannot take the place of concrete types.
So Nothing is a very good type to use for something that contains nothing, or emits nothing, so you can still use it in cases with more specific types, whereas Unit typically (but not always) means some side-effect is being performed.
Nathaniel Fischer
@kag0
Is there any more documentation for Local? Especially how it could be used with other effect monads like cats IO as discussed here monix/monix#880
etienne
@crakjie

@sloshy I agree with that but the problem is due to covarience Observable[Nothing] <: Observable[Int] wich is a problem when you relie on the compiler to detect errors
exemple

def fun() : Observable[Int] = {
  Observable(1,2,3,4).completed
}

val consumer : Consumer[Int] = ...
fun().consumeWith(consumer).runToFuture

Here the consumer will never see any number wich can be quite confusing when looking at return types.

Piotr Gawryś
@Avasil

@kag0 Not much. There were a lot of changes recently and we're spending most of our efforts to push for 3.0.0 release. I have plans to write more docs/blog post with example repo but I won't find time to do it before October. Feel free to ask questions, at the very least I can copy answers to the docs

As long as you use TracingScheduler it should work for Future but you will have to call Local.isolate { f } for the ones that should be completely isolated from the rest. In the current model we pretty much always share locals unless we tell otherwise. We can do it withisolate, bind etc. which is also called in Task during running it so it works in many cases without any additional effort. We're probably going to stick with it for 3.0.0, gather feedback and reevaluate it for 3.x

I don't know about IO/ZIO, I don't expect it to fully work without any extra support.

@crakjie The behavior is expected (it's mentioned in the scaladoc). I guess it can be confusing but I'd say it's like Observable.empty[Int] or List.empty[Int]
Nathaniel Fischer
@kag0
@Avasil so I'm trying it out, does this look right?
implicit val sched = TracingScheduler(ExecutionContext.global)
val local = Local(0)

val f1 = for {
  _ <- Future.successful("shift")
  _ <- Future.successful{
    println(Thread.currentThread.getName + s" 1- ${local()}")
    assert(local() == 0)
    local := 50
  }
  _ <- Future.successful{
    println(Thread.currentThread.getName + s" 1- ${local()}")
    assert(local() == 50)
  }
  _ <- Local.isolate(Future.successful{
    println(Thread.currentThread.getName + s" 1- ${local()}")
    assert(local() == 50)
    local := 100
  })
  _ <- Future.successful{
    println(Thread.currentThread.getName + s" 1- ${local()}")
    assert(local() == 50)
  }
} yield ()

val f2 = Local.isolate(for {
  _ <- Future.successful("shift")
  _ <- Future.successful{
    println(Thread.currentThread.getName + s" 2- ${local()}")
    assert(local() == 0)
    local := 5
  }
  _ <- Future.successful{
    println(Thread.currentThread.getName + s" 2- ${local()}")
    assert(local() == 5)
  }
  _ <- Local.isolate(Future.successful{
    println(Thread.currentThread.getName + s" 2- ${local()}")
    assert(local() == 5)
    local := 10
  })
  _ <- Future.successful{
    println(Thread.currentThread.getName + s" 2- ${local()}")
    assert(local() == 5)
  }

} yield ())

Await.result(f1, Duration.Inf)
Await.result(f2, Duration.Inf)
Piotr Gawryś
@Avasil
yeah
is that what you expect?
Nathaniel Fischer
@kag0
I think I find myself expecting the outer Local.isolate to be implied. But it makes sense when I think about it, and that could be invoked as part of a framework or something and logic code wouldn't need to worry about it.
I'm pleasantly surprised it works so cleanly.
Swapping the futures for IOs does not seem to work so well. I was hoping a IO.contextShift(TracingScheduler(ExecutionContext.global)) would do it but it seems like not.
Piotr Gawryś
@Avasil

Local.isolate won't work for lazy structures, Task uses:

    Task {
      val current = Local.getContext()
      Local.setContext(current.mkIsolated)
      current
    }.bracket(_ => task)(backup => Task(Local.setContext(backup)))

it might help a bit but there is also some logic encoded in Task run-loop itself

automatically isolating outer Future would be awesome but challenging to implement (we have to somehow detect that it is the first Future). Maybe one day x)
Nathaniel Fischer
@kag0
So how would one rewrite the above test to use Tasks rather than Futures?
Is it as simple as swapping all the Locals for TaskLocals? Because that wouldn't be too bad to start with Future and migrate later.
Piotr Gawryś
@Avasil
Yes, and if you call the Task separately, let's say runToFuture instead of Task.parZipthen you get outer isolate for free
in many cases it's probably all you need if you want to set sth like correlationId
Nathaniel Fischer
@kag0
Yes, that's exactly what I need to set 🙄 There are very few other cross cutting concerns I'd want to put in something like a Local, but when you need it nothing's better.
I'm interested to watch how it evolves in 3.x. It would be great if local context propagation made its way into cats, but I don't know how likely that really is.
Piotr Gawryś
@Avasil
I feel like many proficient cats users/maintainers prefer to use Reader / ApplicativeLocal for tracing so there is not enough demand to incorporate it into c.e.IO directly. I think @oleg-py who came up with our current Local model was thinking about supporting IO / F[_] without any support there so who knows, maybe it will happen one day
Nathaniel Fischer
@kag0
That makes sense. For me (not a super heavy cats user) I'd rather log a warning if a correlation id is missing on the thread than add a new type everywhere. Tracing just doesn't warrant the rigor.
JosDenmark
@JosDenmark

Hi all,

My team is under the impression that (using the default global scheduler) calls to parTraverse make use of "green threads" and I'd like to evaluate the validity of that claim. I've seen elsewhere that JVM doesn't support green threads in the sense that JVM can't spawn hundreds of thousands of micro-threads that will magically make over-parallelized code more efficient. Is there a preferred resource for describing Task's relationship to green threading? What is Fiber's role?

toxicafunk
@toxicafunk
generally speaking, a Fiber is a green thread, but afaik monix does not implement them
Fabio Labella
@SystemFw
that's not true :)

. I've seen elsewhere that JVM doesn't support green threads in the sense that JVM can't spawn hundreds of thousands of micro-threads that will magically make over-parallelized code more efficient

You are confusing concurrency and parallelism here. Threads and green threads are primarily about concurrency, which is a related but separate thing from parallelism

. I've seen elsewhere that JVM doesn't support green threads I

The JVM doesn't support green threads in the sense that when you spawn a JVM thread (the Thread class), it maps one to one with an operating system thread

since operating system threads have a high cost of switching between them, you can only have so many of them, they are a scarce resource
in turn, this means that if you block a JVM thread (Thread) waiting for a result (e.g. an HTTP call), there is only so many you can block before you exhaust them all
green threads otoh are more lightweight, so you can spawn many more, and they all run using a small number of JVM threads
or, if you run on Javascript, only one thread
libraries that interact with cats-effect, so cats-effect itself, fs2 or Monix, all support this model of concurrency, which is represented by the Fiber interface
toxicafunk
@toxicafunk
oh, u get it through cats-effect :-D
Fabio Labella
@SystemFw
well, in the case of Monix is not like you get it from cats-effect, Monix.Task has it and it conforms to the cats-effect interface
Ryan Peters
@sloshy
It can be implemented differently at the library level - what's required of projects like Monix is that they merely implement the same typeclasses if you want interop, and maybe some conversion methods between other types where appropriate
Fabio Labella
@SystemFw
cats-effect IO also has it, with a very similar implementation, since Alex very very heavily contributed to it
toxicafunk
@toxicafunk
I remember that much
but wasn't sure on the Fiber part
nice to know :thumbsup:
Fabio Labella
@SystemFw
feel free to ask more questions if you have further doubts
JosDenmark
@JosDenmark

Thanks for your help

You are confusing concurrency and parallelism here. Threads and green threads are primarily about concurrency, which is a related but separate thing from parallelism

I don't think I am. Single threaded systems can be concurrent.

libraries that interact with cats-effect, so cats-effect itself, fs2 or Monix, all support this model of concurrency, which is represented by the Fiber interface

So, can you definitively claim that usages of parTraverse (and similar methods) do not make use of green threading because they do not use the Fiber interface?

Sorry, I'll be more specific.
*usages of parTraverse defined in the Parallel[Task, Task.Par] instance
Fabio Labella
@SystemFw

I don't think I am. Single threaded systems can be concurrent.

ofc they can (I mention it at the end of my answer with JS), I'm referring to the part when you talk about efficiency of parallel computations

So, can you definitively claim that usages of parTraverse (and similar methods) do not make use of green threading because they do not use the Fiber interface?

parTraverse does use green threading. I can be more specific, I was only giving an intro

Ryan Peters
@sloshy
When you run parTraverse it does not return a fiber directly as it runs all tasks to completion. You would get a fiber if you started the tasks independently though.
JosDenmark
@JosDenmark
@SystemFw Please be more specific :)
Fabio Labella
@SystemFw
tl;dr a single Task has a runloop that executes it. Each of these running loops is a "green thread", in the sense that many of them can be multiplexed on the same thread pool (so, N: M threading, where M << N and can be even 1). Fiber is a type of handle over a runloop that let's you interrupt it or wait asynchronously for its completion. start: F[Fiber[F, A]] is the cats-effect method from the Concurrent typeclass that describes spawning one of these runloops. There are other, higher-level ways of tapping into this functionality. parTraverse is one of them
monix.Task participates in these abstractions, plus it has some other methods and implementation details which are unique to it and cats.effect.IO does not have