Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 18 01:05
    codecov-commenter commented #622
  • Sep 18 01:05
    codecov-commenter commented #621
  • Sep 18 01:04
    codecov-commenter commented #622
  • Sep 18 01:04
    codecov-commenter commented #621
  • Sep 18 01:03
    scala-steward closed #604
  • Sep 18 01:03
    scala-steward commented #604
  • Sep 18 01:03
    scala-steward opened #624
  • Sep 18 01:03
    scala-steward closed #612
  • Sep 18 01:03
    scala-steward commented #612
  • Sep 18 01:03
    scala-steward opened #623
  • Sep 18 01:02
    scala-steward closed #611
  • Sep 18 01:02
    scala-steward commented #611
  • Sep 18 01:02
    scala-steward opened #622
  • Sep 18 01:02
    scala-steward closed #603
  • Sep 18 01:01
    scala-steward commented #603
  • Sep 18 01:01
    scala-steward opened #621
  • Sep 15 21:07
    scala-steward closed #618
  • Sep 15 21:07
    scala-steward commented #618
  • Sep 15 21:06
    scala-steward opened #620
  • Sep 11 18:57
    codecov-commenter commented #619
Adelbert Chang
@adelbertc
also im trying to repeatedly run a () => Future[A] to get an infinite Enumerator[Future, A] - is this provided somewhere, or is this consciously not provided for Future
Travis Brown
@travisbrown
@adelbertc for iteratee-core it's not really (much) of an issue—you just bring your own type (the more task-like the better).
in iteratee-files we have an interim solution where you have modules that play a role similar to Capture and Async type classes. You create an instance of a module and get file I/O enumerators, etc., and part of creating the module instance is that you have to handle effect capture.
@adelbertc you're probably looking for Enumerator.liftMEval. I was skeptical about providing first-class support in core for effect capture for types like Future, but Oscar talked me into a couple of concessions, and liftMEval is one of them.
Adelbert Chang
@adelbertc
ah yes, perhaps that
i shall try that
how do i "run" an Enumerator
Adelbert Chang
@adelbertc
oh looks like its toVector
hm Enumerator.liftMEval(Eval.always { Future { println("hi"); 5 }}).take(5).toVector doesnt seemt o do what i think it does
in that it only evaluates once
Travis Brown
@travisbrown
@adelbertc once you've got a future you're back in the horrible world of future semantics, but if you stick with Enumerator[Future, A] you're okay.
Adelbert Chang
@adelbertc
so what you're saying is i should use scalaz.concurrent.Task if i want something sane
Travis Brown
@travisbrown
well, iteratee.io does its best—it just can't help you once you run the iteratee if what you're running it in is Future.
@adelbertc e.g.
scala> import cats.Eval, cats.instances.future._
import cats.Eval
import cats.instances.future._

scala> import io.iteratee.Enumerator
import io.iteratee.Enumerator

scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global

scala> import scala.concurrent.Future
import scala.concurrent.Future

scala> val enum = Enumerator.liftMEval(Eval.always { Future { println("hi"); 5 }}).take(5)
enum: io.iteratee.Enumerator[scala.concurrent.Future,Int] = io.iteratee.Enumeratee$$anon$25@ddca1ad

scala> enum.toVector
res0: scala.concurrent.Future[Vector[Int]] = List()

scala> hi
enum.toVector
hi
res1: scala.concurrent.Future[Vector[Int]] = List()
(note that enum.toVector is just a convenience method for having the enumerator run an iteratee that collects all values.)
Adelbert Chang
@adelbertc
@travisbrown so the API i have is basically () => Future[List[A]] and i want to keep doing that "forever"
so i have a Stream[A]
er, Enumerator[Future, A]
or Enumerator[Task, A] if thats less terrible
Travis Brown
@travisbrown
@adelbertc hmm, that sounds like a repeatM, which we don't have (at the moment). you could repeat and flatten, or generateM with Some, or iterateM and ignore the last value.
Adelbert Chang
@adelbertc
ahh generateM - let me give that a shot
sweet that works
thanks!
Travis Brown
@travisbrown
@adelbertc :+1:, but we should think about a repeatM.
Adelbert Chang
@adelbertc
would calling .toVector.void on an Enumerator[Task, A] that is infinite
cause it to actually attempt to collect the results and thereby OOMing
Travis Brown
@travisbrown
@adelbertc yeah, you'd probably want to plug it into a different iteratee (a fold, etc.).
Adelbert Chang
@adelbertc
ah yes good call
apologies for the dumb questions, still wrapping my head around the iteratee model
Travis Brown
@travisbrown
no worries! but yeah, toVector generally isn't as useful as it looks.
Adelbert Chang
@adelbertc
sounds good
so if i just intend to use it tos ide effect
looks like i can just pipe it into a
fold(())((_, _) => ())
Travis Brown
@travisbrown
@adelbertc or foreach(_ => ())
Adelbert Chang
@adelbertc
aha. yes. or that.
Travis Brown
@travisbrown
@adelbertc I'm curious about the use case, though—typically you'd have the processing happen in an iteratee and plug your enumerator into that, rather than layering all the processing directly onto the enumerator.
Adelbert Chang
@adelbertc
@travisbrown its very possible im structing it wrong
Travis Brown
@travisbrown
I'm not sure "wrong" is the right word, but depending on how you compose the pieces they can be more or less reusable.
Adelbert Chang
@adelbertc
the use case is effectively i have A => Task[Unit], and i want to continuously pull off a queue (an Enumerator[Task, A]i believe) and run that function on elements
so the Enumerator is for all intents and purposes infinite
Travis Brown
@travisbrown
that sounds like creating a foreachM iteratee might be a better fit. then if you need to filter or transform the data, etc., you can plug a new piece in between the enumerator and iteratee more cleanly.
Adelbert Chang
@adelbertc
so would i have something like Enumerator[Task, RawData], a parser that looks like Iteratee[Task, RawData, ActualData], and then the thing that runs the task like Iteratee[Task, ActualData, Unit] via foreach ?
and then at the top level glue those pieces together?
Travis Brown
@travisbrown
well, the parser would be Enumeratee[Task, RawData, ActualData], but yeah, that sounds right.
Adelbert Chang
@adelbertc
looks up Enumeratee
Travis Brown
@travisbrown
…and foreachM if you need side effects in the iteratee.
Adelbert Chang
@adelbertc
and right
ah*
alright let me give this a go