Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 18 01:05
    codecov-commenter commented #622
  • Sep 18 01:05
    codecov-commenter commented #621
  • Sep 18 01:04
    codecov-commenter commented #622
  • Sep 18 01:04
    codecov-commenter commented #621
  • Sep 18 01:03
    scala-steward closed #604
  • Sep 18 01:03
    scala-steward commented #604
  • Sep 18 01:03
    scala-steward opened #624
  • Sep 18 01:03
    scala-steward closed #612
  • Sep 18 01:03
    scala-steward commented #612
  • Sep 18 01:03
    scala-steward opened #623
  • Sep 18 01:02
    scala-steward closed #611
  • Sep 18 01:02
    scala-steward commented #611
  • Sep 18 01:02
    scala-steward opened #622
  • Sep 18 01:02
    scala-steward closed #603
  • Sep 18 01:01
    scala-steward commented #603
  • Sep 18 01:01
    scala-steward opened #621
  • Sep 15 21:07
    scala-steward closed #618
  • Sep 15 21:07
    scala-steward commented #618
  • Sep 15 21:06
    scala-steward opened #620
  • Sep 11 18:57
    codecov-commenter commented #619
RomanIakovlev
@RomanIakovlev
Yeah, and thanks @travisbrown for mentioning the reservoir sampling. I didn’t know about such term.
Travis Brown
@travisbrown
yesterday on the plane I threw this together: #153
it allows us to work with strings as chunks without creating Seq[Char]s all over the place.
any feedback appreciated, but the motivation will be clearer once I open the iteratee-parsing PR (should be today or tomorrow).
Travis Brown
@travisbrown
I'm planning to publish 0.8.0 today, with some small additions (#157), version updates for Monix and Twitter Util, and 2.12 support for iteratee-twitter (but not #153). If anyone would like to see any other changes in this version, please let me know asap.
Adelbert Chang
@adelbertc
Was the Task type removed from Iteratee?
I could've sworn the library provided a Task type
are you just using Eval for effect capture? suspicious eyes
@travisbrown
Adelbert Chang
@adelbertc
also im trying to repeatedly run a () => Future[A] to get an infinite Enumerator[Future, A] - is this provided somewhere, or is this consciously not provided for Future
Travis Brown
@travisbrown
@adelbertc for iteratee-core it's not really (much) of an issue—you just bring your own type (the more task-like the better).
in iteratee-files we have an interim solution where you have modules that play a role similar to Capture and Async type classes. You create an instance of a module and get file I/O enumerators, etc., and part of creating the module instance is that you have to handle effect capture.
@adelbertc you're probably looking for Enumerator.liftMEval. I was skeptical about providing first-class support in core for effect capture for types like Future, but Oscar talked me into a couple of concessions, and liftMEval is one of them.
Adelbert Chang
@adelbertc
ah yes, perhaps that
i shall try that
how do i "run" an Enumerator
Adelbert Chang
@adelbertc
oh looks like its toVector
hm Enumerator.liftMEval(Eval.always { Future { println("hi"); 5 }}).take(5).toVector doesnt seemt o do what i think it does
in that it only evaluates once
Travis Brown
@travisbrown
@adelbertc once you've got a future you're back in the horrible world of future semantics, but if you stick with Enumerator[Future, A] you're okay.
Adelbert Chang
@adelbertc
so what you're saying is i should use scalaz.concurrent.Task if i want something sane
Travis Brown
@travisbrown
well, iteratee.io does its best—it just can't help you once you run the iteratee if what you're running it in is Future.
@adelbertc e.g.
scala> import cats.Eval, cats.instances.future._
import cats.Eval
import cats.instances.future._

scala> import io.iteratee.Enumerator
import io.iteratee.Enumerator

scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global

scala> import scala.concurrent.Future
import scala.concurrent.Future

scala> val enum = Enumerator.liftMEval(Eval.always { Future { println("hi"); 5 }}).take(5)
enum: io.iteratee.Enumerator[scala.concurrent.Future,Int] = io.iteratee.Enumeratee$$anon$25@ddca1ad

scala> enum.toVector
res0: scala.concurrent.Future[Vector[Int]] = List()

scala> hi
enum.toVector
hi
res1: scala.concurrent.Future[Vector[Int]] = List()
(note that enum.toVector is just a convenience method for having the enumerator run an iteratee that collects all values.)
Adelbert Chang
@adelbertc
@travisbrown so the API i have is basically () => Future[List[A]] and i want to keep doing that "forever"
so i have a Stream[A]
er, Enumerator[Future, A]
or Enumerator[Task, A] if thats less terrible
Travis Brown
@travisbrown
@adelbertc hmm, that sounds like a repeatM, which we don't have (at the moment). you could repeat and flatten, or generateM with Some, or iterateM and ignore the last value.
Adelbert Chang
@adelbertc
ahh generateM - let me give that a shot
sweet that works
thanks!
Travis Brown
@travisbrown
@adelbertc :+1:, but we should think about a repeatM.
Adelbert Chang
@adelbertc
would calling .toVector.void on an Enumerator[Task, A] that is infinite
cause it to actually attempt to collect the results and thereby OOMing
Travis Brown
@travisbrown
@adelbertc yeah, you'd probably want to plug it into a different iteratee (a fold, etc.).
Adelbert Chang
@adelbertc
ah yes good call
apologies for the dumb questions, still wrapping my head around the iteratee model
Travis Brown
@travisbrown
no worries! but yeah, toVector generally isn't as useful as it looks.
Adelbert Chang
@adelbertc
sounds good
so if i just intend to use it tos ide effect
looks like i can just pipe it into a
fold(())((_, _) => ())
Travis Brown
@travisbrown
@adelbertc or foreach(_ => ())
Adelbert Chang
@adelbertc
aha. yes. or that.
Travis Brown
@travisbrown
@adelbertc I'm curious about the use case, though—typically you'd have the processing happen in an iteratee and plug your enumerator into that, rather than layering all the processing directly onto the enumerator.
Adelbert Chang
@adelbertc
@travisbrown its very possible im structing it wrong
Travis Brown
@travisbrown
I'm not sure "wrong" is the right word, but depending on how you compose the pieces they can be more or less reusable.
Adelbert Chang
@adelbertc
the use case is effectively i have A => Task[Unit], and i want to continuously pull off a queue (an Enumerator[Task, A]i believe) and run that function on elements
so the Enumerator is for all intents and purposes infinite