travisbrown/iteratee

Iteratees for cats

Adelbert Chang

@adelbertc

so what you're saying is i should use scalaz.concurrent.Task if i want something sane

Travis Brown

@travisbrown

well, iteratee.io does its best—it just can't help you once you run the iteratee if what you're running it in is Future.

@adelbertc e.g.

scala> import cats.Eval, cats.instances.future._
import cats.Eval
import cats.instances.future._

scala> import io.iteratee.Enumerator
import io.iteratee.Enumerator

scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global

scala> import scala.concurrent.Future
import scala.concurrent.Future

scala> val enum = Enumerator.liftMEval(Eval.always { Future { println("hi"); 5 }}).take(5)
enum: io.iteratee.Enumerator[scala.concurrent.Future,Int] = io.iteratee.Enumeratee$$anon$25@ddca1ad

scala> enum.toVector
res0: scala.concurrent.Future[Vector[Int]] = List()

scala> hi
enum.toVector
hi
res1: scala.concurrent.Future[Vector[Int]] = List()

(note that enum.toVector is just a convenience method for having the enumerator run an iteratee that collects all values.)

Adelbert Chang

@adelbertc

@travisbrown so the API i have is basically () => Future[List[A]] and i want to keep doing that "forever"

so i have a Stream[A]

er, Enumerator[Future, A]

or Enumerator[Task, A] if thats less terrible

Travis Brown

@travisbrown

@adelbertc hmm, that sounds like a repeatM, which we don't have (at the moment). you could repeat and flatten, or generateM with Some, or iterateM and ignore the last value.

Adelbert Chang

@adelbertc

ahh generateM - let me give that a shot

sweet that works

thanks!

Travis Brown

@travisbrown

@adelbertc :+1:, but we should think about a repeatM.

Adelbert Chang

@adelbertc

would calling .toVector.void on an Enumerator[Task, A] that is infinite

cause it to actually attempt to collect the results and thereby OOMing

Travis Brown

@travisbrown

@adelbertc yeah, you'd probably want to plug it into a different iteratee (a fold, etc.).

Adelbert Chang

@adelbertc

ah yes good call

apologies for the dumb questions, still wrapping my head around the iteratee model

Travis Brown

@travisbrown

no worries! but yeah, toVector generally isn't as useful as it looks.

Adelbert Chang

@adelbertc

sounds good

so if i just intend to use it tos ide effect

looks like i can just pipe it into a

fold(())((_, _) => ())

Travis Brown

@travisbrown

@adelbertc or foreach(_ => ())

Adelbert Chang

@adelbertc

aha. yes. or that.

Travis Brown

@travisbrown

@adelbertc I'm curious about the use case, though—typically you'd have the processing happen in an iteratee and plug your enumerator into that, rather than layering all the processing directly onto the enumerator.

Adelbert Chang

@adelbertc

@travisbrown its very possible im structing it wrong

Travis Brown

@travisbrown

I'm not sure "wrong" is the right word, but depending on how you compose the pieces they can be more or less reusable.

Adelbert Chang

@adelbertc

the use case is effectively i have A => Task[Unit], and i want to continuously pull off a queue (an Enumerator[Task, A]i believe) and run that function on elements

so the Enumerator is for all intents and purposes infinite

Travis Brown

@travisbrown

that sounds like creating a foreachM iteratee might be a better fit. then if you need to filter or transform the data, etc., you can plug a new piece in between the enumerator and iteratee more cleanly.

Adelbert Chang

@adelbertc

so would i have something like Enumerator[Task, RawData], a parser that looks like Iteratee[Task, RawData, ActualData], and then the thing that runs the task like Iteratee[Task, ActualData, Unit] via foreach ?

and then at the top level glue those pieces together?

Travis Brown

@travisbrown

well, the parser would be Enumeratee[Task, RawData, ActualData], but yeah, that sounds right.

Adelbert Chang

@adelbertc

looks up Enumeratee

Travis Brown

@travisbrown

…and foreachM if you need side effects in the iteratee.

Adelbert Chang

@adelbertc

and right

ah*

alright let me give this a go

hold my beer

Travis Brown

@travisbrown

an enumeratee is a transformer that can attach contramap-style to an iteratee (resulting in an iteratee), or map-style to an enumerator (resulting in an enumerator).

it doesn't have to maintain a 1:1 input-to-output mapping, though.

Adelbert Chang

@adelbertc

ayyyyy it works

and i like it a lot more

thank you sir, you are a gentleman and a scholar

Travis Brown

@travisbrown

:+1:

Travis Brown

@travisbrown

@adelbertc I'll be curious to hear what you think about the iteratee model in general once you've spent some time with it. I've been (slowly) working on a blog post comparing it and fs2.

Adelbert Chang

@adelbertc

sounds good

dwhitney

@dwhitney

it seems that I am just following @adelbertc around since he's in all of the same gitter channels I am :)

Adelbert Chang

@adelbertc

;)

Where communities thrive

People

Repo info

Activity