just makes me uneasy that we still need a concept of stack-safe monad independent of
tailRecM
but that's not iteratees' fault.
has more to do with how MTL works
(i.e. i think other people will get bitten by this in other libraries as well.)
@johnynek this is pretty similar to the change you reviewed in #129, but with a couple of additional API changes. if you have any concerns, let me know—I'd like to get this published today so I can also publish the new circe release.
For instance, in few places you could do some ugly thing: get an Iterator and then transform as building rather than a few .map/traverse calls.
Just for the better understanding: results of methods like
io.iteratee.files.SuspendableFileModule#readBytesFromStream are not supposed to be consumed multiple times, right? I.e. if I create an Enumerator instance and consume it in a certain way (e.g. taking head of it), it doesn’t make sense to keep that instance, because I’ll not be able to consume it again.
If the above is correct, I need another piece of advice. I have a big zipped json file (hundreds of MB zipped), which contains an array of json objects. In the tests I want to take an arbitrary entry from that file and do something with it. I want to get the length of the array first, and generate a random number from 0 to length to get the entry for test. However it seems I’ll have to read through the file twice: once to get the length, then to get the entry. Since the file is huge, it takes time, and tests are running slower. Maybe there’s a way to do this in one go?
will try to reproduce
it allows us to work with strings as chunks without creating
Seq[Char]s all over the place.
any feedback appreciated, but the motivation will be clearer once I open the iteratee-parsing PR (should be today or tomorrow).
I could've sworn the library provided a
Task type
are you just using
Eval for effect capture? suspicious eyes
@travisbrown
in iteratee-files we have an interim solution where you have modules that play a role similar to
Capture and Async type classes. You create an instance of a module and get file I/O enumerators, etc., and part of creating the module instance is that you have to handle effect capture.
@adelbertc you're probably looking for
Enumerator.liftMEval. I was skeptical about providing first-class support in core for effect capture for types like Future, but Oscar talked me into a couple of concessions, and liftMEval is one of them.
i shall try that
how do i "run" an
Enumerator
hm
Enumerator.liftMEval(Eval.always { Future { println("hi"); 5 }}).take(5).toVector doesnt seemt o do what i think it does
in that it only evaluates once