Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Feb 04 21:11
    scala-steward closed #643
  • Feb 04 21:11
    scala-steward commented #643
  • Feb 04 21:11
    scala-steward opened #647
  • Jan 31 23:50
    codecov-commenter commented #646
  • Jan 31 23:50
    codecov-commenter commented #645
  • Jan 31 23:47
    codecov-commenter commented #646
  • Jan 31 23:47
    codecov-commenter commented #645
  • Jan 31 23:44
    codecov-commenter commented #646
  • Jan 31 23:43
    codecov-commenter commented #645
  • Jan 31 23:36
    scala-steward opened #646
  • Jan 31 23:36
    scala-steward closed #629
  • Jan 31 23:36
    scala-steward commented #629
  • Jan 31 23:36
    scala-steward opened #645
  • Jan 28 02:19
    codecov-commenter commented #644
  • Jan 28 02:16
    scala-steward closed #636
  • Jan 28 02:16
    scala-steward commented #636
  • Jan 28 02:16
    scala-steward opened #644
  • Jan 20 00:48
    scala-steward closed #639
  • Jan 20 00:48
    scala-steward commented #639
  • Jan 20 00:48
    scala-steward opened #643
P. Oscar Boykin
@johnynek
I am thinking of a case where I have an Enumerator on a Free monad and I want to convert that to given monad. But maybe I just need to convert the function used to generate the Enumerator (but if I can do that with a FunctionK it seems like many Enumerators would also have that property.
If I actually look on something other than my phone I might make some headway. ;)
Travis Brown
@travisbrown
@johnynek you'd think it would be easy to lift an enumerator into a larger context, and I've never been able to come up with a clear intuition about why it's not. Scalaz's EnumeratorP is one attempt to address this—you define an EnumeratorP[E, F] and then you can apply it to get an EnumeratorT for any larger monad.
@johnynek I asked a question about this a few months ago and got no response: https://twitter.com/travisbrown/status/709108132476293120
RomanIakovlev
@RomanIakovlev

Hello everyone. I’m getting strange runtime errors from monix+iteratee+circe combination, and it’s probably the binary incompatibility between those. I have the following now:

val monixVersion = "2.0.4"
libraryDependencies ++= Seq(
  "io.monix"          %% "monix",
  "io.monix"          %% "monix-cats"
).map(_ % monixVersion)

val circeVersion = "0.5.3"
libraryDependencies ++= Seq(
  "io.circe" %% "circe-core",
  "io.circe" %% "circe-generic",
  "io.circe" %% "circe-parser",
  "io.circe" %% "circe-streaming",
  "io.circe" %% "circe-literal"
).map(_ % circeVersion)

val iterateeVersion = "0.6.1"
libraryDependencies ++= Seq(
  "io.iteratee" %% "iteratee-files",
  "io.iteratee" %% "iteratee-core",
  "io.iteratee" %% "iteratee-monix"
).map(_ % iterateeVersion)

Are these supposed to work together?

These are the latest
I’ll cross-post to circe, it might be a more active room
Ghost
@ghost~55118a7f15522ed4b3ddbe95
https://github.com/travisbrown/iteratee/blob/master/files/src/main/scala/io/iteratee/files/FileModule.scala#L27 - shouldn't r.close() here be properly captured? Is there really a need to do a NonFatal pattern matching considering that any adequate monad shouldn't catch fatal exception? And is non-exhaustive matching intentional?
Ghost
@ghost~55118a7f15522ed4b3ddbe95
  protected final def bracket[R <: Closeable, A](fr: F[R])(f: R => F[A]): F[A] = {
    F.flatMap(fr) { r =>
      F.handleErrorWith(f(r)) { e =>
        F.flatMap(F.handleError(close(r))(_ => ()))(_ => F.raiseError(e))
      }
    }
  }
Something like this maybe?
Travis Brown
@travisbrown
@alexknvl you'd need a catchNonFatal in there, but yeah, that looks reasonable, and is probably preferable for the sake of consistency. I don't see a problem with the current implementation, though, apart from the fact that it's maybe a little less elegant.
Ghost
@ghost~55118a7f15522ed4b3ddbe95
@travisbrown My concern is that r.close() is inside the pure part of Throwable => F[A], not inside of its result F[A]. Imagine if I could stop the world at handleErrorWith and then call handle function twice, it would do r.close() twice, which is a side-effect.
catchNonFatal should be used only for side-effect free functions that can throw exceptions (e.g. they might divide by zero but shouldn't do println) AFAIU, since it does pure(a) which is strict in its argument and therefore executes whatever side-effects a: => A might have.
Travis Brown
@travisbrown
heads up—tailRecM isn't necessary stack safe when you have thousands of iteratees sequenced on a single chunk if the underlying flatMap isn't stack-safe: #142. it's fixed in #141, but I'm not planning to backport it to an 0.6.2 unless someone is particularly interested, since it's somewhat unlikely people will run into this.
Erik Osheim
@non
@travisbrown uh oh.
Travis Brown
@travisbrown
@non do you need it fixed in 0.6? I'm happy to do it.
Erik Osheim
@non
nah, i think it's probably fine.
just makes me uneasy that we still need a concept of stack-safe monad independent of tailRecM
but that's not iteratees' fault.
has more to do with how MTL works
(i.e. i think other people will get bitten by this in other libraries as well.)
Travis Brown
@travisbrown
@non in this case it was definitely a bug—there was an opportunity for recursion outside of the context of the underlying tailRecM (it was just hidden in the way chunks were processed).
Erik Osheim
@non
ah ok, i see.
Travis Brown
@travisbrown
@non also it got caught by the stack-safety laws as soon as I updated to Cats 0.8.0.
Erik Osheim
@non
well, that's something at least! :)
Travis Brown
@travisbrown
okay, here's my latest shot at the Step change I've been trying to get done for a couple of weeks: #146
@johnynek this is pretty similar to the change you reviewed in #129, but with a couple of additional API changes. if you have any concerns, let me know—I'd like to get this published today so I can also publish the new circe release.
P. Oscar Boykin
@johnynek
I'll take a look.
Travis Brown
@travisbrown
thanks.
P. Oscar Boykin
@johnynek
@travisbrown so, how much do you want me to share allocation anxiety?
For instance, in few places you could do some ugly thing: get an Iterator and then transform as building rather than a few .map/traverse calls.
Travis Brown
@travisbrown
@johnynek the more the better, I guess, but if you're basically okay with the API I might want to push some improvements like that to after 0.8.0.
P. Oscar Boykin
@johnynek
Okay. I'll focus on the non-internal details.
Travis Brown
@travisbrown
@johnynek Any thoughts on #146?
Travis Brown
@travisbrown
iteratee.io 0.7.1 is out with Scala 2.12.0 artifacts.
RomanIakovlev
@RomanIakovlev
Yay!
RomanIakovlev
@RomanIakovlev
Just for the better understanding: results of methods like io.iteratee.files.SuspendableFileModule#readBytesFromStream are not supposed to be consumed multiple times, right? I.e. if I create an Enumerator instance and consume it in a certain way (e.g. taking head of it), it doesn’t make sense to keep that instance, because I’ll not be able to consume it again.
RomanIakovlev
@RomanIakovlev
If the above is correct, I need another piece of advice. I have a big zipped json file (hundreds of MB zipped), which contains an array of json objects. In the tests I want to take an arbitrary entry from that file and do something with it. I want to get the length of the array first, and generate a random number from 0 to length to get the entry for test. However it seems I’ll have to read through the file twice: once to get the length, then to get the entry. Since the file is huge, it takes time, and tests are running slower. Maybe there’s a way to do this in one go?
Travis Brown
@travisbrown
@RomanIakovlev All enumerators provided by the library should be side-effect free, so you can re-use them however you need.
@RomanIakovlev for that application specifically, it sounds like reservoir sampling would do what you're looking for (it's not provided by iteratee, but it's a pretty straightforward algorithm).
RomanIakovlev
@RomanIakovlev
Okay, I need to take a deeper look. I’ve seen some stream already closed exceptions when tried to reuse the enumerators, but it might be just my sloppy coding skills.
Travis Brown
@travisbrown
hmm, if you can reproduce I'd definitely be interested in making sure it's not a logic problem.
RomanIakovlev
@RomanIakovlev
sure
will try to reproduce
RomanIakovlev
@RomanIakovlev
Yeah, and thanks @travisbrown for mentioning the reservoir sampling. I didn’t know about such term.
Travis Brown
@travisbrown
yesterday on the plane I threw this together: #153
it allows us to work with strings as chunks without creating Seq[Char]s all over the place.
any feedback appreciated, but the motivation will be clearer once I open the iteratee-parsing PR (should be today or tomorrow).
Travis Brown
@travisbrown
I'm planning to publish 0.8.0 today, with some small additions (#157), version updates for Monix and Twitter Util, and 2.12 support for iteratee-twitter (but not #153). If anyone would like to see any other changes in this version, please let me know asap.
Adelbert Chang
@adelbertc
Was the Task type removed from Iteratee?
I could've sworn the library provided a Task type