Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • May 05 15:10
    scala-steward opened #497
  • May 04 09:28
    codecov-commenter commented #496
  • May 04 09:28
    codecov-commenter commented #496
  • May 04 09:25
    codecov-commenter commented #496
  • May 04 09:20
    scala-steward closed #494
  • May 04 09:20
    scala-steward commented #494
  • May 04 09:20
    scala-steward opened #496
  • May 03 22:54
    codecov-commenter commented #495
  • May 03 22:52
    codecov-commenter commented #495
  • May 03 22:46
    scala-steward opened #495
  • May 02 23:24
    codecov-commenter commented #494
  • May 02 23:24
    codecov-commenter commented #494
  • May 02 23:21
    codecov-commenter commented #494
  • May 02 23:16
    scala-steward closed #493
  • May 02 23:16
    scala-steward commented #493
  • May 02 23:16
    scala-steward opened #494
  • May 02 11:26
    codecov-commenter commented #493
  • May 02 11:23
    codecov-commenter commented #493
  • May 02 11:23
    codecov-commenter commented #493
  • May 02 11:17
    scala-steward closed #492
John Sullivan
@sullivan-
Hey @travisbrown, I just wanted to let you know I got a version now that produces a reusable Enumerator from an Iterator! (I'm not expecting you to care that much, I'm just excited and have to share :smile: ) https://github.com/longevityframework/unblocking/blob/master/core/src/main/scala/unblocking/ToCatsEnumerator.scala
Travis Brown
@travisbrown
@sullivan- :+1:
John Sullivan
@sullivan-
it occurred to me that converters from/to org.reactivestreams.Publisher[A] would probably be much more sensible than converters from/to () => Iterator[A] with Closeable
Travis Brown
@travisbrown
@sullivan- I'd like to have a reactivestreams conversions module—just haven't had the time to work on one myself (hint, hint :smile:).
John Sullivan
@sullivan-
I'm probably going to write one soon. I'd be happy to contribute it to your codebase. I'd probably need your assistance in figuring out where to put impls and tests, and how to integrate these things with your codebase. How about I write an implementation in my own repo first, and then if you like it, we can talk about how to get it into yours?
There is also this whole TCK Java-API testing framework thingie that I'm not sure how much I want to deal with.. I guess I ought to take a look in the least http://www.reactive-streams.org/reactive-streams-tck-1.0.0-javadoc/
Travis Brown
@travisbrown
@sullivan- :+1:, sounds good to me!
John Sullivan
@sullivan-

@travisbrown would you consider this a bug? I've managed to get Iteratee.foreach to replay some elements twice

https://gist.github.com/sullivan-/4561d9fce97ca0fafdc187d17755bf8f

(If you think it's a bug, I have a simple fix. Alas no test yet.)

Travis Brown
@travisbrown
@sullivan- taking a look now. at a glance I'd say all bets are off if you have mutable state in your enumerator, but the duplication still looks odd.
John Sullivan
@sullivan-
i don't think that embedded iterator is the problem...

my fix is in Done.bind method, line 106 Step.scala

as you can see, f(value) is computed twice. if i extract local, the problem goes away

John Sullivan
@sullivan-
I now have a test that shows it out; unfortunately the test stack overflows for twitter:test/test .. looking into it
  "foreach" should "perform an operation on all values in a grouped stream" in forAll { (eav: EnumeratorAndValues[Int]) =>
     var total = 0
     val iteratee = foreach[Vector[Int]](is => total += is.sum)

     val eavg = EnumeratorAndValues(
       eav.enumerator.grouped(3),
       eav.values.grouped(3).toVector)

     assert(eavg.resultWithLeftovers(iteratee) === F.pure(((), Vector.empty)) && total === eavg.values.flatten.sum)
   }
(in IterateeSuite.scala)
John Sullivan
@sullivan-
weird, i am unable to get twitter:test/test to stack overflow a second time
John Sullivan
@sullivan-
I put in a PR, maybe a better spot to continue conversation travisbrown/iteratee#180
Travis Brown
@travisbrown
@sullivan- got sidetracked just now, but thanks much for the PR—I'll try to get it reviewed this afternoon.
@sullivan- btw I'm considering a new release soon-ish—there are a few little things I've added locally for a personal project that I'll try to push this week, and there are new Monix and Twitter Util releases.
@sullivan- so if there's anything you'd like to see added, now's the time :)
John Sullivan
@sullivan-
I wish I had more for you Travis :smile: reactive streams stuff turned out to be a bit more involved than I expected .. and I'm otherwise sidetracked.
Thanks for you consideration re: PR
John Sullivan
@sullivan-
@travisbrown I still have a lot to learn. The PR I gave you yesterday not stack safe. Unfortunately it only shows up sporadically in the tests - normally for JS. I just updated the PR with a stack safe version
just saw your comment on the PR, will take a look
Travis Brown
@travisbrown
@sullivan- this stuff can be so confusing. thanks for the PR and the fix.
John Sullivan
@sullivan-
Very happy to contribute. It's just about the best way for me to learn this stuff. Thanks for the great project.
John Sullivan
@sullivan-
@travisbrown any chance of a iteratee.io release that includes that PR of mine you merged recently? I'd like to do a release of my own some time soon that depends on that change. Thanks!
Travis Brown
@travisbrown
@sullivan- hmm, I really wanted to get that SO sorted first but haven't had time (I'm at NE Scala right now). Is it just the foreach bug? That should be backported anyway, so maybe we could just cut an 0.9.1 today with that and not publish 0.10.0 with a potential stack bug?
John Sullivan
@sullivan-
@travisbrown that would be totally fine with me. but i can also wait a few days, no big deal. enjoy the conference, i wish i was there. thanks!
Travis Brown
@travisbrown
@sullivan- okay, so I've got a fix for the SO bug (which is a real bug in sequenceI that existed before the foreach fix).
I should be able to get 0.10.0 published tonight.
John Sullivan
@sullivan-
awesome thanks Travis!
John Sullivan
@sullivan-
I'm curious to take a look at your fix..
Travis Brown
@travisbrown
@sullivan- I've just opened #185, which I think is almost ready to go—there's just a weird test failure to address first.
Travis Brown
@travisbrown
Okay! 0.10.0 is on its way to Maven Central: https://github.com/travisbrown/iteratee/releases/tag/v0.10.0
Travis Brown
@travisbrown
Would anyone object if I publish releases more frequently / for more minor stuff? e.g. I've got a few little additions since 0.10 that'd be convenient for me to have in a side project, but they're not binary compatible thanks to pre-2.12 traits.
Travis Brown
@travisbrown
I'm thinking of moving iteratee-files to add a cats-effect dependency (and use it for "suspendible" file modules). Any objections?
P. Oscar Boykin
@johnynek
I'd be a little bummed I think. Wouldn't this mean you can't use Try or Future?
To me, it seems like MonadError plus a few lazy params could do the trick
(Even ApplicativeError in many cases)
By lazy I mean call-by-name or use of Eval.
Travis Brown
@travisbrown
@johnynek No, what I was thinking was that SuspendableFileModule would remain basically unchanged, but that there would be non-module versions of the reading / writing methods for type constructors with the appropriate cats-effect type class instances.
P. Oscar Boykin
@johnynek
That sounds fine then.
Sergey Kolbasov
@sergeykolbasov

@travisbrown Hi there

I have a question regarding .ensure in Enumerator and Iteratee.
Is it called after last step processed or failed?

Let's say, I have an Enumerator of futures, and I want to close connection on reading failure or when I'm done with stream.

Currently it seems like .ensure is called before the last element is processed (if I add it here):
https://github.com/ImLiar/finch/blob/246fd7ccf80e9da9d8fedd409fbf09ce2d7bda18/iteratee/src/main/scala/io/finch/iteratee/package.scala#L40

I got an exception every time when I write inside of the readerWriter,
https://github.com/ImLiar/finch/blob/246fd7ccf80e9da9d8fedd409fbf09ce2d7bda18/iteratee/src/test/scala/io/finch/iteratee/EnumerateEndpointSpec.scala#L45 saying that writer is already closed

Sergey Kolbasov
@sergeykolbasov
I guess I should use ensureEval instead /_-
Sergey Kolbasov
@sergeykolbasov

Yeah, ensureEval with Eval.later did a trick. Now data is properly discarded.

I wonder if it's a normal behaviour for .ensure to have call-by-value argument with Eval.now under the hood?

Travis Brown
@travisbrown
@ImLiar yeah, I'll take a closer look but I think this should depend on the strict/non-strictness of the context type constructor.
My general approach so far has been to make non-strict contexts the first-class citizens, with things like Option and Try supported by explicitly lazy methods with the Eval suffix.
I think I'd like to stick to this approach (unless everyone else strongly disagrees), but it should be better documented.
Travis Brown
@travisbrown
(the primary motivation here is that I don't want to penalize people who are doing the right thing by using Monix's Task or Rerunnable or (soon) cats.effect.IO with extra overhead)
Sergey Kolbasov
@sergeykolbasov

@travisbrown

Could you give a hint if it's possible to make an accumulator in the middle of a stream that will push accumulated results down to rest of a stream each X seconds?

So I need a batching, essentially