Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Feb 01 10:11
    @SystemFw banned @Hudsone_gitlab
  • Jan 31 2019 04:19
    404- forked
    404-/fs2
  • Jan 31 2019 03:01
    SethTisue commented #1232
  • Jan 30 2019 17:22
  • Jan 30 2019 13:45
  • Jan 30 2019 10:48
    pchlupacek commented #1406
  • Jan 30 2019 10:47
    pchlupacek commented #1406
  • Jan 30 2019 10:39
    pchlupacek commented #1407
  • Jan 30 2019 09:58
    lJoublanc commented #870
  • Jan 30 2019 09:42
    vladimir-popov commented #1407
  • Jan 30 2019 08:10
    vladimir-popov closed #1407
  • Jan 30 2019 08:10
    vladimir-popov commented #1407
  • Jan 29 2019 19:20
    SystemFw commented #1407
  • Jan 29 2019 19:20
    SystemFw commented #1407
  • Jan 29 2019 18:57
    SystemFw commented #1406
  • Jan 29 2019 17:47
    pchlupacek commented #1406
  • Jan 29 2019 17:42
    pchlupacek commented #1406
  • Jan 29 2019 17:39
    pchlupacek commented #1407
  • Jan 29 2019 17:39
    vladimir-popov edited #1407
  • Jan 29 2019 17:38
    vladimir-popov commented #1406
Mark Tomko
@mtomko
It is the one you don't love. That method has the advantage of being very fast, but you definitely have to hold your nose to use it.
wedens
@wedens
it makes things more modular (reusable/composable)
Fabio Labella
@SystemFw
agreed with @wedens there
Mark Tomko
@mtomko
oh, right, because then I can just map over the string stream...
Fabio Labella
@SystemFw
especially because there are ready-made methods to convert from InputStreams and/or iterators
Mark Tomko
@mtomko
and I have other line-oriented file formats.
Fabio Labella
@SystemFw
so you could
io.file.readAll[IO](fileName).through(text.uft8Decode).through(text.lines).evalMap(yourFormat.convert[IO])
where convert will be convert[F[_]: ApplicativeError](s: String): F[Format]
Mark Tomko
@mtomko
I had had something similar to that for a different file format and it turned out to be somewhat slow, which was why I tried dropping down to BufferedReader.
Fabio Labella
@SystemFw
right, fair enough, although have you tried increasing the buffer size for readAll?
Mark Tomko
@mtomko
I did in that case; possibly not enough.
Fabio Labella
@SystemFw
ok, fair :)
Mark Tomko
@mtomko
I don't have the timings in front of me, but it seemed like I was reading 10 or 20 records per ms, when I'm used to 500 or 600; there was a possibly confounding factor there, in that that file format is read 4 lines at a time, so I had to write a pull that may have turned out to be very inefficiently done.
Fabio Labella
@SystemFw
mm
how long ago was this? (we did fix a few bugs in the past few months)
but from here, that seems too slow
Mark Tomko
@mtomko
Not terribly long ago. My bet is my Pull was not well done.
Fabio Labella
@SystemFw
anyway, we can possibly help either figuring that out, or writing a lower level version
for the grouping, you could try segmentN(4) after lines, instead of a custom pull
Mark Tomko
@mtomko
That looks promising
I'll rework it with segmentN and post again if it's still slow.
I need to turn in, I really appreciate all the help and patience.
Thanks everyone!
Fabio Labella
@SystemFw
:) :wave:
Alex Henning Johannessen
@ahjohannessen
@SystemFw On your slides wrt. fs2 + declarative control flow you use Stream.sleep in some examples. I cannot find sleep on Stream, just on Scheduler.
Fabio Labella
@SystemFw
Yeah, it's not released yet
Use Scheduler or Timer
Scheduler sleep gives you a stream, Timer sleep gives you an F that you'll need to eval
Alex Henning Johannessen
@ahjohannessen
Aah :) which release is that?
Fabio Labella
@SystemFw
Well, which one are you asking about?
But basically in 0.10.3 we have added the latest cats effect, with enhanced concurrency features
But we're not taking advantage of them yet
There's a branch, however
So in this case Scheduler will be removed since we have Timer in cats effect, and sleep will go on Stream
We are still deciding when and how to release that (new milestone? 0.11?)
Alex Henning Johannessen
@ahjohannessen
Ah great, yeah I meant branch :)
Michael Pilquist
@mpilquist
I'm still hoping for fs2 1.0 in mid-May once cats-effect 1.0 is out
Peter Aaser
@PeterAaser
today my brakes failed mid traffic
I wish hyundai would use fs2
maybe then their cars wouldn't suffer random brake failures
Gavin Bisesi
@Daenyth
Can someone explain segments vs chunks?
and how that all interacts?
Fabio Labella
@SystemFw
@Daenyth yeah, I can do it in a bit
Gavin Bisesi
@Daenyth
thanks!
Specifically I ask because I have a Stream[IO, ByteString] where ByteString is an akka rope structure like ByteVector, and I want to get to bytes so I can use text stuff with it
and I have
byteStream
      .mapChunks(c => c.map(bs => bs.toVector).toSegment)
      .flatMap(Stream.emits(_).covary[IO])
but it feels wrong
Fabio Labella
@SystemFw
why does it feel wrong?
Christopher Davenport
@ChristopherDavenport
tldr: chunk - index based access
Gavin Bisesi
@Daenyth
@SystemFw it feels like I should be able to do this in a single step