Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Ywen
    @YPares
    My 2 cents is that it better encodes what the function does without locking anyone or incurring overhead, because it's just a newtype
    Torsten Scholak
    @tscholak
    right, but we suspect that our users will have a hard time wrapping their heads around IO and State, so we were looking for ways where we make this optional, maybe even preferred, but not mandatory.
    Ywen
    @YPares
    you can have a look at http://hackage.haskell.org/package/capability or http://hackage.haskell.org/package/polysemy if you need to deal with patterns like that often
    Torsten Scholak
    @tscholak
    ah, thanks! I'm familiar with polysemy
    algebraic effects is another option, yes
    Ywen
    @YPares
    polysemy is maybe a bit overkill for the usecase, but capability would make your type sigs just (HasState "rngSeed" Seed m) => X -> m Y
    I'm not suggesting reader-soup which I developped for porcupine, because as the name suggests it deals only with reader-like effects
    My usual way of approaching it is to be really dumb: use StateT IO Stuff when can then always easily be converted to whatever the app needs
    Torsten Scholak
    @tscholak
    I'll tell the team about (HasState "rngSeed" Seed m) => X -> m Y. I'm not the best person to judge beginner-friendliness of these things. I have been using extensible effects, free monads, and even recursion schemes, and don't see the learning curve anymore
    Ywen
    @YPares
    @tscholak If you're interested, you can have a look at https://www.tweag.io/posts/2018-10-04-capability.html
    Torsten Scholak
    @tscholak
    thanks!
    Ywen
    @YPares
    The goal of it is to have abstract signatures where effects are tagged (so you can easily disambiguate between two reader effects with the same type e.g.) and actually implement them concisely with a concrete type with DerivingVia
    The only drawback is when you want to run part of the effects (for instance if you want some local state or reader in your code, but not in your whole stack)
    For that I think a polysemy approach is better suited (thats' why I developed the reader-soup approach, because when I did it polysemy wasn't known yet)
    Torsten Scholak
    @tscholak
    ah, extensible effects solves partially running of effects quite elegantly
    Ywen
    @YPares
    yes, and maybe capability could too with some extra newtypes or support for vinyl records or HLists. I'm discussing that with people at tweag these days
    Torsten Scholak
    @tscholak
    you need something like FastSum
    https://hackage.haskell.org/package/fastsum
    Ywen
    @YPares
    @tscholak Yep, it looks exactly like vinyl
    Torsten Scholak
    @tscholak
    :+1:
    cool, thanks for the chat!
    Ywen
    @YPares
    you're welcome :)
    Ywen
    @YPares
    @/all Our discussion @tscholak had me thinking. I was wondering if you guys could be interested in a chan for pro/semi-pro haskell devs (or serious enthusiasts) to specifically talk about architecturing Haskell programs, and more specifically composing effects. So a scope broader than this gitter or dataHaskell, but narrower than r/haskell or #haskell on IRC. Also I think small scale should be better, so I think it should be on invitation first to see how is goes first.
    Could be on gitter, discord or slack. I envision the products of these discussions could be a wiki detailing some use cases, the patterns that were followed, some guidelines re what is the best effect lib to architecture you program in such specific case, etc.
    This would facilitate the discussion around conventions and interoperability between libs and toolkits.
    Stefan Dresselhaus
    @Drezil
    @YPares i like the idea. I was already suggesting developer-blogs etc. at our company to present/document what and how we (or better i -.-) do things.
    Torsten Scholak
    @tscholak
    @YPares sounds great!
    @gelisam gave a talk recently on different effect “patterns” in haskell
    Ywen
    @YPares
    Okay :) I opened a lobby at https://gitter.im/pure-fp-architects/community . I invited you two to the main room
    But the lobby is public, it's for people who want to manifest they are interested in the discussion. I shared the link on https://gitter.im/dataHaskell/Lobby
    Torsten Scholak
    @tscholak
    @YPares how difficult would it be to give porcupine distributed computing capabilities as provided by Dask, https://distributed.dask.org/en/latest/ ?
    Ywen
    @YPares
    @tscholak So there's already https://github.com/tweag/funflow/tree/master/funflow-jobs that exists, but I never used it with porcupine. I added a few days ago an entry about distributed computing to the FAQ in porcupine's README
    funflow-jobs is pretty rough though for the moment I think
    Torsten Scholak
    @tscholak
    I’m fighting with Dask at work since yesterday. there’s a lot that just doesn’t work, and you can’t reason about behaviour well enough to debug strategically
    3 replies
    Ywen
    @YPares
    what's the use case? (if discloseable)
    Torsten Scholak
    @tscholak
    a trivial map job, open a bunch files, do a transformation, save to files. embarrassingly parallel, currently also embarrassingly broken
    I want to be able to scale seamlessly between prototyping on a laptop and running it on our cluster
    everything written in python...
    configuration in yaml
    Ywen
    @YPares
    Problems arise when you try to distribute it? Or even in the one-node case?
    Torsten Scholak
    @tscholak
    even in the one thread case...
    Ywen
    @YPares
    Oh damn...
    Torsten Scholak
    @tscholak
    code behaves differently when wrapped into dask delayed computations
    and it’s hard to figure out why
    Ywen
    @YPares
    Why was Dask chosen over, say, Spark?
    Torsten Scholak
    @tscholak
    dask is perceived to be more leightweight and more “pythonic”
    also, it was my idea to give it a try...
    the current alternative is a bash script ;)
    a bash script that splits up the work, dispatches and launches jobs on the cluster
    Ywen
    @YPares
    @tscholak Yes, that's exactly what we had too at my client, some custom solution to distribute bash/docker commands. We switched to celery last year, but now we think we would have fared better with a simple job queue (like RQ) on top of rabbit or redis
    Tim Pierson
    @o1lo01ol1o
    It's worth mentioning transient again here. However, it's currently rough in terms of developer UX and the materials are not super accessible. (It's basically ContT over IO with semantics similar to ListT. However, composing all the streaming in a distributed setting is never trivial.) The best source of current information is probably via the author in the gitter: https://gitter.im/Transient-Transient-Universe-HPlay/Lobby. I know he's working on a new release.
    Ywen
    @YPares
    @o1lo01ol1o Will have a look :)