dataHaskell/Lobby

Knowledge base: http://datahaskell.org/docs

Riley Moher

@avatR630

Hello everyone. I'm currently learning Haskell while working on my research centered on semantic data types. My background is more in first order logic, ontologies, and data science, but what I'm learning about Haskell seems very interesting to me so far! Just checking out what tools currently exist for data science with haskell

Austin Huang

@austinvhuang

Thanks for taking over ploylyhs @JonathanReeve

there were some other promising modules in diffusionkinetics, i'm kind of bummed it's stalled but i understand

Austin Huang

@austinvhuang

@ocramz how is the dataframe library going?

i was wondering - i'm not an expert in this space but i always figured dataframe libraries would be implemented as struct-of-arrays rather than array-of-structs to be cache & vectorization friendly.

Marco Z

@ocramz

hey @austinvhuang , yes that's how Arrow and similar work. Heidi was more of an experiment on API ergonomics than performance

Austin Huang

@austinvhuang

makes sense @ocramz. are you (or anyone else) looking into arrow-based dataframe implementations? that seems like it'd be a useful direction but the repos i've seen don't look active.

Marco Z

@ocramz

@austinvhuang Arrows inactive? looks quite the contrary .. https://ursalabs.org/blog/ursa-computing/

Austin Huang

@austinvhuang

Arrow is doing great. Haskell integration with arrow is what's missing.

Casey Kneale

@caseykneale

Hi everyone. So I am like 20 minutes into learning Haskell. Why isn't it used for data stuff already? It looks so promising it's crazy.

Austin Huang

@austinvhuang

The biggest challenge is the ecosystem needs to be implemented, differentiated, and matured. Python as a language isn't inherently suited for data science, but the fact that pandas, sklearn, pyspark, pytorch, tensorflow, seaborn, plotly, etc. exist and have had rough edges smoothed out by years of use cases makes it the defacto standard.

Marco Z

@ocramz

and also because most haskellers are a bunch of perfectionists, so they can never agree on the Right Abstraction ;)

BTW welcome @caseykneale ! Don't hesitate to ask here if you have any questions

Marco Z

@ocramz

Happy new year all!

Adam Conner-Sax

@adamConnerSax

Happy New Year!

Anurag Hooda

@specdrake

Hello, I am really excited about using Haskell for data analysis.

Came to know about DataHaskell through reddit.

The biggest challenge is the ecosystem needs to be implemented, differentiated, and matured. Python as a language isn't inherently suited for data science, but the fact that pandas, sklearn, pyspark, pytorch, tensorflow, seaborn, plotly, etc. exist and have had rough edges smoothed out by years of use cases makes it the defacto standard.

yeah exactly

Marco Z

@ocramz

Welcome @specdrake 🙌

Marco Z

@ocramz

@specdrake what kind of problems do you work on usually? or, what made you interested in DataHaskell?

Tony Day

@tonyday567

https://www.reddit.com/r/haskell/comments/l2yypi/chartsvg020/?utm_source=share&utm_medium=web2x&context=3

https://hackage.haskell.org/package/chart-svg

Justin Le

@mstksg

b/ 1

Tony Day

@tonyday567

I'm pretty happy with chart-svg. It's a solid design, easy api and the svg's are tight.

Yves Parès

@YPares

Hi ppl! Is anyone knowledgable with existing SQL libs for Haskell? I'd like to do the following:

Support both sqlite and postgres backends
CREATE tables from a HashMap Text TypeRep, supposing each TypeRep can correspond to a Sql numeric/string type (not looking for an ORM)
I can do that with some boilerplate with HDBC or {postgres,sqlite}-simple, but I wanted to know if there was somewhere a lib that could limit the amount of SQL code to write manually.
Selda ( https://hackage.haskell.org/package/selda ) looks promising but it generates tables out of plain old records through Generic. Pretty cool, but I don't have such static datatypes (my schemas will be generated semi-dynamically at runtime, albeit in a deterministic fashion, hence the hashmaps).
opaleye looked interesting too, but opaleye-sqlite seems not maintained anymore.

The rest of the queries to perform are fairly mundane insert/select-based stuff, on classical sql number/string types, nothing fancy

Yves Parès

@YPares

To give a bit more detail, my use of SQL would be to index some datasets that we produce and store on S3. Each dataset will be produced in some context, and that context comes with metadata bits (numbers, text values etc) that we want to use later to query through all the datasets that we produced that match some metadata filters. But these metadata bits aren't known at compile-time (they're similar to the way Katip context Items work, for those who know katip), although they are known in advance (ie. before running the program's logic per se) and they don't change from one run to the next unless we edit the code.

Tony Day

@tonyday567

hey team dataHaskell, I’m looking for blog ideas. Anything interesting?

Scott Edwards

@scottedwards2000

what's been the effect of Julia on this project? I know it's not truly functional like Haskell, but has it's growth affected some of the impetus to develop packages in Haskell for data science? I hope not, because Haskell is special and it would be wonderful to see it become a popular choice in data science.

Tony Day

@tonyday567

Hi Scott, this is more a community hub than a project these days. It terms of growth in use of Haskell for data sciences, I would say that the territory is dominated by the python cluster:

Python, TensorFlow, Pandas, Torch, Keras

So I would tag Julia as a bit player just like us. I don't know too much about the language, but I read that it is the direct opposite to Haskell in many ways when it comes to typing etc. Which makes it a very useful comparator. Anything the reduces the empire of python is a good thing when it comes to alternative approaches becoming more acceptable.

Evgeny Pogrebnyak

@epogrebnyak

Julia is a more affected by lisp than Haskell in language design, and it is dynamically typed - you can start writing in Julia without mentioning types. The package manager there is very friendly too, so is community - people like sharing when they write something in a new language.

Agustin Jimenez

@agustinjimenez_gitlab

Hi! someone here?

rohakapo

@rohakapo

Yea

Fraser Mince

@frasermince

Hey everyone! I've been doing some deep learning in python for a while and got interested in jumping back in in Haskell. Is anyone currently trying to participate in kaggle with something like hasktorch?

Anurag Hooda

@specdrake

@specdrake what kind of problems do you work on usually? or, what made you interested in DataHaskell?

Hey, I usually work on machine learning problems with python mostly computer vision stuff, I was looking for good data science and ML libraries for haskell

Agustin Jimenez

@agustinjimenez_gitlab

@rohakapo sorry, I'll turn on the notifications.

I'm very beginner at haskell and also I'm very beginner at DataScience.

But I love haskell and this language give me more satisfaction than anything in IT.

I'm looking to make a carer in data science. I'm from python backend word.

Any advice?

idontgetoutmuch

@idontgetoutmuch

What do folks use for drawing charts these days? I sometimes use charts-diagrams but iirc it takes a fair bit of boilerplate to draw a simple chart. I'm inclined to try inline-r with ggplot.

Aleksey Khudyakov

@Shimuuar

Chart's API is broken enough but plots themselves are decent. I don't like anything else so I just forked Chart https://github.com/Shimuuar/Chart-B/tree/ChartB (branch Chart-B). It's very much UUU (unfinished, undocumented, unstable)

1 reply

Adam Conner-Sax

@adamConnerSax

@idontgetoutmuch I’ve been using hvega https://hackage.haskell.org/package/hvega (my target is producing html). Also some boilerplate but I like the grammar-of-graphics way of structuring things.

idontgetoutmuch

@idontgetoutmuch

@adamConnerSax I'd forgotten about that - thanks for reminding me

Kevin Brubeck Unhammer

@unhammer

+1 on hvega

idontgetoutmuch

@idontgetoutmuch

How does one render in hvega? Do I just write an hmtl file and use a browser?

idontgetoutmuch

@idontgetoutmuch

Doug Burke

@DougBurke

Suggestions for improvements to the hvega docs - in particular for the "how do I actually display the damn thing" - are appreciated at https://github.com/DougBurke/hvega . I'd particularly like to improve the IHaskell story, but that (I believe) requires IHaskell work which I haven't had the energy to dig into.

Doug Burke

@DougBurke

If you create JSON output - I see I don't really document how to do this but spec = A.encodeToLazyText (fromVL vl) should do - then save it to a file you can then use https://github.com/DougBurke/vega-view (or the vega desktop application) to view it (it runs a web server that will let you view the .vg.json files in the working directory as well as allowing you to drag-n-drop files onto the browser).

idontgetoutmuch

@idontgetoutmuch

@DougBurke nice package - I am using my browser - it's pretty similar to the julia experience with gadfly (not that I am a big user of julia or gadfly)

dracon98

@dracon98

Hi I am new to Haskell and data science. I wonder whats the advantage of implementing machine learning through DataHaskell compared to the standard implementation?

Austin Huang

@austinvhuang

+1 hvega from me too. vega-lite itself has been such a boon to other languages, thanks for your work on hvega @DougBurke.

@YPares did you ever find a sql solution? i'm using sqlite-simple a lot... i can live with sqlite-simple but i think i have to bite the bullet on some minimal string builder on my current project. it'd be nice to have more lightweight string-builder middle-ground options between hard-coding queries as strings and full-on orm commitment.

@frasermince can ping us in the hasktorch slack if you'd like to get a hasktorch kaggle effort going. will say kaggle is probably a space where haskell has few comparative advantages (there's no penalty to scripts being hacky/throwaway as long as they score the test data, this is not true of "real-world" ml).

Where communities thrive

People

Repo info

Activity