These are chat archives for non/algebra

4th
Jan 2016
Rüdiger Klaehn
@rklaehn
Jan 04 2016 09:24
It seems that cats will stop depending on algebra in the near future. non/cats#762
I really hope that the goal of not duplicating basic typeclasses is retained. So algebra would have to depend on cats, and spire in turn would have to depend on algebra and indirectly on cats.
Rüdiger Klaehn
@rklaehn
Jan 04 2016 09:35
I am writing an immutable unboxed collections library (kind of similar to non/debox, but for immutable collections), and not knowing which typeclasses to use from where is a major pain point.
Pascal Voitot
@mandubian
Jan 04 2016 09:57
nice lib!
Rüdiger Klaehn
@rklaehn
Jan 04 2016 10:06
I spent 6 hours yesterday trying to get rid of the ClassTag in many places, but at the end of the day it was futile. But I do like the spit of partial/total collections. Much nicer than having a scala.collection.immutable.Map and not knowing if apply is total or partial.
Rüdiger Klaehn
@rklaehn
Jan 04 2016 11:06
I opened a few tickets on non/cats to ease the transition from cats depends on algebra to algebra depends on cats. It would be good to get this done soon. non/cats#776 non/cats#777
Denis Rosset
@denisrosset
Jan 04 2016 11:30
@rklaehn great work! How did you solve the specialization problem, for example Map[Int, Int] being specialized and Map[Int, AnyRef] not at all ? (I opened an issue in non/debox)
I solved this using macro machinery in https://github.com/denisrosset/metal, but the trickery is quite heavy
I also opened an issue about the consequences of algebra depending on cats in non/algebra#142
I start to be worried about ecosystem fragmentation in the pure-FP side of Scala.
Pascal Voitot
@mandubian
Jan 04 2016 11:49
@denisrosset I would talk about mind evolution & fresh air in the ecosystem...
Rüdiger Klaehn
@rklaehn
Jan 04 2016 12:31
@denisrosset actually, I did not solve it at all. My main goal is compact in-memory representation, and that is taken care of by the (extremely annoying) ClassTag instances.
One thing I might try is to not specialize on the value at all, since the specialization is most beneficial for the binary search during the lookup. Short-lived boxing is something you kind of have to live with on the JVM sometimes.
Regarding the fragmentation: I am worried too. But I hope/expect that cats will do everything to make depending on cats-kernel as painless as possible for algebra and spire. I think it would be great if all three projects (cats/algebra/spire) would sort this out once and for all, preferably this month. The current state of having to import typeclass instances from all over the place is really less than optimal.
Denis Rosset
@denisrosset
Jan 04 2016 13:19
@mandubian : the pure-FP vs. standard library fragmentation is already there, and given Martin Odersky last post, I don't see Scala's stdlib adopting Task-like effect representations or the like. There is already fragmentation between algebird/spire/breeze for example, and I don't feel "algebra depends on cats" helping to reduce that fragmentation.
The algebird/spire/breeze fragmentation is the one I'm concerned about BTW, because I use Scala a lot for numerics / finite group computations. For the rest, cats vs. scalaz , simple vs. comprehensive Scala stdlib, I don't have a strong opinion. Most of the users of my libraries will call them from Matlab using very dumb Java interfaces anyway.
Denis Rosset
@denisrosset
Jan 04 2016 13:24
If my libraries were in a movie, it would start like "A Beautiful Mind" and end like "Idiocracy".
Rüdiger Klaehn
@rklaehn
Jan 04 2016 13:29
I am concerned about cats and algebra/spire using the same typeclasses, since I would like to use both in the same project, and also have some common typeclasses to use in basic libraries. I was actually quite happy with cats depending on algebra. But if it is the other way round I hope it can also be made to work.
Rüdiger Klaehn
@rklaehn
Jan 04 2016 13:40
@denisrosset I tried a similar trick as you do in metal (having a data structure with non-specialized type parameter, and having the specialization on the method). But I could not get it to work because I was not willing to use macros. I have to take a closer look at the library. The focus seems to be slightly different than abc. You spend much more effort avoiding even short-lived boxing, while I have efficient bulk operations (e.g. Set.union) and more typeclass support.
Denis Rosset
@denisrosset
Jan 04 2016 13:45
@rklaehn, actually my goal was to not box at all, and not even allocate. I have macros for the loop methods such as foreach or forall that eliminate the closures completely.
I split metal in two parts: core is about the capabilities of objects and the macro machinery, while library provides the implementation of mutable and immutable versions of hash maps, sets, and the like.
If you want to use the core part, I'm open about splitting it in a different common project.
Pascal Voitot
@mandubian
Jan 04 2016 13:46
@denisrosset there are a lot of discussions on Task/IO in scala community, even in Haskell community... there is no perfect representation for now... so it's not surprising that it won't appear in stdlib before long...
Denis Rosset
@denisrosset
Jan 04 2016 13:51
@mandubian : I quite like that I can write Scala code in a quick'n'dirty imperative way, while I can request a straight-jacket for other parts of my code. I deal with numerics in the quantum physics community, it's difficult for me to ask my users to use Scala. It would be impossible to ask them to use Scala + cats for example. Everybody is using Matlab or Python, they need a familiar context to make the switch into. So the versatility of Scala is a huge +1 for me.
Pascal Voitot
@mandubian
Jan 04 2016 14:08
@denisrosset yes I know... that's why scala is interesting right now :) I have the luck to work on greenfield stuff with people receptive to FP concepts...
InTheNow
@InTheNow
Jan 04 2016 14:24

... can write Scala code in a quick'n'dirty imperative way....

I agree that this is the best answer, for science and engineering. In the past, C/C++ tried to be the way to go, but is really too much if it's not your main job - same for quants in finance. But python/matlab are/were based on C/C++ but hid the details unless really needed. I think the same is needed for scala, except that the division has to be self imposed; unless, of course, there's more than one scala ;)

Denis Rosset
@denisrosset
Jan 04 2016 14:29
@InTheNow at the same time, you can tell those people "my library is based on sound algebraic principles, here are the laws they obey, here are the cumulative test results checking 100'000 random cases" and people are safe, warm and comfortable to take your stuff and use it
InTheNow
@InTheNow
Jan 04 2016 14:29
After all, Linux managed that - eg Linux , on Android and http://www.top500.org/statistics/list/ - sort by OS
@denisrosset Yep. Another interesting angle that I read from the world of "Data Science" is that work must be repeatable, so for example the use of git is required. I wonder how far the use of dynamic languages, and the loading of libs dynamically, fits into repeatable if you don't know what libs you actually used? Same goes to scala, btw
Also, matlab/python have the advantage of being taught at university - getting scala into courses is the real nut to crack
Pascal Voitot
@mandubian
Jan 04 2016 15:19
@InTheNow the people you've seen at MFGLabs were doing mainly python & C & R... just because all libs have been plugged on those langs... and because they are mathematicians, not full-time developers which is not exactly the same job
InTheNow
@InTheNow
Jan 04 2016 15:21
@mandubian I wasn't actually referring to them specifically, but yes - it's the same
Pascal Voitot
@mandubian
Jan 04 2016 15:21
yes sure, I took an example we know both :)
InTheNow
@InTheNow
Jan 04 2016 15:21
:)
Another example is @antonkulaga, a good scala dev currently in Paris do work in Biology - all in MatLab. I believe he has learnt to hate sbt the hard way ;)
Pascal Voitot
@mandubian
Jan 04 2016 15:25
It's still a very different approach to write code that can run 24/24 in production than to run an algorithm whatever duration & resources it takes... but I believe it should be the same :)
InTheNow
@InTheNow
Jan 04 2016 15:29
If cern can do it in C++/Python, I'm sure scala can too ;) https://root.cern.ch/
interpreted C++ before LLVM, not bad
Pascal Voitot
@mandubian
Jan 04 2016 15:35
at least in a compiled & typed language, it will be better :)
InTheNow
@InTheNow
Jan 04 2016 15:47
talking of science, anyone come across https://try.jupyter.org/ ?
Supports scala, too
Rüdiger Klaehn
@rklaehn
Jan 04 2016 15:51
They used this in a spark course I did last year. I found the lack of type safety in python quite annoying.
InTheNow
@InTheNow
Jan 04 2016 15:58
But in theory, scala could have been used?
Rüdiger Klaehn
@rklaehn
Jan 04 2016 16:01
Yes. But the autograder for that course used python, and I took this as an opportunity to learn python a bit. Much nicer than javascript for a dynamic language, I must say.
InTheNow
@InTheNow
Jan 04 2016 16:01
eg like in the haskell notebook, where the haskell can be run in the cells - almost like dynamic tut
Rüdiger Klaehn
@rklaehn
Jan 04 2016 16:02
InTheNow
@InTheNow
Jan 04 2016 16:03
ooh - loads of new toys :)
InTheNow
@InTheNow
Jan 04 2016 16:11
Straying of the topic of algebra, but those most interested in promoting scala to a larger (python) audience should really get scala working in these sorts of projects
Iscala is over a year old, uses 2.10 and has not scala.js support for gui
InTheNow
@InTheNow
Jan 04 2016 17:09
Actually, current scala is via spark from IBM
or jupyter-scala that even shows simulacrum support ;) (and psp)
Pascal Voitot
@mandubian
Jan 04 2016 17:39
you've got Andy Petrella doing great job too https://github.com/andypetrella/spark-notebook
spark from IBM :(
boohoooooo why do BS companies need to put their big fingers in everything like that :(
InTheNow
@InTheNow
Jan 04 2016 17:42
jupyter-scala is from @alexarchambault so that should make up for it :)
Pascal Voitot
@mandubian
Jan 04 2016 17:44
yes too, good guy ;)
there are too many good guys here around anyway, stop being so good and do mediocre things sometimes!
InTheNow
@InTheNow
Jan 04 2016 23:55
That jupyter project got $6M in funding last July