These are chat archives for symengine/symengine

11th
Mar 2017
Shikhar Jaiswal
@ShikharJ
Mar 11 2017 13:26
@isuruf Regarding symengine/symengine.py#76, ImmutableDenseMatrix should be a separate class in itself right? (like MutableDenseMatrix, deriving from MatrixBase)
Isuru Fernando
@isuruf
Mar 11 2017 14:17
They both should derive from DenseMatrixBase and MutableDenseMatrix would have methods that mutate the matrix.
berceanu
@berceanu
Mar 11 2017 17:23
could you guys recommend a matrix class for doing linear algebra in C++?
there seems to be no consensus
people recommend armadillo, mtl4, blitz, etc
i'm coming from python/numpy and am quite new to C++
Isuru Fernando
@isuruf
Mar 11 2017 17:25
eigen?
berceanu
@berceanu
Mar 11 2017 17:26
can it do numpy-style slicing?
A[2:5, :]
Isuru Fernando
@isuruf
Mar 11 2017 17:28
don't think so
berceanu
@berceanu
Mar 11 2017 17:29
pff
what solution did you use in symengine?
Isuru Fernando
@isuruf
Mar 11 2017 17:30
our own implementation. most linear algebra libraries have routines that are incompatible with symbolic expressions
berceanu
@berceanu
Mar 11 2017 17:31
ok, makes sense
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:31
(that's where Julia is nice. Generic fallbacks "just work" :smile: )
berceanu
@berceanu
Mar 11 2017 17:31
what makes less sense is that there is still no standard for matrices in C++ as far as number-crunching is concerned
in some ways it's worse for numerics than fortran
Isuru Fernando
@isuruf
Mar 11 2017 17:32
@ChrisRackauckas, generic fallbacks in julia are still written with numerical entities in mind.
@berceanu, yep.
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:34
yes, but they don't have to specialize to the type, so "matrix of symbolic expressions" just works because of the operations that are defined by the symbolic expression. I guess you can do something like that with C++ templates?
Isuru Fernando
@isuruf
Mar 11 2017 17:34
Yes, eigen is C++ template library like that.
berceanu
@berceanu
Mar 11 2017 17:37
apparently armadillo is more matlab-like
Isuru Fernando
@isuruf
Mar 11 2017 17:40
that's great. boost's multi-array has slicing memory views etc
berceanu
@berceanu
Mar 11 2017 17:41
so did numpy guys implement their own array type in C++?
Isuru Fernando
@isuruf
Mar 11 2017 17:41
i think so
Scott P. Jones
@ScottPJones
Mar 11 2017 17:42
@berceanu Why are you moving to C++? Have you considered other languages which might be a better fit for your work (and for you, coming from Python)?
berceanu
@berceanu
Mar 11 2017 17:43
I would much rather work in Python and/or Julia
or even Fortran
but I have to do large-scale distributed particle-in-cell simulations on a HPC
Scott P. Jones
@ScottPJones
Mar 11 2017 17:44
hehe, then I'd recommend going with Julia - it's rapidly "firming up" as it gets closer to v1.0 ;-)
berceanu
@berceanu
Mar 11 2017 17:44
and most of the codes are in C++
we're talking huge codes here, not something I can rewrite in one weekend
Scott P. Jones
@ScottPJones
Mar 11 2017 17:45
But that shouldn't be that much of an issue - why rewrite? (we use a number of C++ libraries, for example gecode, for constraint programming)
berceanu
@berceanu
Mar 11 2017 17:45
what do you mean it shouldn't be an issue?
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:45
it depends on whether you're using the big C++ code as a black box or not.
berceanu
@berceanu
Mar 11 2017 17:46
there's two parts to this, running the codes on the cluster, in which i don't care about the code and i'm using it as a black box
Scott P. Jones
@ScottPJones
Mar 11 2017 17:46
That you shouldn't have to rewrite the C++ code - of course, @ChrisRackauckas is correct, that it does depend on how you are using it
berceanu
@berceanu
Mar 11 2017 17:46
and then there's the need to implement new things
for example i need to use http://geant4.web.cern.ch/geant4/
a C++ code that simulates particle-matter interactions
Scott P. Jones
@ScottPJones
Mar 11 2017 17:47
implementing new things - well, after decades of C/C++, and the last 2 years using Julia, I'd definitely go for Julia to implement anything new
berceanu
@berceanu
Mar 11 2017 17:48
sure, me too
but how do you implement a new scattering mechanism in a million lines C++ code from 1996 using Julia
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:49
@ScottPJones scientists try to take the path of least resistance. Writing a Clang wrapper is time we don't have, even if it is mostly automatic.
Remember, there is no academic credit for software development
berceanu
@berceanu
Mar 11 2017 17:49
unfortunately, yes
that also explains the quality of most scientific codes out there
which are quick hacks put together by generations of phd students
but anyway, back to the issue
say you are given a CERN C++ library, probably using MPI, probably having more than 1 million lines of code
and you need to implement something new in it
as much as I want to use Julia, I don't see how it would help me here
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:53
agreed
unless you write a plugin interface
which is a software engineering job that would get you no academic credit.
berceanu
@berceanu
Mar 11 2017 17:53
well
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:54
and people would probably hate that your contribution adds a Julia dependency
berceanu
@berceanu
Mar 11 2017 17:54
exactly
also, it depends on the time it takes to write the interface
if overall implementing the interface and then using julia is faster than changing the original C++
might be worth a shot
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:55
I'm sure it's much quicker than you think, but there's still language mixing issues that, unfortunately, most people would be uncomfortable with.
berceanu
@berceanu
Mar 11 2017 17:56
but it's a problem julia has to solve somehow if it wants to succeed
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:56
For these kinds of things, I just go middle of the road. Eigen is widely used, Eigen is probably a good idea. Pound out a solution without being proud of the code, but get the paper done.
berceanu
@berceanu
Mar 11 2017 17:57
yup
sounds good
anyway, i was saying, julia is addressing high-performance computing
looks like it can do slicing
I don't know about views
berceanu
@berceanu
Mar 11 2017 17:57
so it has to deal with this, the fact that almost all existing codes are in fortran and C/C++
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:57
That syntax is pretty scary though
berceanu
@berceanu
Mar 11 2017 17:58
:)
Scott P. Jones
@ScottPJones
Mar 11 2017 17:58
Of course, you can make a potential future career out of Julia development, doing nice things for scientists, like @ChrisRackauckas has been doing ;-)
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:58
I started from scratch and build a hundred thousand line of code library in my free time, and got academic blowback for spending the 6 months or so doing it.
Scott P. Jones
@ScottPJones
Mar 11 2017 17:59
I thought that Cxx.jl pretty much solved the issues of using C++ code directly though?
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 17:59
It sounds like @berceanu 's issue is that it's a methods project
berceanu
@berceanu
Mar 11 2017 18:00
wait, so they gave you a hard time instead of thanking you?
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:00
normally what one does it develops a method, does small scale tests, then incorporates it as a change to a larger project to show that it does make it better/faster/less error, and that's the paper.
I assume that's what you're doing?
@berceanu you can't get high-impact papers for software development.
Scott P. Jones
@ScottPJones
Mar 11 2017 18:01
Silly old guard academics ;-) Hopefully they are now seeing the error of their ways, and will be more encouraging of the transformative nature of what Chris has been doing.
berceanu
@berceanu
Mar 11 2017 18:01
well, no, but you can produce high impact research in the long run
that's more or less the jist of it, yes
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:01
I did get one paper out of this which is submitted to a scientific software journal (JORS, IF ~3), but it put behind some things I am submitting to Cell Systems (IF >> 3)
advisor and committee doesn't care about software. They want papers.
Anything that delays a paper is bad.
berceanu
@berceanu
Mar 11 2017 18:02
reminds me of my phd days, when my supervisor got pissed for converting her fortran code to julia, reducing its size 10-fold
"it's nice, but this is not fortran"
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:03
^
berceanu
@berceanu
Mar 11 2017 18:03
"is this what you've been doing instead of working on that calculation"
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:03
^
berceanu
@berceanu
Mar 11 2017 18:03
yeah
the best argument is "ok, it's spaghetti code, but it works, ok?? no need to change it."
i think mostly their fear is they somehow lose control of their code
Francesco Biscani
@bluescarni
Mar 11 2017 18:05
my personal view is that academia is fundamentally a highly reactionary environment
berceanu
@berceanu
Mar 11 2017 18:05
it's completely cemented
well, that's also a good thing sometimes
when i think of the other end of the spectrum, like, i dunno, web development
which is more trendy than women's fashion
and you have a new framework coming out each week
Isuru Fernando
@isuruf
Mar 11 2017 18:06
:smile:
berceanu
@berceanu
Mar 11 2017 18:07
i'd hate to work in such an environment
Shikhar Jaiswal
@ShikharJ
Mar 11 2017 18:07
:smile:
Francesco Biscani
@bluescarni
Mar 11 2017 18:07
:)
berceanu
@berceanu
Mar 11 2017 18:07
but, yea. there's people still writing new Fortran 77 code
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:08
I think it will keep happening until research software engineering grows to become its own force in science, along with software sustainability initiatives.
You can still publish papers about algorithms without sharing the code, so that tells you where we're at.
berceanu
@berceanu
Mar 11 2017 18:09
yea. it's also a problem of funding in the end
academic research is severely underfunded
so there is so much competition to get the scraps (grants) from the government
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:10
When software sustainability initiatives enforce that published code has to be opensource, available in some version-controlled repo, CI tested, etc. in order to get papers and grants, research software engineering will be something that people in academia care about
berceanu
@berceanu
Mar 11 2017 18:10
that any delay in deliverables (papers) is unacceptable
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:10
yeah, and right now bad (or even wrong) software doesn't delay papers.
berceanu
@berceanu
Mar 11 2017 18:11
well, papers seem to come out, more than ever
reproducibility goes down the drain though
but hey, who cares about that as long as i've got my science paper right
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:12
it's definitely a local minima, but only the people who are on the journal and grant review boards have the power to make the software reproducibility matter...
berceanu
@berceanu
Mar 11 2017 18:12
and do they have any incentive to do so?
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:12
No, because they're likely too old to know what Github, version control, and CI testing are.
@isuruf is SymEngine funded at all? Is it through SymPy through NumFocus?
berceanu
@berceanu
Mar 11 2017 18:13
also, it's not like Nature will make more money if software in their papers gets published
Isuru Fernando
@isuruf
Mar 11 2017 18:13
Only through GSoC
berceanu
@berceanu
Mar 11 2017 18:14
@ChrisRackauckas my problem is that i've somehow manadged to avoid C++ until now
and after using Julia and Python/numpy, it feels, well..
let's say you can tell it was not meant for this
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:15
Slicing
RowVectorXf v = RowVectorXf::LinSpaced(20,0,19);
cout << "Input:" << endl << v << endl;
Map<RowVectorXf,0,InnerStride<2> > v2(v.data(), v.size()/2);
cout << "Even:" << v2 << endl;
It feels like that?
berceanu
@berceanu
Mar 11 2017 18:16
is there also slicing for humans?
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:17
Armadillo looks better for that
berceanu
@berceanu
Mar 11 2017 18:17
yea, i saw it's more matlab like
but i bet they would get pissed for introducing another external dependency
Scott P. Jones
@ScottPJones
Mar 11 2017 18:18
I keep feeling I'd rather retire or slit my wrists rather than have to be programming full-time in C++ rather than Julia
berceanu
@berceanu
Mar 11 2017 18:18
hahah
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:18
what library do they use internally for their linear algebra?
they must use one already
and that's probably what you have to use.
You could also go the route of making the modification, getting the paper, and never submitting the code upstream
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:19
that just feels wrong though, but I know that's academically acceptable
berceanu
@berceanu
Mar 11 2017 18:19
wow they're even on github
i mean, it's not like we're meant to make their code faster or anything
we need to implement some new functionality
would it be nice if everyone benefited? yup
to give them some credit, the initial release was in 1998
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:22
oh great
berceanu
@berceanu
Mar 11 2017 18:22
no Julia back then, end even Python was in its infancy
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:22
it's one of these beasts
this looks like is where their matrix type comes from
berceanu
@berceanu
Mar 11 2017 18:23
like, their own internal implementation, right?
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:23
not necessarily, but pretty close
berceanu
@berceanu
Mar 11 2017 18:24
and here i am doing a course in modern C++
it's probably going to drag me back if anything
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:24
CLHEP looks be the only dependency that they mention, so they might be pretty strict about dependencies
(don't worry, CLHEP had a workshop in 2003, maybe they'll have another?)
berceanu
@berceanu
Mar 11 2017 18:24
first commit: "added Julia to the dependency list"
"The HepMatrix class does all the obvious things, in all the obvious ways. "
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:27
wait, I'm totally not done shitting on this
berceanu
@berceanu
Mar 11 2017 18:27
there you have it, all the doc you ever wanted
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:27
Simplified version of CLHEP HepMatrix class
that's right, they don't just use the CLHEP matrices
they build their own internal version using CLHEP matrices
berceanu
@berceanu
Mar 11 2017 18:28
because, why not
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:28
so it is both an internal version AND an unknown dependency, together as one great world
oh god, you're on your own.
berceanu
@berceanu
Mar 11 2017 18:28
tnx man :)
i'll report back in a couple of months
// - Imported from CLHEP and modified: P. Arce May 2007
sounds like copy-paste to me
is there a way to see how many LOC their repo has?
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:32
I only know how to do that after cloning it
BTW, it has its own license
berceanu
@berceanu
Mar 11 2017 18:32
1994
even better
how do you get LOC after cloning?
Christopher Rackauckas
@ChrisRackauckas
Mar 11 2017 18:34
git ls-files | xargs cat | wc -l or Atom has a tool for it.
berceanu
@berceanu
Mar 11 2017 18:35
Atom is not CERN-approved i think
it would be frowned upon
3'430'220
looks like hello world to me
try interfacing this with Julia :)