These are chat archives for elemental/chat

29th
Apr 2017
Hari Sundar
@hsundar
Apr 29 2017 05:43 UTC
Is El::Multiplynot implemented for DistSparseMatrices? I get undefined references during linking
Jack Poulson
@poulson
Apr 29 2017 14:20 UTC
It is implemented: can you share the linker error that shows the full signature?
With that said, yhe distributed sparse matrix vector products arent particularly fast due to the 1d distribution
Jed Brown
@jedbrown
Apr 29 2017 14:23 UTC
What sort of sparsity pattern? 1D distribution should be good for problems that partition well -- bounded degree, modest expansion factor.
Hari Sundar
@hsundar
Apr 29 2017 15:06 UTC
main.cpp:(.text.startup+0xc87): undefined reference to `void El::Multiply<double>(El::OrientationNS::Orientation, double, El::DistSparseMatrix<double> const&, El::AbstractDistMatrix<double> const&, double, El::AbstractDistMatrix<double>&)'
main.cpp:(.text.startup+0xce9): undefined reference to `void El::Multiply<double>(El::OrientationNS::Orientation, double, El::DistSparseMatrix<double> const&, El::AbstractDistMatrix<double> const&, double, El::AbstractDistMatrix<double>&)'
collect2: error: ld returned 1 exit status
this is with version 0.87.7
Jack Poulson
@poulson
Apr 29 2017 16:33 UTC
The link error is due to the AbstractDistMatrix arguments, as it is implemented for the deprecated DistMultiVec class. You can redistribute between them with Copy.
Jed: random nonzero patterns tend to be very problematic as well.
But one can get pretty far with 1d.
Really what I wanted to say is that the distributed sparse support is a mess.
And, honestly, I think that I am getting ready to stick a fork in Elemental. Basically everything deserves a rewrite.
The software engineering quality doesnt meet my standards anymore
Jed Brown
@jedbrown
Apr 29 2017 16:37 UTC
Jack: matrices with bounded degree and modest expansion factor can and should be reordered. (Performance is much better than using a random ordering with any distribution.)
Interesting. What would you do differently?
Jack Poulson
@poulson
Apr 29 2017 16:38 UTC
Make MPI support downstream from the main library for starters
Jed Brown
@jedbrown
Apr 29 2017 16:38 UTC
Sometimes I fantasize about a successor to PETSc, but I'm very wary of second system syndrome.
Jack Poulson
@poulson
Apr 29 2017 16:38 UTC
Write my own METIS and AMD
Support for algebraic combinatorics
Jed Brown
@jedbrown
Apr 29 2017 16:39 UTC
Yeah, the community needs clean and robust preconditioners.
Jack Poulson
@poulson
Apr 29 2017 16:39 UTC
Not rush anything and do real code review and design docs
Jed Brown
@jedbrown
Apr 29 2017 16:40 UTC
With MPI, do you mean to abstract parallelism so it can run on top of different models or to write a sequential/shared-mem library?
Erm, I meant partitioners. Fingers got away from me.
Jack Poulson
@poulson
Apr 29 2017 16:41 UTC
I mean to have standalone releases of sequential and shared memory functionality
Jed Brown
@jedbrown
Apr 29 2017 16:42 UTC
What is wrong with using a mock MPI? Or is this a development objective, to make new implementations easier and possibly the interface simpler?
My concern is that focusing on the sequential case (which, let's face it, is still the majority of science) greatly increases the incremental complexity.
The holy grail, of course, is to make experimentation easy while still having a relatively easy ramp up to scalability.
Jack Poulson
@poulson
Apr 29 2017 16:44 UTC
I don't see a reason not to burden the math exploration with the distributed weight
Jed Brown
@jedbrown
Apr 29 2017 16:44 UTC
Maybe that's a pipe dream.
Jack Poulson
@poulson
Apr 29 2017 16:44 UTC
err, remove "not"
If nothing else, 99% of use cases of math dont need distributed functionality, so dont burden those users
Jed Brown
@jedbrown
Apr 29 2017 16:47 UTC
Is it first and foremost a math tool or an engineering/science tool?
Perhaps the latter is 70%, for example.
And a much larger and richer ($) community.
Jack Poulson
@poulson
Apr 29 2017 17:14 UTC
I dont see any forfeiture of capability from having the MPI component as a downstream package. And, to be clear, I receive 0 dollars in funding for Elemental since leaving academia
But fast iteration of math exploration is useful as a lot of exploration can be done on a laptop
Jed Brown
@jedbrown
Apr 29 2017 17:15 UTC
Ah, you're not working on Elemental while at Google?
Jack Poulson
@poulson
Apr 29 2017 17:15 UTC
It has nothing to do with my job.
Jed Brown
@jedbrown
Apr 29 2017 17:16 UTC
Ah, didn't know that.
I don't think sequential AMG gets you much closer to parallel AMG.
For example. In contrast, a parallel AMG can run sequentially with little overhead.
Jack Poulson
@poulson
Apr 29 2017 17:18 UTC
But sequential SDPs do
The difference is in whether the high level algorithm is even settled
Jed Brown
@jedbrown
Apr 29 2017 17:19 UTC
In a lot of domains that I work with, if you think about how the pieces fit together up-front, the parallel implementation is only marginally more complicated than sequential, but you can't build the parallel thing from the sequential.
High level interface or high level implementation?
Jack Poulson
@poulson
Apr 29 2017 17:20 UTC
High level impl
My thesis is if there is any algorithmic uncertainty (e.g., for stability reasons) that one should settle this first.
Parallelism should come after this step
Cases where achieving parallelism requires accuracy tradeoffs are in the middle but are not the norm
Jed Brown
@jedbrown
Apr 29 2017 17:23 UTC
And likely no single implementation is appropriate for all inputs/requirements, but a well-defined interface allows users to get their work done now and provides use cases for better implementations that may come in the future. It also allows other members of the community an environment in which to effectively disseminate their research results.
DD methods for PDEs have been an active research area for decades, but there is a significant amount of research that may never have happened if not for PETSc, simply because the work would have been academic instead of deployed to real users. It still isn't "settled", but I'd say it helped to have the interfaces.
Jack Poulson
@poulson
Apr 29 2017 17:25 UTC
Also, to be clear, I am not arguing that PETSc should make a similar decision
DD is firmly in the "middle"
Jed Brown
@jedbrown
Apr 29 2017 17:27 UTC
Shortcomings in existing methods often are not understood until they have users. Having users provides a deployment opportunity for fundamental research. I think there is a lot of value in a package that facilitates such research, rather than waiting to get it perfect before deploying.
I'm talking generally about the relationship between software and research, not specific packages.
Jack Poulson
@poulson
Apr 29 2017 17:28 UTC
Need to go though. Currently marching at climate march.
Jed Brown
@jedbrown
Apr 29 2017 17:29 UTC
Ah, nice. It's snowing here.
(A bit sad the Climate March has a statement against nuclear energy. Regardless of your perspective on nuclear, climate isn't a reason to avoid it.)
Jack Poulson
@poulson
Apr 29 2017 18:29 UTC
I didnt know about that stance :-(
Jed Brown
@jedbrown
Apr 29 2017 18:34 UTC
I guess all I saw was this, which has an Anti-Nuclear section. https://peoplesclimate.org/lineup/