These are chat archives for elemental/chat

20th
Aug 2016
Jack Poulson
@poulson
Aug 20 2016 05:19
Elemental's distributed SVDs now support Divide-and-Conquer
Ryan H. Lewis
@rhl-
Aug 20 2016 15:09
isn't deflation good?
Jack Poulson
@poulson
Aug 20 2016 15:09
yes, it is often argued that D&C is sub-cubic because of deflation
but one has to have very large problems for that to occur as the precision increases
or have low-rank matrices
Ryan H. Lewis
@rhl-
Aug 20 2016 15:12
@poulson there is something wrong with my elemental rpm: it doesn't install :(
elena got me sick so I've been a bit AWOL
Jack Poulson
@poulson
Aug 20 2016 15:13
sorry to hear that!
hopefully you both are feeling better
Ryan H. Lewis
@rhl-
Aug 20 2016 15:38
@poulson elemental package seems to install properly in fedora 22, but not fedora 24
Jack Poulson
@poulson
Aug 20 2016 16:37
what about it?
Ryan H. Lewis
@rhl-
Aug 20 2016 17:29
it works :)
something is wrong on the latest fedora though
Jack Poulson
@poulson
Aug 20 2016 17:30
ah, well the first part is good news!
Ryan H. Lewis
@rhl-
Aug 20 2016 17:30
i'm going to start a redhat ticket for inclusion in the mainline repositories
Jack Poulson
@poulson
Aug 20 2016 17:30
I see DoubleDouble and QuadDouble are supported as well
great; just in time for the D&C SVD to make it in
Ryan H. Lewis
@rhl-
Aug 20 2016 17:30
the build is in the elemental.spec file
@poulson time to cut a release :)
Jack Poulson
@poulson
Aug 20 2016 17:52
I'm going to try to get sequential and distributed D&C Hermitian eigensolvers in by the end of the weekend
Ryan H. Lewis
@rhl-
Aug 20 2016 17:53
libpmrr.so apparently contains code that calls exit
Jack Poulson
@poulson
Aug 20 2016 17:53
hmm
you may want to contact Paolo Bientinesi about this
Ryan H. Lewis
@rhl-
Aug 20 2016 17:54
ok
Jack Poulson
@poulson
Aug 20 2016 17:54
all the more reason to have backup eigensolvers
I was excited for a second
Ryan H. Lewis
@rhl-
Aug 20 2016 17:56
haha
where are documentation sources?
Jack Poulson
@poulson
Aug 20 2016 17:57
you may want to file an issue on one of these repos: https://github.com/HPAC
I have a fuzzy vision in my head of completely redoing both the webpage and docs in a tutorial format with the more impressive capabilities of the library
as no-frills and short as possible
there is just too much stuff to document everything without a bigger community
Ryan H. Lewis
@rhl-
Aug 20 2016 17:59
i think a tutorial of elemental within python would be interesting
especially elemental within python + mpi
Jack Poulson
@poulson
Aug 20 2016 18:00
better numpy integration is needed first
it is always the first thing asked for
and I am not a pythonista, so I doubt I'm the best person to do that
Ryan H. Lewis
@rhl-
Aug 20 2016 18:00
right
im trying to find the ridge regression docs
Ryan H. Lewis
@rhl-
Aug 20 2016 18:08
@poulson : elemental/elemental-web#3
Ryan H. Lewis
@rhl-
Aug 20 2016 19:07
@poulson: ping ?
Ryan H. Lewis
@rhl-
Aug 20 2016 19:13
@poulson trying to polish off my sparse multiply branch. I am uber confused atm. would appreciate your help
Ryan H. Lewis
@rhl-
Aug 20 2016 19:26
I'm having trouble again with the macro system within Multiply.cpp
Jack Poulson
@poulson
Aug 20 2016 19:36
back
what are you confused about?
Ryan H. Lewis
@rhl-
Aug 20 2016 19:37
So, I added MultiplyCSR specialization for multiplies against a Graph
essentially dropped the values array
Jack Poulson
@poulson
Aug 20 2016 19:37
OK
Ryan H. Lewis
@rhl-
Aug 20 2016 19:37
it seems that there are a number of specializations for the usual case
I also added a SparseMatrix * Matrix -> Matrix
or DistMatrix
cant remember
anyways
that one is not working
Jack Poulson
@poulson
Aug 20 2016 19:38
what do you mean by "not working"?
Ryan H. Lewis
@rhl-
Aug 20 2016 19:38
when you instantiate all the special templates
it cannot deduce types
i can find the error momentarily
I just commented that one out and pushed it
to see if I have broken anything
Jack Poulson
@poulson
Aug 20 2016 19:39
the types of what cannot be deduced?
which routine?
Ryan H. Lewis
@rhl-
Aug 20 2016 19:40
like BigInt and such
I actually am in the middle of a build locally
so I can't reproduce the bug
Jack Poulson
@poulson
Aug 20 2016 19:40
I assume you're not using the Intel compilers
Ryan H. Lewis
@rhl-
Aug 20 2016 19:40
Oh, and I changed something from DistMultiVec to DistMatrix
Screen Shot 2016-08-20 at 12.41.30 PM.png
I had to change : /Users/rhl/code/Elemental/src/blas_like/level3/Multiply.cpp:687:37: error: no member named 'FirstLocalRow' in 'El::DistMatrix<El::BigInt, El::DistNS::Dist::MC, El::DistNS::Dist::MR, El::DistWrapNS::DistWrap::ELEMENT>'
const Int firstLocalRow = X.FirstLocalRow();
to use X.RowShift()
not sure if this is correct, but it seems right from documentation
Jack Poulson
@poulson
Aug 20 2016 19:43
yes, the member functions are different
Ryan H. Lewis
@rhl-
Aug 20 2016 19:43
did I make the appropriate change?
Jack Poulson
@poulson
Aug 20 2016 19:43
you will need to ensure that the DistMatrix<T,VC,STAR,BLOCK> passed in has the right distribution block size as well
Ryan H. Lewis
@rhl-
Aug 20 2016 19:44
Ok, we will get to that
Jack Poulson
@poulson
Aug 20 2016 19:44
switching that implementation is not likely to be a simple search and replace
Ryan H. Lewis
@rhl-
Aug 20 2016 19:44
Maybe I should have done that separately.
Well, it's a learning experience one way or the other
Jack Poulson
@poulson
Aug 20 2016 19:44
yes, incremental changes are much easier
Ryan H. Lewis
@rhl-
Aug 20 2016 19:45
Oh, found the traceback
Jack Poulson
@poulson
Aug 20 2016 19:51
did you update the prototypes in the headers?
Ryan H. Lewis
@rhl-
Aug 20 2016 19:51
eh, which header?
Jack Poulson
@poulson
Aug 20 2016 19:52
wherever MultiplyCSR is declared
Ryan H. Lewis
@rhl-
Aug 20 2016 19:52
didnt see a header
looks like its cpp only
/Users/rhl/code/Elemental/include
$ grep -ir MultiplyCSR *
$
Multiply exists in headers
Ryan H. Lewis
@rhl-
Aug 20 2016 19:57
looking at the diff from master, im not understanding something about the pre-existing: template<typename T>
 void Multiply
( Orientation orientation,
  T alpha, const SparseMatrix<T>& A, const Matrix<T>& X,
  T beta,                                  Matrix<T>& Y )
it doesn't seem like it would have compiled
I made this change to its implementation:
 template<typename T>
 void MultiplyCSR
@@ -544,22 +598,43 @@ void Multiply
           LogicError("X and Y must have the same width");
     )
     MultiplyCSR
-    ( orientation, A.Height(), A.Width(), X.Width(),
+    ( orientation, A.Height(), A.Width(), 
+      alpha, A.LockedOffsetBuffer(), 
+             A.LockedSourceBuffer(), 
+             A.LockedTargetBuffer(),
+             X.LockedBuffer(), 
+      beta,  Y.Buffer() );
+}
Signature of existing MultiplyCSR:
template<typename T>
void MultiplyCSR
( Orientation orientation,
  Int m, Int n,
  T alpha,
  const Int* rowOffsets,
  const Int* colIndices,
  const T*   values,
  const T*   x,
  T beta,
        T*   y )
Jack Poulson
@poulson
Aug 20 2016 20:01
there is a missing BigInt specialization of MultiplyCSR
might be my fault
Ryan H. Lewis
@rhl-
Aug 20 2016 20:01
did I delete it?
Jack Poulson
@poulson
Aug 20 2016 20:02
no
I will add it real quick
Ryan H. Lewis
@rhl-
Aug 20 2016 20:02
Do you think those specializations can be cleaned up? they all look very very similar.
@poulson can you add it to my branch?
Jack Poulson
@poulson
Aug 20 2016 20:02
you should be able to pull
and, no, I can't edit your PR
it doesn't let me
Ryan H. Lewis
@rhl-
Aug 20 2016 20:03
oh, fascinating. ok
ill merge it
Jack Poulson
@poulson
Aug 20 2016 20:03
give me five minutes to commit
Ryan H. Lewis
@rhl-
Aug 20 2016 20:03
kk
there appears to be no signficant difference between Int, Quad, Complex<Quad> impl
this code can all be squashed down to a single function
it looks like the MKL call is the only distinction
Jack Poulson
@poulson
Aug 20 2016 20:06
there are three impl's required
I'm doing so with different type constraints
Ryan H. Lewis
@rhl-
Aug 20 2016 20:07
why is there full specialization at all?
aside from MKL you are just changing the types
Jack Poulson
@poulson
Aug 20 2016 20:08
I'm eliminating the conjugation branch for integral types in my commit as well
Ryan H. Lewis
@rhl-
Aug 20 2016 20:08
cant you just define Conj(Int x){ return x; }
or whatever
Jack Poulson
@poulson
Aug 20 2016 20:09
there is an outer branch
Ryan H. Lewis
@rhl-
Aug 20 2016 20:09
why remove it manually?
the compiler can figure this out
one branch in a template > zero branches + duplicate impl per type
Jack Poulson
@poulson
Aug 20 2016 20:13
committed
I left out the integral specialization
Ryan H. Lewis
@rhl-
Aug 20 2016 20:17
@poulson yeah, there we go :)
I see there was a multiply RHS hiding under all those
im rebuilding now
Ryan H. Lewis
@rhl-
Aug 20 2016 20:22
@poulson new error, pushed it. gotta run out. looks like your template stuff is conflicting with my new additions
Ryan H. Lewis
@rhl-
Aug 20 2016 21:56
@poulson: back
Jack Poulson
@poulson
Aug 20 2016 22:02
hello
what do you mean by the "template stuff" conflicting?
Ryan H. Lewis
@rhl-
Aug 20 2016 22:02
i'm trying to debug now
did you test your changes?
Jack Poulson
@poulson
Aug 20 2016 22:03
which ones?
Ryan H. Lewis
@rhl-
Aug 20 2016 22:03
to Multiply.cpp ?
Jack Poulson
@poulson
Aug 20 2016 22:04
it was copy and paste
is there a reason you're asking?
Ryan H. Lewis
@rhl-
Aug 20 2016 22:04
hm?
Jack Poulson
@poulson
Aug 20 2016 22:04
are you running into a bug or something?
Ryan H. Lewis
@rhl-
Aug 20 2016 22:04
yeah.. hang on
Jack Poulson
@poulson
Aug 20 2016 22:05
the tests/lapack_like/LeastSquares driver is fine
Ryan H. Lewis
@rhl-
Aug 20 2016 22:05
my sparse overloads aren't resolving...
Jack Poulson
@poulson
Aug 20 2016 22:06
if you paste the compile errors I'll take a look
I used to be good at this kind of stuff
i'm out of practice.
nvm, I see it
I lost LDim on X and Y
Jack Poulson
@poulson
Aug 20 2016 22:12
ah, okay
glad to play the rubber duck
Ryan H. Lewis
@rhl-
Aug 20 2016 22:12
nope
still broken
i need to train myself to use an actual rubber duck
ive pushed so travis should also rebuild soon
Ryan H. Lewis
@rhl-
Aug 20 2016 22:27
ah, sweet. I think I fixed it
Jack Poulson
@poulson
Aug 20 2016 22:27
what was the issue?
Ryan H. Lewis
@rhl-
Aug 20 2016 22:27
I had borked the Sparse*Matrix -> Matrix
I just copied it back from master
Jack Poulson
@poulson
Aug 20 2016 22:27
ah
Ryan H. Lewis
@rhl-
Aug 20 2016 22:27
it seems to have worked.
what were you saying about removing Multivec?
can you walk me through those changes?
i'll also add unit tests now for my new code
Jack Poulson
@poulson
Aug 20 2016 22:29
removing DistMultiVec is a large amount of work
at least a full weekend of work
Ryan H. Lewis
@rhl-
Aug 20 2016 22:29
from that one method?
Jack Poulson
@poulson
Aug 20 2016 22:29
removing it in general
Ryan H. Lewis
@rhl-
Aug 20 2016 22:29
oh, yeah, I just wanted to do it in that one spot.
crud, I have linker errors.
oh it expects multivec
oh, ok, let me try and undo those changes.
Now I see what I wanted
I ultimately wanted DistSparse * DistMatrix -> DistMatrix
thats why I was doing this
Jack Poulson
@poulson
Aug 20 2016 22:33
a lot of things are tied together
Ryan H. Lewis
@rhl-
Aug 20 2016 22:39
can you remind of of matrices in your generators which are sparse?
Jack Poulson
@poulson
Aug 20 2016 22:39
Helmholtz
Laplace
Identity
Ryan H. Lewis
@rhl-
Aug 20 2016 22:39
ah yeah
Jack Poulson
@poulson
Aug 20 2016 22:40
you could also load a MATRIX_MARKET one
I now have one or two checked in
Ryan H. Lewis
@rhl-
Aug 20 2016 22:40
well, I am writing unit tests
do your generators generate sparse matrices?
Jack Poulson
@poulson
Aug 20 2016 22:42
some of them, yes
Ryan H. Lewis
@rhl-
Aug 20 2016 22:42
not seeming the ones you just mentioned with sparse documentation
Jack Poulson
@poulson
Aug 20 2016 22:43
assuming that because it's not in the docs implies that it doesn't exist is a really bad idea at this point
Ryan H. Lewis
@rhl-
Aug 20 2016 22:43
yeah, ok
i understand
Jack Poulson
@poulson
Aug 20 2016 22:43
unfortunately nothing magically keeps the docs updated
:(
Ryan H. Lewis
@rhl-
Aug 20 2016 22:43
want to tell me one that you recall outputs any sparse matrix?
Jack Poulson
@poulson
Aug 20 2016 22:43
just do Laplace
Ryan H. Lewis
@rhl-
Aug 20 2016 22:43
how about Identity?
Jack Poulson
@poulson
Aug 20 2016 22:43
err, Laplacian
look in src/matrices/deterministic/pde/
Ryan H. Lewis
@rhl-
Aug 20 2016 22:46
Identity is a good one actually
then I can test that Sparse Matrix = Graph Matrix
lol that is funny
Sparse(I) x B = Dense(I) x B for any RHS
Jack Poulson
@poulson
Aug 20 2016 22:50
yes, it is probably a bad thing if that doesn't hold :-)
I would test a diagonal matrix though
at least
with an arbitrary diagonal
tridiagonal would be better yet
and 3D Laplace would be even better
Ryan H. Lewis
@rhl-
Aug 20 2016 22:51
once I get Identity working i'll add a more different sparse matrix
are you using a unit test framework? not seeming it in Gemm.cpp
Jack Poulson
@poulson
Aug 20 2016 22:52
no I am not
but CTest is used on top of the drivers
Ryan H. Lewis
@rhl-
Aug 20 2016 22:52
uh, no assertions?
Jack Poulson
@poulson
Aug 20 2016 22:53
exceptions are thrown if conditions are not met
Ryan H. Lewis
@rhl-
Aug 20 2016 22:53
I see.
how do you add/subtract two matrices?
nvm I am an idiot
Jack Poulson
@poulson
Aug 20 2016 23:04
:-p
Ryan H. Lewis
@rhl-
Aug 20 2016 23:04
I have two tests
one for identity
Jack Poulson
@poulson
Aug 20 2016 23:04
Axpy is the more general way
Ryan H. Lewis
@rhl-
Aug 20 2016 23:04
Oh, tahts what I did
Ryan H. Lewis
@rhl-
Aug 20 2016 23:09
how do I make it build the tests again?
EL_TESTS=On?
Jack Poulson
@poulson
Aug 20 2016 23:10
yes
-D EL_TESTS=ON
Ryan H. Lewis
@rhl-
Aug 20 2016 23:10
is Gemm.cpp a test or an example?
Jack Poulson
@poulson
Aug 20 2016 23:11
it's a test but there isn't yet a failure condition on the measured error
Ryan H. Lewis
@rhl-
Aug 20 2016 23:22
/Users/rhl/code/Elemental/include/El/macros/GuardAndPayload.h:19:136: warning: self-comparison always evaluates to false [-Wtautological-compare]
else if( A.ColDist() == MC && A.RowDist() == STAR ) { auto& ACast = static_cast<const DistMatrix<T,MC,STAR>&>(A); if( VR != MC || STAR != STAR || reinterpret_cast<const DistMatrix<T,VR,STAR>>(&A) != this ) this = ACast; else LogicError("Tried to construct DistMatrix with itself"); }
have you seen this?
Jack Poulson
@poulson
Aug 20 2016 23:24
have you opened up GuardAndPayload.h?
this is a prime example of an overzealous compiler warning
Ryan H. Lewis
@rhl-
Aug 20 2016 23:24
nope
Jack Poulson
@poulson
Aug 20 2016 23:25
those comparisons appear after macro expansion
I'm effectively implementing multiple dispatch via macros
Ryan H. Lewis
@rhl-
Aug 20 2016 23:26
you appear to do it in a way which generates self comparisons..
presumably from lines like this: ELSEIF_GUARD_AND_PAYLOAD(STAR,STAR) ?
Jack Poulson
@poulson
Aug 20 2016 23:27
it likely occurs within a template function
Ryan H. Lewis
@rhl-
Aug 20 2016 23:27
sure
Jack Poulson
@poulson
Aug 20 2016 23:27
one could manually implement each combination, but that is crazy
it is just the compiler being overzealous
Ryan H. Lewis
@rhl-
Aug 20 2016 23:27
I mean, what is weird is it could just remove the comparison
if ( false || SOMETHING) == if( SOMETHING)
Jack Poulson
@poulson
Aug 20 2016 23:28
in that particular case, yes
consider:
template<typename T,Dist colDistA,Dist rowDistB,Dist colDistB,Dist rowDistB>
void Foo( const DistMatrix<T,colDistA,rowDistA>& A, const DistMatrix<T,colDistB,rowDistB>& B ) {
  // Do something
}
the distributions of A and B are known at compile time, but that doesn't mean that one should write custom code for all 13*13 valid combinations of the four distribution parameters
Ryan H. Lewis
@rhl-
Aug 20 2016 23:29
not sure what that has to do with the warnings.
but, I agree with you.
Jack Poulson
@poulson
Aug 20 2016 23:30
insert a GuardAndPayload into that function and its what you get
check out the Copy implementations in blas_like/level1/
they are essentially the above scenario
Ryan H. Lewis
@rhl-
Aug 20 2016 23:30
I see
Jack Poulson
@poulson
Aug 20 2016 23:31
if you want to branch based upon whether or not colDistA == colDistB, etc., then you get those warnings
Ryan H. Lewis
@rhl-
Aug 20 2016 23:31
right right
I see
but wait
I don't completely follow
well, maybe you need C++17 for this
but
yeah, I guess in C++17 you can say:
Jack Poulson
@poulson
Aug 20 2016 23:32
why?
Ryan H. Lewis
@rhl-
Aug 20 2016 23:32
if constexpr (colDistA == colDistB) { do something; }
I wonder how much smaller we could make elemental if we tried.
Jack Poulson
@poulson
Aug 20 2016 23:33
I continually try to whiddle it down
but any extra help is appreciated
Ryan H. Lewis
@rhl-
Aug 20 2016 23:34
yeah, well, as soon as I merge multiply sparse matrices
we can look at something more complex
I would like to see the matrix clases settled
Jack Poulson
@poulson
Aug 20 2016 23:40
suggestions and PR's are always welcome
Ryan H. Lewis
@rhl-
Aug 20 2016 23:40
@poulson yeah, I mean, i'm interested in collaborating on that, but, I want to clear my plate first.
Jack Poulson
@poulson
Aug 20 2016 23:40
templated functions cannot involve implicit conversions
and 1.0 is a double
you want to use T(1) instead
Ryan H. Lewis
@rhl-
Aug 20 2016 23:41
wtf. I modify the test file and it makes me rebuild all of El
unless that is what git commit did
Jack Poulson
@poulson
Aug 20 2016 23:42
yes
Ryan H. Lewis
@rhl-
Aug 20 2016 23:42
wtf git commit?
Jack Poulson
@poulson
Aug 20 2016 23:42
I would be okay with changing the way the Git revision is incorporated
Ryan H. Lewis
@rhl-
Aug 20 2016 23:42
wait what
Jack Poulson
@poulson
Aug 20 2016 23:42
the build system polls the Git revision
Ryan H. Lewis
@rhl-
Aug 20 2016 23:42
ugh.
Jack Poulson
@poulson
Aug 20 2016 23:42
and writes it into a config file
Ryan H. Lewis
@rhl-
Aug 20 2016 23:42
that should be off by default.
Jack Poulson
@poulson
Aug 20 2016 23:42
it would perhaps make more sense to write it into a text file
Ryan H. Lewis
@rhl-
Aug 20 2016 23:47
how do you make a matrix have size m x n
Jack Poulson
@poulson
Aug 20 2016 23:49
A.Resize( m, n )
or Zeros( A, m, n )
depending upon whether you want to force zero initialization
Ryan H. Lewis
@rhl-
Aug 20 2016 23:53
Ok, so, I have my test harness working for identity matrix
I'm not sure how to add more than binary matrices in the unit tests without more effort
do you have a random binary matrix generator?
want to review my PR or do you want me to add more tests?