These are chat archives for elemental/chat

19th
Oct 2016
Jack Poulson
@poulson
Oct 19 2016 14:53
@rhl- I can't seem to remember why I disabled it; it would be a good idea to re-enable it with the appropriate flags (e.g., -g for GNU)
Ryan H. Lewis
@rhl-
Oct 19 2016 16:04
@poulson I see. I'm not sure what you mean. I see that you are setting compilation flags using ${LANG}_FLAGS. I'm quite sure this is un-necessary. CMake does this for you.
I am noticing that somehow this morning my El build is broken due to those changes in CMakeLists.txt
Ryan H. Lewis
@rhl-
Oct 19 2016 16:10
It looks like various parts of the library are using C++14 features somehow.
/Users/rlewis/code/Elemental/include/El/core/types.hpp:227:35: warning: variable templates are a C++14 extension [-Wc++14-extensions]
template<Dist U,Dist V> constexpr Dist DiagCol() { return ( U==STAR ? V : U ); }
I tried doing add_definitions(-std=c++14) and this broke things horridly.
Jack Poulson
@poulson
Oct 19 2016 19:40
Is this the Intel compiler?
Ryan H. Lewis
@rhl-
Oct 19 2016 21:37
No it's clang. It seems to disappear as soon as you set CMAKE_BUILD_TYPE to either Debug or Release.
I'm not sure why.
Jeff Hammond
@jeffhammond
Oct 19 2016 22:29
i'd love any help anyone can give me w.r.t. elemental/Elemental#185. CMake is completely unhelpful.
Ryan H. Lewis
@rhl-
Oct 19 2016 22:29
@jeffhammond reading it now ..
@jeffhammond: i am hoping for us to move up to C++14.. haha
you need to look at the internals of: cmake/detect/MPI.cmake
Jeff Hammond
@jeffhammond
Oct 19 2016 22:31
look at http://en.cppreference.com/w/cpp/compiler_support. three compilers support C++14 right now and one of them requires Windows
Ryan H. Lewis
@rhl-
Oct 19 2016 22:32
aren't the other two gcc and clang?
Jeff Hammond
@jeffhammond
Oct 19 2016 22:32
yes, they are, but moving to C++14 right now would be a clear statement that Elemental is not interested in supporting any HPC vendor compiler
Ryan H. Lewis
@rhl-
Oct 19 2016 22:33
@jeffhammond sure. I agree we wouldn't want that..
Jeff Hammond
@jeffhammond
Oct 19 2016 22:33
and there is at least one downstream ISV that must use Intel compilers.
Ryan H. Lewis
@rhl-
Oct 19 2016 22:33
@jeffhammond can I replicated without a cray compiler?
Jeff Hammond
@jeffhammond
Oct 19 2016 22:34
you might be able to replicate without the Cray compiler itself, but i think this is specific to the Cray programming environment wrappers
i can get you a NERSC account in a day if you want to fight with this, but i'd prefer that you invent a time machine and kill whatever person created CMake ;-)
Ryan H. Lewis
@rhl-
Oct 19 2016 22:35
haha. I used to have a NERSC account. It might still be active..
Jeff Hammond
@jeffhammond
Oct 19 2016 22:35
so there is no CMake log for the mpi.h faiure
it's ridiculous that it tells me to look in C++ detection output, which is unrelated
i set -DMPI_C_INCLUDE_PATH=$CRAY_MPICH_DIR/include
is that not correct?
Ryan H. Lewis
@rhl-
Oct 19 2016 22:36
I'm not 100% sure, but, that doesn't seem right.
Ah, I bet its failing right here: find_package(MPI)
Jeff Hammond
@jeffhammond
Oct 19 2016 22:37
yes it tells me CMake Error at cmake/detect/MPI.cmake:69 (message):
Could not find mpi.h
Call Stack (most recent call first):
CMakeLists.txt:361 (include)
and then "See also "/global/homes/j/jhammond/ELEMENTAL/git/build-cray/CMakeFiles/CMakeOutput.log"."
can you add message ("MPI_FOUND=${MPI_FOUND}) just below find_package or something
Jeff Hammond
@jeffhammond
Oct 19 2016 22:38
not until just now but you can see on the ticket what options i set
Ryan H. Lewis
@rhl-
Oct 19 2016 22:41
@jeffhammond according to the docs you need to set MPI_C_LIBRARIES to the list of things to link against.
However, this is likely unrelated to the actual failure.
Jeff Hammond
@jeffhammond
Oct 19 2016 22:41
cc is equivalent to mpicc on other platforms
no lib or inc options should be required ever
Ryan H. Lewis
@rhl-
Oct 19 2016 22:41
well, the documentation says the variable needs to be set
it doesnt say to what.
Jeff Hammond
@jeffhammond
Oct 19 2016 22:42
i have never before needed to set it when i set the MPI compilers
-DMPI_CXX_COMPILER=CC -DMPI_C_COMPILER=cc -DMPI_Fortran_COMPILER=ftn
that should do it
does everywhere else
Ryan H. Lewis
@rhl-
Oct 19 2016 22:42
try removing the explicit setting of the include path
Jeff Hammond
@jeffhammond
Oct 19 2016 22:42
honestly, at this point, i just want CMake to tell me what the error is
yeah that's where i started and it failed
Ryan H. Lewis
@rhl-
Oct 19 2016 22:43
this is the piece of CMake that is giving you a problem:
foreach(MPI_PATH ${MPI_C_INCLUDE_PATH})
if(EXISTS "${MPI_PATH}/mpi.h")
set(MPI_HEADER_PATH ${MPI_PATH})
endif()
endforeach()
Jeff Hammond
@jeffhammond
Oct 19 2016 22:44
but i set MPI_C_INCLUDE_PATH such that $MPI_C_INCLUDE_PATH/mpi.h was correct
jhammond@edison12:~> ll $CRAY_MPICH_DIR/include/mpi.h
-rwxr-xr-x 1 root root 127K Jun 27 10:30 /opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/include/mpi.h
Ryan H. Lewis
@rhl-
Oct 19 2016 22:49
can you just print ${MPI_PATH}/mpi.h and see what it is?
Jeff Hammond
@jeffhammond
Oct 19 2016 22:55
i added the appropriate command at line 69 but it had no effect. i'm trying other things now.
Jeff Hammond
@jeffhammond
Oct 19 2016 23:05
what's unfortunate is that the mpi.h existence check is to accommodate OpenMPI's bugs.
Ryan H. Lewis
@rhl-
Oct 19 2016 23:05
what do you mean?
what was ${MPI_PATH} set to?
Jeff Hammond
@jeffhammond
Oct 19 2016 23:05
look at cmake/detect/MPI.cmake
Ryan H. Lewis
@rhl-
Oct 19 2016 23:05
I am..
Jeff Hammond
@jeffhammond
Oct 19 2016 23:06
message(STATUS "JEFF: ${MPI_C_INCLUDE_PATH}")
message(STATUS "JEFF: ${MPI_PATH}")
message(STATUS "JEFF: ${MPI_HEADER_PATH}")
gave me
-- JEFF:
-- JEFF:
-- JEFF: MPI_HEADER_PATH-NOTFOUND
Ryan H. Lewis
@rhl-
Oct 19 2016 23:06
instead put: message(STATUS "JEFF: ${MPI_PATH}") on line 62
Jeff Hammond
@jeffhammond
Oct 19 2016 23:08
but you can see that lines 46-90 are specifically for OpenMPI
Cray MPI (MPICH-based) should not need that
would be better to test if OMPI used and only then look for mpi.h
so my next attempt is to just disable all of that code
Ryan H. Lewis
@rhl-
Oct 19 2016 23:11
sure, I was just suggesting to see what that is
might suggest what your problem is.
Jeff Hammond
@jeffhammond
Oct 19 2016 23:15
yeah i will debug later once everything else works
Jeff Hammond
@jeffhammond
Oct 19 2016 23:43
i got the build working just fine but then i hit Cray compiler ICE. FML
Jeff Hammond
@jeffhammond
Oct 19 2016 23:58
back to MPI debugging:
i made it print this out:
-- Jeff: MPI_C_INCLUDE_PATH=
-- Jeff: MPI_HEADER_PATH=MPI_HEADER_PATH-NOTFOUND
this is how i cmaked:
jhammond@edison12:~/ELEMENTAL/git/build-cray> rm -rf * && cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/ELEMENTAL/install-cray -DBUILD_SHARED_LIBS=OFF -DCMAKE_CXX_COMPILER_ID=Cray -DCMAKE_C_COMPILER_ID=Cray -DCMAKE_Fortran_COMPILER_ID=Cray -DCMAKE_CXX_COMPILER=CC -DCMAKE_C_COMPILER=cc -DCMAKE_Fortran_COMPILER=ftn -DMPI_CXX_COMPILER=CC -DMPI_C_COMPILER=cc -DMPI_Fortran_COMPILER=ftn -DMPI_C_INCLUDE_PATH=$CRAY_MPICH_DIR/include -DMPI_C_COMPILE_FLAGS="" -DMPI_C_LINK_FLAGS="" -DMPI_C_LIBRARIES="" -DCMAKE_CXX_FLAGS="-h std=c++11 -h omp " -DCMAKE_C_FLAGS="-h std=c11 -h omp " -DMATH_LIBS="-L${CRAY_LIBSCI_PREFIX_DIR}/lib -lsci_cray_mpi_mp -lsci_cray_mp " -DEL_HYBRID=TRUE -DEL_DISABLE_PARMETIS=TRUE -DCMAKE_BUILD_TYPE=Debug -DEL_USE_64BIT_INTS=ON -DEL_USE_64BIT_BLAS_INTS=OFF -DINSTALL_PYTHON_PACKAGE=OFF -DEL_DISABLE_QUAD=TRUE
please tell me why -DMPI_C_INCLUDE_PATH=$CRAY_MPICH_DIR/include is not effective