These are chat archives for bluescarni/pagmo_reborn

8th
Jun 2016
Dario Izzo
@darioizzo
Jun 08 2016 07:05
@bluescarni daje!!!! So ready to merge into master?
@CoolRunning I think, then, you cannot do it with max_element as you indeed would have to iterate along the "wrong" direction .....
Marcus Märtens
@CoolRunning
Jun 08 2016 07:18
The only solution I can think of is like weird iterator arithmetrics, and I would rather avoid that.
Dario Izzo
@darioizzo
Jun 08 2016 07:18
no no for God health
the point is to simplify not make it unreadable .... I just though a one liner could be used, but I think I was wrong
Marcus Märtens
@CoolRunning
Jun 08 2016 07:25
Okay ^^
Francesco Biscani
@bluescarni
Jun 08 2016 07:49
maybe I should have some bits of testing implemented before the merge? I mean, we can merge now but then I am immediately going to open a python_testing or something like that branch :)
Dario Izzo
@darioizzo
Jun 08 2016 08:37
@bluescarni I still need a few days to finish cmaes, but then I would like to be able to do also the python part ..... otherwise I keep adding stuff and you have to keep doing expositions :)
Francesco Biscani
@bluescarni
Jun 08 2016 08:38
yeah I figured.. pygmo is becoming a sizable portion of the whole
so I have a stub of a testing module. I'll try to make the CI build the bindings as well and then I'll merge
Dario Izzo
@darioizzo
Jun 08 2016 08:41
yay
Francesco Biscani
@bluescarni
Jun 08 2016 08:44
mhm what is the alpine image we are using
I need to try locally
Dario Izzo
@darioizzo
Jun 08 2016 08:45
I think you can get the docker image of pagmo directly no?
Francesco Biscani
@bluescarni
Jun 08 2016 08:45
yes what is the name?
pagmo/ci-alpine:v1
Dario Izzo
@darioizzo
Jun 08 2016 08:46
pagmo/ci-alpine:v1
yes
Francesco Biscani
@bluescarni
Jun 08 2016 08:47
did you inherit from a docker image with python 3?
Dario Izzo
@darioizzo
Jun 08 2016 08:48
I do not remember, I tried many things but I do not remember what was the last one I used
I think so though ...
Francesco Biscani
@bluescarni
Jun 08 2016 08:48
because python 2 support I still need to do
Dario Izzo
@darioizzo
Jun 08 2016 08:48
there is either no python or python 3
Francesco Biscani
@bluescarni
Jun 08 2016 08:49
right
we need to build the bindings as part of the pages as well if we want sphinx to work properly
Dario Izzo
@darioizzo
Jun 08 2016 08:49
of course ...
Can you help to understand how they do the Dynamic stuff?
Francesco Biscani
@bluescarni
Jun 08 2016 08:50
what do you mean?
Dario Izzo
@darioizzo
Jun 08 2016 08:50
In the template they have int, but if you pass the special value dynamic the matrix becames run-time allocated
typedef Matrix<double, Dynamic, Dynamic> MatrixXd;
This is an isntantiation of a dynamically allocated matrix. they obtain it passing Dynamic as a template argument to the class Matrix<type, int int>
Francesco Biscani
@bluescarni
Jun 08 2016 08:51
might be that Dynamic is just an alias for an int special value?
const int Dynamic = -1;
Dario Izzo
@darioizzo
Jun 08 2016 08:52
That is what I thought, but then serialization should work , instead it doeas not if Dynamic
is passed
Francesco Biscani
@bluescarni
Jun 08 2016 08:53
how does it fail?
Dario Izzo
@darioizzo
Jun 08 2016 08:53
not finding load / save
Francesco Biscani
@bluescarni
Jun 08 2016 08:53
can you paste the code and the error
Dario Izzo
@darioizzo
Jun 08 2016 08:53
    template <class Archive, class S, int R, int C> inline
    void CEREAL_SAVE_FUNCTION_NAME(Archive &ar, Eigen::Matrix<S,R,C> const &cb)
    {
        // Let's first save the dimension
        auto nrows = cb.rows();
        auto ncols = cb.cols();
        ar << nrows;
        ar << ncols;
        //And then the numbers
        for (decltype(nrows) i = 0; i < nrows; ++i) {
            for (decltype(nrows) j = 0; j < ncols; ++j) {
                ar << cb(i,j);
            }
        }
    }
    template <class Archive, class S, int R, int C> inline
    void CEREAL_LOAD_FUNCTION_NAME(Archive &ar, Eigen::Matrix<S,R,C> &cb)
    {
        decltype(cb.rows()) nrows;
        decltype(cb.cols()) ncols;
        // Let's first restore the dimension
        ar >> nrows;
        ar >> ncols;
        cb.resize(nrows,ncols);
        //And then the numbers
        for (decltype(nrows) i = 0; i < nrows; ++i) {
            for (decltype(nrows) j = 0; j < ncols; ++j) {
                ar >> cb(i,j);
            }
        }
    }
thats the implementation of the serialization
now the code:
    auto b = Eigen::VectorXd::Zero(10);
    serialize_deserialize_compare(a);
with:
template <typename T>
void serialize_deserialize_compare(T &object)
{
    auto object_copy(object);
    std::stringstream ss;
    {
    cereal::JSONOutputArchive oarchive(ss);
    oarchive(object_copy);
    }
    // Change the content of p before deserializing.
    object_copy = T{};
    {
    cereal::JSONInputArchive iarchive(ss);
    iarchive(object_copy);
    }
    auto nrows = object.rows();
    auto ncols = object.cols();
    BOOST_CHECK_EQUAL(nrows, object_copy.rows());
    BOOST_CHECK_EQUAL(ncols, object_copy.cols());
    //And then we check
    for (decltype(nrows) i = 0; i < nrows; ++i) {
        for (decltype(nrows) j = 0; j < ncols; ++j) {
            BOOST_CHECK_EQUAL(object_copy(i,j), object(i,j));
        }
    }
}
and the error:
/home/dario/Documents/PaGMOreborn/tests/../include/external/cereal/archives/../cereal.hpp:448:9: error: static assertion failed: cereal could not find any output serialization functions for the provided type and archive combination. 

 Types must either have a serialize function, load/save pair, or load_minimal/save_minimal pair (you may not mix these). 
 Serialize functions generally have the following signature: 

 template<class Archive> 
   void serialize(Archive & ar) 
   { 
     ar( member1, member2, member3 ); 
   } 


         static_assert(traits::detail::count_output_serializers<T, ArchiveType>::value != 0,
         ^~~~~~~~~~~~~
But I did not want you to go through all of this, I just needed to know if they could have done some special template trick there with Dynamic, something I was not aware existed in the esoteric c++14...
If not I can exclude a number of things and look in the correct direction to debug
Francesco Biscani
@bluescarni
Jun 08 2016 08:58
try std::cout << Dynamic << '\n'; :)
Dario Izzo
@darioizzo
Jun 08 2016 08:58
good suggestion ...
Francesco Biscani
@bluescarni
Jun 08 2016 08:58
or std::cout << std::is_integral<decltype(Dynamic)>::value << '\n';
or std::cout << typeid(Dynamic).string() << '\n';
but reading the source code of Eigen it does look like it's an int
can you get more context on that error
like the full lof
log
Dario Izzo
@darioizzo
Jun 08 2016 08:59
do you really want to know>
?
Francesco Biscani
@bluescarni
Jun 08 2016 09:00
sure
Dario Izzo
@darioizzo
Jun 08 2016 09:01
std::cout << Eigen::Dynamic << '\n'; -> -1
full log:
Scanning dependencies of target eigen3_serialization
[ 50%] Building CXX object tests/CMakeFiles/eigen3_serialization.dir/eigen3_serialization.cpp.o
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp: In member function ‘void matrix_serialization_test::test_method()’:
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp:55:10: warning: variable ‘a’ set but not used [-Wunused-but-set-variable]
     auto a = Eigen::MatrixXd::Identity(8, 8);
          ^
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp: In instantiation of ‘void serialize_deserialize_compare(T&) [with T = Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >]’:
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp:58:36:   required from here
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp:22:17: error: no matching function for call to ‘Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >::CwiseNullaryOp(<brace-enclosed initializer list>)’
     object_copy = T{};
     ~~~~~~~~~~~~^~~~~
In file included from /usr/include/eigen3/Eigen/Core:298:0,
                 from /usr/include/eigen3/Eigen/Dense:1,
                 from /home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp:8:
/usr/include/eigen3/Eigen/src/Core/CwiseNullaryOp.h:57:5: note: candidate: Eigen::CwiseNullaryOp<NullaryOp, MatrixType>::CwiseNullaryOp(Eigen::CwiseNullaryOp<NullaryOp, MatrixType>::Index, Eigen::CwiseNullaryOp<NullaryOp, MatrixType>::Index, const NullaryOp&) [with NullaryOp = Eigen::internal::scalar_constant_op<double>; PlainObjectType = Eigen::Matrix<double, -1, 1>; Eigen::CwiseNullaryOp<NullaryOp, MatrixType>::Index = long int]
     CwiseNullaryOp(Index nbRows, Index nbCols, const NullaryOp& func = NullaryOp())
     ^~~~~~~~~~~~~~
/usr/include/eigen3/Eigen/src/Core/CwiseNullaryOp.h:57:5: note:   candidate expects 3 arguments, 0 provided
/usr/include/eigen3/Eigen/src/Core/CwiseNullaryOp.h:49:7: note: candidate: Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >::CwiseNullaryOp(const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >&)
 class CwiseNullaryOp : internal::no_assignment_operator,
       ^~~~~~~~~~~~~~
/usr/include/eigen3/Eigen/src/Core/CwiseNullaryOp.h:49:7: note:   candidate expects 1 argument, 0 provided
/usr/include/eigen3/Eigen/src/Core/CwiseNullaryOp.h:49:7: note: candidate: Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >::CwiseNullaryOp(Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >&&)
/usr/include/eigen3/Eigen/src/Core/CwiseNullaryOp.h:49:7: note:   candidate expects 1 argument, 0 provided
In file included from /home/dario/Documents/PaGMOreborn/tests/../include/external/cereal/archives/binary.hpp:32:0,
                 from /home/dario/Documents/PaGMOreborn/tests/../include/serialization.hpp:11,
                 from /home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp:10:
/home/dario/Documents/PaGMOreborn/tests/../include/external/cereal/archives/../cereal.hpp: In instantiation of ‘ArchiveType& cereal::OutputArchive<ArchiveType, Flags>::processImpl(const T&) [with T = Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >; typename cereal::traits::detail::EnableIfHelper<(cereal::traits::has_invalid_output_versioning<T, ArchiveType>::value || ((! cereal::traits::is_output_serializable<T, ArchiveType>::value) && ((!(Flags & AllowEmptyClassElision)) || ((Flags & AllowEmptyClassElision) && (! std::is_empty<T>::value)))))>::type <anonymous> = (cereal::traits::detail::type)0; ArchiveType = cereal::JSONOutputArchive; unsigned int Flags = 0u]’:
/home/dario/Documents/PaGMOreborn/tests/../include/external/cereal/archives/../cereal.hpp:333:9:   required from ‘void cereal::OutputArchive<ArchiveType, Flags>::process(T&&) [with
he is trimming my paste :(
Francesco Biscani
@bluescarni
Jun 08 2016 09:03
so that's the first error:
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp: In instantiation of ‘void serialize_deserialize_compare(T&) [with T = Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >]’:
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp:58:36:   required from here
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp:22:17: error: no matching function for call to ‘Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >::CwiseNullaryOp(<brace-enclosed initializer list>)’
     object_copy = T{};
     ~~~~~~~~~~~~^~~~~
the rest is garbage until this is fixed
Dario Izzo
@darioizzo
Jun 08 2016 09:04
infact the code works for other cases where Dynamic is not used
I noticed, but how can there be no default constructor if there is for matrix<class, int , int> ?
Francesco Biscani
@bluescarni
Jun 08 2016 09:04
because that's not a matrix<class,int,int>
Dario Izzo
@darioizzo
Jun 08 2016 09:05
what is not?
Francesco Biscani
@bluescarni
Jun 08 2016 09:05
/home/dario/Documents/PaGMOreborn/tests/eigen3_serialization.cpp:22:17: error: no matching function for call to ‘Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, -1, 1> >::CwiseNullaryOp(<brace-enclosed initializer list>)’
look at the type
Eigen::CwiseNullaryOp
that's no matrix
Dario Izzo
@darioizzo
Jun 08 2016 09:06
???????????????
    auto b = Eigen::VectorXd::Zero(10);
    //serialize_deserialize_compare(a);
    serialize_deserialize_compare(b);
b is auto from Eigen::VectorXd::Zero(10)
Francesco Biscani
@bluescarni
Jun 08 2016 09:06
yeps
which is not a matrix :)
Dario Izzo
@darioizzo
Jun 08 2016 09:06
that is passed to the templated function so T is whatever auto is right?
what is it then?
Francesco Biscani
@bluescarni
Jun 08 2016 09:07
ready for the spiegone
Dario Izzo
@darioizzo
Jun 08 2016 09:07
if I must :)
go
Francesco Biscani
@bluescarni
Jun 08 2016 09:08
Eigen uses a technique called expression templates
when you add two matrices, for instance, the result is not a matrix object but an unnamed type which is implicitly convertible to a matrix
so if you write in C++11 auto a = matrix1 + matrix2;, a is not a matrix
nice eh?
Dario Izzo
@darioizzo
Jun 08 2016 09:09
lol ... I suspecetd
Francesco Biscani
@bluescarni
Jun 08 2016 09:09
I think the Zero() function also returns an expression rather than a matrix
Dario Izzo
@darioizzo
Jun 08 2016 09:09
so I can "just" change auto to the correct type and should work ....
Francesco Biscani
@bluescarni
Jun 08 2016 09:10
this expression template stuff does not play well with templates and auto, as you discovered
yes
Dario Izzo
@darioizzo
Jun 08 2016 09:10
what is exactly an expression template?
Francesco Biscani
@bluescarni
Jun 08 2016 09:11
it's an expression tree built at compile time... it's a compile-time analogous to the data structures you use in computer algebra system to represent arbitrary mathematical expressions
Dario Izzo
@darioizzo
Jun 08 2016 09:12
woah ... cannot even parse your answer
BoooooMMMM
nvm
Francesco Biscani
@bluescarni
Jun 08 2016 09:12
so something like a + b * c becomes something like Add<a,Mul<b,c>>
Dario Izzo
@darioizzo
Jun 08 2016 09:12
ah ok that clearer
Francesco Biscani
@bluescarni
Jun 08 2016 09:12
you build a complicated template that represents the structure of your expression
Dario Izzo
@darioizzo
Jun 08 2016 09:12
do you use it in Piranha?
Francesco Biscani
@bluescarni
Jun 08 2016 09:13
which allows you to do intelligent stuff like looping only once instead of doing three loops (if a,b,c are vectors for instance)
no it would be useless there
and it destroys the type system, as you have seen
Dario Izzo
@darioizzo
Jun 08 2016 09:13
cool stuff though .... I wish I could understand this as ddeply as you do .... !
Francesco Biscani
@bluescarni
Jun 08 2016 09:14
I don't understand it really, as I have never implemented it :) I just know what it is
but I hate it
Marcus Märtens
@CoolRunning
Jun 08 2016 09:14
I read about it when I was studying how Sympy works
Francesco Biscani
@bluescarni
Jun 08 2016 09:14
it's the type of stuff that C++ coders really love to put everywhere to show how cool the language is
Marcus Märtens
@CoolRunning
Jun 08 2016 09:15
Those expression trees
Francesco Biscani
@bluescarni
Jun 08 2016 09:15
yes it's a compile-time analogue of a generic expression tree
Marcus Märtens
@CoolRunning
Jun 08 2016 09:15
But I also did not like it
Francesco Biscani
@bluescarni
Jun 08 2016 09:15
you could write something liek SymPy at compile time in C++, if you were crazy enough
Marcus Märtens
@CoolRunning
Jun 08 2016 09:16
That sounds crazy
Francesco Biscani
@bluescarni
Jun 08 2016 09:17
it is... but you could do automatic differentiation at compile time for instance :)
Marcus Märtens
@CoolRunning
Jun 08 2016 09:17
Sounds like compiler-abuse
Moving all the computation into compilation ^^
Francesco Biscani
@bluescarni
Jun 08 2016 09:18
it would be nice for something like automatic differentiation (which would be really symbolic differentiation done at compile time rather than "automatic differentiation" in the proper sense)
because that's exactly the type of tedious stuff you don't want to do by hand
Marcus Märtens
@CoolRunning
Jun 08 2016 09:19
True
But then you hope that the compilers are doing it correct
Francesco Biscani
@bluescarni
Jun 08 2016 09:19
well you'd have to debug it clearly :)
Marcus Märtens
@CoolRunning
Jun 08 2016 09:19
^^
Francesco Biscani
@bluescarni
Jun 08 2016 09:20
but all this stuff does not play well with modern C++
because of auto, templates, etc.
but C++ is the only language that allows you to do this
there was a proposal to augment the auto keyword so that it becomes an overloadable operator
so you can decide what your class converts-to when used with auto
but I don't know about the status of this proposal
Marcus Märtens
@CoolRunning
Jun 08 2016 09:28
Everytime I hear about these peculiarities of C++, I think the interview with Stroustrup wasn't a joke at all.
Francesco Biscani
@bluescarni
Jun 08 2016 09:29
meh... as they say, there are languages which everyone complains about and languages that nobody uses :)
it's a large complex beast, but you don't really have to know about all this shit
Dario Izzo
@darioizzo
Jun 08 2016 09:29

If I activate in CMake the option PAGMO_ENABLE_EIGEN3 then in the code I can

  #ifdef PAGMO_ENABLE_EIGEN3
  #include <Eigen/Dense>
  #endif

right?

wtf
Francesco Biscani
@bluescarni
Jun 08 2016 09:30
it was like watching a train wreck happening in real time :D
Dario Izzo
@darioizzo
Jun 08 2016 09:30
ang going on
Francesco Biscani
@bluescarni
Jun 08 2016 09:30
:)
yes you can do that, but as I said on the comment these defines should go into a config header eventually
Dario Izzo
@darioizzo
Jun 08 2016 09:31
but its ignored ... :(
cannot make it work
Francesco Biscani
@bluescarni
Jun 08 2016 09:32
ah it's because you did not add the definition to the compiler command line
you just enabled the cmake option
Dario Izzo
@darioizzo
Jun 08 2016 09:32
so its not added automagically .... I need to add it manually in CMake?
Is that how we want to do it?
Francesco Biscani
@bluescarni
Jun 08 2016 09:33
as I said, it should be in a config.hpp header
Dario Izzo
@darioizzo
Jun 08 2016 09:33
I do not really get your idea, for the time being can I add an ADD_DEFINITION manally ?
Francesco Biscani
@bluescarni
Jun 08 2016 09:34
the idea is that if you add the ADD_DEFINITION manually, then evey pagmo user has to remember to add that to the compile options when using pagmo
which is not what we want
but ok for the time being
if you create at configure time a header called config.hpp which contains all the defines #define PAGMO_USE_EIGEN3 etc. then the user has to do nothing, just include the pagmo headers as usual
https://github.com/bluescarni/piranha/blob/master/src/config.hpp.in it's something like this.. then the variables inside the @@@@ get replaced at configure time by the corresponding CMake variables
then whenever you need to use eigen stuff, you do:
#include <pagmo/config.hpp>

...

#if defined(PAGMO_ENABLE_EIGEN3)

#endif
Francesco Biscani
@bluescarni
Jun 08 2016 09:39
so it's a bit like storing pieces of the CMake configuration in a file, if you want
but do the ADD_DEFINITION stuff now, I'll do this bullshit later
Dario Izzo
@darioizzo
Jun 08 2016 09:40
:) thanks ....
works!!
Francesco Biscani
@bluescarni
Jun 08 2016 10:22
shitty shitty... so alpine does not have numpy in its default repos
it has it in the unstable repo
but only for python2...
Dario Izzo
@darioizzo
Jun 08 2016 10:23
shall we ubuntu?
Francesco Biscani
@bluescarni
Jun 08 2016 10:23
and if you try to install it via pip, it fails because alpine uses a special C library
which is not supported by numpy
Dario Izzo
@darioizzo
Jun 08 2016 10:23
lets ubuntu since the image is in electra its downloaded only once
the reason for alpine was to try using the gitlab CI runners, but since you set up nicely electra ....
lets go mainstream
Francesco Biscani
@bluescarni
Jun 08 2016 10:23
it's emma but ok
just to remind you :)
so ubuntu 16 I guess
so how do I upload it once I have set it up?
Dario Izzo
@darioizzo
Jun 08 2016 10:26
We have a docker user called pagmo I guess
Let me see if I manage to retrieve our credential
would have been smart to put it somewhere ....
which I did
but I forgot where
so it would have been smart to put somewhere the info on where I put the docker credential
and so on ....
Francesco Biscani
@bluescarni
Jun 08 2016 10:27
so much rust
Dario Izzo
@darioizzo
Jun 08 2016 10:28
:) inizia ad accumularsi
user: pagmo
pass: pagmo4docker
Francesco Biscani
@bluescarni
Jun 08 2016 10:29
it's gonna take forever to upload it from home I gess
oh well
Dario Izzo
@darioizzo
Jun 08 2016 10:30
once you set up everything in docker (i.e. you created your image) to upload it to our docker user follow:
its straight forward, except for how to name the container (like versions and stuff)
Francesco Biscani
@bluescarni
Jun 08 2016 10:31
it's hundreds of megs of stuff, maybe I should do it another time
clang, gcc, boost
che palle
maybe I can do it from the office compute
Dario Izzo
@darioizzo
Jun 08 2016 10:32
If you tell me exactly whats needed I could do it ....
Francesco Biscani
@bluescarni
Jun 08 2016 10:32
via ssh
Dario Izzo
@darioizzo
Jun 08 2016 10:32
or from emma
Francesco Biscani
@bluescarni
Jun 08 2016 10:32
might be faster from my workstation there
Dario Izzo
@darioizzo
Jun 08 2016 10:32
:)
Francesco Biscani
@bluescarni
Jun 08 2016 10:32
I don't recall esa network to be particularly fast
Dario Izzo
@darioizzo
Jun 08 2016 10:32
emma is super fast .... the problem is thats in the firewall
electra is not though ....
electra is lightning
Francesco Biscani
@bluescarni
Jun 08 2016 10:33
yeah but docker is not setup on electra, good luck with that
Dario Izzo
@darioizzo
Jun 08 2016 10:33
2 min to download GoT
I can do it tonight from home if you tell me what is needed exactly
Francesco Biscani
@bluescarni
Jun 08 2016 10:34
I am doing it from my work pc
Dario Izzo
@darioizzo
Jun 08 2016 10:34
k
good luck
Francesco Biscani
@bluescarni
Jun 08 2016 10:38
it's like 10x faster
between ssd and faster network
Dario Izzo
@darioizzo
Jun 08 2016 10:38
nice
Francesco Biscani
@bluescarni
Jun 08 2016 10:40
holy fuck it's like 40MB/s... must be a proxy
Dario Izzo
@darioizzo
Jun 08 2016 11:35
back from lunch ... :)
Francesco Biscani
@bluescarni
Jun 08 2016 11:50
Dario Izzo
@darioizzo
Jun 08 2016 11:51
yeah!
real time peep show
in the meantime:
    population preserve_elitism(population &pop, const vector_double &x_champ, const vector_double &f_champ, const vector_double::size_type idx_champ)
    {
        auto f_champ = pop.get_f()[idx_champ];
        if (f_champ[0] < f[0])
        {
            pop.set_xf(idx_champ, x_champ, f_champ);
        }
        return pop;
    }
is this making a copy of pop at return?
like if I write: return preserve_elitism(pop, x,f,idx);
am I making a copy of pop?
SUCCESS!!!
Francesco Biscani
@bluescarni
Jun 08 2016 11:53
yes you are making a copy
Dario Izzo
@darioizzo
Jun 08 2016 11:53
can I avoid to?
Francesco Biscani
@bluescarni
Jun 08 2016 11:53
sure you can return void :)
or return a reference
Dario Izzo
@darioizzo
Jun 08 2016 11:54
can I avoid to returning population&
ok
Francesco Biscani
@bluescarni
Jun 08 2016 11:54
so if you want a return value and don't want the copy it has to be population &
Dario Izzo
@darioizzo
Jun 08 2016 12:00
I think I will just docment that CMA-ES may return a population which is worse as its not elitist.
but hey the new docker works!!!
Congrats
nicely done
Francesco Biscani
@bluescarni
Jun 08 2016 12:01
it's failing now because I forgot to install g++
updating the image
https://gitlab.com/EuropeanSpaceAgency/PaGMOreborn/builds/1684016 now it's running... before it was using a mix of GCC and clang
k more stuff missing
Francesco Biscani
@bluescarni
Jun 08 2016 12:09
missing curl
Dario Izzo
@darioizzo
Jun 08 2016 12:10
why would you need curl ... ? thats wget right?
Francesco Biscani
@bluescarni
Jun 08 2016 12:10
it's the shit codecov script
this docker technology is pretty cool though
Dario Izzo
@darioizzo
Jun 08 2016 12:11
indeed
I also had some satisfaction
to do it with osx and windows would be great!
Francesco Biscani
@bluescarni
Jun 08 2016 12:12
what do you mean?
Dario Izzo
@darioizzo
Jun 08 2016 12:12
to have docker images with osx and windows and use them ...
Francesco Biscani
@bluescarni
Jun 08 2016 12:14
you need a virtual machine for that
Dario Izzo
@darioizzo
Jun 08 2016 12:15
that is the whole point of docker right?
why is this only 42% ?
Francesco Biscani
@bluescarni
Jun 08 2016 12:18
well you can see it yourself in the file below
Dario Izzo
@darioizzo
Jun 08 2016 12:18
I see nothing there, just the code
Francesco Biscani
@bluescarni
Jun 08 2016 12:18
the colors
Dario Izzo
@darioizzo
Jun 08 2016 12:18
all white
Screenshot from 2016-06-08 14-18-57.png
Francesco Biscani
@bluescarni
Jun 08 2016 12:19
I see nicely colored lines in the file
Dario Izzo
@darioizzo
Jun 08 2016 12:19
actually all green if I toggle the non hits, semi hits and hits
Francesco Biscani
@bluescarni
Jun 08 2016 12:20
maybe if you scroll down...
Dario Izzo
@darioizzo
Jun 08 2016 12:20
all green
Francesco Biscani
@bluescarni
Jun 08 2016 12:20
there's two functions all red her in the bottom
force_bounds_reflection and the other
Dario Izzo
@darioizzo
Jun 08 2016 12:20
all green there too
Screenshot from 2016-06-08 14-20-01.png
so its 100% !!
fuck this shit .. I go back to code
Francesco Biscani
@bluescarni
Jun 08 2016 12:25
I'll just disable it
Dario Izzo
@darioizzo
Jun 08 2016 12:25
no no , keep it
Dario Izzo
@darioizzo
Jun 08 2016 12:26
I can read the logs and fix some tests
Francesco Biscani
@bluescarni
Jun 08 2016 12:26
it's shit, completely unusable.. we should do manually but ok
Dario Izzo
@darioizzo
Jun 08 2016 12:27
I think I can use it (suboptimal) but I found already a few things I did not notice
as I seem to be unable to run the coverage with clang ... this is my only option ...
Francesco Biscani
@bluescarni
Jun 08 2016 12:27
why do you want to run it with clang? the tool (gcov) explicitly works only with GCC
and it needs to be a specific GCC version that matches the gcov version
Dario Izzo
@darioizzo
Jun 08 2016 12:28
can you teach me how to do it?
Francesco Biscani
@bluescarni
Jun 08 2016 12:28
compile with --coverage flag
run the tests
Dario Izzo
@darioizzo
Jun 08 2016 12:28
thats what I did .... but nothing happened ...
I will try again
Francesco Biscani
@bluescarni
Jun 08 2016 12:28
http://ltp.sourceforge.net/coverage/lcov.php then run the two commands here
lcov and genhtml
it has to be GCC though
we can just automate this, taking care of removing the coverage for stuff outside pagmo (STL) and the external deps (cereal), then we upload to generated html
it's not ideal but better than this shit
Dario Izzo
@darioizzo
Jun 08 2016 12:47
why, it looks pretty good
Screenshot from 2016-06-08 14-47-03.png
then I can interactively navigate ...
Francesco Biscani
@bluescarni
Jun 08 2016 12:49
it's ok but there's not automation or integration with the rest of the ci
on travis with codecov what happens is that if the code coverage decreases, then the CI fails globally
Dario Izzo
@darioizzo
Jun 08 2016 12:49
right .... lets keep the codecov for the time being... they will fix it eventually ....
Francesco Biscani
@bluescarni
Jun 08 2016 12:49
here there needs to be someone who keeps on checking the thing
Dario Izzo
@darioizzo
Jun 08 2016 12:49
and for the next month we can use this locally to complete the tests
Francesco Biscani
@bluescarni
Jun 08 2016 12:49
I would not hold my breath for that
Dario Izzo
@darioizzo
Jun 08 2016 12:50
no? come on there are abvious bugs ....
they must have noticed right?
So they either shut down the service or they fix it ...
Francesco Biscani
@bluescarni
Jun 08 2016 12:50
they are basically the only ones having some support for gitlab
it works ok from github I think
the gitlab integration is the problem
Dario Izzo
@darioizzo
Jun 08 2016 12:50
ah its connected with gitlab ... then maybe its no priority for them ....
you did not like the css stuff on sphinx? I see its reverted
Francesco Biscani
@bluescarni
Jun 08 2016 12:56
yes I think it's better not to touch until we have a precise idea
I'm changing the tests to use the header-only version of the boost unit test, let's see how it goes
ubuntu clang was complaining as usual about random bullshit
Francesco Biscani
@bluescarni
Jun 08 2016 13:01
alright it looks like the previous setup of the CI is fully restored and we can avoid now linking to the stupid boost library
probably should check appveyor as well sigh
ah maybe not necessary
Dario Izzo
@darioizzo
Jun 08 2016 13:03
any restriction with the use of the header only version?
Francesco Biscani
@bluescarni
Jun 08 2016 13:03
you cannot build tests which are made of more than 1 cpp, which we never do and have no reason for doing
Dario Izzo
@darioizzo
Jun 08 2016 13:03
if thats all .... its a great step forward !
Francesco Biscani
@bluescarni
Jun 08 2016 13:05
so shall I merge once appveyor is done?
Dario Izzo
@darioizzo
Jun 08 2016 13:05
yep go
Francesco Biscani
@bluescarni
Jun 08 2016 13:06
fuck it merging now
Dario Izzo
@darioizzo
Jun 08 2016 13:07
:)
how were the docs online if the merge was not done?
Francesco Biscani
@bluescarni
Jun 08 2016 13:07
fuck this shit
I enabled momentarily the upload from the branch
Dario Izzo
@darioizzo
Jun 08 2016 13:07
Do we get the docs of the latest build whatever branch?
ah ok...
Francesco Biscani
@bluescarni
Jun 08 2016 13:08
no only master normally
so now for the unit test headers, #include <boost/test/included/unit_test.hpp> this is the one to use
only change required
Dario Izzo
@darioizzo
Jun 08 2016 13:10
did you change it already to all?
Francesco Biscani
@bluescarni
Jun 08 2016 13:10
yes
Dario Izzo
@darioizzo
Jun 08 2016 13:10
so I just merge master now?
Francesco Biscani
@bluescarni
Jun 08 2016 13:10
yep
there's something fishy as well in the breathe/sphinx interaction in ubuntu
it gives some warnings that are not there on my system
I don't know if it is due to doxygen, sphinx or breathe
who knows
Dario Izzo
@darioizzo
Jun 08 2016 13:12
I do
its sphinx complaining that two signatures are equal. Did you forget to import the bootstrap correctly?
like leaf some commented lines ...
I had these errors yesterday when I was trying different emplates
Francesco Biscani
@bluescarni
Jun 08 2016 13:13
I don't see those warnings locally
and believe me I've been running sphinx many times in the last week
Dario Izzo
@darioizzo
Jun 08 2016 13:13
did you install the sphinx template remotely?
Francesco Biscani
@bluescarni
Jun 08 2016 13:13
(nor on my work pc)
Dario Izzo
@darioizzo
Jun 08 2016 13:13
in the new docker image
Francesco Biscani
@bluescarni
Jun 08 2016 13:13
I installed everything via ubuntu packages, they might be outdated
Dario Izzo
@darioizzo
Jun 08 2016 13:14
maybe a pip install sphinx-bootstrap
would solve it?
Francesco Biscani
@bluescarni
Jun 08 2016 13:14
maybe but I am out of steam for today, I am already ready to punch someone
I'll do it eventually
but not today
Dario Izzo
@darioizzo
Jun 08 2016 13:16
Access Denied lol ... profanity
Francesco Biscani
@bluescarni
Jun 08 2016 13:16
lol... must be because it's a russian website
as if he did resolve the overloads correctly
Francesco Biscani
@bluescarni
Jun 08 2016 13:17
std::pair:vector_double.vector_double:CR.detail::random_engine_typeR it seems the problem is with the RNG types
are they even documented? I can't see them in the docs
maybe they are not explicitly referenced in the sphinx side
Dario Izzo
@darioizzo
Jun 08 2016 13:18
boh, fuck it
Francesco Biscani
@bluescarni
Jun 08 2016 13:19
it's all very brittle, but what can we do
doxygen gives new and creatively different warnings each time you update it
it's all shit
Dario Izzo
@darioizzo
Jun 08 2016 13:19
its a one time setup in theory ... we will eventually converge
Francesco Biscani
@bluescarni
Jun 08 2016 13:19
doxygen does not steadily improves unfortunately, it often regresses
Dario Izzo
@darioizzo
Jun 08 2016 13:20
Nah, its just that we are getting 'tired' .... its a marathon and we are only approaching half way
doxygen is like latex to me
I just wonder why nobody is redoing it from scratch, but better
Francesco Biscani
@bluescarni
Jun 08 2016 13:21
there's hope... I think that doxygen with the clang parser should improve much with respect to not choking on strange C++ code
it's clang after all
the problem then moves to the xml export -> breahte -> sphinx
so plenty of possibilities to fuck things up
Dario Izzo
@darioizzo
Jun 08 2016 13:22
yay
at the end life is a workaround
Francesco Biscani
@bluescarni
Jun 08 2016 13:23
a slow bitrot you mean
that's frigging amazing...for some reason boost on windows does not have the all-in-one header
Francesco Biscani
@bluescarni
Jun 08 2016 13:28
whyyyyyy
Francesco Biscani
@bluescarni
Jun 08 2016 13:37
actually the header is there... sigh... I need to log onto appevyor and understand what is going on
Dario Izzo
@darioizzo
Jun 08 2016 13:38
noooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
Marcus Märtens
@CoolRunning
Jun 08 2016 13:40
:fearful:
Francesco Biscani
@bluescarni
Jun 08 2016 13:48
ahh ok I think I might know
Dario Izzo
@darioizzo
Jun 08 2016 13:50
what is it?
curiosity kills the cat
Francesco Biscani
@bluescarni
Jun 08 2016 13:51
so the thing is that since we don't use the unit test library any more, we are not calling FindBoost anywhere
so it does not even try to locate the headers
Dario Izzo
@darioizzo
Jun 08 2016 13:52
but it works on linux ....
Francesco Biscani
@bluescarni
Jun 08 2016 13:52
on linux it's ok because boost is installed in standard paths
Dario Izzo
@darioizzo
Jun 08 2016 13:52
so its recognizes as a system header ....
makes sense
I finished cmaes and the test!!! Same convergence as the algo in the old PaGMO !!
Docs missing, then I am good to learn the python madness :)
It was horrible though .... but now its over its OVER :fire:
Francesco Biscani
@bluescarni
Jun 08 2016 13:55
:+1:
another step in the marathon
Francesco Biscani
@bluescarni
Jun 08 2016 14:36
this is getting more and more hilarious
Dario Izzo
@darioizzo
Jun 08 2016 14:37
what is?
Francesco Biscani
@bluescarni
Jun 08 2016 14:37
so basically this new header thingie triggered an abysmally idiotic behaviour in MSVC
which we just avoided by sheer luck so far
MSVC has these fantastic min/max macros
that fuck up your code completely... you can't use std::min/max or std::numeric_limits::max() etc.
everything breaks down
unless you add a compiler flag that disables this idiocy
so MSVC in its default configuration rejects plain, standard compliant C++ code
Dario Izzo
@darioizzo
Jun 08 2016 14:39
If you need to use the min or max operations, use std::min() or std::max() from the <algorithm> header.
Really?
who would have thought. A Macro is so much better
Francesco Biscani
@bluescarni
Jun 08 2016 14:40
of course... it was breaking down on std::numeric_limits<int>::max()
Dario Izzo
@darioizzo
Jun 08 2016 14:40
explains a lot on how widows is coded :)
Francesco Biscani
@bluescarni
Jun 08 2016 14:40
it was parsing max() as a macrod
how fucking retarded is this
Dario Izzo
@darioizzo
Jun 08 2016 14:41
even for my standards
its as bad as using std in a header
Francesco Biscani
@bluescarni
Jun 08 2016 14:41
btw std::min/max still break, unless you invoke them std::min<int> and similar
Dario Izzo
@darioizzo
Jun 08 2016 14:41
yes you need to disable the macro, then you can use std::min right?
Francesco Biscani
@bluescarni
Jun 08 2016 14:42
yes you need a special compiler definition
Dario Izzo
@darioizzo
Jun 08 2016 14:42
thats what the guy was saying .... disable it and then you have to use std::min ... what a pity :)
did you also install eigen in the VM?
eigen3
?
Francesco Biscani
@bluescarni
Jun 08 2016 14:43
nopew
you want me to do it?
Dario Izzo
@darioizzo
Jun 08 2016 14:43
when you are there .. we should activate all options right?
Marcus Märtens
@CoolRunning
Jun 08 2016 14:43
strange - didn't I recently used std::max?
Dario Izzo
@darioizzo
Jun 08 2016 14:44
maybe you did not include windows headers
Francesco Biscani
@bluescarni
Jun 08 2016 14:44
it depends if some windows headers are included before you use it
brittle as fuck
we should test both with and without eigen3, but of course this won't scale when we have many options
Marcus Märtens
@CoolRunning
Jun 08 2016 14:44
it is used in the hypervolume all the time
Dario Izzo
@darioizzo
Jun 08 2016 14:45
what windows header exactly defines the macro? I want to see it
Marcus Märtens
@CoolRunning
Jun 08 2016 14:45
I think there are some standard headers that are included by MSVC by default somehow
or maybe not - dunno. It just didn't appear to me as a problem.
Dario Izzo
@darioizzo
Jun 08 2016 14:46
if that is the case we have been extremely "lucky" to not encounter this before
Francesco Biscani
@bluescarni
Jun 08 2016 14:46
well it's fixed now, I wish I could smoke enough pot to forget about all this horribleness
Dario Izzo
@darioizzo
Jun 08 2016 14:46
try something stronger :)
Francesco Biscani
@bluescarni
Jun 08 2016 14:47
I could try to dry and sniff Ben's poop
Dario Izzo
@darioizzo
Jun 08 2016 14:47
can you imagine the meeting where this was decided? .... like "hey guys I had an idea ... why don'e we define a macro to get max?"
Marcus Märtens
@CoolRunning
Jun 08 2016 14:47
:poop:
Dario Izzo
@darioizzo
Jun 08 2016 14:47
"that seems like a good idea indeed!"
Francesco Biscani
@bluescarni
Jun 08 2016 14:47
lol
it appears to be somewhat working codecov now: https://codecov.io/gl/EuropeanSpaceAgency/PaGMOreborn
Dario Izzo
@darioizzo
Jun 08 2016 14:49
why? what changed?
Francesco Biscani
@bluescarni
Jun 08 2016 14:49
the fuck if I know
Dario Izzo
@darioizzo
Jun 08 2016 14:49
89.89% shame on us !
Francesco Biscani
@bluescarni
Jun 08 2016 14:49
it still fucked up with the commit list and the timezone
but at least there seems to be some data in it
Marcus Märtens
@CoolRunning
Jun 08 2016 14:49
That looks fancy
And way too professional
All those integration testing tools for a bunch of dirty hackers like us :smile:
Dario Izzo
@darioizzo
Jun 08 2016 14:50
So the yellow means that the line is hit but only one conditional jump is taken?
line 27, I see it yellow now
why?
Francesco Biscani
@bluescarni
Jun 08 2016 14:52
I would not put much faith in the yellow stuff, last time I checked it was broken
Dario Izzo
@darioizzo
Jun 08 2016 14:52
so only red then?
Francesco Biscani
@bluescarni
Jun 08 2016 14:52
yep
the MSVC build takes 13 minutes vs only 2 minutes of the linux ones, and it's only gonna get worse
Dario Izzo
@darioizzo
Jun 08 2016 14:55
are they not releasing a better compiler soon?
Francesco Biscani
@bluescarni
Jun 08 2016 14:55
they started bundling clang with MSVC, so you actually can use it right now
but there's little cmake support for it at the moment
so it gets terribly confused
Dario Izzo
@darioizzo
Jun 08 2016 14:56
what do you mean by bundling?
Dario Izzo
@darioizzo
Jun 08 2016 14:56
they copy clang and rename it to MSCV?
Francesco Biscani
@bluescarni
Jun 08 2016 14:56
you can tick an option in MSVC and it uses clang
instead of cl.exe
but it's not completely working properly right now, it's in kind of beta status
Dario Izzo
@darioizzo
Jun 08 2016 14:57
I just do not see the point, so they have a compiler with an option to use another compiler that actually works?
Francesco Biscani
@bluescarni
Jun 08 2016 14:58
exactly... the existing MSVC compiler is so shit that it's better for them to just switch to clang rather than trying to make it better
Dario Izzo
@darioizzo
Jun 08 2016 14:58
but they still want to call it mscv ....
Francesco Biscani
@bluescarni
Jun 08 2016 14:59
of course... people don't use MSVC for the compiler certainly
they use it for the IDE
Dario Izzo
@darioizzo
Jun 08 2016 15:00
l' anello del conte
Now all makes sense!!!
No more basit face.
Francesco Biscani
@bluescarni
Jun 08 2016 15:00
lol
anyway, what a productive day!!!!
I'll sleep like a stone with the knowledge that I contributed to society today
Dario Izzo
@darioizzo
Jun 08 2016 15:00
it was, in a way ...
Francesco Biscani
@bluescarni
Jun 08 2016 15:01
well some bullshit has been settled
at least that
maybe you can try exposing the cmaes so we see how horrible I did it
Dario Izzo
@darioizzo
Jun 08 2016 15:03
looking forward to it ... finishing the c++ docs right now ....
Francesco Biscani
@bluescarni
Jun 08 2016 15:10
how fucking cool is that
I also added Eigen3 to the packages in the docker image
Dario Izzo
@darioizzo
Jun 08 2016 15:31
pretty awesome!
and all branches are there too!!!
Dario Izzo
@darioizzo
Jun 08 2016 20:32
@bluescarni oi
Francesco Biscani
@bluescarni
Jun 08 2016 20:32
alright
Dario Izzo
@darioizzo
Jun 08 2016 20:33
:) Why two constructors for de algos?
in the python exposition ...
I do not get this logic of having .def and then many inits
also sea ..
Francesco Biscani
@bluescarni
Jun 08 2016 20:34
it's because default arguments have to be limited to compile-time constants, so I expose 1 constructor with the seed unspecified ( == random), and the other with the seed to be specified
Dario Izzo
@darioizzo
Jun 08 2016 20:36
so if the kwarg seed is not specified the seed will be random ... ok
Francesco Biscani
@bluescarni
Jun 08 2016 20:36
yes... in python you just see 1 ctor
Dario Izzo
@darioizzo
Jun 08 2016 20:36
then are we going to have one only file with all in core.cpp?
Is it not going to be too long
Francesco Biscani
@bluescarni
Jun 08 2016 20:36
I split up the docstrings only so far
Dario Izzo
@darioizzo
Jun 08 2016 20:37
At the end I would advocate to split in core, problems, algorithms, metproblems etc.
Francesco Biscani
@bluescarni
Jun 08 2016 20:37
we can split up some more stuff later, but there's some concerns with numpy bullshit
Dario Izzo
@darioizzo
Jun 08 2016 20:37
its also less of a problem for memory during compilation
Francesco Biscani
@bluescarni
Jun 08 2016 20:37
on the other hand it's faster like this :)
Dario Izzo
@darioizzo
Jun 08 2016 20:38
i just fear its going to be an issue with memory
it was for PaGMO legacy
I had to split everything
and it was a painful day
Francesco Biscani
@bluescarni
Jun 08 2016 20:38
I know
Dario Izzo
@darioizzo
Jun 08 2016 20:38
Also, can I add some structure to seoarate the different lines / sectins?
I know you can parse clang and gcc output errors, so for you its not a problem
Francesco Biscani
@bluescarni
Jun 08 2016 20:39
aren't they separated already?
Dario Izzo
@darioizzo
Jun 08 2016 20:39
but human beings tend to have a better reading with some structure
    na.def_pickle(null_algorithm_pickle_suite());
    auto de_ = pygmo::expose_algorithm<de>("de","__init__(gen = 1, F = 0.8, CR = 0.9, variant = 2, ftol = 1e-6, tol = 1e-6, seed = random)\n\n"
        "Differential evolution algorithm.\n\n");
    de_.def(bp::init<unsigned int,double,double,unsigned int,double,double>((bp::arg("gen") = 1u, bp::arg("F") = .8,
        bp::arg("CR") = .9, bp::arg("variant") = 2u, bp::arg("ftol") = 1e-6, bp::arg("tol") = 1E-6)));
    de_.def(bp::init<unsigned int,double,double,unsigned int,double,double,unsigned>((bp::arg("gen") = 1u, bp::arg("F") = .8,
        bp::arg("CR") = .9, bp::arg("variant") = 2u, bp::arg("ftol") = 1e-6, bp::arg("tol") = 1E-6, bp::arg("seed"))));
    pygmo::expose_algo_log(de_,"");
    de_.def("get_seed",&de::get_seed);
    auto sea_ = pygmo::expose_algorithm<sea>("sea","__init__(gen = 1, seed = random)\n\n"
        "(N+1)-ES simple evolutionary algorithm.\n\n");
    sea_.def(bp::init<unsigned>((bp::arg("gen") = 1u)));
    sea_.def(bp::init<unsigned,unsigned>((bp::arg("gen") = 1u, bp::arg("seed"))));
    pygmo::expose_algo_log(sea_,"");
    sea_.def("get_seed",&sea::get_seed);
    auto sade_ = pygmo::expose_algorithm<sade>("sade","__init__(gen = 1, variant = 2, variant_adptv = 1, "
        "ftol = 1e-6, xtol = 1e-6, memory = False, seed = random)\n\n"
        "Self-adaptive differential evolution (jDE and iDE).\n\n");
    sade_.def(bp::init<unsigned,unsigned,unsigned,double,double,bool>((bp::arg("gen") = 1u, bp::arg("variant") = 2u,
        bp::arg("variant_adptv") = 1u, bp::arg("ftol") = 1e-6, bp::arg("xtol") = 1e-6, bp::arg("memory") = false)));
    sade_.def(bp::init<unsigned,unsigned,unsigned,double,double,bool,unsigned>((bp::arg("gen") = 1u, bp::arg("variant") = 2u,
        bp::arg("variant_adptv") = 1u, bp::arg("ftol") = 1e-6, bp::arg("xtol") = 1e-6, bp::arg("memory") = false,
        bp::arg("seed"))));
    pygmo::expose_algo_log(sade_,"");
    sade_.def("get_seed",&sade::get_seed);
    auto de1220_ = pygmo::expose_algorithm<de1220>("de1220","__init__(gen = 1, allowed_variants = [2,3,7,10,13,14,15,16], "
        "variant_adptv = 1, ftol = 1e-6, xtol = 1e-6, memory = False, seed = random)\n\n"
        "Self-adaptive differential evolution (DE 1220 aka pDE).\n\n");
    de1220_.def("__init__",bp::make_constructor(&de1220_init_0,bp::default_call_policies(),
        (bp::arg("gen") = 1u,bp::arg("allowed_variants") = de1220_allowed_variants(),bp::arg("variant_adptv") = 1u,
        bp::arg("ftol") = 1e-6, bp::arg("xtol") = 1e-6, bp::arg("memory") = false)));
    de1220_.def("__init__",bp::make_constructor(&de1220_init_1,bp::default_call_policies(),
        (bp::arg("gen") = 1u,bp::arg("allowed_variants") = de1220_allowed_variants(),bp::arg("variant_adptv") = 1u,
        bp::arg("ftol") = 1e-6, bp::arg("xtol") = 1e-6, bp::arg("memory") = false, bp::arg("seed"))));
    pygmo::expose_algo_log(de1220_,"");
    de1220_.def("get_seed",&de1220::get_seed);
Francesco Biscani
@bluescarni
Jun 08 2016 20:40
yes it's a group of all the algorithm expositions, which is separated from the problem expositions by a section
Dario Izzo
@darioizzo
Jun 08 2016 20:40
a bit thick ... maybe some comments to separate the end of one and the beg of another?
Francesco Biscani
@bluescarni
Jun 08 2016 20:40
but if you want go ahead
Dario Izzo
@darioizzo
Jun 08 2016 20:40
I want
Francesco Biscani
@bluescarni
Jun 08 2016 20:41
it's like 3 lines per algorithm but ok whatever
the docstrings are going to be split off so it should end up being even shorter
Dario Izzo
@darioizzo
Jun 08 2016 20:53
    // CMA-ES
    auto cmaes_ = pygmo::expose_algorithm<cmaes>("cmaes","__init__(gen = 1, cc = -1, cs = -1, c1 = -1, cmu = -1, "
        "sigma0 = -1, ftol = 1e-6, xtol = 1e-6, memory = false, seed = random)\n\n"
        "Covariance Matrix Evolutionary Strategy (CMA-ES).\n\n");
    cmaes_.def("__init__", bp::init<unsigned,double,double,double,double,double,double,double,bool>(
        (bp::arg("gen") = 1u, bp::arg("cc") = -1, bp::arg("cs") = -1, bp::arg("c1") = -1, bp::arg("cmu") = -1,
         bp::arg("sigma0") = 0.5, bp::arg("ftol") = 1e-6, bp::arg("xtol") = 1e-6, bp::arg("memory") = false))
     );
    cmaes_.def("__init__",bp::init<unsigned,double,double,double,double,double,double, double,bool,unsigned>((bp::arg("gen") = 1u,
         bp::arg("cc") = -1, bp::arg("cs") = -1, bp::arg("c1") = -1, bp::arg("cmu") = -1,
         bp::arg("sigma0") = 0.5, bp::arg("ftol") = 1e-6, bp::arg("xtol") = 1e-6, bp::arg("memory") = false, bp::arg("seed"))));
    pygmo::expose_algo_log(cmaes_,"");
    cmaes_.def("get_seed",&cmaes::get_seed);
@bluescarni do you spot the error?
[ 33%] Building CXX object pygmo/CMakeFiles/core.dir/core.cpp.o
In file included from /Users/darioizzo/Documents/PaGMOreborn/pygmo/core.cpp:4:
In file included from /usr/local/include/boost/python/args.hpp:25:
In file included from /usr/local/include/boost/python/object_core.hpp:20:
/usr/local/include/boost/python/def_visitor.hpp:43:29: error: no matching member function for call to 'visit'
        v.derived_visitor().visit(c, name, options);
        ~~~~~~~~~~~~~~~~~~~~^~~~~
/usr/local/include/boost/python/def_visitor.hpp:74:29: note: in instantiation of function template specialization
      'boost::python::def_visitor_access::visit<boost::python::def_visitor<boost::python::init<unsigned int, double, double, double, double, double,
      double, double, bool, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_> >, boost::python::class_<pagmo::cmaes,
      boost::python::detail::not_specified, boost::python::detail::not_specified, boost::python::detail::not_specified>,
      boost::python::detail::def_helper<const char *, boost::python::detail::not_specified, boost::python::detail::not_specified,
      boost::python::detail::not_specified> >' requested here
        def_visitor_access::visit(*this, c, name, options);
                            ^
The error message does not really help me much :(
Francesco Biscani
@bluescarni
Jun 08 2016 20:54
not sure, but my first comment is about those default values
Dario Izzo
@darioizzo
Jun 08 2016 20:54
what about it?
Francesco Biscani
@bluescarni
Jun 08 2016 20:54
if you have double precision def values, they should be written as 1. and not 1
not sure if it matters
Dario Izzo
@darioizzo
Jun 08 2016 20:55
compiling
nope same
at the end of the message there is :
Francesco Biscani
@bluescarni
Jun 08 2016 20:56
can you post the full log somewhere
Dario Izzo
@darioizzo
Jun 08 2016 20:56
/Users/darioizzo/Documents/PaGMOreborn/pygmo/core.cpp:600:12: note: in instantiation of function template specialization
      'boost::python::class_<pagmo::cmaes, boost::python::detail::not_specified, boost::python::detail::not_specified,
      boost::python::detail::not_specified>::def<boost::python::init<unsigned int, double, double, double, double, double, double, double, bool, unsigned
      int, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_> >' requested here
    cmaes_.def("__init__",bp::init<unsigned,double,double,double,double,double,double, double, bool, unsigned>((bp::arg("gen") = 1u,
           ^
/usr/local/include/boost/python/init.hpp:143:10: note: candidate function template not viable: requires single argument 'cl', but 3 arguments were
      provided
    void visit(classT& cl) const
the full?
ok
Francesco Biscani
@bluescarni
Jun 08 2016 20:58
what's on line 597
the first or second init
Dario Izzo
@darioizzo
Jun 08 2016 20:58
the first
the second is three lines after and its referenced in the error log too
Francesco Biscani
@bluescarni
Jun 08 2016 20:59
ah ok... you should do .def(init<..) not .def("__init__",init<..)
the .def("__init__" needs to be used in conjunction with bp::make_constructor and you should not use it unless you need to
Dario Izzo
@darioizzo
Jun 08 2016 21:00
ah ok you have to give the name only if you make_constructor then?
k
compiling
Francesco Biscani
@bluescarni
Jun 08 2016 21:00
you need that bullshit only when you have a constructor argument that does not have a conversion from C++ registered (e.g., vector_double)
Dario Izzo
@darioizzo
Jun 08 2016 21:01
works!
So now ... the docstring
ALl in one file ?
Francesco Biscani
@bluescarni
Jun 08 2016 21:02
it's a bunch of trivial non-template functions that just return strings, it takes nothing to compile
Dario Izzo
@darioizzo
Jun 08 2016 21:02
so? all in one file?
Francesco Biscani
@bluescarni
Jun 08 2016 21:03
it works for me but if you want to split it up with some arbitrary categorization go ahead :)
keep in mind you need also the corresponding header file
Dario Izzo
@darioizzo
Jun 08 2016 21:03
also, why did you split h and cpp ?
Francesco Biscani
@bluescarni
Jun 08 2016 21:04
because like that if I need to change the docstring I just need to recompile the cpp, not the core.cpp
which is what takes all the time
Dario Izzo
@darioizzo
Jun 08 2016 21:04
clever ...
what did your mother feed you with?
Francesco Biscani
@bluescarni
Jun 08 2016 21:04
all the other functions are templates so they have to stay in headers unfortunately
sviluppina e ovomaltina
just managed to compile and run pygmo on the windows VM
can probably be added to the appveyor
Dario Izzo
@darioizzo
Jun 08 2016 21:07
could we also ship the bundle and upload it somewhere?
Francesco Biscani
@bluescarni
Jun 08 2016 21:07
sure if you wanna engage in some windows scripting :)
Dario Izzo
@darioizzo
Jun 08 2016 21:07
I'd love to
Francesco Biscani
@bluescarni
Jun 08 2016 21:07
but rather than that I'd try to understand how to make these damn wheels
Dario Izzo
@darioizzo
Jun 08 2016 21:07
Its what would really make a difference. A rolling windows distribution ....
its so cool that I would also engage in win scripting
wheels -> i tried ... could not
Francesco Biscani
@bluescarni
Jun 08 2016 21:09
the issue I see with wheels is that it seems they assume you are starting with a python project already in distutils/setuptool
what we want is just zip up a directory essentially
no setup.py bullshit
we can do a normal zip of course, but that's no pip install pygmo
so basically you list the dll as "package data", you add the pure .py files, some metadata and it looks good to go
I don't see any dependency checking though, that needs probably to be specified somewhere
we depend on numpy
Francesco Biscani
@bluescarni
Jun 08 2016 21:14
I think this should work
we could bundle the same way all the rest of the dependencies on windows
nlopt, ipopt (the open source flavour), GSL
Dario Izzo
@darioizzo
Jun 08 2016 21:17
more questions, in the init we write unsigned
but then in the docs Args we write (int) and overflow in the throws .. correct?
Francesco Biscani
@bluescarni
Jun 08 2016 21:18
yeah
Dario Izzo
@darioizzo
Jun 08 2016 21:18
also for the seed?
Francesco Biscani
@bluescarni
Jun 08 2016 21:18
yes
Dario Izzo
@darioizzo
Jun 08 2016 21:18
k
seems easy enough -> mah ... I do not believe
but we must try
MUST
Francesco Biscani
@bluescarni
Jun 08 2016 21:19
I'll have a whl in half an hour
Dario Izzo
@darioizzo
Jun 08 2016 21:19
I wait
Dario Izzo
@darioizzo
Jun 08 2016 21:29
pygmo::expose_algo_log(cmaes_,"");
in the "" can i actually put a docstring?
Francesco Biscani
@bluescarni
Jun 08 2016 21:30
lol my wheel works actually
I packaged it and installed it afterwards, worked fine... import pygmo
Dario Izzo
@darioizzo
Jun 08 2016 21:31
really?
Francesco Biscani
@bluescarni
Jun 08 2016 21:32
yeah you can put a docstring there, it's empty because I am lazy :)
yeah it does work
Dario Izzo
@darioizzo
Jun 08 2016 21:32
SO the wheel ... how can one i nstall it?
Francesco Biscani
@bluescarni
Jun 08 2016 21:32
pip install pygmo-....whl at the moment
as it's not uploaded anywhere
there's still something to work out
as probably it needs to be restricted to 64 bit archs and python 3.5
but it works here
Dario Izzo
@darioizzo
Jun 08 2016 21:33
so if I have a python 3.5 on a 64 bit I can pip install the wheel?
Francesco Biscani
@bluescarni
Jun 08 2016 21:33
it bundles core.pyd, the boost python dll and the .py files
if I send you the wheel file yes
I don't know if there's any bullshit related to MSVC though... we can try
maybe it needs some libraries from it
Dario Izzo
@darioizzo
Jun 08 2016 21:34
and this wheel can be packaged automatically?
Lets try tomorrow at work ...
Francesco Biscani
@bluescarni
Jun 08 2016 21:34
sure, it's a python command
Dario Izzo
@darioizzo
Jun 08 2016 21:34
I have windows there with py35
how does python know what and how to package?
I do not want to know ...
Francesco Biscani
@bluescarni
Jun 08 2016 21:35
from setuptools import setup, Distribution

class BinaryDistribution(Distribution):
    def is_pure(self):
        return False

setup(
    name='pygmo',
    version='2.0',
    description='PyGMO',
    packages=['pygmo'],
    package_data={
        'pygmo': ['boost_python-vc140-mt-1_61.dll', 'core.pyd'],
    },
    install_requires=['numpy'],
    distclass=BinaryDistribution
)
it's not rocket science...
this is the setup.py
Dario Izzo
@darioizzo
Jun 08 2016 21:36
it seems awesome to me .... btw, I pushed cmaes .... its done .... ready for the code review
will remove the WIP
Francesco Biscani
@bluescarni
Jun 08 2016 21:37
c:\Python35\python.exe setup.py bdist_wheel
c:\Python35\Scripts\pip install pygmo-2.0-py3-none-any.whl
Dario Izzo
@darioizzo
Jun 08 2016 21:38
the library dll ... where does he look for it?
Francesco Biscani
@bluescarni
Jun 08 2016 21:38
the boost python you mean?
Dario Izzo
@darioizzo
Jun 08 2016 21:38
yes
Francesco Biscani
@bluescarni
Jun 08 2016 21:38
in the same dir where core.pyd is
Dario Izzo
@darioizzo
Jun 08 2016 21:38
which is where setup is?
Francesco Biscani
@bluescarni
Jun 08 2016 21:39
no... setup sits one level above the pygmo subdirectory, which contains the dlls and the python files
basically what you would have after a make install is in the pygmo subdirectory
above it, you create the setup.py and run that command
Dario Izzo
@darioizzo
Jun 08 2016 21:40
but the dll are not there necessarily
like nlopt and stuff
Francesco Biscani
@bluescarni
Jun 08 2016 21:40
yes you have to copy them into the dir and then add them as data files inside the setup.py
same for boost python
Dario Izzo
@darioizzo
Jun 08 2016 21:41
cool
sounds easy
why did we not do it before?
Francesco Biscani
@bluescarni
Jun 08 2016 21:42
when I searched for this I did not find the page I found now
which is from 2014
Dario Izzo
@darioizzo
Jun 08 2016 21:42
i also remember horror when trying to make sense of the process
but its me ....
So, at each commit we can create a wheel, name it with an incteased version. 2.1, 2.2, 2.3 and upload it to PyPI
done
correct?
Francesco Biscani
@bluescarni
Jun 08 2016 21:44
well it seemed like one needed to rewrite the build system in setuptools but that's not the case... also, we did not have appveyor and windows scripting available. as usual, things trickle down to a critical threshold which opens up new ways
yeah
Dario Izzo
@darioizzo
Jun 08 2016 21:44
wow
this is going to be legendary
Francesco Biscani
@bluescarni
Jun 08 2016 21:44
I'd do a release at each master push and develop only in branches
Dario Izzo
@darioizzo
Jun 08 2016 21:44
exactly
Francesco Biscani
@bluescarni
Jun 08 2016 21:44
just give it the datestamp as releas number and fuck it
pygmo-20160608
Dario Izzo
@darioizzo
Jun 08 2016 21:45
like it
but we need a way to check the version
Francesco Biscani
@bluescarni
Jun 08 2016 21:45
it's like ubuntu's but fancier :)
what do you mean?
Dario Izzo
@darioizzo
Jun 08 2016 21:45
like pygmo.__version__
Francesco Biscani
@bluescarni
Jun 08 2016 21:46
ah yeah syre
sure
we also need to do packages for multiple python versions
Dario Izzo
@darioizzo
Jun 08 2016 21:46
to help us debugging the idiotic requests from users
All the world will use PaGMO!!!! We need the fundation
Francesco Biscani
@bluescarni
Jun 08 2016 21:46
that we need to add on the C++ side as well
Dario Izzo
@darioizzo
Jun 08 2016 21:46
maybe 10 euros a year will be donated
Francesco Biscani
@bluescarni
Jun 08 2016 21:47
we should get money from the companies, since they seem the most interested in this stuff
Dario Izzo
@darioizzo
Jun 08 2016 21:47
they are interested since its free :)
Francesco Biscani
@bluescarni
Jun 08 2016 21:48
it's peanuts money for them, and they care much more about the support rather than the price
Dario Izzo
@darioizzo
Jun 08 2016 21:48
so we can sell it for 1 euros, then support it
:)
Francesco Biscani
@bluescarni
Jun 08 2016 21:48
if we knew someone with a bit of an enterprise streak we could make money from this
Dario Izzo
@darioizzo
Jun 08 2016 21:49
I know ....
sucks admitting that those guys are useful
Or we wait for google to offer us 100000000000000 EUR to sell
to which we, of course, say no way!
Francesco Biscani
@bluescarni
Jun 08 2016 21:49
meh I am past that, fuck everything :) think that they would pay you for open source development
in the end people keep on devloping GCC regardless of what other people use it for
Dario Izzo
@darioizzo
Jun 08 2016 21:50
I think that if we get a user base occasions will arise
and already now PaGMO has a user base
with this and some ads it will fly
Francesco Biscani
@bluescarni
Jun 08 2016 21:50
I'd much rather work on pagmo full time than doing full time my current job :)
Dario Izzo
@darioizzo
Jun 08 2016 21:51
then you would hate it ....
already now I sometime do !
Francesco Biscani
@bluescarni
Jun 08 2016 21:51
I don't hate it otherwsie there's now way I'd do it.. .just need to switch tom something else every once in a while
Dario Izzo
@darioizzo
Jun 08 2016 21:52
ANyway .. its late ... going to bed. Tomorrow wheels and I will check if the python expo I did actually works
Francesco Biscani
@bluescarni
Jun 08 2016 21:52
I cannot do anything fulltime I think, needs always some breathing toom
Dario Izzo
@darioizzo
Jun 08 2016 21:52
on OSX I cannot :)
yeah me too ...
Francesco Biscani
@bluescarni
Jun 08 2016 21:52
ok I'll send you the bullshit now
Dario Izzo
@darioizzo
Jun 08 2016 21:52
right ...
Francesco Biscani
@bluescarni
Jun 08 2016 22:08
I think I managed to create a wheel that's platform-specific right now, as it should be
this is pretty frigging awesome