These are chat archives for bluescarni/pagmo_reborn

28th
Jun 2016
Dario Izzo
@darioizzo
Jun 28 2016 07:44
The error it gives is:
/home/dario/Documents/PaGMOreborn/tests/../include/algorithms/../utils/generic.hpp:239:106: error: no matching function for call to ‘std::vector<long unsigned int>::erase(long unsigned int*, long unsigned int*)’
                 neigh_idxs[i].erase(neigh_idxs[i].data() + k, neigh_idxs[i].data() + neigh_idxs[i].size());
followed by:
In file included from /usr/include/c++/6.1.1/vector:64:0,
                 from /usr/include/boost/test/utils/runtime/errors.hpp:29,
                 from /usr/include/boost/test/utils/runtime/argument.hpp:20,
                 from /usr/include/boost/test/unit_test_parameters.hpp:19,
                 from /usr/include/boost/test/impl/compiler_log_formatter.ipp:23,
                 from /usr/include/boost/test/included/unit_test.hpp:15,
                 from /home/dario/Documents/PaGMOreborn/tests/cmaes.cpp:2:
/usr/include/c++/6.1.1/bits/stl_vector.h:1147:7: note: candidate: std::vector<_Tp, _Alloc>::iterator std::vector<_Tp, _Alloc>::erase(std::vector<_Tp, _Alloc>::const_iterator) [with _Tp = long unsigned int; _Alloc = std::allocator<long unsigned int>; std::vector<_Tp, _Alloc>::iterator = __gnu_cxx::__normal_iterator<long unsigned int*, std::vector<long unsigned int> >; typename std::_Vector_base<_Tp, _Alloc>::pointer = long unsigned int*; std::vector<_Tp, _Alloc>::const_iterator = __gnu_cxx::__normal_iterator<const long unsigned int*, std::vector<long unsigned int> >; typename __gnu_cxx::__alloc_traits<typename std::_Vector_base<_Tp, _Alloc>::_Tp_alloc_type>::const_pointer = const long unsigned int*]
       erase(const_iterator __position)
       ^~~~~
/usr/include/c++/6.1.1/bits/stl_vector.h:1147:7: note:   candidate expects 1 argument, 2 provided
/usr/include/c++/6.1.1/bits/stl_vector.h:1174:7: note: candidate: std::vector<_Tp, _Alloc>::iterator std::vector<_Tp, _Alloc>::erase(std::vector<_Tp, _Alloc>::const_iterator, std::vector<_Tp, _Alloc>::const_iterator) [with _Tp = long unsigned int; _Alloc = std::allocator<long unsigned int>; std::vector<_Tp, _Alloc>::iterator = __gnu_cxx::__normal_iterator<long unsigned int*, std::vector<long unsigned int> >; typename std::_Vector_base<_Tp, _Alloc>::pointer = long unsigned int*; std::vector<_Tp, _Alloc>::const_iterator = __gnu_cxx::__normal_iterator<const long unsigned int*, std::vector<long unsigned int> >; typename __gnu_cxx::__alloc_traits<typename std::_Vector_base<_Tp, _Alloc>::_Tp_alloc_type>::const_pointer = const long unsigned int*]
       erase(const_iterator __first, const_iterator __last)
       ^~~~~
/usr/include/c++/6.1.1/bits/stl_vector.h:1174:7: note:   no known conversion for argument 1 from ‘long unsigned int*’ to ‘std::vector<long unsigned int>::const_iterator {aka __gnu_cxx::__normal_iterator<const long unsigned int*, std::vector<long unsigned int> >}’
tests/CMakeFiles/cmaes.dir/build.make:62: recipe for target 'tests/CMakeFiles/cmaes.dir/cmaes.cpp.o' failed
I think no known conversion for argument 1 from ‘long unsigned int*’ to ‘std::vector<long unsigned int>::const_iterator
is the issue right?
    std::vector<std::vector<vector_double::size_type> > neigh_idxs;
FYI
Francesco Biscani
@bluescarni
Jun 28 2016 08:38
right, that's not going to work with vector::erase()
I assumed you were using std::erase()
actually I mean std::remove()
Francesco Biscani
@bluescarni
Jun 28 2016 08:52
I don't have a solution which is safe and quick
the safe way would be to extend safe_cast() to deal with signed - unsigned conversions, and then use it to convert k to the distance type of the iterator
Dario Izzo
@darioizzo
Jun 28 2016 08:57
FOr the time being I put a static_cast there with a TODO.
Dario Izzo
@darioizzo
Jun 28 2016 11:36
@bluescarni
hoi are u there?
Francesco Biscani
@bluescarni
Jun 28 2016 11:37
yep
Dario Izzo
@darioizzo
Jun 28 2016 11:37
I need to produce a pure python utility. Like a function that plots points in the Pareto sense ..... how would you structure such a thing? Given that there will be many
?
Francesco Biscani
@bluescarni
Jun 28 2016 11:38
you mean from the point of view of including it in pagmo?
Dario Izzo
@darioizzo
Jun 28 2016 11:38
yes, from the point of view of the directory structure, file .. etc...
These will be all utils shipped with pygmo and only accessible from python.
I imagine all graphical stuff will go there
Like plotting the network for topologies etc.
Francesco Biscani
@bluescarni
Jun 28 2016 11:39
I think that attempts at categorization early in the life of a project invariably end up being shitty choices, so for now I'd just put it under the pygmo. module
Dario Izzo
@darioizzo
Jun 28 2016 11:40
and one single file called python_utils.py ?
or already multi_objective.py
or whatever?
Francesco Biscani
@bluescarni
Jun 28 2016 11:41
I can't recall on top of my head how it is handled in Python... if you put them in a separate file can they still appear under pygmo. or do they need their own submodule
Dario Izzo
@darioizzo
Jun 28 2016 11:41
I guess it depends how you import the file
Francesco Biscani
@bluescarni
Jun 28 2016 11:41
brb
Dario Izzo
@darioizzo
Jun 28 2016 11:41
from core.py
k
btw ...
Screenshot from 2016-06-28 13-41-39.png
whats with the functions hits? I could not understand what they are and why should we care / not care.
Francesco Biscani
@bluescarni
Jun 28 2016 12:08
I vaguely remember that they had something to do with the number of times templates are instantiated and I never cared
core is a compiled module
so you cannot really add stuff to it
Dario Izzo
@darioizzo
Jun 28 2016 12:09
i meant __init__.py
I say one thing, think another and write down a third .. :)
Francesco Biscani
@bluescarni
Jun 28 2016 12:10
I find the python module hierarchy import stuff a mess personally
from pippo import pluto vs import pippo.pluto vs the all the stuff about the __all__ variable in __init__.py
Dario Izzo
@darioizzo
Jun 28 2016 12:11
I was going to ask
Francesco Biscani
@bluescarni
Jun 28 2016 12:11
absolute imports vs relative imports
Dario Izzo
@darioizzo
Jun 28 2016 12:11
What is our intended use?
Francesco Biscani
@bluescarni
Jun 28 2016 12:11
and then they say C++ is complicated
Dario Izzo
@darioizzo
Jun 28 2016 12:11
I say from pygmo.core import *
or import pygmo as pg
right?
Francesco Biscani
@bluescarni
Jun 28 2016 12:12
I don't know what's the current status among pythonistas, but I think the first version is to be tolerated only in interactive code
it's like using namespace std;
Dario Izzo
@darioizzo
Jun 28 2016 12:12
I love that!
Francesco Biscani
@bluescarni
Jun 28 2016 12:13
I liked the idea of having always everything available in pygmo.
it reduces typing
Dario Izzo
@darioizzo
Jun 28 2016 12:13
if we do the first thing, then we get algorithms, otherwise we get pg.core.algorithms.
not sure why
but its like this oon my system
Francesco Biscani
@bluescarni
Jun 28 2016 12:14
algorithms is created as a submodule in the core as far as I remember
Dario Izzo
@darioizzo
Jun 28 2016 12:14
indeed
Francesco Biscani
@bluescarni
Jun 28 2016 12:14
__all__ = ['core','test']
try to add algorithm to that to see if anything changes
Dario Izzo
@darioizzo
Jun 28 2016 12:16
yep, now we can do from pygmo import * and get algorithms in the root nmspc
Francesco Biscani
@bluescarni
Jun 28 2016 12:16
it's just horrible, and it will require some reading to understand what is really going on
Dario Izzo
@darioizzo
Jun 28 2016 12:16
should I leave it? also with problems?
Francesco Biscani
@bluescarni
Jun 28 2016 12:16
I don't understand fully the consequences of doing so
Dario Izzo
@darioizzo
Jun 28 2016 12:16
then I leave oit as is
we always have time to study
Francesco Biscani
@bluescarni
Jun 28 2016 12:17
IMO better not to touch just because "it seems to work", but whatever
Dario Izzo
@darioizzo
Jun 28 2016 12:17
right
i do not care at the moment, i can always from pygmo.core import * and be happy for now
Francesco Biscani
@bluescarni
Jun 28 2016 12:18
the shitty thing is that I read about this at one point, but never logged what I learned anywhere so now I'd need to re-learn it
there was a rationale for some decisions taken in the past
Dario Izzo
@darioizzo
Jun 28 2016 12:19
I actually also remember you trying to explain it to me, but as always I did not listen :)
Francesco Biscani
@bluescarni
Jun 28 2016 12:19
:) I am also sure there are subtle differencese Python 2 vs 3
fucking assholes
Dario Izzo
@darioizzo
Jun 28 2016 12:19
In the later PaGMO I adopted an evolutionary approach to decide how to do the import ....
fitness being actually being able to use it as I was intending .... you can imagine the horror it produced
Francesco Biscani
@bluescarni
Jun 28 2016 12:20
but that's the thing about python right... it's not "easier", it just gets you quick to something that kinda works but which most likely is pretty shit
Dario Izzo
@darioizzo
Jun 28 2016 12:21
yeah!
rapid prototyping ....
Francesco Biscani
@bluescarni
Jun 28 2016 12:21
yeah that never gets fixed.. the eternal TODO/FIXME
Dario Izzo
@darioizzo
Jun 28 2016 12:42
Help ..... help ....
    moead_.def(bp::init<unsigned,std::string, unsigned, double, double, double, double, unsigned, bool, unsigned>(
        (bp::arg("gen") = 1u, bp::arg("weight_generation") = "grid", bp::arg("neighbours") = 20u, bp::arg("CR") = 1., bp::arg("F") = 0.5,
         bp::arg("eta_m") = 20, bp::arg("realb") = 0.9, bp::arg("limit") = 2u, bp::arg("preserve_diversity") = true))
     );
do you see anything wrong here? It compiles, but when I do:
In [1]: import pygmo as pg

In [2]: pg.algorithms.moead(2, "giggi")
---------------------------------------------------------------------------
ArgumentError                             Traceback (most recent call last)
<ipython-input-2-4696f40fd495> in <module>()
----> 1 pg.algorithms.moead(2, "giggi")

ArgumentError: Python argument types in
    moead.__init__(moead, int, str)
did not match C++ signature:
    __init__(_object*, unsigned int gen=1, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > weight_generation='grid', unsigned int neighbours=20, double CR=1.0, double F=0.5, double eta_m=20, double realb=0.9, unsigned int limit=2, bool preserve_diversity=True, unsigned int seed)
    __init__(_object*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > gen=1, unsigned int weight_generation='grid', double neighbours=20, double CR=1.0, double F=0.5, double eta_m=20, unsigned int realb=0.9, bool limit=2, unsigned int preserve_diversity=True)
    __init__(_object*)
he does not recognize the signature.
The exposed constructor has signature:
    moead(unsigned int gen = 1u,
            std::string weight_generation = "grid",
            population::size_type neighbours = 20u,
            double CR = 1.0,
            double F = 0.5,
            double eta_m = 20.,
            double realb = 0.9,
            unsigned int limit = 2u,
            bool preserve_diversity = true,
            unsigned int seed = pagmo::random_device::next()
            ) :
Francesco Biscani
@bluescarni
Jun 28 2016 12:45
why does it have 3 signatures? and the second signature looks fishy.... std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > gen=1 this looks wrong
Dario Izzo
@darioizzo
Jun 28 2016 12:46
    auto moead_ = pygmo::expose_algorithm<moead>("moead", pygmo::moead_docstring().c_str());
    moead_.def(bp::init<unsigned,std::string, unsigned, double, double, double, double, unsigned, bool, unsigned>(
        (bp::arg("gen") = 1u, bp::arg("weight_generation") = "grid", bp::arg("neighbours") = 20u, bp::arg("CR") = 1., bp::arg("F") = 0.5,
         bp::arg("eta_m") = 20, bp::arg("realb") = 0.9, bp::arg("limit") = 2u, bp::arg("preserve_diversity") = true))
     );
     moead_.def(bp::init<unsigned,std::string, unsigned, double, double, double, double, unsigned, bool, unsigned>(
         (bp::arg("gen") = 1u, bp::arg("weight_generation") = "grid", bp::arg("neighbours") = 20u, bp::arg("CR") = 1., bp::arg("F") = 0.5,
          bp::arg("eta_m") = 20, bp::arg("realb") = 0.9, bp::arg("limit") = 2u, bp::arg("preserve_diversity") = true, bp::arg("seed")))
      );
the two signatures I thought were necessary for the seed to be random
Francesco Biscani
@bluescarni
Jun 28 2016 12:46
yes but for some reason it looks like the second signature has a string gen = 1 as parameter
Dario Izzo
@darioizzo
Jun 28 2016 12:47
aaahh right thanks!
trying without the unsigned
on the first signature
Francesco Biscani
@bluescarni
Jun 28 2016 12:48
yep
Dario Izzo
@darioizzo
Jun 28 2016 12:49
one day I will learn .. and you will be proud :)
it just not today !
Francesco Biscani
@bluescarni
Jun 28 2016 12:50
just pattern matching :)
Dario Izzo
@darioizzo
Jun 28 2016 13:07

I need to expose:

fnds_return_type fast_non_dominated_sorting (const std::vector<vector_double> &obj_list)

I guess I have to do a wrapper and use vvd_to_a for the argument. The return value is an std::tuple. Do I need to unpack it and then make tuple? Or its automatic?

Francesco Biscani
@bluescarni
Jun 28 2016 13:07
I think I wrote a converter from C++ tuple, it should be there
Dario Izzo
@darioizzo
Jun 28 2016 13:08
ok I look better
Francesco Biscani
@bluescarni
Jun 28 2016 13:08
fnds_return_type what are the tuple types in here?
Dario Izzo
@darioizzo
Jun 28 2016 13:08
std::tuple<std::vector<std::vector<vector_double::size_type>>,std::vector<std::vector<vector_double::size_type>>,std::vector<vector_double::size_type>,std::vector<vector_double::size_type>>;
but you don't need that converter
it's too generic, it was written for the log stuff
return boost::make_tuple(....); will be enough
and you need a new utility conversion for a vector<vector<size_type>>, which I don't think is available
Dario Izzo
@darioizzo
Jun 28 2016 13:10
and std::vector<std::vector<vector_double::size_type>> has already a converter?
Francesco Biscani
@bluescarni
Jun 28 2016 13:10
no
Dario Izzo
@darioizzo
Jun 28 2016 13:10
also std::vector<vector_double::size_type> right?
Francesco Biscani
@bluescarni
Jun 28 2016 13:10
I think there's something for that, let me check
https://gitlab.com/EuropeanSpaceAgency/PaGMOreborn/blob/master/pygmo/common_utils.hpp#L152 so there's this for unsigned long long, it should probably be made generic to deal with all unsigned types
Dario Izzo
@darioizzo
Jun 28 2016 13:12
-> template?
Francesco Biscani
@bluescarni
Jun 28 2016 13:13
yes, something like:
template <typename T>
inline bp::object vuint_to_a(const std::vector<T> &v);
Dario Izzo
@darioizzo
Jun 28 2016 13:14
needs to be active only when the type is unsigned ...
correct?
Francesco Biscani
@bluescarni
Jun 28 2016 13:14
I'd hope the name gives it away, but yeah.. you can put an enabler for that
Dario Izzo
@darioizzo
Jun 28 2016 13:14
:) just to practice ...
Francesco Biscani
@bluescarni
Jun 28 2016 13:15
template <typename T>
inline bp::object vvuint_to_a(const std::vector<std::vector<T>> &v);
and finally this
actually all the functions v_to_a could probably be rolled in a single template
it would shorten things quite a bit
but just to the unsigned for now
// Convert a vector of vectors of doubles into a numpy array.
inline bp::object vvd_to_a(const std::vector<pagmo::vector_double> &v)
{
    // The dimensions of the array to be created.
    const auto nrows = v.size();
    const auto ncols = nrows ? v[0].size() : 0u;
    npy_intp dims[] = {boost::numeric_cast<npy_intp>(nrows),boost::numeric_cast<npy_intp>(ncols)};
    // Attempt creating the array.
    PyObject *ret = PyArray_SimpleNew(2,dims,NPY_DOUBLE);
    if (!ret) {
        pygmo_throw(PyExc_RuntimeError,"couldn't create a NumPy array: the 'PyArray_SimpleNew()' function failed");
    }
    // Hand over to BP for exception-safe behaviour.
    bp::object retval{bp::handle<>(ret)};
    if (nrows) {
        auto data = static_cast<double *>(PyArray_DATA((PyArrayObject *)(ret)));
        for (const auto &i: v) {
            if (i.size() != ncols) {
                pygmo_throw(PyExc_ValueError,"cannot convert a vector of vector_double to a NumPy 2D array "
                    "if the vector_double instances don't have all the same size");
            }
            std::copy(i.begin(),i.end(),data);
            data += ncols;
        }
    }
    return retval;
}

// Convert a vector of ull into a numpy array.
inline bp::object vull_to_a(const std::vector<unsigned long long> &v)
{
    // The dimensions of the array to be created.
    npy_intp dims[] = {boost::numeric_cast<npy_intp>(v.size())};
    // Attempt creating the array.
    PyObject *ret = PyArray_SimpleNew(1,dims,cpp_npy<unsigned long long>::value);
    if (!ret) {
        pygmo_throw(PyExc_RuntimeError,"couldn't create a NumPy array: the 'PyArray_SimpleNew()' function failed");
    }
    if (v.size()) {
        // Copy over the data.
        std::copy(v.begin(),v.end(),static_cast<unsigned long long *>(PyArray_DATA((PyArrayObject *)(ret))));
    }
    // Hand over to boost python.
    return bp::object(bp::handle<>(ret));
}
basically you need the unsigned int versions of these
you need to convert one member of the tuple to a numpy array right? and that's the vector of vectors (aka, 2D array)?
Dario Izzo
@darioizzo
Jun 28 2016 13:19
I guess ...
Francesco Biscani
@bluescarni
Jun 28 2016 13:19
is that a real matrix (N x M) or just a list of vectors (in C++ I mean)
because if it is a matrix it makes sense to return it as a 2D numpy array, otherwise it's better a list of 1D arrays
Dario Izzo
@darioizzo
Jun 28 2016 13:20
I posted the return value def above, it contains vectors of vectors of indexes
ans stuff like that
Francesco Biscani
@bluescarni
Jun 28 2016 13:21
I cannot know from that if it represents a matrix of something or a list of vectors
returning stuff as N-dimensional numpy arrays makes sense only if you are representing some dense matrix-like entity
so that accessing them with numpy slices/indices brings advantages
if it's a list of 1D vectors with different sizes I don't think it makes sense to turn it into a 2D dense matrix
Dario Izzo
@darioizzo
Jun 28 2016 13:28
checking ...
Francesco Biscani
@bluescarni
Jun 28 2016 13:29
the non dominated fronts, an std::vector<std::vector<vector_double::size_type>> containing the non dominated fronts. Example {{1,2},{3},{0}}
the domination list, an std::vector<std::vector<size_type>> containing the domination list, i.e. the indexes of all individuals dominated by the individual at position ii. Example {{},{},{0,3},{0}}
it looks like a list of vectors to me
Dario Izzo
@darioizzo
Jun 28 2016 13:29
indeed
Francesco Biscani
@bluescarni
Jun 28 2016 13:31
then you need just one conversion function from vuint to numpy array, then you do this:
bp::list dominated_fronts;
for (...) {
    dominate_fronts.append(vuint_to_a(...));
}
...
return bp::make_tuple(...,dominated_fronts,...);
Dario Izzo
@darioizzo
Jun 28 2016 13:31
k
vull_to_a and vuint_to_a can be reduced to the same template right?
Francesco Biscani
@bluescarni
Jun 28 2016 13:34
yeah
vd_to_a as well
Dario Izzo
@darioizzo
Jun 28 2016 13:34
name? v_to_a?
Francesco Biscani
@bluescarni
Jun 28 2016 13:36
yeah why not
Dario Izzo
@darioizzo
Jun 28 2016 13:38
I am just cautious as its strange you did not do it ... maybe some hidden reason?
Francesco Biscani
@bluescarni
Jun 28 2016 13:39
it just grew organically
Dario Izzo
@darioizzo
Jun 28 2016 13:51
Does this make sense?
// Convert a vector of doubles into a numpy array.
template <typename T>
    using vd_to_a_enabler = std::enable_if_t<std::is_arithmetic<T>::value,int>;

template <typename T, vd_to_a_enabler<T> = 0>>
inline bp::object v_to_a(const std::vector<T> &v)
{
    // The dimensions of the array to be created.
    npy_intp dims[] = {boost::numeric_cast<npy_intp>(v.size())};
    // Attempt creating the array.
    PyObject *ret = PyArray_SimpleNew(1,dims,NPY_DOUBLE);
    if (!ret) {
        pygmo_throw(PyExc_RuntimeError,"couldn't create a NumPy array: the 'PyArray_SimpleNew()' function failed");
    }
    if (v.size()) {
        // Copy over the data.
        std::copy(v.begin(),v.end(),static_cast<double *>(PyArray_DATA((PyArrayObject *)(ret))));
    }
    // Hand over to boost python.
    return bp::object(bp::handle<>(ret));
}
Francesco Biscani
@bluescarni
Jun 28 2016 13:52
you need to change all the types
this is correct only if T == double
Dario Izzo
@darioizzo
Jun 28 2016 13:54
you mean in the statc_cast?
Francesco Biscani
@bluescarni
Jun 28 2016 13:54
PyObject *ret = PyArray_SimpleNew(1,dims,NPY_DOUBLE); needs to be PyObject *ret = PyArray_SimpleNew(1,dims,cpp_npy<T>::value);
that as well
Dario Izzo
@darioizzo
Jun 28 2016 13:55
This message was deleted
sooo accurate
Francesco Biscani
@bluescarni
Jun 28 2016 13:56
it's really easy... PyArray_SimpleNew(1,dims,NPY_DOUBLE); asks to create a new numpy array with the following characteristics:
  • the shape of the array has dimension 1 (unidimensional array)
  • the size of the only dimension is in the dims variable,
  • the type of the contained type in the array is NPY_DOUBLE
since this is C and not C++, types are represented by global integral values called NPY_DOUBLE, NPY_INT, NPY_UINT, etc.
Dario Izzo
@darioizzo
Jun 28 2016 13:57
cpp_npy<T>::value
Francesco Biscani
@bluescarni
Jun 28 2016 13:57
and there is a structure on top of the utils file that establish the mapping between these numerical constants and C++ types
that one exactly
you need to add something on top
because I enabled the mapping only for integral types
// Map C++ types to NPY_ types.
template <typename T>
struct cpp_npy {};

#define PYGMO_CPP_NPY(from,to) \
template <> \
struct cpp_npy<from> \
{ \
    static constexpr auto value = to; \
};

// We only need integral types at the moment.
PYGMO_CPP_NPY(unsigned char,NPY_UBYTE)
PYGMO_CPP_NPY(unsigned short,NPY_USHORT)
PYGMO_CPP_NPY(unsigned,NPY_UINT)
PYGMO_CPP_NPY(unsigned long,NPY_ULONG)
PYGMO_CPP_NPY(unsigned long long,NPY_ULONGLONG)
PYGMO_CPP_NPY(signed char,NPY_BYTE)
PYGMO_CPP_NPY(short,NPY_SHORT)
PYGMO_CPP_NPY(int,NPY_INT)
PYGMO_CPP_NPY(long,NPY_LONG)
PYGMO_CPP_NPY(long long,NPY_LONGLONG)
so you need to append this:
PYGMO_CPP_NPY(double,NPY_DOUBLE)
and after you fix the static_cast<double *> as well it should be good to go
Dario Izzo
@darioizzo
Jun 28 2016 14:01
it compiles indeed, but then ... it compiled also before when it was total rubbish :)
Francesco Biscani
@bluescarni
Jun 28 2016 14:01
I mean this numpy stuff is really brain damaged, it's just creating arrays essentially
there's nothing to it
create a pointer to a new numpy array, fill the array with data, then wrap it in a boost python object (aka smart pointer)
Dario Izzo
@darioizzo
Jun 28 2016 14:02
the general overview is clear to me .. its the details that are aramaic
Francesco Biscani
@bluescarni
Jun 28 2016 14:03
it's because C is utter shit
Dario Izzo
@darioizzo
Jun 28 2016 14:03
anyway, I guess you will have to make a tough code review when all of this is finished ....
Francesco Biscani
@bluescarni
Jun 28 2016 14:03
I can review the python bits
boost python I mean
Dario Izzo
@darioizzo
Jun 28 2016 14:03
also the c++ may have hidden treasures
I have been writing a ton of stuff .... for moead to be properly included
Francesco Biscani
@bluescarni
Jun 28 2016 14:04
it will be lost in the volume of code and my ignorance about stuff
Dario Izzo
@darioizzo
Jun 28 2016 14:04
I know .... only the user testing will tell if all of this makes sense
Francesco Biscani
@bluescarni
Jun 28 2016 14:05
is it that different from pagmo legacy?
Dario Izzo
@darioizzo
Jun 28 2016 14:05
a bit yes
more than usual
This message was deleted
Francesco Biscani
@bluescarni
Jun 28 2016 14:13
vd_to_a_enabler should also probably be renamed to v_to_a_enabler
Dario Izzo
@darioizzo
Jun 28 2016 14:14
This message was deleted
yes that I did
Dario Izzo
@darioizzo
Jun 28 2016 14:33
I need also a to_vvd utility converting a bp::object to std::vector<std::vector<double>>
Francesco Biscani
@bluescarni
Jun 28 2016 14:33
it's not there already?
Dario Izzo
@darioizzo
Jun 28 2016 14:33
that would be cool ... let me recheck
Francesco Biscani
@bluescarni
Jun 28 2016 14:34
no doesn't look like
Dario Izzo
@darioizzo
Jun 28 2016 14:34
not in common_utils.hpp
Francesco Biscani
@bluescarni
Jun 28 2016 14:34
that's uglier to write
Dario Izzo
@darioizzo
Jun 28 2016 14:35
yap, I do not think I should ....
Francesco Biscani
@bluescarni
Jun 28 2016 14:35
unless it's a list of vectors...
Dario Izzo
@darioizzo
Jun 28 2016 14:35
no, in this case its the input points, so they all have the same dimension
Francesco Biscani
@bluescarni
Jun 28 2016 14:36
so it should accept a bp::object both in form of a list of lists and a numpy array then?
Dario Izzo
@darioizzo
Jun 28 2016 14:36
I guess thats the pygmo standard right?
Francesco Biscani
@bluescarni
Jun 28 2016 14:37
yep
Dario Izzo
@darioizzo
Jun 28 2016 14:37
the numpy array is an array of arrays?
Francesco Biscani
@bluescarni
Jun 28 2016 14:37
no it's a 2D array
it's actually more efficient than vector<vector<>>
a single memory allocation and all the values in the same memory chunk
Dario Izzo
@darioizzo
Jun 28 2016 14:39
indeed. but we anyway convert back and forth so is all for nothing
Francesco Biscani
@bluescarni
Jun 28 2016 14:39
yeah I was just saying.. in general it's better for multidimensional dense data to store it contiguously and create the N-dimensional array view on top of it
eigen does this as well, but it's only 2d I guess?
Dario Izzo
@darioizzo
Jun 28 2016 14:41
Matrix ... think so
Francesco Biscani
@bluescarni
Jun 28 2016 14:42
would it make sense to enforce the 2d array input on the python side? what are the dimensions of this matrix?
and where is it used?
Dario Izzo
@darioizzo
Jun 28 2016 14:42
Its the collection of all fitness in a population
or cpould be the collection of all chromosomes
or IDSs
Francesco Biscani
@bluescarni
Jun 28 2016 14:43
ah but so you get already a numpy array when you get it out of the population
Dario Izzo
@darioizzo
Jun 28 2016 14:43
so, yes, I do not see a problem in asking a 2D structure
Francesco Biscani
@bluescarni
Jun 28 2016 14:43
the pop accessors in python all return numpy arrays
Dario Izzo
@darioizzo
Jun 28 2016 14:43
At the moment the population is not involved as we have to expose the utils which are orthogonal
Francesco Biscani
@bluescarni
Jun 28 2016 14:44
but you would apply it to some entity extracted from the population no?
Dario Izzo
@darioizzo
Jun 28 2016 14:44
but yes one could first extract from the population too and then pass it to this function
yes you should be able to, but also use it independently.
Francesco Biscani
@bluescarni
Jun 28 2016 14:45
right ok... I see the list syntax to be useful mainly for small interactive examples, but probably it should accept lists and arrays for consistency with the rest
Dario Izzo
@darioizzo
Jun 28 2016 14:47
I mean fro the user point of view:
ret = non_dom_sort([[1,2],[3,2],[2,2],[4,4],[1,1]])
ret = non_dom_sort(array([[1,1],[2,2],[3,4],[0,1]]))
should both work right?
Francesco Biscani
@bluescarni
Jun 28 2016 14:48
yes.. I am just saying that I wouldn't expect the user to fill in manually long lists of numbers in typical use
Dario Izzo
@darioizzo
Jun 28 2016 14:48
indeed, so also:
ret = non_dom_sort(pop.get_f())
should work
I guess the above is the typical use correct?
Francesco Biscani
@bluescarni
Jun 28 2016 14:49
don't ask me, I don't do optimisation :)
I can make the version that does both
Dario Izzo
@darioizzo
Jun 28 2016 14:50
would be cool because I would just screw up all
i pushed my last moead branch version
better start from there?
Francesco Biscani
@bluescarni
Jun 28 2016 14:51
right... but I can't work on this for at least a couple of days, too much stuff happening
Dario Izzo
@darioizzo
Jun 28 2016 14:52
yeah ok np ...
I wait
Francesco Biscani
@bluescarni
Jun 28 2016 14:52
I need to finish the research statement for the application and my bosses here are breaking my balls because frankly I am not doing much
Dario Izzo
@darioizzo
Jun 28 2016 14:53
Did they say anything?
Francesco Biscani
@bluescarni
Jun 28 2016 14:53
not really, but I have tasks and I am just postponing and really zero motivation
Dario Izzo
@darioizzo
Jun 28 2016 14:54
I also have a ton of shit that I am delaying .... and should be done before Ileave for summer .... but wont
Francesco Biscani
@bluescarni
Jun 28 2016 14:54
can spend only so much time on shitty cmake macros, shitty student code and shitty multidimensional array stuff before wanting to kill myself
I'll have 2 weeks of holiday starting next week
going to Italy
Dario Izzo
@darioizzo
Jun 28 2016 14:55
finally!! with family?
Francesco Biscani
@bluescarni
Jun 28 2016 14:55
I really need to get the fuck out of this job
yeps
Dario Izzo
@darioizzo
Jun 28 2016 14:55
mountains?
Francesco Biscani
@bluescarni
Jun 28 2016 14:55
grandparents :) drop the beast, relax
Dario Izzo
@darioizzo
Jun 28 2016 14:55
cool!
Francesco Biscani
@bluescarni
Jun 28 2016 14:55
you going to sardinia?
Dario Izzo
@darioizzo
Jun 28 2016 14:55
yep and maremma
Francesco Biscani
@bluescarni
Jun 28 2016 14:56
cooll... the usual 10 weeks of vacation?
:)
Dario Izzo
@darioizzo
Jun 28 2016 14:56
of course
from the 22nd to the 19th
August- January ... :)
Francesco Biscani
@bluescarni
Jun 28 2016 14:56
lol
btw it's ok for you tomorrow to send my bullshit letter?
Dario Izzo
@darioizzo
Jun 28 2016 14:57
np, but I need some details .. like to who?
Francesco Biscani
@bluescarni
Jun 28 2016 14:57
it's just the email address of the secretar, I'll send it to you
Dario Izzo
@darioizzo
Jun 28 2016 14:57
ok... and the content?
Francesco Biscani
@bluescarni
Jun 28 2016 14:58
wtf
Dario Izzo
@darioizzo
Jun 28 2016 14:58
Same as before + updates? Tailored to cel mech?
Francesco Biscani
@bluescarni
Jun 28 2016 14:58
yes
the page is not available any more, I hope they did not cancelled it or something
Dario Izzo
@darioizzo
Jun 28 2016 14:58
ok np...
Francesco Biscani
@bluescarni
Jun 28 2016 14:58
I'll send you a copy of the announcement
Dario Izzo
@darioizzo
Jun 28 2016 14:59
ok great
Francesco Biscani
@bluescarni
Jun 28 2016 14:59
thanks
need to catch the train, see you later
Dario Izzo
@darioizzo
Jun 28 2016 14:59
later
Marcus Märtens
@CoolRunning
Jun 28 2016 16:06
@darioizzo I am considering passing by on thursday. What do you think? Will you be around?
Dario Izzo
@darioizzo
Jun 28 2016 16:09
@CoolRunning I am actually busy on thursday for the Asteroid Day ......http://asteroidday.org/ I will be answering questions on reddit
Marcus Märtens
@CoolRunning
Jun 28 2016 16:11
I am an asteroid - AMA?
Dario Izzo
@darioizzo
Jun 28 2016 16:11
@bluescarni One more problem with the current python stuff. If the logs contain non arithmetic values (like an std::vector) the current set up fails. Is there an easy fix? Or we should just prevents logs to have anything else but basic types?
@CoolRunning I also signed the declaration ... :)
Marcus Märtens
@CoolRunning
Jun 28 2016 16:11
There is a declaration?
Dario Izzo
@darioizzo
Jun 28 2016 16:12
Have fun:
I am also there :)
in the scientists !!
Dario Izzo
@darioizzo
Jun 28 2016 16:13
:) knew this one ...
Marcus Märtens
@CoolRunning
Jun 28 2016 16:13
One of "the door"-classics
Dario Izzo
@darioizzo
Jun 28 2016 16:13
going home now .... later
Marcus Märtens
@CoolRunning
Jun 28 2016 16:14
Bummer, still. I could have made a point to my supervisor I have to go in order to "prepare" for this important talk I will be giving.
Another day then.
Dario Izzo
@darioizzo
Jun 28 2016 16:14
I am just the whole day in a meeting room writing on reddit
Marcus Märtens
@CoolRunning
Jun 28 2016 16:16
Yeah, will try again after Poznan, before you disappear ;)
Dario Izzo
@darioizzo
Jun 28 2016 18:38
@bluescarni On my osx the PYGMO_INSTALL_PATH is empty .... and does not appear on the list of variable that can be set manually .... as a consequence is really difficult to get the files installed where they should
Dario Izzo
@darioizzo
Jun 28 2016 18:44
Since there are a few things accumulating I will list them here for future reference:
  • In common_utils.hpp we need a pagmo::to_vvd able to convert a bp::object (list of lists or numpy.array) to a c++ vector<vector<double>>
  • The utility to convert the algorithm logs is right now limited to arithmetic values. It could make sense to add the possibility to include a std::vector<double> in the log entry. Or decide we will not and thus change the logging strategy for moead and all future algos that require this.
  • Build system in OSX does not allow to set easily the PYGMO_INSTALL_PATH nor it detects correctly in a homebrew system.