These are chat archives for bluescarni/pagmo_reborn

30th
Jun 2016
Francesco Biscani
@bluescarni
Jun 30 2016 10:56
@darioizzo so I implemented the to_vvd() functionality in the moad branch
regarding the OSX stuff, feel free to provide a cmake implementation specialised for OSX if you want in the meantime
for the tuple stuff, I think you can get away by writing a small wrapper. Something like this:
template <typename Algo>
inline bp::list my_new_log_getter(const Algo &a)
{
    bp::list retval;
    for (const auto &t: a.get_log()) {
        retval.append(bp::make_tuple(std::get<0>(t),std::get<1>(t),vd_to_a(std::get<2>(t)),...));
    }
    return retval;
}
Dario Izzo
@darioizzo
Jun 30 2016 11:02
Love you!!!
Francesco Biscani
@bluescarni
Jun 30 2016 11:02
probably need a different one for each different tuple log type, not sure how easy it is to generalise
Dario Izzo
@darioizzo
Jun 30 2016 11:03
I tried yesterday to look into the t-_vvd stuff, oh man
Francesco Biscani
@bluescarni
Jun 30 2016 11:03
yeah it should probably be generalised a bit, but I am not sure how
I am also slightly worried about all the copying around we do with vectors and stuff
Dario Izzo
@darioizzo
Jun 30 2016 11:05
Me too, but thats done mainly on stuff that needs no optimization. The main bottleneck is on fitness
if thats quick enough, the rest should not worry us
Francesco Biscani
@bluescarni
Jun 30 2016 11:05
I was thinking that if you have something like a linear programming problem, which basically just computes a linear combination of the values of a vector as objfun, you will have a measurable overhead regardless of dimensions
I think that the only way of solving this would be to have a special vector class instead of vector_double which allows to steal the allocated memory and re-use it to init numpy arrays and similar
so you don't copy stuff around actually, a bit like the move semantics stuff
but anyway, for the future
Dario Izzo
@darioizzo
Jun 30 2016 11:06
True, you mentioned that already. But we also discussed that PyGMO is oriented to fitness computation that do more than just summing ...
... at least in phase 1
Francesco Biscani
@bluescarni
Jun 30 2016 11:07
it does not need changing the interface we have, which I think is a nice one
yeah
Dario Izzo
@darioizzo
Jun 30 2016 11:08
Will try to get the moead pull request finalized ..... this will probably be my last before the "new" year, except for small refinements and docs
interface -> love it
I wrote a few examples tutorials, enjoyed it a lot
(in python I mean)
Francesco Biscani
@bluescarni
Jun 30 2016 11:09
yeah it's good
I'll keep on writing tests on the Python stuff as I find time
Dario Izzo
@darioizzo
Jun 30 2016 12:08
>>> from pygmo.core import *
>>> ndf, dl, dc, ndr = fast_non_dominated_sorting([[2,3],[-1,2],[-3,2],[0,5],[1,1]])
>>> print(ndf)
[array([2, 4], dtype=uint64), array([1], dtype=uint64), array([0, 3], dtype=uint64)]
>>> ndf
[array([2, 4], dtype=uint64), array([1], dtype=uint64), array([0, 3], dtype=uint64)]
>>> ndf[0]
array([2, 4], dtype=uint64)
>>> print(dl)
[array([], dtype=uint64), array([0, 3], dtype=uint64), array([0, 1, 3], dtype=uint64), array([], dtype=uint64), array([0], dtype=uint64)]
>>> print(dc)
[3 1 0 2 0]
>>> print(ndr)
[2 1 0 2 0]
Is this the behavior we want? I mean the dtype=uint64
its only in the first two vvd, not in the last two vd
Should we not just convert all to doubles?
Francesco Biscani
@bluescarni
Jun 30 2016 12:10
I don't follow... the idea is to convert to dtype float the vector_double and similar, and dtype int when it's an array of indices or similar
you'd rather have only dtype double for everything?
Dario Izzo
@darioizzo
Jun 30 2016 12:12
I do not know, I was just pointing at an inconsistency between dc and ndf[0]
Francesco Biscani
@bluescarni
Jun 30 2016 12:12
what are those quantities representing?
Dario Izzo
@darioizzo
Jun 30 2016 12:12
indexes
ints
both
domination count (dc)
and non dominated front (ndf)
Francesco Biscani
@bluescarni
Jun 30 2016 12:13
but why would you want them to be represented as list/arrays of doubles?
Dario Izzo
@darioizzo
Jun 30 2016 12:13
Sorry, I may be confusing you (as usual) ... my point is that:
>>> ndf[0]
array([2, 4], dtype=uint64)
while:
>>> dc
[3 1 0 2 0]
while i would be expecting both to be array or list I guess .... but dunno if we care of outputing also dc as one
Francesco Biscani
@bluescarni
Jun 30 2016 12:15
In [2]: array([4,5,6])
Out[2]: array([4, 5, 6])

In [3]: print(array([4,5,6]))
[4 5 6]
Dario Izzo
@darioizzo
Jun 30 2016 12:15
"All lists of indexes shall be arrays" type of rule ... is it worth enforcing?
Francesco Biscani
@bluescarni
Jun 30 2016 12:15
I think it's just printing
print the types of those objects
Dario Izzo
@darioizzo
Jun 30 2016 12:15
>>> type(ndr)
<class 'numpy.ndarray'>
>>> type(ndf[0])
<class 'numpy.ndarray'>
>>>
lol
Francesco Biscani
@bluescarni
Jun 30 2016 12:16
so they are the same type
Dario Izzo
@darioizzo
Jun 30 2016 12:16
happy? so you lost 3 minutes of your life!!
Francesco Biscani
@bluescarni
Jun 30 2016 12:16
anything to keep me away from my shit tasks :)
Dario Izzo
@darioizzo
Jun 30 2016 12:16
thanks for putting it that way :)
Dario Izzo
@darioizzo
Jun 30 2016 12:53
the test_vvd can be removed right?
Francesco Biscani
@bluescarni
Jun 30 2016 12:53
no I will use to build python tests on top of it
it's hidden anyway
name starts with an underscore, it won't show up normally
Dario Izzo
@darioizzo
Jun 30 2016 12:56
ok .... good I asked :)
Francesco Biscani
@bluescarni
Jun 30 2016 12:59
I am not 100% sure on that uint stuff, we might need to change it eventually
Dario Izzo
@darioizzo
Jun 30 2016 12:59
what is that you do not like?
Francesco Biscani
@bluescarni
Jun 30 2016 12:59
so the problem is that by default numpy uses signed ints. if you write array([1,2,3]), it's an array of some signed int type
you can write array([1,2,3],dtype='uint64') to force it to uint
which is of course poop
so some of the utils in pygmo accept as input numpy arrays of signed int, and the convert it internally to size_type vectors (as expected by pagmo)
but as you saw above some other exported functions return arrays of uint
this is a bit inconsistent, and on top of that at one point we might need to send the output of some functions to input of other functions
Dario Izzo
@darioizzo
Jun 30 2016 13:02
yep, risks to be confusing and poop ...
Francesco Biscani
@bluescarni
Jun 30 2016 13:02
here we would have some errors due to signed/unsigned conversions
not sure what's best to do... we could force always signed ints on the python side
Dario Izzo
@darioizzo
Jun 30 2016 13:04
i'd like that. only floats and int in python side
would require to deal with overflows, but probably is minor
Francesco Biscani
@bluescarni
Jun 30 2016 13:04
so basically it'd be a matter of changing those conversion functions to output numpy int arrays
there's already overflow check when the conversion is done internally
Dario Izzo
@darioizzo
Jun 30 2016 13:05
ah then ... its straight forward
Francesco Biscani
@bluescarni
Jun 30 2016 13:08
where are those quantities above coming from?
Dario Izzo
@darioizzo
Jun 30 2016 13:08
which ones?
the dl etc.?
Francesco Biscani
@bluescarni
Jun 30 2016 13:08
I mean where are those uint vector generated
Dario Izzo
@darioizzo
Jun 30 2016 13:08
from the newly exposed fast_non_dominated_sorting
not pushed yet
Francesco Biscani
@bluescarni
Jun 30 2016 13:09
you are using the v_to_a method in there right?
Dario Izzo
@darioizzo
Jun 30 2016 13:09
// Wrappers for utils/multi_objective stuff
// fast_non_dominated_sorting
static inline bp::object fast_non_dominated_sorting_wrapper(const bp::object &x)
{
    auto fnds = fast_non_dominated_sorting(pygmo::to_vvd(x));
    // the non-dominated fronts
    auto ndf = std::get<0>(fnds);
    bp::list ndf_py;
    for (const std::vector<vector_double::size_type> &front : ndf) {
        ndf_py.append(pygmo::v_to_a(front));
    }
    // the domination list
    auto dl = std::get<1>(fnds);
    bp::list dl_py;
    for (const auto &item : dl) {
        dl_py.append(pygmo::v_to_a(item));
    }
    return bp::make_tuple(ndf_py, dl_py, pygmo::v_to_a(std::get<2>(fnds)), pygmo::v_to_a(std::get<3>(fnds)));
}
yes
Francesco Biscani
@bluescarni
Jun 30 2016 13:10
right ok so the problem is that v_to_a deduces the array type from the type of std::get<2>(fnds), which is an unsigned vector
Dario Izzo
@darioizzo
Jun 30 2016 13:10
indeed as its templated now
Francesco Biscani
@bluescarni
Jun 30 2016 13:10
so you'll have to do the conversion yourself inside there before returning
Dario Izzo
@darioizzo
Jun 30 2016 13:12
casting each component? without making a copy right?
Francesco Biscani
@bluescarni
Jun 30 2016 13:13
std::vector<std::make_signed_t<size_type>> new_vec;
for (const auto &n: old_vec) {
    new_vec.push_back(boost::numeric_cast<std::make_signed_t<size_type>>(n))
}
something like this
Dario Izzo
@darioizzo
Jun 30 2016 13:13
boost is ok as its core.cpp right?
Francesco Biscani
@bluescarni
Jun 30 2016 13:13
yes
Dario Izzo
@darioizzo
Jun 30 2016 13:14
in c++ we will have to put boost in when doing topologies anyway I fear
boost graph ...
Francesco Biscani
@bluescarni
Jun 30 2016 13:14
here they use a library called lemon for graph stuff
Dario Izzo
@darioizzo
Jun 30 2016 13:14
let me check
Francesco Biscani
@bluescarni
Jun 30 2016 13:14
probably overkill
and most likely shittier code quality than boost graph
also last release 2014
wtf
Dario Izzo
@darioizzo
Jun 30 2016 13:16
we will see when it time I guess ...
also porting all pagmo legacy code on topos changing the graph library is 1 order of magnitude more difficult
Francesco Biscani
@bluescarni
Jun 30 2016 13:17
yep.. and I really see no benefit
The Boost Graph Library is a header-only library and does not need to be built to be used. The only exceptions are the GraphViz input parser and the GraphML parser.
not even need to link actually... I thought it was not header only
Dario Izzo
@darioizzo
Jun 30 2016 13:18
we could just copy it then into our repo. Also given that there was a small bugfix we had to do on the adj list ...
do not ask why or when o whether it was really necessary ... probably not
Francesco Biscani
@bluescarni
Jun 30 2016 13:19
that probably will not work, I am sure it drags half boost headers as dependency... and the bugfix was only for serialization, which we have to rewrite anyway
Dario Izzo
@darioizzo
Jun 30 2016 13:19
you remember? wow
I think I have alzheimer
Francesco Biscani
@bluescarni
Jun 30 2016 13:20
you take too long vacations :)
Francesco Biscani
@bluescarni
Jun 30 2016 14:10
you gotta start answering man
Dario Izzo
@darioizzo
Jun 30 2016 14:10
at 16:45 :)
we all are doomed
Francesco Biscani
@bluescarni
Jun 30 2016 14:10
ah right ok
Dario Izzo
@darioizzo
Jun 30 2016 14:10
we at ESA are building bunkers to protect us from the next one
Francesco Biscani
@bluescarni
Jun 30 2016 14:10
it's gonna be tough :)
Dario Izzo
@darioizzo
Jun 30 2016 14:10
No, we cannot deflect them
We are all going to die soon
Francesco Biscani
@bluescarni
Jun 30 2016 14:10
lol
"it's the gay people's fault, god hates them"
Dario Izzo
@darioizzo
Jun 30 2016 14:11
thats a good one !!
if you are not gay the asteroid will not hit you
are you?
so much room to get ESA in trouble ...