These are chat archives for bluescarni/pagmo_reborn

18th
May 2016
Dario Izzo
@darioizzo
May 18 2016 09:15
Hello and good morning! I have just committed a finished algorithm called sea (Simple evolutionary algorithm). It will eventually go in the tutorials. The code shows how to deal with screen output, with stochastic algorithms and how to add the possibility to log the single algorithm run. As sea is rather basic, no stopping criteria are present except the generation number.
Dario Izzo
@darioizzo
May 18 2016 09:23
Some comments I had while coding:
1 - I think its nice the way we can log the internal algorithm work via a local log and, later, an extract (see https://gitlab.com/EuropeanSpaceAgency/PaGMOreborn/blob/master/tests/sea.cpp)
2 - In sea the stopping criteria issue was not present, so I did not come out with any proposal there.
3 - There are two ways to code an algorithm. a) at the beginning of evolve you get_x and get_f, do your stuff, at the end set_xf on the population copy. b) use and act directly on the population object. The reason that, at the moment, b) may be easier is that best_idx, worst_idx etc. are only provided as methods of population, not as general utilities. One may think to move them out of pop so that one could use them also if doing a)
Marcus Märtens
@CoolRunning
May 18 2016 09:30
Hey! Today we have the webex, right?
Dario Izzo
@darioizzo
May 18 2016 09:30
indeed ... you should have received an invitation
Its at 14:00 right?
Francesco Biscani
@bluescarni
May 18 2016 09:33
you have any idea on how this works? I can't understand if I need a plugin, an extensions or it is supposed to work out of the box
Dario Izzo
@darioizzo
May 18 2016 09:33
I hope you open chrome to the link and the plugin downloads
if not installed
Francesco Biscani
@bluescarni
May 18 2016 09:35
Dario Izzo
@darioizzo
May 18 2016 09:36
you modified the const ref stuff?
Francesco Biscani
@bluescarni
May 18 2016 09:36
yep
Dario Izzo
@darioizzo
May 18 2016 09:36
we can live with that ....
Francesco Biscani
@bluescarni
May 18 2016 09:37
it's still correct :) but we should probably consider having a mingw build as well, just if things go south with MSVC
I doubt we will make more extreme use of C++, but who knows
Dario Izzo
@darioizzo
May 18 2016 09:38
Its funny to have found a compiler bug ..... should we tell them?
Francesco Biscani
@bluescarni
May 18 2016 09:38
it's shooting fishes in a barrel with MSVC... to be honest, they do acknowledge that the expression SFINAE support is not complete yet
in fact I am kinda surprised we arrived this far with it
Dario Izzo
@darioizzo
May 18 2016 09:39
k
Francesco Biscani
@bluescarni
May 18 2016 09:41
I am wondering about the log stuff.. it seems like its meaning is going to be rather different depending on the algorithm, so why are we putting it into the general algorithm interface? as opposed to putting it into the concrete problems?
Dario Izzo
@darioizzo
May 18 2016 09:42
We are....
It is in the concrete problems. Then you can extract it :)
Right now an example is in sea.cpp
Francesco Biscani
@bluescarni
May 18 2016 09:43
but we do have a verbosity method in the generic algorithm interface... I mean, what is it supposed to do from the point of view of a general algorithm? what kind of general behaviour we can expect when we do set_verbosity() on a pagmo::algorithm?
Dario Izzo
@darioizzo
May 18 2016 09:44
If implemented, the concrete algorithm will change the verbosity of its screen output and (if implemented) logs
Francesco Biscani
@bluescarni
May 18 2016 09:45
I guess what I am trying to say is that the algorithm interface exist to enable communication between problems, islands, etc. When you do a fitness computation, you know you are getting out from the method a vector representing the fitness, but when you do set_verbose() it's not clear what the post-condition will be
Dario Izzo
@darioizzo
May 18 2016 09:47
I guess the reason to force it into algorithm is to avoid that the user then codes set_screen_output, set_verbose, verbosity, screen_log ...... according to his pleasure. Like this we force him to use one name and prototype. Also, We can query from the algorithm if such a method exist
Say I have a vector of algorithms, I can switch all output off bu set_verbose(0u) ..... Given that the algorithms are using it correctly (which we can only guarantee in our algo suite)
Francesco Biscani
@bluescarni
May 18 2016 09:50
Alright... the thing is, I am not sure about the role of these setters once we start doing stuff with islands etc. Because we cannot really extract references from the island once we begin doing parallel computations (and also the python exposition is a problem), so I am not sure there will be a way to change problem/algorithm properties once we stuff them in the island
we could duplicate the setters from the island I suppose, dunno
I have a feeling these properties will be set once and for all from the constructor
so I was wondering if instead of having a set_verbosity() method we should instead have an enum with various levels of verbosity, and then let the algorithm constructors use it
Dario Izzo
@darioizzo
May 18 2016 09:53
The way I see it is that the use case of algo.evolve is: algo development/debug. algo benchmarking. In this context is very useful to have seed control and verbosity control. The use case of isl.evolve arch.evolve is: solve a problem, study migration. In which case that requirment is lower
Francesco Biscani
@bluescarni
May 18 2016 09:53
the problem/algorithm interface is going to be largely unmutable once we start using islands
fair enough
just thinking ahead :)
Dario Izzo
@darioizzo
May 18 2016 09:55
indeed it will be confusing .... We could just say that once in island/archi verbosity is 0u
Francesco Biscani
@bluescarni
May 18 2016 09:55
well we can still provide a verbosity argument in the ctor, that we can do
btw it does work the keyword arguments in Python
Dario Izzo
@darioizzo
May 18 2016 09:56
from pybind11 ?
Francesco Biscani
@bluescarni
May 18 2016 09:56
yes, but there's similar support in boost python: http://www.boost.org/doc/libs/1_37_0/libs/python/doc/v2/args.html
Dario Izzo
@darioizzo
May 18 2016 09:56
can I try it?
Francesco Biscani
@bluescarni
May 18 2016 09:57
sure
Dario Izzo
@darioizzo
May 18 2016 09:57
which class has it?
Francesco Biscani
@bluescarni
May 18 2016 09:57
inv = _core.inventory(weeks = 10)
Dario Izzo
@darioizzo
May 18 2016 09:58
not compiling anymore :(
Francesco Biscani
@bluescarni
May 18 2016 09:58
still the submodule stuff I guess
I haven't touched the python bindings since the weekend
Dario Izzo
@darioizzo
May 18 2016 09:59
should I update the submodule then?
Francesco Biscani
@bluescarni
May 18 2016 09:59
it's another computer this one ?
Dario Izzo
@darioizzo
May 18 2016 09:59
No.
But I updated the submodule this morning as he was showing new commits
in git status
so I typed git submodule update
Francesco Biscani
@bluescarni
May 18 2016 10:00
something happeend, let me check
Dario Izzo
@darioizzo
May 18 2016 10:00
and he did something
Francesco Biscani
@bluescarni
May 18 2016 10:01
holy fuck this submodule stuff is horrid
it still compiles here though
Dario Izzo
@darioizzo
May 18 2016 10:01
lol
Francesco Biscani
@bluescarni
May 18 2016 10:02
lol I did a submodule update and now it breaks
holy fuck
k let me switch to subtrees
holy fuck
fucking fuck
Dario Izzo
@darioizzo
May 18 2016 10:02
thats what I did ... I think he is checking out the very latest pybind11 and
ok I wait
Marcus Märtens
@CoolRunning
May 18 2016 10:04
sounds like stuff is seriously breaking... :worried:
Dario Izzo
@darioizzo
May 18 2016 10:04
the sky is falling on our heads
Francesco Biscani
@bluescarni
May 18 2016 10:07
so try now, it's better if you just erase the pagmo dir and re-clone it
because otherwise you have to deinit the submodule and crap like that
now you don't have to do anything special with subtrees
just clone as usual, and do cmake
In [4]: _core.inventory?
Docstring:      <no docstring>
Init docstring:
__init__(*args, **kwargs)
Overloaded function.

1. __init__(pygmo._core.inventory) -> NoneType

Default constructor.

2. __init__(pygmo._core.inventory, pygmo._core.inventory) -> NoneType

Copy constructor.

3. __init__(self : pygmo._core.inventory, weeks : int = 4, sample_size : int = 10) -> NoneType

Constructor from weeks and sample size (seed is randomly-generated).

4. __init__(self : pygmo._core.inventory, weeks : int = 4, sample_size : int = 10, seed : int) -> NoneType

Constructor from weeks, sample size and seed.
File:           ~/repos/PaGMOreborn/build_clang/pygmo/_core.so
Type:           type
that's the automatically generated help output when you use kwargs
Dario Izzo
@darioizzo
May 18 2016 10:19
works ...
one thing ...before with the python namespaces we could get a list of all problems available and algorithm available by tabbing after having typed problem
That was very cool.
If we have no namespaces the list of possible problems or algorithms or topologies will not be as easily accessible ..... not cool :( Could be worth to have namespaces only for that reason
lunch now ... later
Francesco Biscani
@bluescarni
May 18 2016 10:31
In python we are not bound to C++ namespaces, we can have the problem submodule if we want. Actually I think we should have both - problems should be accessible both from pygmo.rosenbrock and pygmo.problems.rosenbrock
it's a rather common pattern in matplotlib and numpy for instance
btw it looks like it is possible to use the Boost.Test library without the need to link... http://www.boost.org/doc/libs/1_55_0/libs/test/doc/html/utf/user-guide/usage-variants/single-header-variant.html
Marcus Märtens
@CoolRunning
May 18 2016 10:35
In the getting started guide of Boost they recommend using it linked
Boost.Test can be used in “header-only” or “separately compiled” mode, although separate compilation is recommended for serious use.
Francesco Biscani
@bluescarni
May 18 2016 10:36
I think we have solved that issue anyway
just throwing it out :)
Marcus Märtens
@CoolRunning
May 18 2016 10:36
I thought the opposite - are we not returning to Boost rather than throwing it out?
Francesco Biscani
@bluescarni
May 18 2016 10:37
the discussion was about Boost.Python, Boost is made of many libraries
and even if we use Boost.Python, that would be limited to when we compile the Python bindings
Marcus Märtens
@CoolRunning
May 18 2016 10:38
Okay - so what will happen to Boost.Test?
Francesco Biscani
@bluescarni
May 18 2016 10:38
nothing, we keep it as it is, since it is working fine
Marcus Märtens
@CoolRunning
May 18 2016 10:38
ah okay
Dario Izzo
@darioizzo
May 18 2016 11:21
namespaces .... cool then we will have them in python hurray!!
Marcus Märtens
@CoolRunning
May 18 2016 11:21
:+1:
Dario Izzo
@darioizzo
May 18 2016 11:54
I am in position .... do we need a phone to be able to connect?
Francesco Biscani
@bluescarni
May 18 2016 11:54
I hope not, have no phone here
Dario Izzo
@darioizzo
May 18 2016 11:55
logging in now .. lets see
Francesco Biscani
@bluescarni
May 18 2016 11:55
lol it needs java
I'll try from the virtual machine
Dario Izzo
@darioizzo
May 18 2016 11:57
I see you :)
Marcus Märtens
@CoolRunning
May 18 2016 11:58
I will also try - one second
Dario Izzo
@darioizzo
May 18 2016 11:59
ok
we are connected
i see you can you hear us?
Marcus Märtens
@CoolRunning
May 18 2016 12:58
coming back one second
Dario Izzo
@darioizzo
May 18 2016 12:58
np
Marcus Märtens
@CoolRunning
May 18 2016 13:00
somehow my micro is not working
I just wanted to know how many objectives he needs :(
Dario Izzo
@darioizzo
May 18 2016 13:00
will ask
Marcus Märtens
@CoolRunning
May 18 2016 13:00
thx
Okay, very interesting :)
Then hypervolume will be needed
My audio settings were screwed :worried:
Marcus Märtens
@CoolRunning
May 18 2016 13:05
But I heard everything
Dario Izzo
@darioizzo
May 18 2016 13:05
:) so whats your take?
Marcus Märtens
@CoolRunning
May 18 2016 13:05
Pretty cool
I am kinda amazed how deeply he embedded PaGMO
Dario Izzo
@darioizzo
May 18 2016 13:06
yep, ... what was all that GUI :)
Francesco Biscani
@bluescarni
May 18 2016 13:06
lol
Marcus Märtens
@CoolRunning
May 18 2016 13:06
And I agree with the current developments :)
Getting some "experts" from Continuum by the beginning of the year is valuable for porting
for implementing the parallelization - i am not so sure whether we should let others touch that part
Better get it done before
And afterwards use the workforce
Dario Izzo
@darioizzo
May 18 2016 13:07
hangouts ?
Marcus Märtens
@CoolRunning
May 18 2016 13:07
yes
let me restart, one minute - should fix my audio issues
brb
Dario Izzo
@darioizzo
May 18 2016 15:11
@bluescarni Do we need the algorithm to be default-constructible, copy-constructible, move-constructible and destructible?
Francesco Biscani
@bluescarni
May 18 2016 15:13
you mean the algorithm class or concrete algorithms?
Dario Izzo
@darioizzo
May 18 2016 15:13
the class
Francesco Biscani
@bluescarni
May 18 2016 15:14
yeah sure... I mean, if it is not destructible what you even gonna do with it :)
Dario Izzo
@darioizzo
May 18 2016 15:15
ok so we need to check that as static assert ... I did not code it ... will put it in and socument
Francesco Biscani
@bluescarni
May 18 2016 15:15
it should be the same as in the problem class
Dario Izzo
@darioizzo
May 18 2016 15:15
yep, thats how I figured
The signature algorithm::evolve(const population &pop)
still has a const reference, I guess thats ok as it avoids one more copy?
Francesco Biscani
@bluescarni
May 18 2016 15:18
I thought we turned that into a pass by copy?
Dario Izzo
@darioizzo
May 18 2016 15:18
In the user implemented yes
Thats in algorithm
        /// Evolve method
        population evolve(const population &pop) const
        {
            return ptr()->evolve(pop);
        }
Francesco Biscani
@bluescarni
May 18 2016 15:19
right ok.. that works
Dario Izzo
@darioizzo
May 18 2016 15:19
The inner method will have as signature evolve(population)
Francesco Biscani
@bluescarni
May 18 2016 16:01
nice
Francesco Biscani
@bluescarni
May 18 2016 16:13
To create an instance of a pagmo::algorithm the user is asked to construct it from a separate object of type T where, at least, the implementation of the following method is provided:
population evolve(population) const;
I'd still put the const reference in the documentation, for consistence with the rest
the fact that you can actually pass by value if you want to avoid having to create a population inside the algorithm is inconsequential, as a signature with const reference will accept anything as input

http://en.cppreference.com/w/cpp/algorithm/transform see for instance the documentation here:

unary operation function object that will be applied.
The signature of the function should be equivalent to the following:
Ret fun(const Type &a);
The signature does not need to have const &.
The type Type must be such that an object of type InputIt can be dereferenced and then implicitly converted to Type. The type Ret must be such that an object of type OutputIt can be dereferenced and assigned a value of type Ret. ​

Marcus Märtens
@CoolRunning
May 18 2016 19:22
Hi! Is there a need for mutable member-variables in the current PaGMO? I know this had some meaning for serialization...
Francesco Biscani
@bluescarni
May 18 2016 20:06
sometimes, but it depends on the specific context.. if it works without mutable it's better, otherwise we can discuss it
Marcus Märtens
@CoolRunning
May 18 2016 20:12
Well, this will be a discussion anyway at some point.
As it happens in the most important algorithm... which happens to do some nasty memory manipulations as well. So this part of the code needs more serious changes, I am afraid... :/
Francesco Biscani
@bluescarni
May 18 2016 20:13
we'll manage, a bit at a time... is it about global variables and stuff like that?
Marcus Märtens
@CoolRunning
May 18 2016 20:15
the variables that are mutable are initialized and freed at specific computational methods... they are otherwise meaningless
I guess we flagged them as mutable so they would not be serialized
which is fine as they seem to be merely placeholders for memory blocks to be allocated etc.
Anyway... a bit at a time would be nice, but this hypervolume code is very interdependent.
Imagine an algorithm for 2 dimensions, another one for 3, another one for 4 and another one for more... all of them more or less recursive and calling methods from each other...
It is a pretty efficient and nice design, but you have to move it as a whole to make it work.
Very interdependent.
Francesco Biscani
@bluescarni
May 18 2016 20:34
right, it's a matter of identifying where to start with the modifications wit minimal disruption
Marcus Märtens
@CoolRunning
May 18 2016 20:37
I think I have an almost functional core by now
But probably this can only be temporary and some algorithms need to be rewritten or adapted at least.
Francesco Biscani
@bluescarni
May 18 2016 20:38
it's good, I can help with the part concerning pointers and C++ bullshit
Marcus Märtens
@CoolRunning
May 18 2016 20:39
Ah, is there a straightforward way to rewrite this boost::bind thingy in std-terms?
            // Bind the object under "this" pointer to the cmp_points method so it can be used as a valid comparator function for std::sort
            // We need that in order for the cmp_points to have acces to the m_current_slice member variable.
            std::sort(points, points + n_points, boost::bind(&wfg::cmp_points, this, _1, _2));
Francesco Biscani
@bluescarni
May 18 2016 20:40
what is wfg::cmp_points? a function?
Marcus Märtens
@CoolRunning
May 18 2016 20:40
Once I have this core working, in theory, all methods from the old interface are implemented. Then I will commit and we should discuss the necessary changes.
Francesco Biscani
@bluescarni
May 18 2016 20:41
std::sort(points, points + n_points, [this](auto a, auto b) {return wfg::cmp_points(this,a,b);});
Marcus Märtens
@CoolRunning
May 18 2016 20:41
Yes, this is bool function that takes two numbers and compares them... kinda - it does a bit more.
Francesco Biscani
@bluescarni
May 18 2016 20:41
ok but it also takes this as input right?
Marcus Märtens
@CoolRunning
May 18 2016 20:42
no, but it needs to have access to a member-variable
Francesco Biscani
@bluescarni
May 18 2016 20:42
the way it was called from bind it looks like a function with 3 arguments
Marcus Märtens
@CoolRunning
May 18 2016 20:42
the function is implemented like this:
    /**
    * Comparison function for WFG. Can't be static in order to have access to member variable m_current_slice.
    */
    bool cmp_points(double* a, double* b) const
    {
        for (int i = m_current_slice - 1; i >= 0; --i) {
            if (a[i] > b[i]) {
                return true;
            }
            else if (a[i] < b[i]) {
                return false;
            }
        }
        return false;
    }
As I understand, it is bound to this since this has access to m_current_slice
Francesco Biscani
@bluescarni
May 18 2016 20:44
and the sort is called from a method of wfg right?
Marcus Märtens
@CoolRunning
May 18 2016 20:44
correct
Francesco Biscani
@bluescarni
May 18 2016 20:44
so it should be: std::sort(points, points + n_points, [this](auto a, auto b) {return this->cmp_points(a,b);});
Marcus Märtens
@CoolRunning
May 18 2016 20:45
That looks like a mess of parenthesis :D
But cool, I will try... but first I have to bash some guys in Doto
I think if you want you can replace boost bindh with std::bind
without changing anything
Marcus Märtens
@CoolRunning
May 18 2016 20:47
Okay, will also try this.
Thing is, the whole blob will probably explode over my head as soon as I try. I changed and ported like 1000 lines of code at once... :D
Anyway, talk to you later