These are chat archives for bluescarni/pagmo_reborn

1st
Jun 2016
Francesco Biscani
@bluescarni
Jun 01 2016 08:30
it's merged
I am starting work on the python side algorithm, hopefully I'll have more time today
Francesco Biscani
@bluescarni
Jun 01 2016 10:42
vector_double::size_type best_idx(double tol = 0.) these methods in population don't seem to do any check on the signedness of tol
is this a problem?
Francesco Biscani
@bluescarni
Jun 01 2016 11:05
In [1]: import pygmo

In [2]: pop = pygmo.population(size=5, p = pygmo.rosenbrock())

In [3]: pop
Out[3]: 
Problem name: Multidimensional Rosenbrock Function
        Global dimension:                       2
        Fitness dimension:                      1
        Number of objectives:                   1
        Equality constraints dimension:         0
        Inequality constraints dimension:       0
        Lower bounds: [-5, -5]
        Upper bounds: [10, 10]

        Has gradient: false
        User implemented gradient sparsity: false
        Has hessians: false
        User implemented hessians sparsity: false

        Function evaluations: 5

Population size: 5

List of individuals: 
#0:
        ID:                     2609652657959607412
        Decision vector:        [7.56848, 8.78272]
        Fitness vector:         [235260]
#1:
        ID:                     5442405231392028002
        Decision vector:        [4.1504, 6.64168]
        Fitness vector:         [11212.4]
#2:
        ID:                     17439113676494910963
        Decision vector:        [-0.124807, 3.81581]
        Fitness vector:         [1445.44]
#3:
        ID:                     472339684928445373
        Decision vector:        [8.48101, -2.59768]
        Fitness vector:         [555458]
#4:
        ID:                     11753646393074009411
        Decision vector:        [3.92593, 3.04497]
        Fitness vector:         [15305.1]

In [4]: algo = pygmo.algorithm(pygmo.de(gen = 100, variant = 4))

In [5]: pop = algo.evolve(pop)

In [6]: pop
Out[6]: 
Problem name: Multidimensional Rosenbrock Function
        Global dimension:                       2
        Fitness dimension:                      1
        Number of objectives:                   1
        Equality constraints dimension:         0
        Inequality constraints dimension:       0
        Lower bounds: [-5, -5]
        Upper bounds: [10, 10]

        Has gradient: false
        User implemented gradient sparsity: false
        Has hessians: false
        User implemented hessians sparsity: false

        Function evaluations: 505

Population size: 5

List of individuals: 
#0:
        ID:                     2609652657959607412
        Decision vector:        [1.10108, 1.20792]
        Fitness vector:         [0.0122044]
#1:
        ID:                     5442405231392028002
        Decision vector:        [1.09597, 1.20494]
        Fitness vector:         [0.0106446]
#2:
        ID:                     17439113676494910963
        Decision vector:        [1.12403, 1.2503]
        Fitness vector:         [0.0326548]
#3:
        ID:                     472339684928445373
        Decision vector:        [1.07761, 1.17103]
        Fitness vector:         [0.0156027]
#4:
        ID:                     11753646393074009411
        Decision vector:        [1.09597, 1.20494]
        Fitness vector:         [0.0106446]

In [7]:
we have the first python evolution
Francesco Biscani
@bluescarni
Jun 01 2016 11:13
I was thinking that the verbosity output might not be seen from python, depending on how you access the python bindings
it works from straight ipython because you are working in a console and thus you see the output of cout
but I doubt it'll work from the notebook for instance
anyway I need to do some work work work shit work this afternoon, but it's good progress. Still todo:
  • finish the population exposition (this requires some extra code to extract a vector of decision vectors or fitnesses as a numpy 2d array - or matrix if you prefer)
  • expose more algorithms/problems (easy)
  • properly document all exposed methods/classes (decisions to make here)
  • testing (boring!!!!!)
Dario Izzo
@darioizzo
Jun 01 2016 11:44
Cool!!!!
love it
Marcus Märtens
@CoolRunning
Jun 01 2016 11:45
Looks great!
Dario Izzo
@darioizzo
Jun 01 2016 11:45
On the verbosity: I guess that if you do work in jupyter you do not want screen output. Also, the results of screen output are stored in the log too and can be accessed after
Marcus Märtens
@CoolRunning
Jun 01 2016 11:45
I vote for np.array rather than np.matrix
Dario Izzo
@darioizzo
Jun 01 2016 11:46
On vector_double::size_type best_idx(double tol = 0.), I had noticed back then and decided not to check. If the tol in negative, simply, all will be unfeasible.
on np.array vs np.matrix .... I have no opinion, not a great user of these numpy data structures
Francesco Biscani
@bluescarni
Jun 01 2016 11:49
I was talking only about np.array, never used the matrix stuff
Dario Izzo
@darioizzo
Jun 01 2016 11:52
So, de does not solve rosenbrock 2D in 500 evaluations ..... :)
Can you show me the output if you do de?
I mean the docstring
Francesco Biscani
@bluescarni
Jun 01 2016 11:58
the docstring?
Dario Izzo
@darioizzo
Jun 01 2016 11:58
de?
what does it do?
Francesco Biscani
@bluescarni
Jun 01 2016 11:59
de gives you the repr, not the docstring
Dario Izzo
@darioizzo
Jun 01 2016 11:59
and can I see it?
Francesco Biscani
@bluescarni
Jun 01 2016 11:59
ah you mean de? from python
Dario Izzo
@darioizzo
Jun 01 2016 11:59
from ipython
Francesco Biscani
@bluescarni
Jun 01 2016 11:59
In [8]: pygmo.de?
Docstring:
__init__(gen = 1, F = 0.8, CR = 0.9, variant = 2, ftol = 1e-6, tol = 1e-6, seed = random)

Differential evolution algorithm.
File:      ~/.local/lib64/python3.4/site-packages/pygmo/core.so
Type:      class
Dario Izzo
@darioizzo
Jun 01 2016 12:00
SO all the docs from c++ are "lost" ... how are we going to dealt with it?
Is there a way to recover them?
Or we need to rewrite?
Francesco Biscani
@bluescarni
Jun 01 2016 12:01
not directly, the C++ docs are comments and you cannot access comments in any shape or form from C++ code
Dario Izzo
@darioizzo
Jun 01 2016 12:01
I was hoping on some new boost magic
Francesco Biscani
@bluescarni
Jun 01 2016 12:02
It's a language thing - but in any case we would need to rewrite them if we want to do them properly
at least the param docs, the types, the exceptions raised etc.
Dario Izzo
@darioizzo
Jun 01 2016 12:02
can we exploit the doxygen parse?
Francesco Biscani
@bluescarni
Jun 01 2016 12:02
the only thing you can salvage is the method description
Dario Izzo
@darioizzo
Jun 01 2016 12:02
I like the signature
__init__(gen = 1, F = 0.8, CR = 0.9, variant = 2, ftol = 1e-6, tol = 1e-6, seed = random)
Francesco Biscani
@bluescarni
Jun 01 2016 12:03
that needs to be inputted by hand when exposing
Dario Izzo
@darioizzo
Jun 01 2016 12:03
:)
pity I was wondering how he would have dealt with the static member defaults
Francesco Biscani
@bluescarni
Jun 01 2016 12:03
if we want to do things properly, we need to replicate and adapt largely the C++ docs
we can always refer to the cpp docs from the docstring btw
it just works when you run it through sphinx
Dario Izzo
@darioizzo
Jun 01 2016 12:04
Ya, but I guess we must have, as a minimum, in the docstring some docs on what are the arguments and what they do
Francesco Biscani
@bluescarni
Jun 01 2016 12:04
yep
of course we could come up with some complicated scheme in which we write the doxygen comments as macros, so we can re-use them from boost.python
that would be a nightmare
Dario Izzo
@darioizzo
Jun 01 2016 12:05
probably no ideal
would force us to document c++ in a weird hybrid way
Francesco Biscani
@bluescarni
Jun 01 2016 12:06
you would need to parse them, remove the * and // characters, translate the param types to corresponding python types..
no thanks
Dario Izzo
@darioizzo
Jun 01 2016 12:06
SO we just rewrite the docstring
one by one
Francesco Biscani
@bluescarni
Jun 01 2016 12:07
I guess it depends on what role you want to give the Python docs.. the way I see it:
  • right now things work decently from the point of view of the generated sphinx docs.. we have shorte r descriptions on ther python classes, with hyperlinks to the C++ docs in case the user wants more
  • if we want the python docstring to be useful interactively, they cannot be too long anyway
to me it's not a big deal frankly
Dario Izzo
@darioizzo
Jun 01 2016 12:08
agree,.
I was just asking to envision the future :)
And we can refer to sphinx in the docstring you said right?
Francesco Biscani
@bluescarni
Jun 01 2016 12:09
inline std::string population_docstring()
{
    return R"(__init__(p, size = 0, seed = random)

The population class.

See also :cpp:class:`pagmo::population`.

)";
}
Dario Izzo
@darioizzo
Jun 01 2016 12:10
which in sphinx will create a hyperlink. in ipython would just show as is
k
whats the R?
also p should this be pygmo.problem ?
Francesco Biscani
@bluescarni
Jun 01 2016 12:21
so the "R" stuff is C++11 raw string literals
p is the name of the construction argument, not its type
its type can be any C++ or Python user-defined problem
Dario Izzo
@darioizzo
Jun 01 2016 12:21
but it is uninformative in the docstring ... maybe call it problem ?
Francesco Biscani
@bluescarni
Jun 01 2016 12:22
it's a balance between shortness of typing and info... maybe prob?
Dario Izzo
@darioizzo
Jun 01 2016 12:22
deal!
Francesco Biscani
@bluescarni
Jun 01 2016 12:22
just so we do it consistently :)
prob and algo it is then
yeah the hyperlink would only be in sphinx
inline std::string rosenbrock_docstring()
{
    return R"(__init__(dim = 2)

The Rosenbrock problem.

:param dim: number of dimensions
:type dim: ``int``
:raises: :exc:`OverflowError` if *dim* is negative or greater than an implementation-defined value
:raises: :exc:`ValueError` if *dim* is less than 2

See :cpp:class:`pagmo::rosenbrock`.

)";
}
another example, here with the complete description of signature and exception raising, in sphinx syntax
Dario Izzo
@darioizzo
Jun 01 2016 12:25
I use more ipython than sphinx ... I am wondering if its not better to type directly in the style we would like to see there ...
Francesco Biscani
@bluescarni
Jun 01 2016 12:27
there's a few alternatives for the sphinx syntax... as I said, decisions :)
the docstring syntax I mean
Francesco Biscani
@bluescarni
Jun 01 2016 12:45
extracting fitness vectors as a 2d array from the pop allows you to quickly get out the feasibility stuff with something like:
>>> pop.get_f()[:,3:5]
if I recall correctly the numpy syntax anyway
Dario Izzo
@darioizzo
Jun 01 2016 12:45
nice
Marcus Märtens
@CoolRunning
Jun 01 2016 12:49
Double-writing the documentation is annoying though... especially since writing documentation is annoying the first place already, though of course necessary... :neutral_face:
Francesco Biscani
@bluescarni
Jun 01 2016 12:50
it is, but I don't see viable alternatives
Marcus Märtens
@CoolRunning
Jun 01 2016 12:50
The slicing with numpy is very useful. I find myself using it more often these days as I grow fond of np.array
Dario Izzo
@darioizzo
Jun 01 2016 12:51
Copied from Matlab to be fair .. :)
Marcus Märtens
@CoolRunning
Jun 01 2016 12:51
It is, true.
Probably the reason, why I had problems with it for such a long time :D
But also matplotlib is a rip-off from Matlab.
Francesco Biscani
@bluescarni
Jun 01 2016 12:52
I am pretty sure matlab is a ripoff of something else still
Marcus Märtens
@CoolRunning
Jun 01 2016 12:52
Which is, why this statemachine is sometimes a bit akward.
Dario Izzo
@darioizzo
Jun 01 2016 12:52
But it slowly diverged .... I think now its more powerful
Marcus Märtens
@CoolRunning
Jun 01 2016 12:52
Isn't Matlab not only some sugarcoating over BLAS or so?
Francesco Biscani
@bluescarni
Jun 01 2016 12:52
it all comes from fortran ultimately :)
Dario Izzo
@darioizzo
Jun 01 2016 12:52
Yes, but we are talking about the notation
they may have invented a[1:4, 3:4]
Its MATrix LABoratory after all
Francesco Biscani
@bluescarni
Jun 01 2016 12:53
https://docs.oracle.com/cd/E19205-01/819-5257/blagu/index.html slicing is in fortran as well apparently
Dario Izzo
@darioizzo
Jun 01 2016 12:53
vs. FORmula TRANslation
Marcus Märtens
@CoolRunning
Jun 01 2016 12:54
There it is - Fortran... well, it brought us to the moon, how bad can it be? ^^
Francesco Biscani
@bluescarni
Jun 01 2016 12:56
I've said it before... python is the modern fortran
at least as far as scientists are concerned :)
Marcus Märtens
@CoolRunning
Jun 01 2016 12:57
It is sort of funny that we developed programming languages that are actually much slower than the old shit
I guess it is about Pareto-optimality also when it comes to them, but would those FORTRAN people have guessed something like Python?
Francesco Biscani
@bluescarni
Jun 01 2016 12:58
as long as you are manipulating arrays, numpy is not too bad.. and the selling point has always been ease of use and speed of iteration with python
Marcus Märtens
@CoolRunning
Jun 01 2016 12:58
I know, but still I would assume that you would try to get more and more efficient (=faster) languages
But I guess FORTRAN, Assembler, C or so are at the cap of what is possible.
At least along this dimension.
Francesco Biscani
@bluescarni
Jun 01 2016 12:59
people still use compiled languages when speed is the most important metric
and if you use C++ you can get nice syntax and max speed
as long as you are up for spending years to learn it that is :)
Marcus Märtens
@CoolRunning
Jun 01 2016 13:00
I just started this journey... I wonder when the regret will turn into bliss
Francesco Biscani
@bluescarni
Jun 01 2016 13:01
good luck, I don't miss the learning times :)
Marcus Märtens
@CoolRunning
Jun 01 2016 13:01
But thinking about types and signs of integers feels so primitive and time-consuming.
Francesco Biscani
@bluescarni
Jun 01 2016 13:02
it's not really, type safety is one of the best things about C++
it's so good that python will adopt it as an opt-in
Marcus Märtens
@CoolRunning
Jun 01 2016 13:03
What did type safety ever do for us? :fearful:
Francesco Biscani
@bluescarni
Jun 01 2016 13:04
code that is not brittle? :)
Marcus Märtens
@CoolRunning
Jun 01 2016 13:05
Still, we are rewriting stuff ^^
But ignore my whinings... I see the point.
So far it hasn't been too bad, though I am also not learning much right now, as this hypervolume stuff is mostly copy&paste work.
Dario Izzo
@darioizzo
Jun 01 2016 13:07
Porting code is horrible ... to learn some of the coolness of the new c++ the best would be to develop from sctartch something ....
Marcus Märtens
@CoolRunning
Jun 01 2016 13:11
Yes, I started some toy-programming as well, but this is even more behind than the porting :D
Anyway, it will come.
I was thinking whether I should rewrite a simulator for epidemics that was done in this group.
It is a piece of Java, which I sometimes touch carefully with a stick... though I would love to see this in C++ with a Python interface.
But maybe that backfires... I am better sticking to my random formulas.
Francesco Biscani
@bluescarni
Jun 01 2016 13:29
for the docstring style, I'd like to have something that also looks good on sphinx. I personally spend more time on the scipy website when I need to read about numpy or ode or whatver than in the command line
I can add a few docstrings in different styles to the bindings and then we can pick one
Dario Izzo
@darioizzo
Jun 01 2016 13:37
ok good ..
how about this
Dario Izzo
@darioizzo
Jun 01 2016 13:41
what about it?
Francesco Biscani
@bluescarni
Jun 01 2016 13:41
does it satisfy your refined fashion sense regarding interactive usage? :)
Dario Izzo
@darioizzo
Jun 01 2016 13:42
not really, I never liked it. But I am a weird guy
Francesco Biscani
@bluescarni
Jun 01 2016 13:42
what would you like then? any example?
Dario Izzo
@darioizzo
Jun 01 2016 13:43
In [2]: algorithm.de?
Init signature: algorithm.de(self, gen=100, f=0.8, cr=0.9, variant=2, ftol=1e-06, xtol=1e-06, screen_output=False)
Docstring:      Differential evolution algorithm.
Init docstring:
Constructs a Differential Evolution algorithm:

USAGE: algorithm.de(gen=1, f=0.5, cr=0.9, variant=2, ftol=1e-6, xtol=1e-6, screen_output = False)

* gen: number of generations
* f: weighting factor in [0,1] (if -1 self-adptation is used)
* cr: crossover in [0,1] (if -1 self-adptation is used)
* variant: algoritmic variant to use (one of [1 .. 10])
        1. DE/best/1/exp
        2. DE/rand/1/exp
        3. DE/rand-to-best/1/exp
        4. DE/best/2/exp
        5. DE/rand/2/exp
        6. DE/best/1/bin
        7. DE/rand/1/bin
        8. DE/rand-to-best/1/bin
        9. DE/best/2/bin
        10. DE/rand/2/bin
* ftol stop criteria on f
* xtol stop criteria on x
File:           /usr/lib/python3.5/site-packages/PyGMO/algorithm/_algorithm.so
Type:           class
Here only the types are missing ...
could be added easily gen: (int) number of generations
Francesco Biscani
@bluescarni
Jun 01 2016 13:49
it needs to be parsable by sphinx as well
Dario Izzo
@darioizzo
Jun 01 2016 13:49
This one is ... see the link
Francesco Biscani
@bluescarni
Jun 01 2016 13:50
it does not seem to me it is identifying any role for the text.. it's just a piece of markdown which is displayed as-is
Dario Izzo
@darioizzo
Jun 01 2016 13:50
yes
Francesco Biscani
@bluescarni
Jun 01 2016 13:50
and the variant: algoritmic variant to use (one of [1 .. 10]) is also displayed wierdly it seems to me
Dario Izzo
@darioizzo
Jun 01 2016 13:50
thats a typo
sphinx recognizing the text having some role is a plus only if we like what he makes of it ...
which I personally do not ... anyway I do not have strong opinions on this, I just would like the output to be as human readable as possible ....
Francesco Biscani
@bluescarni
Jun 01 2016 13:52
how the rosenbrock docstring I pasted above is rendered in sphinx is different, in the sense that for instance it groups together on the same line the name of the function argument and its type, giving specific predictable styling to each
that's why I linked to the scipy stuff, because it looks more human-friendly and (I assume) it is also sphinx friendly
but maybe not, who knows
Dario Izzo
@darioizzo
Jun 01 2016 13:53
Well :dim:, :raise: is not really human readable syntax right?
Its more a computer parsing syntax
And makes sense if rendered in sphinx ...
outside? less so
Francesco Biscani
@bluescarni
Jun 01 2016 13:54
I am used to parsing C++ template metaprogramming, so I might be a bit biased
:)
but the point still stands, I think we need not to sacrifice either output
Dario Izzo
@darioizzo
Jun 01 2016 13:55
The scipy thing also is not recognizing roles for text right?
Its just a wall of text ....
Francesco Biscani
@bluescarni
Jun 01 2016 13:55
I am not sure, if you see how it is rendered it seems so http://docs.scipy.org/doc/scipy-0.17.0/reference/generated/scipy.integrate.odeint.html
but I am not 100% confident it is sphinx generated
in any case there's more than 1 type of sphinx syntax you can have in the docstring
Dario Izzo
@darioizzo
Jun 01 2016 13:56
I think you are saying the same thing as I am ....
Francesco Biscani
@bluescarni
Jun 01 2016 13:56
y0 : array look how this is rendered: y0 is bold, array italic
without any indication in the original docstring
Dario Izzo
@darioizzo
Jun 01 2016 13:57
The scipy stuff is basically what I also posted above for de. There is some REST use (asterisx headings bold etc.) but not any kind of syntax that is parsable from sphinx and rendered ugly
Francesco Biscani
@bluescarni
Jun 01 2016 13:57
no it's not... in the scipy syntax sphinx recognizes that y0 is an argument, and array a type
Dario Izzo
@darioizzo
Jun 01 2016 13:58
No its just a list I think and its rendered accordingly and conveniently
look here
Dario Izzo
@darioizzo
Jun 01 2016 13:58
Let me try to render that in a non sphinx env ..
So they are saying the same discussion ..
one line of thinking is that the sphinx markup shouldn’t kill the docstring that a user might read through the __doc__ attribute
Francesco Biscani
@bluescarni
Jun 01 2016 14:01
yep
Dario Izzo
@darioizzo
Jun 01 2016 14:01
I like the google style stuff
Francesco Biscani
@bluescarni
Jun 01 2016 14:03
so apprently sphinx does not parse specially the googley stuff, according to this: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/
we might consider using it
As of Sphinx 1.3, the napoleon extension will come packaged with Sphinx under sphinx.ext.napoleon. The sphinxcontrib.napoleon extension will continue to work with Sphinx <= 1.2. oh cool
so it's builtin now
Dario Izzo
@darioizzo
Jun 01 2016 14:04
lets go for it
it looks the solution we seek
Francesco Biscani
@bluescarni
Jun 01 2016 14:04
Dario Izzo
@darioizzo
Jun 01 2016 14:04
Are you tired of writing docstrings that look like this:
:)
googley which is what I actually wrote above for de :)
Francesco Biscani
@bluescarni
Jun 01 2016 14:05
it's not but ok :)
Dario Izzo
@darioizzo
Jun 01 2016 14:05
it is
no its not
but it is
no
Francesco Biscani
@bluescarni
Jun 01 2016 14:05
lol
Dario Izzo
@darioizzo
Jun 01 2016 14:05
yes
well at least our discussions are also made by others ... which make me think they are less pointless than expected
Francesco Biscani
@bluescarni
Jun 01 2016 14:06
I never start pointless discussions ;)
Francesco Biscani
@bluescarni
Jun 01 2016 14:12
inline std::string rosenbrock_docstring()
{
    return R"(__init__(dim = 2)

The Rosenbrock problem.

Args:
    dim (int): number of dimensions

Raises:
    OverflowError: if `dim` is negative or greater than an implementation-defined value
    ValueError: if `dim` is less than 2

See :cpp:class:`pagmo::rosenbrock`.

)";
this gets parsed nicely now by sphinx as well
Dario Izzo
@darioizzo
Jun 01 2016 14:12
yeah!!!
Francesco Biscani
@bluescarni
Jun 01 2016 14:13
In [1]: import pygmo

In [2]: pygmo.rosenbrock?
Docstring:
__init__(dim = 2)

The Rosenbrock problem.

Args:
    dim (int): number of dimensions

Raises:
    OverflowError: if `dim` is negative or greater than an implementation-defined value
    ValueError: if `dim` is less than 2

See :cpp:class:`pagmo::rosenbrock`.
File:      ~/.local/lib64/python3.4/site-packages/pygmo/core.so
Type:      class

In [3]:
Dario Izzo
@darioizzo
Jun 01 2016 14:13
love it
Francesco Biscani
@bluescarni
Jun 01 2016 14:14
it similar to doxygen ain'it
Dario Izzo
@darioizzo
Jun 01 2016 14:14
and for population what would it look like?
yes
Francesco Biscani
@bluescarni
Jun 01 2016 14:15
I need to adapt it
inline std::string population_docstring()
{
    return R"(__init__(prob, size = 0, seed = random)

The population class.

Args:
    prob: a user-defined problem
    size (int): the number of individuals
    seed (int): the random seed (if not specified, randomly-generated)

See also :cpp:class:`pagmo::population`.

)";
something like that
Dario Izzo
@darioizzo
Jun 01 2016 14:18
maybe explaining what a user-defined problem is? like a link to the page where its requirements will be documented?
Francesco Biscani
@bluescarni
Jun 01 2016 14:18
when the page exists we can add the link :)
http://imgur.com/iIvBsgl that's how the sphinx generated output looks for rosenbrok
Dario Izzo
@darioizzo
Jun 01 2016 14:22
done
Francesco Biscani
@bluescarni
Jun 01 2016 14:23
I also added the problems/algorithms submodules fuck yeah
Dario Izzo
@darioizzo
Jun 01 2016 14:24
e dajeeeee
Francesco Biscani
@bluescarni
Jun 01 2016 14:24
In [3]: pygmo.problems.
pygmo.problems.hock_schittkowsky_71  pygmo.problems.null_problem          pygmo.problems.rosenbrock            pygmo.problems.translate

In [3]: pygmo.problems.
Dario Izzo
@darioizzo
Jun 01 2016 14:24
legen ... wait for it ....
DARYYYY
Francesco Biscani
@bluescarni
Jun 01 2016 14:24
the high five to end all high fives
the highest of fives
Dario Izzo
@darioizzo
Jun 01 2016 14:25
problems or problem ?
in PaGMO legacy its problem
Francesco Biscani
@bluescarni
Jun 01 2016 14:25
problem would clash with the problem class
Dario Izzo
@darioizzo
Jun 01 2016 14:25
auch
so its pygmo.problem(pygmo.problems.ciccio(2))
Francesco Biscani
@bluescarni
Jun 01 2016 14:29
pygmo.problem(pygmo.ciccio(2))
the problems submodules contains just links to the classes
Dario Izzo
@darioizzo
Jun 01 2016 14:29
ah ok ...
Francesco Biscani
@bluescarni
Jun 01 2016 15:56
so is it intended that a population can be constructed both from a problem and a user-defined problem?
constructor from problem -> creates a copy of the input problem into the population
constructor from user-defined problem (UDP) -> creates a problem in the population from the supplied UDP
Francesco Biscani
@bluescarni
Jun 01 2016 16:19
actually I think I'd prefer forcing the constructor from problem only on the python side
the reason is the following: for each C++ problem we expose, we need to add a constructor to population templated on that C++ type
that is what I am doing now anyway
but imagine once we get to island, which will have a ctor similar to
template <typename Prob, typename Algo>
island(const Prob &p, const Algo &a, ...)
this would mean that on the Python side each time we add an exposed C++ problem, we need to expose nn new constructors of island, where nn is the total number of exposed C++ algorithms
or maybe for the island/population ctor we need to write a small wrapper directly in Python that does the construction of an intermediate problem/algorithm, so that in C++ we just need to expose the ctor from problem/algorithm
maybe this is the way to go
Dario Izzo
@darioizzo
Jun 01 2016 17:28
The reason for the two overloads is to allow the syntax: population(ackley(5u), 20u) as well as population(problem(ackley(5u)), 20)
For the island, the constructor may only work on population, algorithm, ... wiuld this solve the issue?
Francesco Biscani
@bluescarni
Jun 01 2016 17:49
I think hijacking the constructor is the way to go
Francesco Biscani
@bluescarni
Jun 01 2016 20:31
alright I committed it
Dario Izzo
@darioizzo
Jun 01 2016 20:31
what?
Francesco Biscani
@bluescarni
Jun 01 2016 20:32
the implementation of the pop ctor from python
Dario Izzo
@darioizzo
Jun 01 2016 20:32
in your branch?
Francesco Biscani
@bluescarni
Jun 01 2016 20:32
yep
Dario Izzo
@darioizzo
Jun 01 2016 20:33
and? which one is it at the end?
Francesco Biscani
@bluescarni
Jun 01 2016 20:33
Dario Izzo
@darioizzo
Jun 01 2016 20:34
how cool is this coming out?
Francesco Biscani
@bluescarni
Jun 01 2016 20:35
it's friggin great
look at this beauty:
In [1]: import pygmo

In [2]: pop = pygmo.population(size=5, prob = 4, seed = -4)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-55e97e1fdb40> in <module>()
----> 1 pop = pygmo.population(size=5, prob = 4, seed = -4)

/home/yardbird/.local/lib64/python3.4/site-packages/pygmo/__init__.py in _population_init(self, prob, size, seed)
    129         # Otherwise, we attempt to create a problem from it. This will
    130         # work if prob is an exposed C++ problem or a Python UDP.
--> 131         prob_arg = problem(prob)
    132     if seed is None:
    133         __original_population_init(self,prob_arg,size)

TypeError: the mandatory 'fitness()' method is missing from the user-defined Python problem '4' of type '<class 'int'>'
dario-izzo-compliant error message
Dario Izzo
@darioizzo
Jun 01 2016 20:36
agree
whats this overflow error when negative numbers are passed to unsigned?
Francesco Biscani
@bluescarni
Jun 01 2016 20:38
it's the type of error generated by Boost.Python when you pass negative python ints to a C++ function which takes unsigned
Dario Izzo
@darioizzo
Jun 01 2016 20:38
k, and whats the error message if:
Francesco Biscani
@bluescarni
Jun 01 2016 20:38
In [3]: pop = pygmo.population(size=5, prob = pygmo.translate(pygmo.rosenbrock(),[1,2]), seed = -3)
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
<ipython-input-3-af07af7fcd45> in <module>()
----> 1 pop = pygmo.population(size=5, prob = pygmo.translate(pygmo.rosenbrock(),[1,2]), seed = -3)

/home/yardbird/.local/lib64/python3.4/site-packages/pygmo/__init__.py in _population_init(self, prob, size, seed)
    133         __original_population_init(self,prob_arg,size)
    134     else:
--> 135         __original_population_init(self,prob_arg,size,seed)
    136 
    137 setattr(population,"__init__",_population_init)

OverflowError: can't convert negative value to unsigned int
Dario Izzo
@darioizzo
Jun 01 2016 20:39
class ciccio:
    def fitness (self, x, y):
        return x*y
Francesco Biscani
@bluescarni
Jun 01 2016 20:39
that's gonna be ugly
Dario Izzo
@darioizzo
Jun 01 2016 20:39
pop = pygmo.population(size=5, prob = ciccio, seed = -4)
try
me
Francesco Biscani
@bluescarni
Jun 01 2016 20:40
In [7]: class prob(object):
    def fitness(self,v,vf):
        return [v[0]*v[0]]
    def get_bounds(self):
        return ([0],[1])
   ...:     

In [8]: pop = pygmo.population(size=5, prob = prob(), seed = 3)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-fc0f16ce3c3d> in <module>()
----> 1 pop = pygmo.population(size=5, prob = prob(), seed = 3)

/home/yardbird/.local/lib64/python3.4/site-packages/pygmo/__init__.py in _population_init(self, prob, size, seed)
    133         __original_population_init(self,prob_arg,size)
    134     else:
--> 135         __original_population_init(self,prob_arg,size,seed)
    136 
    137 setattr(population,"__init__",_population_init)

TypeError: fitness() missing 1 required positional argument: 'vf'

In [9]:
Dario Izzo
@darioizzo
Jun 01 2016 20:41
wow ... slightly misleading :)
fitness() missing 1 required positional argument: 'vf'
yeah right thats the problem
Francesco Biscani
@bluescarni
Jun 01 2016 20:41
I had some code that tried to do better than that, but it was problematic
Dario Izzo
@darioizzo
Jun 01 2016 20:41
ANd what is the error if:
Francesco Biscani
@bluescarni
Jun 01 2016 20:42
the issue is that you risk of catching TypeErrors which might be generated inside the fitness function
Dario Izzo
@darioizzo
Jun 01 2016 20:42
class ciccio(object):
    def fitness (self, x):
        return x
Francesco Biscani
@bluescarni
Jun 01 2016 20:42
and producing error messages that are even worse
I think the only solution would be to inspect the signature of the fitness() function somehow
that one works, if the bounds have size 1
Dario Izzo
@darioizzo
Jun 01 2016 20:43
really? cool
how? x is no array
Francesco Biscani
@bluescarni
Jun 01 2016 20:44
x could be anything
Dario Izzo
@darioizzo
Jun 01 2016 20:44
ah right so:
class ciccio(object):
    def fitness (self, x):
        return x[0]
I meant this
Francesco Biscani
@bluescarni
Jun 01 2016 20:45
In [11]: class prob(object):
    def fitness(self,v):
        return v[0]
    def get_bounds(self):
        return ([0],[1])
   ....: 

In [12]: pop = pygmo.population(size=5, prob = prob(), seed = 3)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-fc0f16ce3c3d> in <module>()
----> 1 pop = pygmo.population(size=5, prob = prob(), seed = 3)

/home/yardbird/.local/lib64/python3.4/site-packages/pygmo/__init__.py in _population_init(self, prob, size, seed)
    133         __original_population_init(self,prob_arg,size)
    134     else:
--> 135         __original_population_init(self,prob_arg,size,seed)
    136 
    137 setattr(population,"__init__",_population_init)

TypeError: cannot convert the type '<class 'numpy.float64'>' to a vector of doubles: only lists of doubles and NumPy arrays of doubles are supported
Dario Izzo
@darioizzo
Jun 01 2016 20:46
decent
and why the previous one would work only with bounds of dim 1?
Francesco Biscani
@bluescarni
Jun 01 2016 20:46
that one I wrote myself :) too bad it uses numpy.float64 instead of just float
so what happens is that the population generates random decision vectors of size 1, because that's the bounds size
Dario Izzo
@darioizzo
Jun 01 2016 20:47
I mean if I define nobj =3 shoud also work for bounds of dim 3 right?
Francesco Biscani
@bluescarni
Jun 01 2016 20:47
it should yeah
Dario Izzo
@darioizzo
Jun 01 2016 20:47
this is super cool, I am wetting myself
Francesco Biscani
@bluescarni
Jun 01 2016 20:47
I know right
Dario Izzo
@darioizzo
Jun 01 2016 20:48
did you try to write a rosenbrock in python and benchmark it vs the one in c++ ?
I mean what is the penalty of defining a problem directly in python?
In a simple case where just a few mult and divs are made?
Francesco Biscani
@bluescarni
Jun 01 2016 20:49
In [15]: pop = pygmo.population(size=5, prob = translate(translate(prob(),[1]),[-1]), seed = 3)

In [16]: pop
Out[16]: 
Problem name: <class '__main__.prob'> [translated] [translated]
        Global dimension:                       1
        Fitness dimension:                      1
        Number of objectives:                   1
        Equality constraints dimension:         0
        Inequality constraints dimension:       0
        Lower bounds: [0]
        Upper bounds: [1]

        Has gradient: false
        User implemented gradient sparsity: false
        Has hessians: false
        User implemented hessians sparsity: false

        Function evaluations: 5

Extra info:

        Translation Vector: [1]
        Translation Vector: [-1]
Population size: 5

List of individuals: 
#0:
        ID:                     13063021645730866819
        Decision vector:        [0.0707249]
        Fitness vector:         [0.0707249]
#1:
        ID:                     9423106023066854165
        Decision vector:        [0.121329]
        Fitness vector:         [0.121329]
#2:
        ID:                     16533689267043216235
        Decision vector:        [0.437062]
        Fitness vector:         [0.437062]
#3:
        ID:                     3822956344058944405
        Decision vector:        [0.0406307]
        Fitness vector:         [0.0406307]
#4:
        ID:                     8131506278483621524
        Decision vector:        [0.0935515]
        Fitness vector:         [0.0935515]
translate translate translate
Dario Izzo
@darioizzo
Jun 01 2016 20:49
coatto
Francesco Biscani
@bluescarni
Jun 01 2016 20:49
I am not at the point of doing benchmarks yet
but it's going to be quite a bit slower
Dario Izzo
@darioizzo
Jun 01 2016 20:50
Would be nice to know how much.
I am tempted to write all racing, ert etc. directly in python
Francesco Biscani
@bluescarni
Jun 01 2016 20:50
eventually... I have not even compiled it yet in non-debug mode :)
but then you can't use those from C++ can you?
Dario Izzo
@darioizzo
Jun 01 2016 20:51
no, who cares?
right?
the sheer elegance of this syntax ....
essential, to the point .... just just beautiful
Francesco Biscani
@bluescarni
Jun 01 2016 20:51
C++ is dying indeed java rust will kill it
java go rust will kill it
Francesco Biscani
@bluescarni
Jun 01 2016 20:53
I heard that before... basically modern C++ plus compile with -Werror -Weverything
there ya go