These are chat archives for bluescarni/pagmo_reborn

1st
Jun 2016
Francesco Biscani
@bluescarni
Jun 01 2016 08:30 UTC
it's merged
I am starting work on the python side algorithm, hopefully I'll have more time today
Francesco Biscani
@bluescarni
Jun 01 2016 10:42 UTC
vector_double::size_type best_idx(double tol = 0.) these methods in population don't seem to do any check on the signedness of tol
is this a problem?
Francesco Biscani
@bluescarni
Jun 01 2016 11:05 UTC
In [1]: import pygmo

In [2]: pop = pygmo.population(size=5, p = pygmo.rosenbrock())

In [3]: pop
Out[3]:
Problem name: Multidimensional Rosenbrock Function
Global dimension:                       2
Fitness dimension:                      1
Number of objectives:                   1
Equality constraints dimension:         0
Inequality constraints dimension:       0
Lower bounds: [-5, -5]
Upper bounds: [10, 10]

Has hessians: false
User implemented hessians sparsity: false

Function evaluations: 5

Population size: 5

List of individuals:
#0:
ID:                     2609652657959607412
Decision vector:        [7.56848, 8.78272]
Fitness vector:         [235260]
#1:
ID:                     5442405231392028002
Decision vector:        [4.1504, 6.64168]
Fitness vector:         [11212.4]
#2:
ID:                     17439113676494910963
Decision vector:        [-0.124807, 3.81581]
Fitness vector:         [1445.44]
#3:
ID:                     472339684928445373
Decision vector:        [8.48101, -2.59768]
Fitness vector:         [555458]
#4:
ID:                     11753646393074009411
Decision vector:        [3.92593, 3.04497]
Fitness vector:         [15305.1]

In [4]: algo = pygmo.algorithm(pygmo.de(gen = 100, variant = 4))

In [5]: pop = algo.evolve(pop)

In [6]: pop
Out[6]:
Problem name: Multidimensional Rosenbrock Function
Global dimension:                       2
Fitness dimension:                      1
Number of objectives:                   1
Equality constraints dimension:         0
Inequality constraints dimension:       0
Lower bounds: [-5, -5]
Upper bounds: [10, 10]

Has hessians: false
User implemented hessians sparsity: false

Function evaluations: 505

Population size: 5

List of individuals:
#0:
ID:                     2609652657959607412
Decision vector:        [1.10108, 1.20792]
Fitness vector:         [0.0122044]
#1:
ID:                     5442405231392028002
Decision vector:        [1.09597, 1.20494]
Fitness vector:         [0.0106446]
#2:
ID:                     17439113676494910963
Decision vector:        [1.12403, 1.2503]
Fitness vector:         [0.0326548]
#3:
ID:                     472339684928445373
Decision vector:        [1.07761, 1.17103]
Fitness vector:         [0.0156027]
#4:
ID:                     11753646393074009411
Decision vector:        [1.09597, 1.20494]
Fitness vector:         [0.0106446]

In [7]:
we have the first python evolution
Francesco Biscani
@bluescarni
Jun 01 2016 11:13 UTC
I was thinking that the verbosity output might not be seen from python, depending on how you access the python bindings
it works from straight ipython because you are working in a console and thus you see the output of cout
but I doubt it'll work from the notebook for instance
anyway I need to do some work work work shit work this afternoon, but it's good progress. Still todo:
• finish the population exposition (this requires some extra code to extract a vector of decision vectors or fitnesses as a numpy 2d array - or matrix if you prefer)
• expose more algorithms/problems (easy)
• properly document all exposed methods/classes (decisions to make here)
• testing (boring!!!!!)
Dario Izzo
@darioizzo
Jun 01 2016 11:44 UTC
Cool!!!!
love it
Marcus Märtens
@CoolRunning
Jun 01 2016 11:45 UTC
Looks great!
Dario Izzo
@darioizzo
Jun 01 2016 11:45 UTC
On the verbosity: I guess that if you do work in jupyter you do not want screen output. Also, the results of screen output are stored in the log too and can be accessed after
Marcus Märtens
@CoolRunning
Jun 01 2016 11:45 UTC
I vote for np.array rather than np.matrix
Dario Izzo
@darioizzo
Jun 01 2016 11:46 UTC
On vector_double::size_type best_idx(double tol = 0.), I had noticed back then and decided not to check. If the tol in negative, simply, all will be unfeasible.
on np.array vs np.matrix .... I have no opinion, not a great user of these numpy data structures
Francesco Biscani
@bluescarni
Jun 01 2016 11:49 UTC
I was talking only about np.array, never used the matrix stuff
Dario Izzo
@darioizzo
Jun 01 2016 11:52 UTC
So, de does not solve rosenbrock 2D in 500 evaluations ..... :)
Can you show me the output if you do de?
I mean the docstring
Francesco Biscani
@bluescarni
Jun 01 2016 11:58 UTC
the docstring?
Dario Izzo
@darioizzo
Jun 01 2016 11:58 UTC
de?
what does it do?
Francesco Biscani
@bluescarni
Jun 01 2016 11:59 UTC
de gives you the repr, not the docstring
Dario Izzo
@darioizzo
Jun 01 2016 11:59 UTC
and can I see it?
Francesco Biscani
@bluescarni
Jun 01 2016 11:59 UTC
ah you mean de? from python
Dario Izzo
@darioizzo
Jun 01 2016 11:59 UTC
from ipython
Francesco Biscani
@bluescarni
Jun 01 2016 11:59 UTC
In [8]: pygmo.de?
Docstring:
__init__(gen = 1, F = 0.8, CR = 0.9, variant = 2, ftol = 1e-6, tol = 1e-6, seed = random)

Differential evolution algorithm.
File:      ~/.local/lib64/python3.4/site-packages/pygmo/core.so
Type:      class
Dario Izzo
@darioizzo
Jun 01 2016 12:00 UTC
SO all the docs from c++ are "lost" ... how are we going to dealt with it?
Is there a way to recover them?
Or we need to rewrite?
Francesco Biscani
@bluescarni
Jun 01 2016 12:01 UTC
not directly, the C++ docs are comments and you cannot access comments in any shape or form from C++ code
Dario Izzo
@darioizzo
Jun 01 2016 12:01 UTC
I was hoping on some new boost magic
Francesco Biscani
@bluescarni
Jun 01 2016 12:02 UTC
It's a language thing - but in any case we would need to rewrite them if we want to do them properly
at least the param docs, the types, the exceptions raised etc.
Dario Izzo
@darioizzo
Jun 01 2016 12:02 UTC
can we exploit the doxygen parse?
Francesco Biscani
@bluescarni
Jun 01 2016 12:02 UTC
the only thing you can salvage is the method description
Dario Izzo
@darioizzo
Jun 01 2016 12:02 UTC
I like the signature
__init__(gen = 1, F = 0.8, CR = 0.9, variant = 2, ftol = 1e-6, tol = 1e-6, seed = random)
Francesco Biscani
@bluescarni
Jun 01 2016 12:03 UTC
that needs to be inputted by hand when exposing
Dario Izzo
@darioizzo
Jun 01 2016 12:03 UTC
:)
pity I was wondering how he would have dealt with the static member defaults
Francesco Biscani
@bluescarni
Jun 01 2016 12:03 UTC
if we want to do things properly, we need to replicate and adapt largely the C++ docs
we can always refer to the cpp docs from the docstring btw
it just works when you run it through sphinx
Dario Izzo
@darioizzo
Jun 01 2016 12:04 UTC
Ya, but I guess we must have, as a minimum, in the docstring some docs on what are the arguments and what they do
Francesco Biscani
@bluescarni
Jun 01 2016 12:04 UTC
yep
of course we could come up with some complicated scheme in which we write the doxygen comments as macros, so we can re-use them from boost.python
that would be a nightmare
Dario Izzo
@darioizzo
Jun 01 2016 12:05 UTC
probably no ideal
would force us to document c++ in a weird hybrid way
Francesco Biscani
@bluescarni
Jun 01 2016 12:06 UTC
you would need to parse them, remove the * and // characters, translate the param types to corresponding python types..
no thanks
Dario Izzo
@darioizzo
Jun 01 2016 12:06 UTC
SO we just rewrite the docstring
one by one
Francesco Biscani
@bluescarni
Jun 01 2016 12:07 UTC
I guess it depends on what role you want to give the Python docs.. the way I see it:
• right now things work decently from the point of view of the generated sphinx docs.. we have shorte r descriptions on ther python classes, with hyperlinks to the C++ docs in case the user wants more
• if we want the python docstring to be useful interactively, they cannot be too long anyway
to me it's not a big deal frankly
Dario Izzo
@darioizzo
Jun 01 2016 12:08 UTC
agree,.
I was just asking to envision the future :)
And we can refer to sphinx in the docstring you said right?
Francesco Biscani
@bluescarni
Jun 01 2016 12:09 UTC
inline std::string population_docstring()
{
return R"(__init__(p, size = 0, seed = random)

The population class.

See also :cpp:class:pagmo::population.

)";
}
Dario Izzo
@darioizzo
Jun 01 2016 12:10 UTC
which in sphinx will create a hyperlink. in ipython would just show as is
k
whats the R?
also p should this be pygmo.problem ?
Francesco Biscani
@bluescarni
Jun 01 2016 12:21 UTC
so the "R" stuff is C++11 raw string literals
p is the name of the construction argument, not its type
its type can be any C++ or Python user-defined problem
Dario Izzo
@darioizzo
Jun 01 2016 12:21 UTC
but it is uninformative in the docstring ... maybe call it problem ?
Francesco Biscani
@bluescarni
Jun 01 2016 12:22 UTC
it's a balance between shortness of typing and info... maybe prob?
Dario Izzo
@darioizzo
Jun 01 2016 12:22 UTC
deal!
Francesco Biscani
@bluescarni
Jun 01 2016 12:22 UTC
just so we do it consistently :)
prob and algo it is then
yeah the hyperlink would only be in sphinx
inline std::string rosenbrock_docstring()
{
return R"(__init__(dim = 2)

The Rosenbrock problem.

:param dim: number of dimensions
:type dim: int
:raises: :exc:OverflowError if *dim* is negative or greater than an implementation-defined value
:raises: :exc:ValueError if *dim* is less than 2

See :cpp:class:pagmo::rosenbrock.

)";
}
another example, here with the complete description of signature and exception raising, in sphinx syntax
Dario Izzo
@darioizzo
Jun 01 2016 12:25 UTC
I use more ipython than sphinx ... I am wondering if its not better to type directly in the style we would like to see there ...
Francesco Biscani
@bluescarni
Jun 01 2016 12:27 UTC
there's a few alternatives for the sphinx syntax... as I said, decisions :)
the docstring syntax I mean
Francesco Biscani
@bluescarni
Jun 01 2016 12:45 UTC
extracting fitness vectors as a 2d array from the pop allows you to quickly get out the feasibility stuff with something like:
>>> pop.get_f()[:,3:5]
if I recall correctly the numpy syntax anyway
Dario Izzo
@darioizzo
Jun 01 2016 12:45 UTC
nice
Marcus Märtens
@CoolRunning
Jun 01 2016 12:49 UTC
Double-writing the documentation is annoying though... especially since writing documentation is annoying the first place already, though of course necessary... :neutral_face:
Francesco Biscani
@bluescarni
Jun 01 2016 12:50 UTC
it is, but I don't see viable alternatives
Marcus Märtens
@CoolRunning
Jun 01 2016 12:50 UTC
The slicing with numpy is very useful. I find myself using it more often these days as I grow fond of np.array
Dario Izzo
@darioizzo
Jun 01 2016 12:51 UTC
Copied from Matlab to be fair .. :)
Marcus Märtens
@CoolRunning
Jun 01 2016 12:51 UTC
It is, true.
Probably the reason, why I had problems with it for such a long time :D
But also matplotlib is a rip-off from Matlab.
Francesco Biscani
@bluescarni
Jun 01 2016 12:52 UTC
I am pretty sure matlab is a ripoff of something else still
Marcus Märtens
@CoolRunning
Jun 01 2016 12:52 UTC
Which is, why this statemachine is sometimes a bit akward.
Dario Izzo
@darioizzo
Jun 01 2016 12:52 UTC
But it slowly diverged .... I think now its more powerful
Marcus Märtens
@CoolRunning
Jun 01 2016 12:52 UTC
Isn't Matlab not only some sugarcoating over BLAS or so?
Francesco Biscani
@bluescarni
Jun 01 2016 12:52 UTC
it all comes from fortran ultimately :)
Dario Izzo
@darioizzo
Jun 01 2016 12:52 UTC
Yes, but we are talking about the notation
they may have invented a[1:4, 3:4]
Its MATrix LABoratory after all
Francesco Biscani
@bluescarni
Jun 01 2016 12:53 UTC
https://docs.oracle.com/cd/E19205-01/819-5257/blagu/index.html slicing is in fortran as well apparently
Dario Izzo
@darioizzo
Jun 01 2016 12:53 UTC
vs. FORmula TRANslation
Marcus Märtens
@CoolRunning
Jun 01 2016 12:54 UTC
There it is - Fortran... well, it brought us to the moon, how bad can it be? ^^
Francesco Biscani
@bluescarni
Jun 01 2016 12:56 UTC
I've said it before... python is the modern fortran
at least as far as scientists are concerned :)
Marcus Märtens
@CoolRunning
Jun 01 2016 12:57 UTC
It is sort of funny that we developed programming languages that are actually much slower than the old shit
I guess it is about Pareto-optimality also when it comes to them, but would those FORTRAN people have guessed something like Python?
Francesco Biscani
@bluescarni
Jun 01 2016 12:58 UTC
as long as you are manipulating arrays, numpy is not too bad.. and the selling point has always been ease of use and speed of iteration with python
Marcus Märtens
@CoolRunning
Jun 01 2016 12:58 UTC
I know, but still I would assume that you would try to get more and more efficient (=faster) languages
But I guess FORTRAN, Assembler, C or so are at the cap of what is possible.
At least along this dimension.
Francesco Biscani
@bluescarni
Jun 01 2016 12:59 UTC
people still use compiled languages when speed is the most important metric
and if you use C++ you can get nice syntax and max speed
as long as you are up for spending years to learn it that is :)
Marcus Märtens
@CoolRunning
Jun 01 2016 13:00 UTC
I just started this journey... I wonder when the regret will turn into bliss
Francesco Biscani
@bluescarni
Jun 01 2016 13:01 UTC
good luck, I don't miss the learning times :)
Marcus Märtens
@CoolRunning
Jun 01 2016 13:01 UTC
But thinking about types and signs of integers feels so primitive and time-consuming.
Francesco Biscani
@bluescarni
Jun 01 2016 13:02 UTC
it's not really, type safety is one of the best things about C++
it's so good that python will adopt it as an opt-in
Marcus Märtens
@CoolRunning
Jun 01 2016 13:03 UTC
What did type safety ever do for us? :fearful:
Francesco Biscani
@bluescarni
Jun 01 2016 13:04 UTC
code that is not brittle? :)
Marcus Märtens
@CoolRunning
Jun 01 2016 13:05 UTC
Still, we are rewriting stuff ^^
But ignore my whinings... I see the point.
So far it hasn't been too bad, though I am also not learning much right now, as this hypervolume stuff is mostly copy&paste work.
Dario Izzo
@darioizzo
Jun 01 2016 13:07 UTC
Porting code is horrible ... to learn some of the coolness of the new c++ the best would be to develop from sctartch something ....
Marcus Märtens
@CoolRunning
Jun 01 2016 13:11 UTC
Yes, I started some toy-programming as well, but this is even more behind than the porting :D
Anyway, it will come.
I was thinking whether I should rewrite a simulator for epidemics that was done in this group.
It is a piece of Java, which I sometimes touch carefully with a stick... though I would love to see this in C++ with a Python interface.
But maybe that backfires... I am better sticking to my random formulas.
Francesco Biscani
@bluescarni
Jun 01 2016 13:29 UTC
for the docstring style, I'd like to have something that also looks good on sphinx. I personally spend more time on the scipy website when I need to read about numpy or ode or whatver than in the command line
I can add a few docstrings in different styles to the bindings and then we can pick one
Dario Izzo
@darioizzo
Jun 01 2016 13:37 UTC
ok good ..
Francesco Biscani
@bluescarni
Jun 01 2016 13:39 UTC
Dario Izzo
@darioizzo
Jun 01 2016 13:41 UTC
Francesco Biscani
@bluescarni
Jun 01 2016 13:41 UTC
does it satisfy your refined fashion sense regarding interactive usage? :)
Dario Izzo
@darioizzo
Jun 01 2016 13:42 UTC
not really, I never liked it. But I am a weird guy
Francesco Biscani
@bluescarni
Jun 01 2016 13:42 UTC
what would you like then? any example?
Dario Izzo
@darioizzo
Jun 01 2016 13:43 UTC
In [2]: algorithm.de?
Init signature: algorithm.de(self, gen=100, f=0.8, cr=0.9, variant=2, ftol=1e-06, xtol=1e-06, screen_output=False)
Docstring:      Differential evolution algorithm.
Init docstring:
Constructs a Differential Evolution algorithm:

USAGE: algorithm.de(gen=1, f=0.5, cr=0.9, variant=2, ftol=1e-6, xtol=1e-6, screen_output = False)

* gen: number of generations
* f: weighting factor in [0,1] (if -1 self-adptation is used)
* cr: crossover in [0,1] (if -1 self-adptation is used)
* variant: algoritmic variant to use (one of [1 .. 10])
1. DE/best/1/exp
2. DE/rand/1/exp
3. DE/rand-to-best/1/exp
4. DE/best/2/exp
5. DE/rand/2/exp
6. DE/best/1/bin
7. DE/rand/1/bin
8. DE/rand-to-best/1/bin
9. DE/best/2/bin
10. DE/rand/2/bin
* ftol stop criteria on f
* xtol stop criteria on x
File:           /usr/lib/python3.5/site-packages/PyGMO/algorithm/_algorithm.so
Type:           class
Here only the types are missing ...
could be added easily gen: (int) number of generations
Francesco Biscani
@bluescarni
Jun 01 2016 13:49 UTC
it needs to be parsable by sphinx as well
Dario Izzo
@darioizzo
Jun 01 2016 13:49 UTC
This one is ... see the link
Francesco Biscani
@bluescarni
Jun 01 2016 13:50 UTC
it does not seem to me it is identifying any role for the text.. it's just a piece of markdown which is displayed as-is
Dario Izzo
@darioizzo
Jun 01 2016 13:50 UTC
yes
Francesco Biscani
@bluescarni
Jun 01 2016 13:50 UTC
and the variant: algoritmic variant to use (one of [1 .. 10]) is also displayed wierdly it seems to me
Dario Izzo
@darioizzo
Jun 01 2016 13:50 UTC
thats a typo
sphinx recognizing the text having some role is a plus only if we like what he makes of it ...
which I personally do not ... anyway I do not have strong opinions on this, I just would like the output to be as human readable as possible ....
Francesco Biscani
@bluescarni
Jun 01 2016 13:52 UTC
how the rosenbrock docstring I pasted above is rendered in sphinx is different, in the sense that for instance it groups together on the same line the name of the function argument and its type, giving specific predictable styling to each
that's why I linked to the scipy stuff, because it looks more human-friendly and (I assume) it is also sphinx friendly
but maybe not, who knows
Dario Izzo
@darioizzo
Jun 01 2016 13:53 UTC
Well :dim:, :raise: is not really human readable syntax right?
Its more a computer parsing syntax
And makes sense if rendered in sphinx ...
outside? less so
Francesco Biscani
@bluescarni
Jun 01 2016 13:54 UTC
I am used to parsing C++ template metaprogramming, so I might be a bit biased
:)
but the point still stands, I think we need not to sacrifice either output
Dario Izzo
@darioizzo
Jun 01 2016 13:55 UTC
The scipy thing also is not recognizing roles for text right?
Its just a wall of text ....
Francesco Biscani
@bluescarni
Jun 01 2016 13:55 UTC
I am not sure, if you see how it is rendered it seems so http://docs.scipy.org/doc/scipy-0.17.0/reference/generated/scipy.integrate.odeint.html
but I am not 100% confident it is sphinx generated
in any case there's more than 1 type of sphinx syntax you can have in the docstring
Dario Izzo
@darioizzo
Jun 01 2016 13:56 UTC
I think you are saying the same thing as I am ....
Francesco Biscani
@bluescarni
Jun 01 2016 13:56 UTC
y0 : array look how this is rendered: y0 is bold, array italic
without any indication in the original docstring
Dario Izzo
@darioizzo
Jun 01 2016 13:57 UTC
The scipy stuff is basically what I also posted above for de. There is some REST use (asterisx headings bold etc.) but not any kind of syntax that is parsable from sphinx and rendered ugly
Francesco Biscani
@bluescarni
Jun 01 2016 13:57 UTC
no it's not... in the scipy syntax sphinx recognizes that y0 is an argument, and array a type
Dario Izzo
@darioizzo
Jun 01 2016 13:58 UTC
No its just a list I think and its rendered accordingly and conveniently
Francesco Biscani
@bluescarni
Jun 01 2016 13:58 UTC
look here
Dario Izzo
@darioizzo
Jun 01 2016 13:58 UTC
Let me try to render that in a non sphinx env ..
So they are saying the same discussion ..
one line of thinking is that the sphinx markup shouldn’t kill the docstring that a user might read through the __doc__ attribute
Francesco Biscani
@bluescarni
Jun 01 2016 14:01 UTC
yep
Dario Izzo
@darioizzo
Jun 01 2016 14:01 UTC
I like the google style stuff
Francesco Biscani
@bluescarni
Jun 01 2016 14:03 UTC
so apprently sphinx does not parse specially the googley stuff, according to this: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/
we might consider using it
As of Sphinx 1.3, the napoleon extension will come packaged with Sphinx under sphinx.ext.napoleon. The sphinxcontrib.napoleon extension will continue to work with Sphinx <= 1.2. oh cool
so it's builtin now
Dario Izzo
@darioizzo
Jun 01 2016 14:04 UTC
lets go for it
it looks the solution we seek
Francesco Biscani
@bluescarni
Jun 01 2016 14:04 UTC
Dario Izzo
@darioizzo
Jun 01 2016 14:04 UTC
Are you tired of writing docstrings that look like this:
:)
googley which is what I actually wrote above for de :)
Francesco Biscani
@bluescarni
Jun 01 2016 14:05 UTC
it's not but ok :)
Dario Izzo
@darioizzo
Jun 01 2016 14:05 UTC
it is
no its not
but it is
no
Francesco Biscani
@bluescarni
Jun 01 2016 14:05 UTC
lol
Dario Izzo
@darioizzo
Jun 01 2016 14:05 UTC
yes
well at least our discussions are also made by others ... which make me think they are less pointless than expected
Francesco Biscani
@bluescarni
Jun 01 2016 14:06 UTC
I never start pointless discussions ;)
Francesco Biscani
@bluescarni
Jun 01 2016 14:12 UTC
inline std::string rosenbrock_docstring()
{
return R"(__init__(dim = 2)

The Rosenbrock problem.

Args:
dim (int): number of dimensions

Raises:
OverflowError: if dim is negative or greater than an implementation-defined value
ValueError: if dim is less than 2

See :cpp:class:pagmo::rosenbrock.

)";
this gets parsed nicely now by sphinx as well
Dario Izzo
@darioizzo
Jun 01 2016 14:12 UTC
yeah!!!
Francesco Biscani
@bluescarni
Jun 01 2016 14:13 UTC
In [1]: import pygmo

In [2]: pygmo.rosenbrock?
Docstring:
__init__(dim = 2)

The Rosenbrock problem.

Args:
dim (int): number of dimensions

Raises:
OverflowError: if dim is negative or greater than an implementation-defined value
ValueError: if dim is less than 2

See :cpp:class:pagmo::rosenbrock.
File:      ~/.local/lib64/python3.4/site-packages/pygmo/core.so
Type:      class

In [3]:
Dario Izzo
@darioizzo
Jun 01 2016 14:13 UTC
love it
Francesco Biscani
@bluescarni
Jun 01 2016 14:14 UTC
it similar to doxygen ain'it
Dario Izzo
@darioizzo
Jun 01 2016 14:14 UTC
and for population what would it look like?
yes
Francesco Biscani
@bluescarni
Jun 01 2016 14:15 UTC
inline std::string population_docstring()
{
return R"(__init__(prob, size = 0, seed = random)

The population class.

Args:
prob: a user-defined problem
size (int): the number of individuals
seed (int): the random seed (if not specified, randomly-generated)

See also :cpp:class:pagmo::population.

)";
something like that
Dario Izzo
@darioizzo
Jun 01 2016 14:18 UTC
maybe explaining what a user-defined problem is? like a link to the page where its requirements will be documented?
Francesco Biscani
@bluescarni
Jun 01 2016 14:18 UTC
http://imgur.com/iIvBsgl that's how the sphinx generated output looks for rosenbrok
Dario Izzo
@darioizzo
Jun 01 2016 14:22 UTC
done
Francesco Biscani
@bluescarni
Jun 01 2016 14:23 UTC
I also added the problems/algorithms submodules fuck yeah
Dario Izzo
@darioizzo
Jun 01 2016 14:24 UTC
e dajeeeee
Francesco Biscani
@bluescarni
Jun 01 2016 14:24 UTC
In [3]: pygmo.problems.
pygmo.problems.hock_schittkowsky_71  pygmo.problems.null_problem          pygmo.problems.rosenbrock            pygmo.problems.translate

In [3]: pygmo.problems.
Dario Izzo
@darioizzo
Jun 01 2016 14:24 UTC
legen ... wait for it ....
DARYYYY
Francesco Biscani
@bluescarni
Jun 01 2016 14:24 UTC
the high five to end all high fives
the highest of fives
Dario Izzo
@darioizzo
Jun 01 2016 14:25 UTC
problems or problem ?
in PaGMO legacy its problem
Francesco Biscani
@bluescarni
Jun 01 2016 14:25 UTC
problem would clash with the problem class
Dario Izzo
@darioizzo
Jun 01 2016 14:25 UTC
auch
so its pygmo.problem(pygmo.problems.ciccio(2))
Francesco Biscani
@bluescarni
Jun 01 2016 14:29 UTC
pygmo.problem(pygmo.ciccio(2))
the problems submodules contains just links to the classes
Dario Izzo
@darioizzo
Jun 01 2016 14:29 UTC
ah ok ...
Francesco Biscani
@bluescarni
Jun 01 2016 15:56 UTC
so is it intended that a population can be constructed both from a problem and a user-defined problem?
constructor from problem -> creates a copy of the input problem into the population
constructor from user-defined problem (UDP) -> creates a problem in the population from the supplied UDP
Francesco Biscani
@bluescarni
Jun 01 2016 16:19 UTC
actually I think I'd prefer forcing the constructor from problem only on the python side
the reason is the following: for each C++ problem we expose, we need to add a constructor to population templated on that C++ type
that is what I am doing now anyway
but imagine once we get to island, which will have a ctor similar to
template <typename Prob, typename Algo>
island(const Prob &p, const Algo &a, ...)
this would mean that on the Python side each time we add an exposed C++ problem, we need to expose $n$ new constructors of island, where $n$ is the total number of exposed C++ algorithms
or maybe for the island/population ctor we need to write a small wrapper directly in Python that does the construction of an intermediate problem/algorithm, so that in C++ we just need to expose the ctor from problem/algorithm
maybe this is the way to go
Dario Izzo
@darioizzo
Jun 01 2016 17:28 UTC
The reason for the two overloads is to allow the syntax: population(ackley(5u), 20u) as well as population(problem(ackley(5u)), 20)
For the island, the constructor may only work on population, algorithm, ... wiuld this solve the issue?
Francesco Biscani
@bluescarni
Jun 01 2016 17:49 UTC
I think hijacking the constructor is the way to go
Francesco Biscani
@bluescarni
Jun 01 2016 20:31 UTC
alright I committed it
Dario Izzo
@darioizzo
Jun 01 2016 20:31 UTC
what?
Francesco Biscani
@bluescarni
Jun 01 2016 20:32 UTC
the implementation of the pop ctor from python
Dario Izzo
@darioizzo
Jun 01 2016 20:32 UTC
Francesco Biscani
@bluescarni
Jun 01 2016 20:32 UTC
yep
Dario Izzo
@darioizzo
Jun 01 2016 20:33 UTC
and? which one is it at the end?
Francesco Biscani
@bluescarni
Jun 01 2016 20:33 UTC
Dario Izzo
@darioizzo
Jun 01 2016 20:34 UTC
how cool is this coming out?
Francesco Biscani
@bluescarni
Jun 01 2016 20:35 UTC
it's friggin great
look at this beauty:
In [1]: import pygmo

In [2]: pop = pygmo.population(size=5, prob = 4, seed = -4)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-55e97e1fdb40> in <module>()
----> 1 pop = pygmo.population(size=5, prob = 4, seed = -4)

/home/yardbird/.local/lib64/python3.4/site-packages/pygmo/__init__.py in _population_init(self, prob, size, seed)
129         # Otherwise, we attempt to create a problem from it. This will
130         # work if prob is an exposed C++ problem or a Python UDP.
--> 131         prob_arg = problem(prob)
132     if seed is None:
133         __original_population_init(self,prob_arg,size)

TypeError: the mandatory 'fitness()' method is missing from the user-defined Python problem '4' of type '<class 'int'>'
dario-izzo-compliant error message
Dario Izzo
@darioizzo
Jun 01 2016 20:36 UTC
agree
whats this overflow error when negative numbers are passed to unsigned?
Francesco Biscani
@bluescarni
Jun 01 2016 20:38 UTC
it's the type of error generated by Boost.Python when you pass negative python ints to a C++ function which takes unsigned
Dario Izzo
@darioizzo
Jun 01 2016 20:38 UTC
k, and whats the error message if:
Francesco Biscani
@bluescarni
Jun 01 2016 20:38 UTC
In [3]: pop = pygmo.population(size=5, prob = pygmo.translate(pygmo.rosenbrock(),[1,2]), seed = -3)
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
<ipython-input-3-af07af7fcd45> in <module>()
----> 1 pop = pygmo.population(size=5, prob = pygmo.translate(pygmo.rosenbrock(),[1,2]), seed = -3)

/home/yardbird/.local/lib64/python3.4/site-packages/pygmo/__init__.py in _population_init(self, prob, size, seed)
133         __original_population_init(self,prob_arg,size)
134     else:
--> 135         __original_population_init(self,prob_arg,size,seed)
136
137 setattr(population,"__init__",_population_init)

OverflowError: can't convert negative value to unsigned int
Dario Izzo
@darioizzo
Jun 01 2016 20:39 UTC
class ciccio:
def fitness (self, x, y):
return x*y
Francesco Biscani
@bluescarni
Jun 01 2016 20:39 UTC
that's gonna be ugly
Dario Izzo
@darioizzo
Jun 01 2016 20:39 UTC
pop = pygmo.population(size=5, prob = ciccio, seed = -4)
try
me
Francesco Biscani
@bluescarni
Jun 01 2016 20:40 UTC
In [7]: class prob(object):
def fitness(self,v,vf):
return [v[0]*v[0]]
def get_bounds(self):
return ([0],[1])
...:

In [8]: pop = pygmo.population(size=5, prob = prob(), seed = 3)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-fc0f16ce3c3d> in <module>()
----> 1 pop = pygmo.population(size=5, prob = prob(), seed = 3)

/home/yardbird/.local/lib64/python3.4/site-packages/pygmo/__init__.py in _population_init(self, prob, size, seed)
133         __original_population_init(self,prob_arg,size)
134     else:
--> 135         __original_population_init(self,prob_arg,size,seed)
136
137 setattr(population,"__init__",_population_init)

TypeError: fitness() missing 1 required positional argument: 'vf'

In [9]:
Dario Izzo
@darioizzo
Jun 01 2016 20:41 UTC
fitness() missing 1 required positional argument: 'vf'
yeah right thats the problem
Francesco Biscani
@bluescarni
Jun 01 2016 20:41 UTC
I had some code that tried to do better than that, but it was problematic
Dario Izzo
@darioizzo
Jun 01 2016 20:41 UTC
ANd what is the error if:
Francesco Biscani
@bluescarni
Jun 01 2016 20:42 UTC
the issue is that you risk of catching TypeErrors which might be generated inside the fitness function
Dario Izzo
@darioizzo
Jun 01 2016 20:42 UTC
class ciccio(object):
def fitness (self, x):
return x
Francesco Biscani
@bluescarni
Jun 01 2016 20:42 UTC
and producing error messages that are even worse
I think the only solution would be to inspect the signature of the fitness() function somehow
that one works, if the bounds have size 1
Dario Izzo
@darioizzo
Jun 01 2016 20:43 UTC
really? cool
how? x is no array
Francesco Biscani
@bluescarni
Jun 01 2016 20:44 UTC
x could be anything
Dario Izzo
@darioizzo
Jun 01 2016 20:44 UTC
ah right so:
class ciccio(object):
def fitness (self, x):
return x[0]
I meant this
Francesco Biscani
@bluescarni
Jun 01 2016 20:45 UTC
In [11]: class prob(object):
def fitness(self,v):
return v[0]
def get_bounds(self):
return ([0],[1])
....:

In [12]: pop = pygmo.population(size=5, prob = prob(), seed = 3)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-fc0f16ce3c3d> in <module>()
----> 1 pop = pygmo.population(size=5, prob = prob(), seed = 3)

/home/yardbird/.local/lib64/python3.4/site-packages/pygmo/__init__.py in _population_init(self, prob, size, seed)
133         __original_population_init(self,prob_arg,size)
134     else:
--> 135         __original_population_init(self,prob_arg,size,seed)
136
137 setattr(population,"__init__",_population_init)

TypeError: cannot convert the type '<class 'numpy.float64'>' to a vector of doubles: only lists of doubles and NumPy arrays of doubles are supported
Dario Izzo
@darioizzo
Jun 01 2016 20:46 UTC
decent
and why the previous one would work only with bounds of dim 1?
Francesco Biscani
@bluescarni
Jun 01 2016 20:46 UTC
that one I wrote myself :) too bad it uses numpy.float64 instead of just float
so what happens is that the population generates random decision vectors of size 1, because that's the bounds size
Dario Izzo
@darioizzo
Jun 01 2016 20:47 UTC
I mean if I define nobj =3 shoud also work for bounds of dim 3 right?
Francesco Biscani
@bluescarni
Jun 01 2016 20:47 UTC
it should yeah
Dario Izzo
@darioizzo
Jun 01 2016 20:47 UTC
this is super cool, I am wetting myself
Francesco Biscani
@bluescarni
Jun 01 2016 20:47 UTC
I know right
Dario Izzo
@darioizzo
Jun 01 2016 20:48 UTC
did you try to write a rosenbrock in python and benchmark it vs the one in c++ ?
I mean what is the penalty of defining a problem directly in python?
In a simple case where just a few mult and divs are made?
Francesco Biscani
@bluescarni
Jun 01 2016 20:49 UTC
In [15]: pop = pygmo.population(size=5, prob = translate(translate(prob(),[1]),[-1]), seed = 3)

In [16]: pop
Out[16]:
Problem name: <class '__main__.prob'> [translated] [translated]
Global dimension:                       1
Fitness dimension:                      1
Number of objectives:                   1
Equality constraints dimension:         0
Inequality constraints dimension:       0
Lower bounds: [0]
Upper bounds: [1]

Has hessians: false
User implemented hessians sparsity: false

Function evaluations: 5

Extra info:

Translation Vector: [1]
Translation Vector: [-1]
Population size: 5

List of individuals:
#0:
ID:                     13063021645730866819
Decision vector:        [0.0707249]
Fitness vector:         [0.0707249]
#1:
ID:                     9423106023066854165
Decision vector:        [0.121329]
Fitness vector:         [0.121329]
#2:
ID:                     16533689267043216235
Decision vector:        [0.437062]
Fitness vector:         [0.437062]
#3:
ID:                     3822956344058944405
Decision vector:        [0.0406307]
Fitness vector:         [0.0406307]
#4:
ID:                     8131506278483621524
Decision vector:        [0.0935515]
Fitness vector:         [0.0935515]
translate translate translate
Dario Izzo
@darioizzo
Jun 01 2016 20:49 UTC
coatto
Francesco Biscani
@bluescarni
Jun 01 2016 20:49 UTC
I am not at the point of doing benchmarks yet
but it's going to be quite a bit slower
Dario Izzo
@darioizzo
Jun 01 2016 20:50 UTC
Would be nice to know how much.
I am tempted to write all racing, ert etc. directly in python
Francesco Biscani
@bluescarni
Jun 01 2016 20:50 UTC
eventually... I have not even compiled it yet in non-debug mode :)
but then you can't use those from C++ can you?
Dario Izzo
@darioizzo
Jun 01 2016 20:51 UTC
no, who cares?
right?
the sheer elegance of this syntax ....
essential, to the point .... just just beautiful
Francesco Biscani
@bluescarni
Jun 01 2016 20:51 UTC
C++ is dying indeed java rust will kill it
java go rust will kill it
Dario Izzo
@darioizzo
Jun 01 2016 20:52 UTC
Francesco Biscani
@bluescarni
Jun 01 2016 20:53 UTC
I heard that before... basically modern C++ plus compile with -Werror -Weverything
there ya go