These are chat archives for bluescarni/pagmo_reborn

23rd
May 2016
Dario Izzo
@darioizzo
May 23 2016 09:15
@bluescarni I am writing the decomposition method using enums to deal with three different available options ... are enums cool nowadays?
Dario Izzo
@darioizzo
May 23 2016 09:21
So the difference is that they are real types not convertible to integers?
Francesco Biscani
@bluescarni
May 23 2016 09:21
yes
Dario Izzo
@darioizzo
May 23 2016 09:22
k
Francesco Biscani
@bluescarni
May 23 2016 09:22
not implicitly
and also you cannot compare enums from different enums classes
Dario Izzo
@darioizzo
May 23 2016 09:24
but
enum class Enumeration {
    Val1,
    Val2,
    Val3 = 100,
    Val4 // = 101
};
what does it now mean val3 = 100 ?
Before it was the corresponding integer representaton. Now ?
Francesco Biscani
@bluescarni
May 23 2016 09:25
each enum value is still associated to an int, but not implicitly convertible
it is useful for serialization for instance
Dario Izzo
@darioizzo
May 23 2016 09:25
I do not see it ...
if they are now real types ... why the associated integer?
Francesco Biscani
@bluescarni
May 23 2016 09:26
because you can serialize it as an int and then restore it from the serialized archive
ar(int(Val3))
Dario Izzo
@darioizzo
May 23 2016 09:28
cereal does this automatically? or I need to provide load save?
Dario Izzo
@darioizzo
May 23 2016 09:31
boh
whatever
Francesco Biscani
@bluescarni
May 23 2016 09:32
cereal::CEREAL_SAVE_MINIMAL_FUNCTION_NAME (Archive const &, T const &t)
Saving for enum types.
Dario Izzo
@darioizzo
May 23 2016 09:50
I do not like enums. Can we use strings?
Like as arguments we accept a string defining the decomposition type?
"weighted", or "tchebicheff" ...
Francesco Biscani
@bluescarni
May 23 2016 09:51
I don't have objections either way, as you prefer
Dario Izzo
@darioizzo
May 23 2016 09:51
string it is
k
Francesco Biscani
@bluescarni
May 23 2016 12:05
the automatic generation of the PyGMO sphinx is coming along quite nicesly
Dario Izzo
@darioizzo
May 23 2016 12:06
what do you like the most?
Francesco Biscani
@bluescarni
May 23 2016 12:06
the methods prototpyes actually look decent, at least for an exposed C++ project
docstring testing is working
automatic linking of classes inside the docstrings
things like these
you can take a look for yourself :)
Dario Izzo
@darioizzo
May 23 2016 12:07
is this online?
or in your branch?
Francesco Biscani
@bluescarni
May 23 2016 12:08
my branch is onine, I just finished pushing
cmake ../ -DCMAKE_BUILD_TYPE=Debug -DBUILD_PYGMO=yes -DCMAKE_INSTALL_PREFIX=/home/username/.local
then make html as usual
well make install first
Dario Izzo
@darioizzo
May 23 2016 12:09
so I need to checkout your branch first right?
Francesco Biscani
@bluescarni
May 23 2016 12:10
git checkout -b boost_python --track origin/boost_python
Dario Izzo
@darioizzo
May 23 2016 12:10
? I actually do git fetch and then checkout
works the same
git fetch
git checkout boost_python
Francesco Biscani
@bluescarni
May 23 2016 12:11
I can't tell what that command is doing
fetch I know
Dario Izzo
@darioizzo
May 23 2016 12:11
dario@localhost ~/Documents/PaGMOreborn/build/tests (git)-[master] % git checkout boost_python
Branch boost_python set up to track remote branch boost_python from origin.
Switched to a new branch 'boost_python'
after the fetch ... :)
I think is a new syntax since a few versions
Francesco Biscani
@bluescarni
May 23 2016 12:12
ah cool didn't know that
Dario Izzo
@darioizzo
May 23 2016 12:12
to avoid the horrid -b -track blah blah
Francesco Biscani
@bluescarni
May 23 2016 12:13
well the one above you still need if you want to checkout from another remote I guess... unless you can do git checkout other_remote/boost_python
they just keep on piling stuff on top of git
Dario Izzo
@darioizzo
May 23 2016 12:14
so, to install locally I select ~/local, then the files go to:
Francesco Biscani
@bluescarni
May 23 2016 12:14
~/.local
notice the dot
Dario Izzo
@darioizzo
May 23 2016 12:14
ah, because by default python look there?
Francesco Biscani
@bluescarni
May 23 2016 12:14
yep
git checkout origin/boost_python this won't work, just checked
or at least it does not do the same thing as git checkout boost_python
whelp
Dario Izzo
@darioizzo
May 23 2016 12:15
:)
which directory has the precedence the .local or system?
Francesco Biscani
@bluescarni
May 23 2016 12:16
.local, that is where pip install --user installs as well
Dario Izzo
@darioizzo
May 23 2016 12:16
so if I install there I will shadow my sysrtem wide PyGMO right?
Francesco Biscani
@bluescarni
May 23 2016 12:16
I believe so yeah
you can rm -fr .local/lib/python3.4/site-packages/pygmo or whatever the directory is called
it's just 2 files in there
Dario Izzo
@darioizzo
May 23 2016 12:18
ImportError: /usr/lib/libboost_python.so.1.60.0: undefined symbol: PyClass_Type
and thats why I hate local installations :)
Francesco Biscani
@bluescarni
May 23 2016 12:18
probably linking to the python3 stuff, it has nothing to do with local install
Dario Izzo
@darioizzo
May 23 2016 12:19
so what is it?
the cmake stuff seems fine
Francesco Biscani
@bluescarni
May 23 2016 12:19
what does arch call the boost python library? can you do ls /usr/lib/libboost_python*?
Dario Izzo
@darioizzo
May 23 2016 12:20
/usr/lib/libboost_python3.a /usr/lib/libboost_python3.so /usr/lib/libboost_python3.so.1.60.0 /usr/lib/libboost_python.a /usr/lib/libboost_python.so /usr/lib/libboost_python.so.1.60.0
Francesco Biscani
@bluescarni
May 23 2016 12:20
/usr/lib/libboost_python.so I assume this is a symlink to the python3 or is it the python2 version?
Dario Izzo
@darioizzo
May 23 2016 12:21
dario@localhost ~/Documents/PaGMOreborn/doc/sphinx (git)-[boost_python] % ls -al /usr/lib/libboost_python*
-rw-r--r-- 1 root root 677772 Jan 24 21:53 /usr/lib/libboost_python3.a
lrwxrwxrwx 1 root root     26 Jan 24 21:50 /usr/lib/libboost_python3.so -> libboost_python3.so.1.60.0
-rwxr-xr-x 1 root root 315208 Jan 24 21:52 /usr/lib/libboost_python3.so.1.60.0
-rw-r--r-- 1 root root 685910 Jan 24 21:53 /usr/lib/libboost_python.a
lrwxrwxrwx 1 root root     25 Jan 24 21:50 /usr/lib/libboost_python.so -> libboost_python.so.1.60.0
-rwxr-xr-x 1 root root 319336 Jan 24 21:52 /usr/lib/libboost_python.so.1.60.0
Francesco Biscani
@bluescarni
May 23 2016 12:21
right looks like the Python2 version has been picked
ls -lsa /usr/lib/libboost_python*
  0 lrwxrwxrwx 1 root root     22 Apr  7 14:39 /usr/lib/libboost_python-2.7-mt.so -> libboost_python-2.7.so
  0 lrwxrwxrwx 1 root root     29 Apr  7 14:39 /usr/lib/libboost_python-2.7.so -> libboost_python-2.7.so.1.58.0
320 -rwxr-xr-x 1 root root 324576 Apr  7 14:39 /usr/lib/libboost_python-2.7.so.1.58.0
  0 lrwxrwxrwx 1 root root     22 Apr  7 14:39 /usr/lib/libboost_python-3.4-mt.so -> libboost_python-3.4.so
  0 lrwxrwxrwx 1 root root     29 Apr  7 14:39 /usr/lib/libboost_python-3.4.so -> libboost_python-3.4.so.1.58.0
324 -rwxr-xr-x 1 root root 328800 Apr  7 14:39 /usr/lib/libboost_python-3.4.so.1.58.0
that's on my gentoo
Dario Izzo
@darioizzo
May 23 2016 12:22
how are we detecting the library name?
Francesco Biscani
@bluescarni
May 23 2016 12:22
try to make it point to /usr/lib/libboost_python3.so.1.60.0
the cmake macros do it
Dario Izzo
@darioizzo
May 23 2016 12:22
mmm ... starnge in PyGMO legacy we do not have this problem
Francesco Biscani
@bluescarni
May 23 2016 12:23
you have the names hardcoded in the cmake
IF(PYTHON_VERSION_MAJOR LESS 3)
    SET(MANDATORY_LIBRARIES ${MANDATORY_LIBRARIES} ${Boost_PYTHON_LIBRARY})
ELSE(PYTHON_VERSION_MAJOR LESS 3)
    SET(MANDATORY_LIBRARIES ${MANDATORY_LIBRARIES} ${Boost_PYTHON3_LIBRARY})
ENDIF(PYTHON_VERSION_MAJOR LESS 3)
something like this
Dario Izzo
@darioizzo
May 23 2016 12:23
so ${Boost_PYTHON3_LIBRARY} points to the lib
why not doing the same ?
Francesco Biscani
@bluescarni
May 23 2016 12:25
I was hoping that by finding the Python interpreter/library beforehand, cmake would pick the correct boost python version
evidently that's not ntecessarily the case, although it works correctly here
I can change that
Dario Izzo
@darioizzo
May 23 2016 12:26
k, for the time being I will point to the correct lib manually
Francesco Biscani
@bluescarni
May 23 2016 12:26
can you paste the cmake output?
Dario Izzo
@darioizzo
May 23 2016 12:26
-- System name: Linux
-- The C++ compiler ID is: GNU
-- Performing Test YACMA_CHECK_CXX_FLAG
-- Performing Test YACMA_CHECK_CXX_FLAG - Success
-- Enabling the '-fdiagnostics-color=auto' compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wall' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wextra' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wnon-virtual-dtor' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wnoexcept' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wlogical-op' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wconversion' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wdeprecated' debug compiler flag.
-- Performing Test YACMA_CHECK_CXX_FLAG
-- Performing Test YACMA_CHECK_CXX_FLAG - Success
-- Enabling the '-ftemplate-depth=1024' compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-pedantic-errors' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wdisabled-optimization' debug compiler flag.
-- Performing Test YACMA_CHECK_CXX_FLAG
-- Performing Test YACMA_CHECK_CXX_FLAG - Success
-- Enabling the '-fvisibility-inlines-hidden' compiler flag.
-- Performing Test YACMA_CHECK_CXX_FLAG
-- Performing Test YACMA_CHECK_CXX_FLAG - Success
-- Enabling the '-fvisibility=hidden' compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-ftemplate-backtrace-limit=0' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-fstack-protector-all' debug compiler flag.
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG
-- Performing Test YACMA_CHECK_DEBUG_CXX_FLAG - Success
-- Enabling the '-Wodr' debug compiler flag.
-- C++14 supported by the compiler, enabling.
-- Thread library: -lpthread
-- POSIX threads detected.
-- Enabling the -pthread compiler flag.
-- POSIX threads affinity extensions detected.
-- Extra compiler flags for threading: -pthread
-- The 'thread_local' keyword is available.
-- Current CXX flags:  -fdiagnostics-color=auto -ftemplate-depth=1024 -fvisibility-inlines-hidden -fvisibility=hidden -std=c++14 -pthread
-- Current CXX debug flags: -g -Wall -Wextra -Wnon-virtual-dtor -Wnoexcept -Wlogical-op -Wconversion -Wdeprecated -pedantic-errors -Wdisabled-optimization -ftemplate-backtrace-limit=0 -fstack-protector-all -Wodr
-- Python interpreter: /usr/bin/python
-- Python libraries: /usr/lib64/libpython3.5m.so
-- Python include dirs: /usr/include/python3.5m
-- Python library version: 3.5.1
-- Generic UNIX platform detected.
-- Python packages dir is: site-packages
-- Python modules install path: lib/python3.5/site-packages
-- NumPy includes dir: /usr/lib/python3.5/site-packages/numpy/core/include
-- Boost version: 1.60.0
-- Found the following Boost libraries:
--   python
-- Setting up the compilation of the Python module "core".
-- Setting up extra compiler flag "-fwrapv" for the Python module "core".
-- Configuring done
-- Generating done
-- Build files have been written to: /home/dario/Documents/PaGMOreborn/build
Francesco Biscani
@bluescarni
May 23 2016 12:27
at least it seems it picks the correct combination interpreter/library/includes
Dario Izzo
@darioizzo
May 23 2016 12:27
-- C++14 supported by the compiler, enabling.
this looks like it could work also for c++ < 14 ..
just being pedantic
Francesco Biscani
@bluescarni
May 23 2016 12:28
it gives a fatal error if that is not the case indeed
Dario Izzo
@darioizzo
May 23 2016 12:29
ah ok
Francesco Biscani
@bluescarni
May 23 2016 12:29
it tests if the -std=c++14 flag is valid
Dario Izzo
@darioizzo
May 23 2016 12:29
so cmake does not generate otherwise ... good
Francesco Biscani
@bluescarni
May 23 2016 12:29
it crashes hard and early
Dario Izzo
@darioizzo
May 23 2016 12:33
nice sphinx :)
Francesco Biscani
@bluescarni
May 23 2016 12:33
it's not too bad is it
the problem docstring can be automatically tested with the doctest module, which compares the displayed output with the actual result of running the code
so you are always sure the examples actually work
I wish there were a way to remove the first argument from the methods' signatures
Dario Izzo
@darioizzo
May 23 2016 12:36
The only thing I do not like is that the docstring like this contains a tutorial.
In interactive shells, the docstring is visualized when hovering or ? or who knows .....
and I would not expect to have a REST formatted tutorial
Francesco Biscani
@bluescarni
May 23 2016 12:37
it's considered good practice to have a small example in each method/function, as additional testing and for educational purposes... but that one specifically in problem might be too big
Dario Izzo
@darioizzo
May 23 2016 12:37
Problem name: Multidimensional Rosenbrock Function
        Global dimension:                       5
        Fitness dimension:                      1
        Number of objectives:                   1
        Equality constraints dimension:         0
        Inequality constraints dimension:       0
        Lower bounds: [-5, -5, -5, -5, -5]
        Upper bounds: [1, 1, 1, 1, 1]
<BLANKLINE>
        Has gradient: false
        User implemented gradient sparsity: false
        Has hessians: false
        User implemented hessians sparsity: false
<BLANKLINE>
        Function evaluations: 0
<BLANKLINE>
>>> p.fitness([1,2,3,4,5])
array([ 14814.])
>>> class prob: # Definition of a pure-Python problem.
...   def fitness(self,dv):
...     return [dv[0]*dv[0]]
...   def get_bounds(self):
...     return ([1],[2])
...   def get_name(self):
...     return "A simple problem"
>>> p = problem(prob()) # Construction from the user-defined Python problem.
>>> p # doctest: +SKIP
Problem name: A simple problem
        Global dimension:                       1
        Fitness dimension:                      1
        Number of objectives:                   1
        Equality constraints dimension:         0
        Inequality constraints dimension:       0
        Lower bounds: [1]
        Upper bounds: [2]
<BLANKLINE>
        Has gradient: false
        User implemented gradient sparsity: false
        Has hessians: false
        User implemented hessians sparsity: false
thats how it appears, fo example, in ipython
<BLANKLINE> is not very appealing
Francesco Biscani
@bluescarni
May 23 2016 12:38
yes... again, it might be too big, and I was not planning to keep the long output lines... I was learning how to do stuff with doctest :)
I'd keep initialization from a rosenbrock say, and a fitness computation
Dario Izzo
@darioizzo
May 23 2016 12:39
Also, do we want to expose to the user the difference between c++ exposed and python exposed?
I mean in the doc now it explicitly say so ... but maybe we could also not care and mirror the c++ folder structure ...
anyway supercool!! I go back to decompose the shit out of problems :)
(commited also the untested verison)
Francesco Biscani
@bluescarni
May 23 2016 12:41
I don't know... the only real difference is that the C++ problems have minimal exposition, whereas the python ones would have more stuff in. Also the Python problem you can derive from, and from the C++ not, so there are some functional differences
alright... making progress!
Dario Izzo
@darioizzo
May 23 2016 12:42
But in the core we will not have python problems ...
only some examples .... but bottomline all will be c++ ... ok
Francesco Biscani
@bluescarni
May 23 2016 12:43
why not? maybe something that uses python-only stuff? but yes, probably not so interesting to have python only probs
Dario Izzo
@darioizzo
May 23 2016 12:43
I see them more in affiliated packages ...
Francesco Biscani
@bluescarni
May 23 2016 12:44
probably yeah... I think we still need to ship a few for testing all the parallel bullshit and for exposition
Dario Izzo
@darioizzo
May 23 2016 13:16
Problem name: ZDT1 [decomposed]
    Global dimension:            5
    Fitness dimension:            1
    Number of objectives:            1
    Equality constraints dimension:        0
    Inequality constraints dimension:    0
    Lower bounds: [0, 0, 0, 0, 0]
    Upper bounds: [1, 1, 1, 1, 1]

    Has gradient: false
    User implemented gradient sparsity: true
    Has hessians: false
    User implemented hessians sparsity: true

    Function evaluations: 0
What do we mean by "User implemented gradient sparsity:"
If I need to write a meta problem that changes the fitness or constraint dimensions
Francesco Biscani
@bluescarni
May 23 2016 13:17
that there is a gradient_sparsity() method in the concrete problem
Dario Izzo
@darioizzo
May 23 2016 13:17
ait
Francesco Biscani
@bluescarni
May 23 2016 13:18
I'd remove that output
Dario Izzo
@darioizzo
May 23 2016 13:18
then I will have to write again the sparisty. As a consequence the user implemented gradient and hessians sparsty will be true
but the "user" will not have implemented them ...
is this not confusing?
Francesco Biscani
@bluescarni
May 23 2016 13:19
I would remove it from the output, it is confusing and of no consequence
any problem will have a sparsity method after you put it into the problem container
Dario Izzo
@darioizzo
May 23 2016 13:19
I would agree ... hoping we are not "missing something"
Francesco Biscani
@bluescarni
May 23 2016 13:19
and it does not really matter if it was automatically generated
we used to have the override sparsity thing
Dario Izzo
@darioizzo
May 23 2016 13:20
ok so I remove it ...
Francesco Biscani
@bluescarni
May 23 2016 13:20
but we decided to simplify
you see that the gradient stuff is recognized doccrectly
Dario Izzo
@darioizzo
May 23 2016 13:20
you agree that in a meta that modifies the nobj or nc I have to reimplement the four methods right?
Francesco Biscani
@bluescarni
May 23 2016 13:20
correctly
Dario Izzo
@darioizzo
May 23 2016 13:20
    bool has_gradient() const
    {
        return false;
    }
    bool has_hessians() const
    {
        return false;
    }
    sparsity_pattern gradient_sparsity() const
    {
        return problem::dense_gradient(1u,get_nx());
    }
    std::vector<sparsity_pattern> hessians_sparsity() const
    {
        return problem::dense_hessians(1u,get_nx());
    }
In decompose, for example I deactivate the gradient and hessians (tchebycheff method is not even differentiable)
via the four methods above ....
any troubles you see?
Francesco Biscani
@bluescarni
May 23 2016 13:21
no, if it is not computable it is not computable
do we need to deactivate methods in translate as well?
or re-implement them?
Dario Izzo
@darioizzo
May 23 2016 13:22
there its ok as reimplementing is easy (its done) and sparisty is invariant
Francesco Biscani
@bluescarni
May 23 2016 13:22
alright
Dario Izzo
@darioizzo
May 23 2016 13:22
Can I move problem::dense_gradient(1u,get_nx()) in details and make it available to all classes?
Francesco Biscani
@bluescarni
May 23 2016 13:23
sure.. it used to be in there but then I moved it into problem... just put it into detail namespace in the same file
Dario Izzo
@darioizzo
May 23 2016 13:23
static
k
I need it from decompose
right now its a private member of problem
Francesco Biscani
@bluescarni
May 23 2016 13:23
maybe it could go even in the public utils, but then it needs to be written, documented and tested properly :)
Dario Izzo
@darioizzo
May 23 2016 13:23
I need it in decompose
Francesco Biscani
@bluescarni
May 23 2016 13:24
cannot be static if it is outside the class
just an inline function
Dario Izzo
@darioizzo
May 23 2016 13:24
yep
Francesco Biscani
@bluescarni
May 23 2016 18:27
in the dense_hessians function, here for (decltype(dim) i = 0u; i<=j; ++i) { is it really a less-than or equal?
Dario Izzo
@darioizzo
May 23 2016 21:07
hey
why do you ask?
Francesco Biscani
@bluescarni
May 23 2016 21:08
always makes me jittery to see loops with leq :)
Dario Izzo
@darioizzo
May 23 2016 21:08
when I wrote it I also wrote tests with, it ... but let me check specifically
I think the equal corrsponds to the elements on the diagonal which are indeed part of the hessian
The hessian is a symmetric matrix, so we only keep the lower triangular part
including diagonal
Francesco Biscani
@bluescarni
May 23 2016 21:09
cool ok