These are chat archives for bluescarni/pagmo_reborn

4th
Jun 2016
Marcus Märtens
@CoolRunning
Jun 04 2016 13:18
This message was deleted

Hi guys!

I report: the minimum functional core for hypervolumes is now complete and commited. I turned the old tests that we had in Python into C++ and they cover all basic functionalities, as far as I see. Now there are a couple of issues/things to discuss and to decide what to prioritize next.

1) The code should be reviewed and potentially refactored. There might be memory manipulation things going on (new, delete[]) especially in hvwfg that we do no longer tolerate from a coding perspective. I have not taken any effort so far to enforce a certain coding style or new concepts. This is almost purely copy&paste work with some renaming going on.

2) The documentation needs to be updated / reviewed.

3) I don't like the fact that the "selection methods" (the ones that will give you the best computational algorithm depending on the points stored in hypervolume) are implemented in hv3d. Hv3d should only contain the implementation of the algorithm for 3 dimensions and not stuff that deals with anything else. However, this issue is difficult because of the circular dependencies.

4) I don't like that all of hypervolume lives in "detail" but hv_algorithm lives in "util". In general, what are our placement rules? What lives in util? What shall live in detail?

5) So far, we can only compute exact hypervolumes, but the approximation algorithms need yet to be ported, if we want them.

6) Current tests are covering functionality, but are very simple and might miss errors that arise in complex geometries. Writing more involved tests however would require the loading of fixtures. We had some comprehensive set of fixtures in the old PaGMO tailored for accuracy that could be run indepedently from the rest of the tests (also, because hv-computations can be very time-consuming and you might want to quickly know whether your tests pass or not). To cut it short: shall we port this test-system as well?

7) There are several algorithms missing, some inferior, some better in certain cornercases. I am indecisive whether we should port them as well.

8) There might be new algorithms out there that outperform the current set. I know for sure that the WFG has improved their algorithm. If we want to be cutting-edge in this field, those developments should be checked.

Francesco Biscani
@bluescarni
Jun 04 2016 19:58
@CoolRunning nice work! I cannot really comment on the algorithmic part, but I will do a code review when the branch is ready and can help on the C++ style
@darioizzo So I finished exposing the C++ algorithms and I have a question
Dario Izzo
@darioizzo
Jun 04 2016 19:58
shoot
Francesco Biscani
@bluescarni
Jun 04 2016 19:59
so for the problems the only extra stuff I exposed (apart from the ctors) are the extra methods, typically the get_best() thingie
Dario Izzo
@darioizzo
Jun 04 2016 19:59
yes ok
Francesco Biscani
@bluescarni
Jun 04 2016 19:59
this is done on an ad-hoc basis depending on the problem
but for the algos there's a bunch of stuff which is not part of the algorithm interface, which sounds like it might be useful, such as
the log stuff
and various getters
do we want this on the python side?
Dario Izzo
@darioizzo
Jun 04 2016 20:01
the log is important as it allow plotting the internals of the algo, so it must be extrated
the getters no .... but which getters are there?
Francesco Biscani
@bluescarni
Jun 04 2016 20:01
get_seed(), get_verobisty()... things like these
Dario Izzo
@darioizzo
Jun 04 2016 20:02
those are part of the algo inteface right?
Francesco Biscani
@bluescarni
Jun 04 2016 20:02
not the getters
Dario Izzo
@darioizzo
Jun 04 2016 20:02
get_seed is not?
Francesco Biscani
@bluescarni
Jun 04 2016 20:02
only the setters
Dario Izzo
@darioizzo
Jun 04 2016 20:02
ah right
Francesco Biscani
@bluescarni
Jun 04 2016 20:02
no... it's not really specified what set_seed() does, it could do nothing
nothing is recorded in the algorithm class
Dario Izzo
@darioizzo
Jun 04 2016 20:02
yeah true ...
is it a pain to expose the getters? they are indeed not super important
we could remove them from the c++ side too, but if is no big deal why?
Francesco Biscani
@bluescarni
Jun 04 2016 20:04
I can do it, it's not hard
it's just work :)
did you take a look at other sphinx themes?
Dario Izzo
@darioizzo
Jun 04 2016 20:06
The reason to have the seed getter is to be able to instantiate a new algorithms using that seed thus repeating the optimzation. The reason to have the verbosity getter is to mah ...
sphinx theme -> yes I looked at them ... not a huge fan ....
rtd or bootstrap are still my favs ....
The theme linked is cluttered and avoid with contrast
Francesco Biscani
@bluescarni
Jun 04 2016 20:12
so it looks like we either need to theme the boostrap, or RTD
Dario Izzo
@darioizzo
Jun 04 2016 20:15
i think so yes
whats your take?
Francesco Biscani
@bluescarni
Jun 04 2016 20:16
I like bootstrap better, but have not CSS/HTML skills nor desire to acquire them. Maybe we could see if anyone ran into the same issue?
Dario Izzo
@darioizzo
Jun 04 2016 20:16
i googled breathe bootstrap c++ ... nothing
id u try?
Francesco Biscani
@bluescarni
Jun 04 2016 20:16
it's not a breathe thing, the exact problem is with normal python docs
Dario Izzo
@darioizzo
Jun 04 2016 20:17
so its a bootstrap thing?
Francesco Biscani
@bluescarni
Jun 04 2016 20:17
yes
Dario Izzo
@darioizzo
Jun 04 2016 20:17
what is the exact complaints we have?
Francesco Biscani
@bluescarni
Jun 04 2016 20:18
that the bootstrap theme is poop when documenting apis basically
Dario Izzo
@darioizzo
Jun 04 2016 20:21
what are other projects doing? they all in rtd ... do you know of any using bootstrap?
Francesco Biscani
@bluescarni
Jun 04 2016 20:21
https://github.com/ryan-roemer/sphinx-bootstrap-theme there's some examples here, look for code documentation sections in the examples
some don't look so bad
maybe we just need to find the right boostwatch theme or something
Dario Izzo
@darioizzo
Jun 04 2016 20:23
where are the examples?
Francesco Biscani
@bluescarni
Jun 04 2016 20:24
section "Introduction and Demos"
Dario Izzo
@darioizzo
Jun 04 2016 20:25
can you post a link to a page that according to you "does not look bad?"
Francesco Biscani
@bluescarni
Jun 04 2016 20:27
well actually the ones that do not look like poop are mostly about functions rather than classes and methods
http://ryan-roemer.github.io/django-cloud-browser/cloud_browser/tags.html if you see here the class doc has the same problem, it's too little indented
dunno
Dario Izzo
@darioizzo
Jun 04 2016 20:29
I am not sure I understand your concern
both links look the same to me
Francesco Biscani
@bluescarni
Jun 04 2016 20:30
I think the main issues that we have now are:
  • poor spacing/separation between functions (see the utils documentation where you had to add the rulers manually)
  • the documentation of methods in classes is poorly indented - it's really hard to understand if we are reading about methods of the class, or if the class is finished and we are reading about free functions or methods of another class
it's just poorly formatted/spaced in my opinion
so I was kinda hoping that other boostrap/bootwatch themes would have these issues ameliorated, but it does not look like it
Dario Izzo
@darioizzo
Jun 04 2016 20:33
On the first point .... how would that appear in rtd ... ?
I mean, if you are documenting a new function it appears obvious that the separation between the previous and following text has to be manually decided no?
I do not hate it particularly, I mean also the rst:
.. cpp_constrained_optimization

Constrained optimization utilities
======================================

A number of utilities to compute quantities that are of relevance to 
constrained optimization tasks.

--------------------------------------------------------------------------

.. doxygenfunction:: pagmo::sort_population_con(const std::vector<vector_double>&, vector_double::size_type, const vector_double&)
   :project: PaGMOreborn

--------------------------------------------------------------------------

.. doxygenfunction:: pagmo::sort_population_con(const std::vector<vector_double>&, vector_double::size_type, double)
   :project: PaGMOreborn
quite clear .... and adding one is just cut and paste anyway, so if you have to cut and paste a few more lines in the same "action" its no biggies
On the second point ... I agree fully. Indentation is there but is not enough. Separation between successive methods is just unsufficient
Francesco Biscani
@bluescarni
Jun 04 2016 20:41
I need to go afk for some, but I'll paste here some screenshots later
just to see how it looks with rtd
Dario Izzo
@darioizzo
Jun 04 2016 20:41
k later
Francesco Biscani
@bluescarni
Jun 04 2016 21:07
with some C++14 kung fu we have now a generic log getter, which actually works :)
the nontrivial bit was to write a function that converts a C++ tuple into a python tuple
Dario Izzo
@darioizzo
Jun 04 2016 21:08
you touvhed the c++ code for the logs?
Francesco Biscani
@bluescarni
Jun 04 2016 21:09
nope
the log is now returned in python as a list of tuples
like in C++ basically
Dario Izzo
@darioizzo
Jun 04 2016 21:10
:) "now" as in differently from before .... which indeed is since before it did not exist
Francesco Biscani
@bluescarni
Jun 04 2016 21:10
yes the "now" bit was confusing
you see that in rtd the rulers are deundant
redundant
Dario Izzo
@darioizzo
Jun 04 2016 21:13
still like their look there ... I would not remove :)
Francesco Biscani
@bluescarni
Jun 04 2016 21:14
really
Dario Izzo
@darioizzo
Jun 04 2016 21:14
sorry :)
Screen Shot 2016-06-04 at 23.13.11.png
This message was deleted
You like this?
Francesco Biscani
@bluescarni
Jun 04 2016 21:15
that's pretty ugly
Dario Izzo
@darioizzo
Jun 04 2016 21:15
lol
but I agree
Francesco Biscani
@bluescarni
Jun 04 2016 21:17
I imagine all the utilities also need to be exposed right?
Dario Izzo
@darioizzo
Jun 04 2016 21:18
yep
those are functions though should be straight forw
Francesco Biscani
@bluescarni
Jun 04 2016 21:19
the params and the return types are the ugly bits
lots of vector<vector<..>> to be converted back and forth to numpy and stuff like that
the way I see RTD is the same way I see standard latex stuff... it's not fashionable, but it's built to be readable and to get the job done (i.e., make documentation accessible and clear)
bootstrap looks good today, but it will get outdated eventually.. RTD is already not fashionable, so one less thing to care about :)
anyway the exposition of the classes is finished, should I proceed to documentation and testing or do the utils first?
Dario Izzo
@darioizzo
Jun 04 2016 21:23
docs and tests I would say
Francesco Biscani
@bluescarni
Jun 04 2016 21:23
booooring.. ok
Dario Izzo
@darioizzo
Jun 04 2016 21:23
Screen Shot 2016-06-04 at 23.22.51.png
this?
Francesco Biscani
@bluescarni
Jun 04 2016 21:24
can you show snippets from class docs and the utils stuff?
I mean class methods rather than the class description
Dario Izzo
@darioizzo
Jun 04 2016 21:24
They are similar to now, but with small css changes we can highlight stuff
Francesco Biscani
@bluescarni
Jun 04 2016 21:25
but then we can tweak the CSS of bootstrap as well then?
Dario Izzo
@darioizzo
Jun 04 2016 21:26
yes that too
Francesco Biscani
@bluescarni
Jun 04 2016 21:26
btw everyone uses bootstrap, we are like the mac assholes who think they are edgy and cool because they have a "think different" sticker on their prius :)
just saying
Dario Izzo
@darioizzo
Jun 04 2016 21:27
ok, ok .... rtd it is then ... we are loosing far too much time on this bllst
Francesco Biscani
@bluescarni
Jun 04 2016 21:27
no try the bootstrap css first, I can fill in the docs without caring how they are rendered
it's not like I woul dbe doing useful stuff now anyway
it's a saturday night and I am destroyed by a long day with 2 babys
Dario Izzo
@darioizzo
Jun 04 2016 21:28
:) ok will give it a try
Miriam and Ben?
Francesco Biscani
@bluescarni
Jun 04 2016 21:28
lol
fabio was visiting with Luca
Dario Izzo
@darioizzo
Jun 04 2016 21:28
cool! ok but then you were 2 vs 2
easy peasy
Francesco Biscani
@bluescarni
Jun 04 2016 21:29
but babies scale nonlinearly
Dario Izzo
@darioizzo
Jun 04 2016 21:29
log?
Dario Izzo
@darioizzo
Jun 04 2016 21:30
1, 1, 2, 12, 288, 34560, 24883200, 125411328000
Sorry I am a nerd
But 1 baby =1, 2 babies =2
I do not see your complain
it's about complexity
the starting point is not necessarily 1 :)
Dario Izzo
@darioizzo
Jun 04 2016 21:31
I see
Dario Izzo
@darioizzo
Jun 04 2016 21:46
Screen Shot 2016-06-04 at 23.45.56.png
better like this?
Francesco Biscani
@bluescarni
Jun 04 2016 21:49
I think the methods' names should be smaller and more indented, but the highlighting is a good idea
Dario Izzo
@darioizzo
Jun 04 2016 21:50
smaller as in same font as the type?
Francesco Biscani
@bluescarni
Jun 04 2016 21:53
how do you paste screenshots in the chat?
Marcus Märtens
@CoolRunning
Jun 04 2016 21:53
@darioizzo any comments on my wall of text?
Dario Izzo
@darioizzo
Jun 04 2016 21:53
a lot of ...
but I wanted first to see the code ....
Marcus Märtens
@CoolRunning
Jun 04 2016 21:54
Alright :)
Dario Izzo
@darioizzo
Jun 04 2016 21:54
On the algorithms (like should we port them or not), its up to you to judge their value.
I guess if a few famous ones are there it would be good to have them
Marcus Märtens
@CoolRunning
Jun 04 2016 21:55
their value is debatable... some of them are historic and outperformed.
Dario Izzo
@darioizzo
Jun 04 2016 21:55
But if they only were added for the purpose of the one paper and nobody uses them, then well ... mmmm
Marcus Märtens
@CoolRunning
Jun 04 2016 21:55
Interesting, if you wanna do benchmarking, but I doubt someone will ever do.
It was good implementing them back then because we did not know the best one
And it was a way of double-checking results
with different algorithms
Dario Izzo
@darioizzo
Jun 04 2016 21:55
If all the best ones are in already, then no need to port them
Marcus Märtens
@CoolRunning
Jun 04 2016 21:55
Well, we could still port the 4d one and the approximation
Dario Izzo
@darioizzo
Jun 04 2016 21:56
on the test suite .... would be nice to have it
In case we overhaul the code it will be important to know if we screw up
Marcus Märtens
@CoolRunning
Jun 04 2016 21:56
If something fundamental breaks - we will notice
But current tests are with point-sets like {1,2},{2,1} and a reference point at {3,3}...
So... not the most difficult case for algorithms
I will wait for your codereviews then... I cannot really judge how much we need/want to refactor what is currently there and what priority that would have
For the time being, I am just happy, the stuff is there :)
And last comment: the last screenshot from Dario: looks good to me.
Dario Izzo
@darioizzo
Jun 04 2016 22:28
Committed changes to the template ... try if you like the look .... its just an attempt we can always remove
Francesco Biscani
@bluescarni
Jun 04 2016 22:37
how did you know which elements to change in the css?
Dario Izzo
@darioizzo
Jun 04 2016 22:38
do you like it?
Francesco Biscani
@bluescarni
Jun 04 2016 22:39
the rulers I'll never like, they are a punch in the eye for my fashion sense :)
Dario Izzo
@darioizzo
Jun 04 2016 22:40
You visualize the page in a browser, then you can inspect the page (right click menu option). Then you can actually play around css and see real time the effect. You can also, hovering a part of the page, get the current css resolve order ... and more
The rulers ... i think they improve the overall structure readability, but I agree that they may look funny .... to an artistic eye
Francesco Biscani
@bluescarni
Jun 04 2016 22:49
this inspect thing is pretty good, never knew it even existed
Dario Izzo
@darioizzo
Jun 04 2016 22:52
its the bread and butter of css html programmers ...
cool tool that will soon make you realize the absurdness of the whole css thing
Francesco Biscani
@bluescarni
Jun 04 2016 22:53
I don't wanna learn, I like my ignorance in such matters :)
Dario Izzo
@darioizzo
Jun 04 2016 22:56
treasure it its a good skill to have "i do not fucking touch html and css"
Francesco Biscani
@bluescarni
Jun 04 2016 22:57
hey I wrote a website last year just via default bootstrap stuff :) never touched anything manually, and it was even fully html5/css compliant :)
Marcus Märtens
@CoolRunning
Jun 04 2016 22:58
@darioizzo have you ever tried firebug from firefox?
It can turn your webpage into 3d where you see the class-hierarchy of the css-divs and stuff... still flashes me.
Dario Izzo
@darioizzo
Jun 04 2016 23:00
yeah ... pretty cool
i always wondered why chrome did not clone it ...
but as I said, the css hierarchy is something to keep as far as possible from .... you would need a pool of lawyer to make sense out of it