These are chat archives for bluescarni/pagmo_reborn

9th
May 2016
Dario Izzo
@darioizzo
May 09 2016 10:15
wtf .. what is this telemetry?
Francesco Biscani
@bluescarni
May 09 2016 10:16
Apparently it's only VS15 feature. It logs at least when your app is executed.
Dario Izzo
@darioizzo
May 09 2016 10:16
You call that a feature? God we're so lucky. /s
App? Ma Vaffanculo
v-App-iartela nel culo!
Francesco Biscani
@bluescarni
May 09 2016 10:17
lol
so glad I stick with linux
Dario Izzo
@darioizzo
May 09 2016 10:20
always ... and more ...
are you going to do the appveyor stuff soon? I am asking because I am curious to see it :)
and because I hate the word Failed
and the color red
Francesco Biscani
@bluescarni
May 09 2016 10:21
I got the stupid boost compiled with the stupid MSVC yesterday, so that's a prereq that's done hopefully
next I need to make a tarbz package of the boost headers + unit test dlls to upload somewhere
so appveyor can fetch it during the build
Dario Izzo
@darioizzo
May 09 2016 10:23
SO at the end one can also make the release files and upload them somewhere right?
Francesco Biscani
@bluescarni
May 09 2016 10:24
yeah eventually... but that requires scripting with windows bullshit, so that's gonna be painful
Dario Izzo
@darioizzo
May 09 2016 10:24
the glorious batch scripting?
Francesco Biscani
@bluescarni
May 09 2016 10:24
yeps
Dario Izzo
@darioizzo
May 09 2016 10:25
bash vs batch -> cannot be that different :)
Francesco Biscani
@bluescarni
May 09 2016 10:25
https://github.com/symengine/symengine/blob/master/appveyor.yml it's not that different from travis or gitlab ci
Francesco Biscani
@bluescarni
May 09 2016 10:30
it's just endless pain and rust to get everything done
Francesco Biscani
@bluescarni
May 09 2016 11:00
lol
Francesco Biscani
@bluescarni
May 09 2016 11:08
the bit at the end about TODD talks is amazing
Dario Izzo
@darioizzo
May 09 2016 11:34
brilliant
Francesco Biscani
@bluescarni
May 09 2016 11:35
coffee cures cancer! or causes cancer? or is it both? or is it both and neither??
I did not know about this p hacking thing
Dario Izzo
@darioizzo
May 09 2016 11:37
p as in p value ... yeah super cool video
:) find a study that suits you the most and just use it !!!!
Francesco Biscani
@bluescarni
May 09 2016 11:38
that's what I always do! I choose to ignore everything else and keep on churning coffee
Dario Izzo
@darioizzo
May 09 2016 11:48
Do you think we should or will introduce some more exceptions? I mean, we are mainly using std::invalid_argument, but we could derive from it and have:
pagmo::invalid_bounds
pagmo::invalid_fitness_dimension
pagmo::invalid_constraint_dimension
etc ...
Francesco Biscani
@bluescarni
May 09 2016 11:48
so there's two things regarding exceptions
Dario Izzo
@darioizzo
May 09 2016 11:48
This would make the tests more reliable as right now we could think to check for a given excpetion to be raised, but we actually pass because somewhere else another exception is
Francesco Biscani
@bluescarni
May 09 2016 11:50
1) the first is the value of having diversified exceptions -> I see this similar to the namespace discussion. If the point is to make things more fine-grained or more "categorized" I am against it in general. I see value in custom exceptions when you want to catch them mostly (e.g., you want to handle exactly the case in which there's an out-of -bound condition rather than a more generic error)
but as long as we don't have concrete examples I would not make them more specific
2) the translation to python
I know how to do it (roughly) in Boost python, but I don't know if there is such a customizable mechanism in place in pybind
http://pybind11.readthedocs.io/en/latest/advanced.html#catching-and-throwing-exceptions I think at the moment pybind is rather hard-coded in this respect
Dario Izzo
@darioizzo
May 09 2016 11:52
Ok. My use case at the moment is in the test when we BOOST_EXCEPTION_THROW, where also when the test passes we are not really sure what exception is thrown as everything is std::invalid_argument
Francesco Biscani
@bluescarni
May 09 2016 11:53
this is what you can do with boost python (mapping c++ exceptions to python ones essentially)
in the tests you could put a check against the error string, so you check that as well :)
Dario Izzo
@darioizzo
May 09 2016 11:56
True, but then, I do not like this string checking too much as I would like to be free to change the message slightly without having to fail a test
Francesco Biscani
@bluescarni
May 09 2016 11:57
I think the main hassle is to code the tests, changing them because you alter slightly the error message would seem rather minor compared to everything else
but if we do want to introduce custom exceptions we can do it
as long as we use them consistently
Dario Izzo
@darioizzo
May 09 2016 11:58
its not the single differential, rather the whole integration that worries me ... I say we leave it as is
Marcus Märtens
@CoolRunning
May 09 2016 13:16
one exception to rule them all
Francesco Biscani
@bluescarni
May 09 2016 14:06
if we implement custom exceptions we should probably derive them from standard ones I guess?
class out_of_bounds: std::invalid_argument
{
/// ...
};
don't know how idiomatic this is
https://isocpp.org/wiki/faq/exceptions#what-to-throw this one as well, straight from the horse's mouth
Francesco Biscani
@bluescarni
May 09 2016 14:57
going hoooome
Dario Izzo
@darioizzo
May 09 2016 14:58
good voyage
Dario Izzo
@darioizzo
May 09 2016 17:23
oh god, the sobol sequence is using a third party code from here http://people.sc.fsu.edu/~jburkardt/cpp_src/sobol/sobol.cpp
We should start talking how we deal with these third party snippets, they cannot be up-to our standards, nor they can be fully made so.
In the code you will see delete[] and new, useless use of pointers and stuff
btw, what is the point of using unsigned int *seed
a pointer to an unsigned .... mah
Suggestions:
1) We just document the code origin and test it thoroughly
2) We do not include it in the core project and we only allow this stuff to be in the affiliated projects
3) We modify the code minimally to at least comply with the basics
Dario Izzo
@darioizzo
May 09 2016 17:28
In case of the discrepancy, if we choose 2) we lose Faure, Sobol and the Latin Hypercube Sampling ..... We basically only remain with Halton, van der Corput and the simplex, which are enough for basic low-dimensional uses.
thoughts?
Marcus Märtens
@CoolRunning
May 09 2016 17:32
The same problem arises in context with the hypervolume, as we use 3rd-party algorithms to compute it in higher dimensions
I think it would be nice to have a solid and slim core of PaGMO that is indeed compliant with high standards of quality and does only include what we ourselves implemented. However, there should be an "extended" version or option in which we sort of relax this constraint and allow for most of the 3rd party shit. I am not so clear how feasible this would be though.
Dario Izzo
@darioizzo
May 09 2016 17:37
We were thinking about the idea of having "affiliated projects" as in separate projects that add functionalities to PaGMO. So we would keep the core nice and clean. The problem I see is that I am now ready to port all the discrepancy thing in PaGMO, if we delay this to when the affiliated projects will be in place (1 year?) I am not so sure I will do it ... !
FOr the hypervolume, how many of the algos are third party? All or a few?
Marcus Märtens
@CoolRunning
May 09 2016 17:38
For the hv, the situation is not so bad
We have more hv algorithms than we actually need
some of them are never used by default, because they suck runtime-wise... the only reason they are included is for benchmark-reasons.
I would get rid of those and only keep the efficient one.
Dario Izzo
@darioizzo
May 09 2016 17:39
So if you had to port only the ones that are nicely coded, we would still have a good hypervolume feature?
Marcus Märtens
@CoolRunning
May 09 2016 17:40
Well... the most general algorithm (wfg) is 3rd party, but I think it can be refactored / reimplemented
I have to have a closer look
For the 4-dimensional case, there is a more efficient algorithm than wfg, which is also 3rd party
not sure its worth though as 4 dimensions is still rather managable
So in a nutshell: depends on WFG
there might be more, which I overlooked though... we had quite the complex collection and optimized the shit out of this ensemble :D
Dario Izzo
@darioizzo
May 09 2016 17:44
k, will see then, let us know
Marcus Märtens
@CoolRunning
May 09 2016 17:45
just checked: WFG wasn't third party, but it contains some delete[] stuff... need to rewrite it.
Dario Izzo
@darioizzo
May 09 2016 17:45
ok, but that is a small refactor
Marcus Märtens
@CoolRunning
May 09 2016 17:45
yeah, shouldn't be a big deal
but 4d is for real
Dario Izzo
@darioizzo
May 09 2016 17:45
for the discrepancy stuff its impossible to refactor, so we either lose it or we need to live with it
Marcus Märtens
@CoolRunning
May 09 2016 17:46
how about a compilation flag?
compile with discrepancy magic or compile without?
Dario Izzo
@darioizzo
May 09 2016 17:48
That would work, but in the long run it would bring to an accumulation of flags and options which I am not sure we want.
I think the "affiliated" project idea is the way to go so that also during GSoC we can have students touch code that is not in core
Marcus Märtens
@CoolRunning
May 09 2016 17:48
sure - this is not a general solution.
what do you mean by "affiliated" project exactly?
Dario Izzo
@darioizzo
May 09 2016 17:50
Francesco introduced me to the idea, he took it from astropy, I think It would be a separate project, but somehow connected via the git submodule or subtree options .... not sure though, we still have to look into how to implement them.
De facto they would be the equivalent of compiling stuff activating an option, but cleaner
Marcus Märtens
@CoolRunning
May 09 2016 17:53
Looks complicated...
Another hypervolume-issue:
The old hv object had a method "get_nadir_point" which is redundant as we have this in multi-objective now
I think I want to overload the nadir-method in multi-objective so that it takes a hypervolume-object and returns the nadir
Is that okay?
Dario Izzo
@darioizzo
May 09 2016 17:57
can you not call it directly with the std::vector<vector_double> ?
Also the current nadir method is O(MN^2), what is the complexity of the one in hypervolume?
It is not difficult to write something O(MNlogN) as one only needs to find the first non dominated front to get the nadir, not all of them as its currently done
Marcus Märtens
@CoolRunning
May 09 2016 18:00
oh... your nadir seems to be different then
Maybe I should take a closer look.
For me, Nadir is simply the maximum in each dimension, which is linear to compute.
Dario Izzo
@darioizzo
May 09 2016 18:01
is that not the ideal?
I am confused
wait
Marcus Märtens
@CoolRunning
May 09 2016 18:01
nadir{{3,4,5},{8,3,1}} = {8,4,5}
given a minimisation problem
ideal would be the minimum in all coordinates.
Dario Izzo
@darioizzo
May 09 2016 18:02
ok, no that is only true as in your case all points are non dominated
If you have a set of points the nadir is the max taken along each direction WITHIN the fisrt non dominated front
Marcus Märtens
@CoolRunning
May 09 2016 18:05
In that case you could have points worse than the nadir in your set of points.
Dario Izzo
@darioizzo
May 09 2016 18:05
In the picture A is the nadir, I the Ideal. A+ is what you refer to, but I do not think it has a name ...
Marcus Märtens
@CoolRunning
May 09 2016 18:06
yes, I refer to A+
adding a small offset to A+, this is commonly used for computations of the hypervolume as reference point
Dario Izzo
@darioizzo
May 09 2016 18:06
ok than you have to give it a name (look in the literature, I am sure it has one) and add it in multiobjective
then you can either wrap it as an overload or call it directly with the signature std::vector<vector_double>
Marcus Märtens
@CoolRunning
May 09 2016 18:10
I have to think about this. This might be a very hv-specific thing. Maybe I leave it as a method for the hv-object
But you are right, the nadir-point is actually defined with respect to the Pareto-front. Interesting, I always believed it was somehwere else :D
So maybe I will no longer call it nadir
but something like ref_point
poor mans ref-point... if you don't know what your ref-point is, you take this one. Mhh... anyway.
Marcus Märtens
@CoolRunning
May 09 2016 18:15
There is another thing considering populations
the old way was as follows:
hv = hypervolume(pop) was extracting and copying by value the fitness coordinates to its internal memory
If the pop evolved somehow, one would need to construct a new hv object
Dario Izzo
@darioizzo
May 09 2016 18:17
I would not offer this constructor anymore.
Just hv = hypervolume(points)
Marcus Märtens
@CoolRunning
May 09 2016 18:18
Not at all? So how do we compute the hypervolume of a population then? We would always need to write a method to get the fitness vectors out.
Dario Izzo
@darioizzo
May 09 2016 18:18
then we can instantiate it as hv = hypervolume(pop.get_f())
in population there is not anymore the notion of an individual
you directly get chromosomes, fitness and so on
Marcus Märtens
@CoolRunning
May 09 2016 18:19
Well, there never was an individuum class ever, true.
Dario Izzo
@darioizzo
May 09 2016 18:19
The fitness contains the objectives and (appended) the constraints
So we could offer an overload hypervolume(pop) that checks that the problem is unconstrained or that only extracts the objectves from the fitness
Or .... we could document it and let it to the user to pass the correct points to the hypervolume object
without offering the overload
whats your take?
Marcus Märtens
@CoolRunning
May 09 2016 18:21
I would like the overload. It is more convenient I guess.
Dario Izzo
@darioizzo
May 09 2016 18:21
ok, then it would look smething ike:
hypervolume(const population &pop) {
     // check that the problem is unconstrained
    m_points = pop.get_f();
}
hypervolume(const std::vector<vector_double> &points) : m_points(points) {
     // check that points make sense
}
right?
modified it :)
Marcus Märtens
@CoolRunning
May 09 2016 18:25
basically yes
Dario Izzo
@darioizzo
May 09 2016 18:25
cool then .. go for it
Marcus Märtens
@CoolRunning
May 09 2016 18:26
I wonder, if you can tell a constructor from a constructor?
Dario Izzo
@darioizzo
May 09 2016 18:26
you mean overload? yes
Marcus Märtens
@CoolRunning
May 09 2016 18:26
but that is probably too nasty ^^
Dario Izzo
@darioizzo
May 09 2016 18:26
call a constructor? too yes
Marcus Märtens
@CoolRunning
May 09 2016 18:26
Because the pop-constructor just needs to check for the constraints and if there are none extract the points
from that point on, it could just hand it over to the point-constructor
Dario Izzo
@darioizzo
May 09 2016 18:27
yep ... can do, better
Marcus Märtens
@CoolRunning
May 09 2016 18:27
Okay - what about the constraints? How to check for them? Are they included already?
Dario Izzo
@darioizzo
May 09 2016 18:38
problem.get_nc() == 0
nc (constraint) nec (equality constraints) nic (inequality constraints)
nc = nic + nec
Marcus Märtens
@CoolRunning
May 09 2016 18:42
Would these constraints necessarily impact the hypervolume? They would invalidate some fitness-points, correct?
Dario Izzo
@darioizzo
May 09 2016 18:49
This message was deleted
No, its just that in the new PaGMO if you have a fitness {1,2,3,3,4}
the first nobj components are objectives, the remaining nc = nf - nobj numbers are constraints
So if you allow the hypervolume to be computed on the whole fitness (i.e. on the concatnation of objectives and constraints)
you would be treating constraint as if they were objectives....
Dario Izzo
@darioizzo
May 09 2016 19:01
If, on the other hand you compute the hypervolume only on the nobj part of the fitness, as you said some points would be invalid as in violating the constraints.
Marcus Märtens
@CoolRunning
May 09 2016 19:01
I see. This will not happen. Only unconstrained populations are allowed. For everything constrained, the user should think more carefully about what he is doing and then he can extract the points and compute the hypervolume on them.
Dario Izzo
@darioizzo
May 09 2016 19:01
Agree
Francesco Biscani
@bluescarni
May 09 2016 19:11
int i4_max ( int i1, int i2 )

//****************************************************************************80
//
//  Purpose:
//
//    I4_MAX returns the maximum of two integers.
//
//  Licensing:
//
//    This code is distributed under the GNU LGPL license. 
//
//  Modified:
//
//    05 May 2003
//
//  Author:
//
//    John Burkardt
//
//  Parameters:
//
//    Input, int I1, I2, two integers to be compared.
//
//    Output, int I4_MAX, the larger of I1 and I2.
//
{
  if ( i2 < i1 ) 
  {
    return i1;
  }
  else 
  {
    return i2;
  }

}
//****************************************************************************80
really...
Marcus Märtens
@CoolRunning
May 09 2016 19:12
Francesco Biscani
@bluescarni
May 09 2016 19:12
  static int atmost;
  static int dim_num_save = 0;
  int i;
  bool includ[LOG_MAX];
  static bool initialized = false;
  int j;
  int j2;
  int k;
  int l;
  static int lastq[DIM_MAX2];
  int m;
  static int maxcol;
  int newv;
  static int poly[DIM_MAX2] =
these static variables it looks like they would be trouble for multithreading
Francesco Biscani
@bluescarni
May 09 2016 19:28
is this code ever run in parallel?
Marcus Märtens
@CoolRunning
May 09 2016 19:30
no... just no.
Francesco Biscani
@bluescarni
May 09 2016 19:36
//
//  Check that the user is not calling too many times!
//
  if ( maxcol < l )
  {
    cout << "\n";
    cout << "I8_SOBOL - Fatal error!\n";
    cout << "  The value of SEED seems to be too large!\n";
    cout << "  SEED =   " << *seed  << "\n";
    cout << "  MAXCOL = " << maxcol << "\n";
    cout << "  L =      " << l << "\n";
    exit ( 2 );
  }
Francesco Biscani
@bluescarni
May 09 2016 19:42
//****************************************************************************80

float r4_abs ( float x )

//****************************************************************************80
//
//  Purpose:
//
//    R4_ABS returns the absolute value of an R4.
//
//  Licensing:
//
//    This code is distributed under the GNU LGPL license. 
//
//  Modified:
//
//    01 December 2006
//
//  Author:
//
//    John Burkardt
//
//  Parameters:
//
//    Input, float X, the quantity whose absolute value is desired.
//
//    Output, float R4_ABS, the absolute value of X.
//
{
  float value;

  if ( 0.0 <= x )
  {
    value = x;
  } 
  else
  {
    value = -x;
  }
  return value;
}
//****************************************************************************80

int r4_nint ( float x )

//****************************************************************************80
//
//  Purpose:
//
//    R4_NINT returns the nearest integer to an R4.
//
//  Example:
//
//        X         R4_NINT
//
//      1.3         1
//      1.4         1
//      1.5         1 or 2
//      1.6         2
//      0.0         0
//     -0.7        -1
//     -1.1        -1
//     -1.6        -2
//
//  Licensing:
//
//    This code is distributed under the GNU LGPL license. 
//
//  Modified:
//
//    14 November 2006
//
//  Author:
//
//    John Burkardt
//
//  Parameters:
//
//    Input, float X, the value.
//
//    Output, int R4_NINT, the nearest integer to X.
//
{
  int value;

  if ( x < 0.0 )
  {
    value = - ( int ) ( r4_abs ( x ) + 0.5 );
  }
  else
  {
    value =   ( int ) ( r4_abs ( x ) + 0.5 );
  }

  return value;
}
holy crap
so it looks like this file is 80% data, 15% duplicated code from the standard library, 5% juice
100% shit code rife with undefined behaviour, assumptions about sizes of integral values, thread unsafety
all in all pretty typical "research code"
anyhow, regarding the affiliated packages... the idea would not be to have them as optional parts in the core pagmo, they should be separate packages relying on functionality provided by pagmo
so they would depend on pagmo, but not vice-versa
it's not clear how this would work in practice, until we actually try this out
Francesco Biscani
@bluescarni
May 09 2016 19:48
ideally they would be separate C++ and python libraries/modules
Marcus Märtens
@CoolRunning
May 09 2016 19:53
Is this actual code that was supposed to be in PaGMO or what were you posting?
Francesco Biscani
@bluescarni
May 09 2016 19:54
it's the sobolev code that dario linked above
Marcus Märtens
@CoolRunning
May 09 2016 19:55
oh fuck
Francesco Biscani
@bluescarni
May 09 2016 19:59
//
//  Here, we have commented out the definition of ATMOST, because
//  in some cases, a compiler was complaining that the value of ATMOST could not
//  seem to be properly stored.  We only need ATMOST in order to specify MAXCOL,
//  so as long as we set MAXCOL (below) to what we expect it should be, we
//  may be able to get around this difficulty.
//  JVB, 24 January 2006.
//
//static long long int atmost = 4611686018427387903;
//
no shit sherlock
that number is 2^62, and you init it as an int
which could be either 32bits or 64bits depending on the platform
static long long int atmost = 4611686018427387903LL;
bug solved
Well, at least it gives me the feeling that my own code is not complete shite...
Francesco Biscani
@bluescarni
May 09 2016 20:01
well the puzzling thing is that this guy took the time to add licensing terms and to upload the code somewhere
Marcus Märtens
@CoolRunning
May 09 2016 20:02
This code almost reads like a parody to me
Francesco Biscani
@bluescarni
May 09 2016 20:02
when I write shit code at least I hide it well in my PC and don't boast it :)
Marcus Märtens
@CoolRunning
May 09 2016 20:03
// increment number
number++;
// number is now one more than it was
Francesco Biscani
@bluescarni
May 09 2016 20:04
should make a function
void increment(int &n)
{
    ++n
}
Marcus Märtens
@CoolRunning
May 09 2016 20:04
haha - broke the syntax highlighting.
Francesco Biscani
@bluescarni
May 09 2016 20:04
lol
Marcus Märtens
@CoolRunning
May 09 2016 20:04
:D