Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 10:56
    stefan-k closed #115
  • 10:56
    stefan-k commented #115
  • May 09 17:18
    fatirishman53 commented #115
  • May 09 05:48
    stefan-k commented #115
  • May 08 15:33
    fatirishman53 commented #115
  • May 08 13:10
    stefan-k commented #115
  • May 08 13:10
    stefan-k commented #115
  • May 08 13:08
    stefan-k commented #115
  • May 08 13:05
    stefan-k reopened #115
  • May 08 13:05
    stefan-k commented #115
  • May 08 08:07
    fatirishman53 commented #115
  • May 08 08:00
    fatirishman53 commented #115
  • May 08 06:42

    stefan-k on master

    Removed extern crate definition… (compare)

  • May 08 06:42
    stefan-k closed #116
  • May 08 06:24
    stefan-k labeled #117
  • May 08 06:24
    stefan-k labeled #117
  • May 08 06:24
    stefan-k opened #117
  • May 08 06:16
    stefan-k closed #115
  • May 08 06:16
    stefan-k commented #115
  • May 08 06:14
    stefan-k opened #116
Stefan Kroboth
@stefan-k
thats strange! I don't see a reason why the code should not take that path (neither in steepest descent nor in hager zhang). I have never used gdb, so I can't really help here. maybe you could try compiling without any optimizations? just wild guess...
Tejas Sanap
@whereistejas
im compiling unoptimized
what do you use to debug?
Stefan Kroboth
@stefan-k
so far print outs served me well
Tejas Sanap
@whereistejas
if im not wrong, we should be getting the minimum of (1, 1) for rosenbrock function in steepestdescent.rs, correct?
was this your reference when you wrote that example: https://pythonhosted.org/algopy/examples/minimization/rosenbrock_banana.html ?
are you getting the correct result when using the more-thuente method in the steepestdescent.rs example?
Stefan Kroboth
@stefan-k
thats the reference: https://en.wikipedia.org/wiki/Rosenbrock_function
Yes, the result is correct (or at least close) for More Thuente
Tejas Sanap
@whereistejas
can you share your terminal output with me, please?
this is what you are getting: param (best): [1.0724442427550391, 1.151531565672518] ?
Stefan Kroboth
@stefan-k
yes
Tejas Sanap
@whereistejas
I get the following result: param (best): [1.437005348473935, 2.0667174296125412] when I give [2.0, 2.0] as init_param
that's pretty bad, isn't it?
Stefan Kroboth
@stefan-k
how many iterations?
Tejas Sanap
@whereistejas
a10
should I increase it to 20 or 30?
Stefan Kroboth
@stefan-k
no, not bad
have a look at the rosenbrock function. it is notoriously difficult to optimize due to the flat valley
Tejas Sanap
@whereistejas
thats difficult?
oh
Stefan Kroboth
@stefan-k
yes, absolutely. there is almost no gradient in the valley
Tejas Sanap
@whereistejas
i see
Stefan Kroboth
@stefan-k
you will need many iterations to find the global optimum
Tejas Sanap
@whereistejas
i can see why numerical optimization is so tricky
Stefan Kroboth
@stefan-k
:D
Tejas Sanap
@whereistejas
its interesting
I can see why people would want to use an machine learning to do something like this, we can basically just offload the strategizing part to the computer
*use machine learning
Stefan Kroboth
@stefan-k
machine learning also needs optimization
Tejas Sanap
@whereistejas
true
Tejas Sanap
@whereistejas
rosenbrock.png
i finally managed to plot the rosenbrock function
so we are looking for the valley in the middle, correct?
but, how does the algorithm know along which plane to search?
does the algorithm infer the plane based on the initial parameters?
Stefan Kroboth
@stefan-k
you are looking for a single point in the banana-shaped valley. Steepest descent computes the gradient at initial position. starting from the initial position, it will use the gradient to define the hyperplane which will cut the cost function. along this 1D cut, it will use a linesearch to find a suitable step length. the algorithm will then move to initial position - step length * gradient. This will be the new position at which it will again compute the gradient, cut the cost function, compute step length, take a step, ... and so on. it will repeat this iteratively until a stopping criterion is met. Nocedal & Wright have very good chapters on line searches and gradient descent and can explain this much better than I can
Tejas Sanap
@whereistejas
hi @stefan-k , I tried working the hager line issue without much success
if you dont mind, i would like to move onto something else
im not going to give up on this issue, but i dont want to spend all my time on this
Stefan Kroboth
@stefan-k
I can understand. What are you interested in?
Tejas Sanap
@whereistejas
i think I can write tests
as you said, this crate needs good tests, right?
Stefan Kroboth
@stefan-k
yes, I think this would be really valuable
Tejas Sanap
@whereistejas
how would you suggest I get started with this?
should we start with testing the solvers or the core logic?
i dont have much experience with unit testing, so this will be a very good experience for me, too
Stefan Kroboth
@stefan-k
An easy start could be OpWrapper ( https://github.com/argmin-rs/argmin/blob/master/src/core/opwrapper.rs ) which has no tests yet. Apart from that, since you are a bit new to optimization, I would suggest to work through a book, maybe wright & nocedal, and during that think of possible tests for a particular solver. I think meaningful tests for the solvers can only be found when one understands the algorithm in detail.
Tejas Sanap
@whereistejas
I agree with you on that 100%. Let me start with OpWrapper. We can take it from there.
Stefan Kroboth
@stefan-k
sounds great!