this is the code that I'm unable to step into
https://github.com/argmin-rs/argmin/blob/dc2a06ce37f6125515ba296091673356997439e7/src/core/executor.rs#L138-L141
https://github.com/argmin-rs/argmin/blob/dc2a06ce37f6125515ba296091673356997439e7/src/core/executor.rs#L138-L141
what do you use to debug?
was this your reference when you wrote that example: https://pythonhosted.org/algopy/examples/minimization/rosenbrock_banana.html ?
are you getting the correct result when using the more-thuente method in the steepestdescent.rs example?
thats the reference: https://en.wikipedia.org/wiki/Rosenbrock_function
Yes, the result is correct (or at least close) for More Thuente
Yes, the result is correct (or at least close) for More Thuente
this is what you are getting:
param (best): [1.0724442427550391, 1.151531565672518] ?
that's pretty bad, isn't it?
should I increase it to 20 or 30?
have a look at the rosenbrock function. it is notoriously difficult to optimize due to the flat valley
oh
I can see why people would want to use an machine learning to do something like this, we can basically just offload the strategizing part to the computer
*use machine learning
i finally managed to plot the rosenbrock function
so we are looking for the valley in the middle, correct?
but, how does the algorithm know along which plane to search?
does the algorithm infer the plane based on the initial parameters?
you are looking for a single point in the banana-shaped valley. Steepest descent computes the gradient at initial position. starting from the initial position, it will use the gradient to define the hyperplane which will cut the cost function. along this 1D cut, it will use a linesearch to find a suitable step length. the algorithm will then move to
initial position - step length * gradient. This will be the new position at which it will again compute the gradient, cut the cost function, compute step length, take a step, ... and so on. it will repeat this iteratively until a stopping criterion is met. Nocedal & Wright have very good chapters on line searches and gradient descent and can explain this much better than I can
if you dont mind, i would like to move onto something else
im not going to give up on this issue, but i dont want to spend all my time on this
as you said, this crate needs good tests, right?
should we start with testing the solvers or the core logic?
i dont have much experience with unit testing, so this will be a very good experience for me, too
An easy start could be
OpWrapper ( https://github.com/argmin-rs/argmin/blob/master/src/core/opwrapper.rs ) which has no tests yet. Apart from that, since you are a bit new to optimization, I would suggest to work through a book, maybe wright & nocedal, and during that think of possible tests for a particular solver. I think meaningful tests for the solvers can only be found when one understands the algorithm in detail.