Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 16:46
    pkairys edited #921
  • 16:43
    pkairys edited #921
  • 16:43
    pkairys edited #921
  • 16:43
    pkairys opened #921
  • May 16 15:24
    kellertuer commented #920
  • May 16 15:22
    Affie commented #920
  • May 16 14:16
    kellertuer commented #920
  • May 15 20:43
    Affie commented #920
  • May 15 17:01
    mateuszbaran commented #920
  • May 15 13:54
    kellertuer commented #920
  • May 15 12:34
    kellertuer commented #920
  • May 15 12:27
    antoine-levitt commented #920
  • May 15 12:27
    antoine-levitt commented #920
  • May 15 12:26
    antoine-levitt commented #920
  • May 15 12:11
    kellertuer commented #920
  • May 15 12:03
    antoine-levitt commented #920
  • May 15 11:56
    kellertuer commented #920
  • May 15 11:53
    kellertuer commented #920
  • May 15 11:50
    antoine-levitt commented #920
  • May 15 07:37
    kellertuer commented #920
Adam
@adamglos92
Hi again. I receive an error Value and slope at step length = 0 must be finite.when using L-BFGS. My function is bounded and all derivatives are bounded. Do You know the reason of such behaviour?
Antoine Levitt
@antoine-levitt
probably it's a nan or something?
print or debug your objective function and see at which point it's evaluated
Adam
@adamglos92
a just printed it. All the values of the objective function, argument and gradient are at most 10 in abs, and there is no NaN or Inf. I forgot to mention that I actually use Fminbox on L-BFGS. May it cause the effect?
Adam
@adamglos92
although I don't think it is the case, as argument is far from boundary for each dimension
Adam
@adamglos92
It seems that passing negative values to for convergence measures may cause this effect
Adam
@adamglos92
Hi again, is there a standard way to work with periodic functions? I wrote a manifold in which I modulo the argument, however, I wonder whether there is a more standard way? Fminbox is not always good, as sometimes I got stuck close to the bounds, which were not a global minimum.
Antoine Levitt
@antoine-levitt
can't you modulo your objective function?
Adam
@adamglos92
what You mean by "modulo your objective function"? You mean modulo inside?
Antoine Levitt
@antoine-levitt
I don't understand the problem actually, what's the problem if you have a periodic function?
Adam
@adamglos92
When I didn't use Fminbox, sometimes arguments just continue to be larger and larger. When I use Fminbox, sometimes I stuck close the boundary. Imagine I optimize the sin function on [0, 2 pi]. Then 0 is local minimum, but it is not interesting. The problem is much worse for multivariate optimization.
Antoine Levitt
@antoine-levitt
I wouldn't use fminbox
It should be fine without it
Adam
@adamglos92
OK, maybe I falsely understood how it worked without minbox, I will recheck it
Christopher Ross
@cpross90

I recently made a post to the Discourse forum about trying to find a way to access the approximated Hessian from a converged solution to use as the preconditioner on a future minimization of a slightly altered initial state; I'm using BFGS to bifurcate from previously minimizers. After looking at Optim.jl, libLBFGS, and LBFGS++ I couldn't find a interface, so I imagine that it is most likely not available in Optim.jl either. I wanted to double check here if that's something viable with the current interface, if there would be any interest in adding it to the package, or if I should try to extend this on my own in Julia somehow.

So far, the only package outside of Julia that implements this is SciPy:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_bfgs.html

Antoine Levitt
@antoine-levitt
this forum is not very used, you might be better off opening an issue
do you want BFGS or LBFGS?
in any case it should simply be a matter of getting out the data you need from the function
I'd suggest hacking it to see if it works
but optim does a good job of storing all its info in a state object
so you just need access to that
in principle I guess the state should just be added to MultivariateOptimizationResults?
Antoine Levitt
@antoine-levitt
or actually you can do it very simply: the optimize function is
optimize(d::D, initial_x::Tx, method::M,
                  options::Options = Options(;default_options(method)...),
                  state = initial_state(method, options, d, initial_x))
so if you just pass a state it'll get mutated and you can get what you want from it after
Christopher Ross
@cpross90
@antoine-levitt Thanks for the response. Sorry about the confusion, LBFGS is what I am trying to work with. That's awesome, I'll try that out now!
Antoine Levitt
@antoine-levitt
cool, let me know how that goes!
jxc100
@jxc100
Is optim reentrant? Can I optimize a function that itself calls optimize()? In a message of May 10, @antoine-levitt mentions a "state" object - is this one global object (ie, non-reentrant) or local to a specific refinement?
Antoine Levitt
@antoine-levitt
It should be ok
Yeah pretty sure it's OK, report an issue if it's not the case
jxc100
@jxc100
Thanks, so far so good!
Ilja Kantorovitch
@IljaK91
Hey everyone, not sure if many people are active here, but is there a way to use box-constrained optimization without providing a gradient? There is no example in the documentation for that.
Antoine Levitt
@antoine-levitt
You'd be better off pinging @pkofod on slack or opening an issue
John Washbourne
@jkwashbourne

Hi All, new here ... We have been using Optim.jl and LBFGS for a while, and for our particular problems the combination of alphaguess = LineSearches.InitialQuadratic() and linesearch = LineSearches.MoreThuente() works best. I thought I would try that for Rosenbrock on the front page, and it beats BFGS pretty handily, at least in terms of operation count, almost 2x. Thought that was worth mentioning.

Where is a pointer to the slack channel please? Cheers

Antoine Levitt
@antoine-levitt
There's no specific slack channel, but there's #math-optimization
In my experiments a simple backtracking worked best, I guess it depends on problems...
John Washbourne
@jkwashbourne
Thanks @antoine-levitt for the reply. Our problem has way costly operations, so we scrutinize. I think I may have to dive down the linesearch rabbit hole eventually as there is some behavior I think I want to change. cheers
Segawa@動く人形キボウちゃん
@segawachobbies_twitter
I started learning the Julia language yesterday. I'm hoping that Julia might be useful for numerical optimization in my research.
One question, is there any way to get or check the Hessian obtained by Automatic Differentiation in Optim.jl in Symbolic form?
Segawa@動く人形キボウちゃん
@segawachobbies_twitter

fun(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2

For example, we want to get the following Hessian from the above objective function.

function fun_hess!(h, x)
h[1, 1] = 2.0 - 400.0 x[2] + 1200.0 x[1]^2
h[1, 2] = -400.0 x[1]
h[2, 1] = -400.0
x[1]
h[2, 2] = 200.0
end

Christopher Rackauckas
@ChrisRackauckas
@segawachobbies_twitter you can use ModelingToolkit to test it
Segawa@動く人形キボウちゃん
@segawachobbies_twitter

@ChrisRackauckas

@segawachobbies_twitter you can use ModelingToolkit to test it

using ModelingToolkit

@variables x[1:2]

result = ModelingToolkit.hessian((1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2, x)

I tried the above code in ModelingToolkit and successfully got the ideal result! Thank you very much.

Segawa@動く人形キボウちゃん
@segawachobbies_twitter
I used ModelingToolkit to find and optimize the gradient and Hesse matrices from functions only.
I compared Automatic differentiation with this method, and the difference in processing speed is about 100 times faster during optimization.
I think this is one of the options for fast optimization by just using the objective function.
using ModelingToolkit
using Optim

@variables x[1:2]

f = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
g = ModelingToolkit.gradient(f, x)
h = ModelingToolkit.hessian(f, x)

buildedF = build_function(f, x[1:2])
buildedG = build_function(g, x[1:2])
buildedH = build_function(h, x[1:2])

newF = eval(buildedF)
newG! = eval(buildedG[2])
newH! = eval(buildedH[2])

initial_x = zeros(2)

@time Optim.minimizer(optimize(newF, initial_x, Newton(); autodiff=:forward))
@time Optim.minimizer(optimize(newF, newG!, newH!, initial_x, Newton()))
Christian Rorvik
@ancapdev
Hi everyone. An open question more than any proposal; Looking at Optim I don't think there's any way to evaluate an objective function over a vector of parameters (e.g., for solvers that work on a population - like particle swarms). My specific use case involves a problem where I can amortize a lot of computation by simultaneously evaluating multiple parameters, but I can't easily factor the problem to extract and pre-compute parameter-independent parts. Has anyone given any thoughts for how the API may be adapted to support this, or have any interest in solutions being contributed?
Antoine Levitt
@antoine-levitt
I don't understand why you can't precompute exactly. You could do this easily with a closure for instance
Christian Rorvik
@ancapdev
For additional context, I have two use cases. One is building signals over high frequency market data, signals may be stateful, i.e., a function over a long history of events, and their state is also a function of parameters under optimisation. Market data is played back through an event processing API, involves building order books a high amount of IO, all which can be amortised over a set of signal parameters, but which cost can't be factored out ahead of time. Second use case is in simulating trading strategies over historical data, which again and even more so doesn't lend itself to precomputing much, but if a set of parameters are available these can be evaluated simultaneously and benefit from amortising shared costs, or even scaled out on a cluster (which is something I've hand built a parallel optimiser for in the past). What I really want from an optimisation package though, is not for it to handle parallelism or simultaneous evaluation of the objective, but to simply give me a set of parameters to evaluate for. Those two problems are fairly orthogonal
Antoine Levitt
@antoine-levitt
OK that makes sense
Gradient type solvers are not parallel but you might have more success with pso type methods
I don't think it's implemented in optim though
Jan Lukas Bosse
@jlbosse
Hey everyone,
I recently implemented the simulatenous stochastic perturbation algorthm (SPSA) and the model gradient descent (MDG, first developed in https://arxiv.org/abs/2005.11011) in a roughly Optim.jl compatible way. Is there interest in adding these methods to Optim.jl? If yes, I would make my code fully compatible to the Optim.jl interface and open a PR
Antoine Levitt
@antoine-levitt
Optim is in the process of being rewritten (allegedly 😂), open an issue on github to see what @pkofod says