Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Horace He
    @Chillee
    Or would they perform about the same?
    Toshihiko Yanase
    @toshihikoyanase

    I think that depends on your hyperparameter. If it is discrete, suggest_discrete_uniform is usually suitable for it.

    Of course, you can manually discretize the return value of suggest_float, but I don't think it is a good approach since the observation (= history of hyperparameters) is different from the values used actually.

    Horace He
    @Chillee
    well, suggest_discrete_uniform is also used for floats, right?
    Toshihiko Yanase
    @toshihikoyanase
    Yes, it is.
    Horace He
    @Chillee
    So, let's say I have a hyperparameter that varies from 0.0 - 1.0
    I'm currently using suggest_uniform
    but I'm considering using suggest_discrete_uniform, largely because it's more interpretable
    Like, if I have x=0.98 after optimization, is it actually just converging to 1.0
    And I started wondering if this has other performance implications
    so, I assume that if you did suggest_categorical on [0.0, 0.1, 0.2...], it would perform substantially worse, right?
    Toshihiko Yanase
    @toshihikoyanase
    I understand. In that case, I usually use suggest_uniform if I were you. This is mainly because the optimal point is not included in the list of possible values.
    Using suggest_categorical likely to show worse performance than suggest_discrete_uniform because the samplers cannot see the distance between values.
    19 replies
    Horace He
    @Chillee
    is there any chance it would perform better? In some sense, you have less possible values to look through, right?
    Toshihiko Yanase
    @toshihikoyanase

    less possible values

    That makes sense. If one of them is the (nearly) optimal point, the search efficiency would be better.

    Horace He
    @Chillee
    I just had an idea - what if I nest optuna :^)
    If you view each Optuna study as an optimization process, I can then just nest it in an outer optuna process and perform pruning strategies :)
    3 replies
    Maarten Haubrich
    @maartenpants
    Hi there, has anyone else experienced this issue with CmaEsSampler and the PostgreSQL storage backend:
    (psycopg2.errors.StringDataRightTruncation) value too long for type character varying(2048)
    [SQL: INSERT INTO trial_system_attributes (trial_id, key, value_json) VALUES (%(trial_id)s, %(key)s, %(value_json)s) RETURNING trial_system_attributes.trial_system_attribute_id]
    8 replies
    It seems the serialized version of the CMA class is too large to fit in the 2048 character column. This code is the culprit:
    optimizer_str = pickle.dumps(optimizer).hex()
    study._storage.set_trial_system_attr(trial._trial_id, "cma:optimizer", optimizer_str)
    Yuhei Fujioka
    @Yuhei-Fujioka

    Hi there, I have a issue with optuna.intergration.lightgbm.
    If I ran bellow code, got TypeError.
    I looked up similar issue, but I didn't found.

    from optuna.integration import lightgbm as lgb
    
    models = []
    best_params_list = []
    best_params, tuning_history = dict(), list()
    
    for train_index, valid_index in skf.split(X_train_valid.values, y_train_valid.values):
        lgb_train = lgb.Dataset(X_train_valid.iloc[train_index, ], y_train_valid.iloc[train_index, ])
        lgb_valid = lgb.Dataset(X_train_valid.iloc[valid_index, ], y_train_valid.iloc[valid_index, ], reference=lgb_train)
    
        model_ = lgb.train(
            lgbm_params, 
            lgb_train,
            valid_sets=lgb_valid,
            num_boost_round=1000,
            early_stopping_rounds=100,
            verbose_eval=10,
            best_params=best_params,
            tuning_history=tuning_history,
            random_state=RANDOM_STATE
            )
    
        models.append(model_)
        best_params_list.append(best_params)
        tuning_history.append(tuning_history)

    TypeError Traceback (most recent call last)

    <ipython-input-27-d60ed207f14a> in <module>()
    16 best_params=best_params,
    17 tuning_history=tuning_history,
    ---> 18 random_state=RANDOM_STATE
    19 )
    20

    /opt/conda/lib/python3.6/site-packages/optuna-1.5.0-py3.6.egg/optuna/integration/_lightgbm_tuner/init.py in train(args, **kwargs)
    34 _imports.check()
    35
    ---> 36 auto_booster = LightGBMTuner(
    args, **kwargs)
    37 auto_booster.run()
    38 return auto_booster.get_best_booster()

    TypeError: init() got an unexpected keyword argument 'best_params'

    5 replies
    Rohan George Thampi
    @rohan-gt
    Hi there, I'm curious about the literature or resources that were referenced for coming up the custom rules and parameter spaces in LightGBMTunerCV. Also are there plans to create similar customized tuners for other algorithms?
    2 replies
    Hideaki Imamura
    @HideakiImamura
    Thanks for your contributions as always, we’ve just released a minor update v2.1.0! This release contains more PRs involving documentation, examples and code fixes than ever before. Here is the release note https://github.com/optuna/optuna/releases/tag/v2.1.0 and our Tweet https://twitter.com/OptunaAutoML/status/1302831704940269568 if you’d like to catch up on the details.
    Hiroyuki Vincent Yamazaki
    @hvy
    🎉
    Philip May
    @PhilipMay
    When I do HP search on small datasets I often do a cross validation. With cross validation I train in the same trial with same HP params but different data (folds) for multiple times (number of folds). The result is the average of all folds. Is there an out of the box mechanism in optuna to prune these folds after a certain number of folds?
    1 reply
    racismnoway
    @racismnoway
    Hey guys! I've posted a question on Stack Overflow asking about HP tuning of a LSTM model which requires CoNLL-U files as input: https://stackoverflow.com/questions/63850904/conll-files-in-hyperparameter-tuning-using-optuna
    1 reply
    Tyler Benster
    @tbenst
    Hello, prospective user here! I need to run my hyperparameter search in a HPC environment, which means using SLURM job scheduler. Just wondering if Optuna is suited for distributed use cases / job schedulers? Many thanks!
    Tyler Benster
    @tbenst
    ^ answer to this is YES! Really easy when using RDB backend. Great design :)
    Hiroyuki Vincent Yamazaki
    @hvy
    Great, sounds like you got it right.
    Hiroyuki Vincent Yamazaki
    @hvy
    Just a heads up, isort was just introduced. For those sitting on ongoing PRs or local changes, it’s a good idea to sync with the latest master to check for violations (isort . --check --diff). I temporarily configured the repository to require that PRs are verified against the latest commit in master so they won’t get merged otherwise. Sorry for the additinal work... optuna/optuna#1695
    dlin1
    @kdlin
    In your github lightgbm_tuner_simple.py demo code, you have this line: prediction = np.rint(model.predict(val_x, num_iteration=model.best_iteration)). Is num_iteration=model.best_iteration really needed? Since the train() has early_stopping_rounds set already, wouldn't the best_iteration be used anyway?
    dlin1
    @kdlin
    For LightGBM Tuner, is LightGBM's Scikit-learn API supported? A reason for using LightGBM's Scikit-learn API, as opposed to it regular training API, is that Scikit-learn API allows one to specify class_weight.
    2 replies
    racismnoway
    @racismnoway
    I've been trying to work out how to optimize the hyperparameters of a Bi-LSTM model for PoS and dependency parsing (https://github.com/datquocnguyen/jPTDP). The model takes CoNLL-U files as input and I am clueless as to how I can go about using them with Optuna. Any idea? All helps hugely appreciated!
    7 replies
    Hiroyuki Vincent Yamazaki
    @hvy
    Sorry, branch protection was set to enforce each PR to be tested against the latest master at all times. This was because isort was introduced and the master could break quite easily in terms of style violations. However, we noticed it puts a lot of burden on the PR author so I disabled this enforcement again. It’s now back to normal
    Giovana
    @giocampos
    Is there anybody who has faced the following problem: AttributeError: module 'optuna' has no attribute 'create_study'? I cannot find out what I'm missing, I just installed Optuna in two computers, one is working fine, but the other I get this message for the code: import optuna and study = optuna.create_study(). Help!
    1 reply
    simenbm
    @simenbm1_twitter
    Hi, I'm having problems getting my study to run using parallel processes. Should it spawn new processes if I specify n_jobs=n>1?
    study = optuna.create_study(direction='maximize')
    objective = MyObjective(x_train, x_val, y_train, y_val)
    study.optimize(objective, n_trials=n_trials, n_jobs=n_jobs)
    simenbm
    @simenbm1_twitter
    never mind, the problem was that my python cache was not updating
    simenbm
    @simenbm1_twitter
    Can I extract the params from a FrozenTrial? If I try to access study.best_trial.params['reg_lambda'] I get AttributeError: 'dict' object has no attribute 'reg_lambda'
    1 reply
    Joris Cadow
    @C-nit_gitlab
    Hi, I was wondering if one can provide some existing results (e.g. from manual hyperparameter tuning) to a study, so that the suggestions for future trials can account for some "trails" outside of optuna? Say one found a good set of hyperparams by chance, my assumption would be that it's beneficial to the optimisation to be aware of it.
    2 replies
    Rounak Agarwal
    @agarwalrounak
    Hi, I was wondering if there are any companies using Optuna in production? I can see a list of projects using Optuna here. I am looking for some more examples or a more in-depth explanation of how a company is using Optuna.
    2 replies
    Nir
    @nirosen_twitter
    Hi, I was looking for examples of pytorch-ignite & Optuna and had trouble finding anything rather than:
    https://github.com/optuna/optuna/blob/master/examples/pytorch_ignite_simple.py
    Do you know some? Thanks (:
    sailesh-reddy
    @sailesh-reddy
    Hi, I have extracted best params for LightGBM regressor using optuna and fed these best params back to lightgbm, but I am getting a very different result when training the model as compared to the value obtained during optuna trials with best params. Can someone help? Thanks very much
    gbm.set_params(**study.best_params,random_state=42)
    gbm.fit(
    X_cv_train,
    y_cv_train,
    feature_name = feature_set,
    categorical_feature = cat_features,
    eval_set=[(X_cv_test, y_cv_test)],
    verbose = True
    )
    Hideaki Imamura
    @HideakiImamura

    Thanks for all your contributions, we’ve just released v2.2.0! Some PRs include the following.

    Also, note that starting from this version, Python 3.5 is no longer officially supported.

    Hiroyuki Vincent Yamazaki
    @hvy
    🎉
    Toshihiko Yanase
    @toshihikoyanase
    :tada:
    Jeff Yang
    @ydcjeff
    Hello, I am going to contribute to optuna and would like to know what pip install -e "[document]" is doing, it installed some dependencies so how it's different installing with regular -r requirements.txt?
    1 reply
    Hiroyuki Vincent Yamazaki
    @hvy
    A fun read if you want to know a little about the internals of the TPE sampler and the recently introduced multivariate parameter https://twitter.com/OptunaAutoML/status/1313388536192614400
    Sergey Kolesnikov
    @Scitator
    Hi guys, thanks for the awesome framework!
    I am author of Catalyst framework - we have recently introduce full Catalyst Config API + Optuna integration
    https://github.com/catalyst-team/catalyst/tree/master/examples/cifar_stages_optuna
    generally speaking, we have plans for active AutoML development with Config API and Optuna support and I wonder if you are interested in such collaboration too
    if so, feel free to join our slack here https://join.slack.com/t/catalyst-team-core/shared_invite/zt-d9miirnn-z86oKDzFMKlMG4fgFdZafw
    I would be happy, if @toshihikoyanase @hvy @HideakiImamura could join :)
    1 reply
    Peter Cotton
    @microprediction
    Hi all. I've been using optuna and a few others. Some initial thoughts here if it is of interest. I'm surely missing something. https://www.microprediction.com/blog/optimize