Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    gxxuej
    @gxxuej
    你好:
    我的工作环境是:
    Anaconda3,Spyder(Python 3.7.7),plotly 4.12.0,matplotlib 3.1.3,pyqlib 0.6.1.dev0.
    运行该行语句:
    My_figure=analysis_model.model_performance_graph(pred_label,graph_names=["pred_ic"],show_notebook=False)
    报错:
    'XAxis' object has no attribute '_gridOnMajor'
    不知道原因,请多多指教!
    谢谢!
    20210323
    you-n-g
    @you-n-g
    @yyll008 How can I reproduce this error ?
    @gxxuej Could you give more details about how to reproduce this bug?
    BigW
    @BigW
    I keep on getting mid way through running backtests using both the qrun and the python run_all_model approach and get a single line saying "killed" has anyone seen these types or errors before?

    code sample below Running the model: LightGBM for iteration 1...
    40048:MainThread INFO - qlib.Initialization - [config.py:276] - default_conf: client.
    40048:MainThread WARNING - qlib.Initialization - [config.py:292] - redis connection failed(host=127.0.0.1 port=6379), cache will not be used!
    40048:MainThread INFO - qlib.Initialization - [init.py:46] - qlib successfully initialized based on client settings.
    40048:MainThread INFO - qlib.Initialization - [init.py:47] - data_path=/home/warren/.qlib/qlib_data/cn_data
    Killed

    Deleting the environment: /tmp/tmp01wsrv_h...
    Retrieving results...
    39831:MainThread ERROR - qlib.workflow - [utils.py:35] - An exception has been raised[ValueError: No valid experiment has been found, please make sure the input experiment name is correct.].
    File "run_all_model.py", line 305, in <module>
    fire.Fire(run) # run all the model
    File "/home/warren/anaconda3/envs/QLIB3TEST/lib/python3.8/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
    File "/home/warren/anaconda3/envs/QLIB3TEST/lib/python3.8/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
    File "/home/warren/anaconda3/envs/QLIB3TEST/lib/python3.8/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(varargs, **kwargs)
    File "run_all_model.py", line 62, in _return_wrapped
    return function_to_decorate(
    args, **kwargs)
    File "run_all_model.py", line 287, in run
    results = get_all_results(folders)
    File "run_all_model.py", line 148, in get_all_results
    exp = R.get_exp(experiment_name=fn, create=False)
    File "/home/warren/anaconda3/envs/QLIB3TEST/lib/python3.8/site-packages/qlib/workflow/init.py", line 239, in get_exp
    return self.exp_manager.get_exp(experiment_id, experiment_name, create)
    File "/home/warren/anaconda3/envs/QLIB3TEST/lib/python3.8/site-packages/qlib/workflow/expm.py", line 149, in get_exp
    exp, is_new = self._get_exp(experiment_id=experiment_id, experiment_name=experiment_name), False
    File "/home/warren/anaconda3/envs/QLIB3TEST/lib/python3.8/site-packages/qlib/workflow/expm.py", line 298, in _get_exp
    raise ValueError(
    ValueError: No valid experiment has been found, please make sure the input experiment name is correct.

    gxxuej
    @gxxuej

    @you-n-g 以下是代码部分:import qlib
    import pandas as pd

    from qlib.config import REG_CN
    from qlib.workflow import R
    from qlib.contrib.report import analysis_model

    provider_uri = "~/.qlib/qlib_data/cn_data" # target_dir
    qlib.init(provider_uri=provider_uri, region=REG_CN)
    my_rid = "3f999767c71b499dbe0dc3c3480067d3"
    recorder = R.get_recorder(my_rid, experiment_name="backtest_analysis")
    pred_df = recorder.load_object("pred.pkl")
    label_df = recorder.load_object("label.pkl")
    label_df.columns = ['label']
    pred_label=pd.concat([label_df,pred_df],axis=1,sort=True).reindex(label_df.index)

    graph_names: list = ["group_return", "pred_ic", "pred_autocorr","pred_turnover"],

    Error : 'XAxis' object has no attribute '_gridOnMajor'

    f1 = analysis_model.model_performance_graph(pred_label,show_notebook=False)

    graph_names=["group_return"],OK !

    f2 = analysis_model.model_performance_graph(pred_label,graph_names=["group_return"],\
    show_notebook=False)

    graph_names=["pred_ic"],Error : 'XAxis' object has no attribute '_gridOnMajor'

    f3 = analysis_model.model_performance_graph(pred_label,graph_names=["pred_ic"],\
    show_notebook=False)
    谢谢了!

    Lespan
    @Lespan
    may i ask if qlib support futures?
    yyll008
    @yyll008

    @you-n-g df = D.features(D.instruments(market="all"), ["$DWJZ", "$LJJZ"],start_time='2018-01-01', end_time='2021-03-22', freq="day")
    market = "all"
    benchmark = "000001"
    data_handler_config = {
    "start_time": "2018-01-01",
    "end_time": "2020-08-01",
    "fit_start_time": "2018-01-01",
    "fit_end_time": "2018-12-31",
    "instruments": market,
    }

    task = {
    "model": {
    "class": "LGBModel",
    "module_path": "qlib.contrib.model.gbdt",
    "kwargs": {
    "loss": "mse",
    "colsample_bytree": 0.8879,
    "learning_rate": 0.0421,
    "subsample": 0.8789,
    "lambda_l1": 205.6999,
    "lambda_l2": 580.9768,
    "max_depth": 8,
    "num_leaves": 210,
    "num_threads": 20,
    },
    },
    "dataset": {
    "class": "DatasetH",
    "module_path": "qlib.data.dataset",
    "kwargs": {
    "handler": {
    "class": "Alpha158",
    "module_path": "qlib.contrib.data.handler",
    "kwargs": data_handler_config,
    },
    "segments": {
    "train": ("2018-01-01", "2018-12-31"),
    "valid": ("2019-01-01", "2019-12-31"),
    "test": ("2020-01-01", "2020-08-01"),
    },
    },
    },
    }

    model initiaiton

    model = init_instance_by_config(task["model"])
    dataset = init_instance_by_config(task["dataset"])

    Lespan
    @Lespan
    @you-n-g Thanks!
    onepointfuck
    @onepointfuck
    How to join the Qlib team and what are the requirements for interns?
    Pengrong Zhu
    @zhupr

    @gxxuej Hi,
    I can’t reproduce this bug in the same environment.

    You can use the latest qlib(https://github.com/microsoft/qlib) for more information: cd scripts && python collect_info.py all

    you-n-g
    @you-n-g
    @onepointfuck You can send your resume to our email qlib@microsoft.com
    gxxuej
    @gxxuej
    @zhupr Thanks!
    Jagadish Kumar E 🇮🇳
    @javajk_twitter
    Hi Team, Currently QLib supports only two regions datasets. Any tentative dates when India region shall be supported and If I have to define India region on my own what all need to be done any pointers shall be helpful
    you-n-g
    @you-n-g
    @javajk_twitter Qlib doesn't limit the region of the datasets.
    You can send PR to create a data collector to crawl data from india market.
    Jagadish Kumar E 🇮🇳
    @javajk_twitter
    @you-n-g : Thanks for your response, How do I raise PR ?
    you-n-g
    @you-n-g
    @javajk_twitter You can raise PR to Qlib like normal opensource project.
    Thanks!
    mo.zi
    @mozi7
    hi, guys, I notice use the pip install is always un-success, could you please tell me how to fix it?
    you-n-g
    @you-n-g

    @mozi7 Can you give us more details about your error?
    Thanks

    We have include pip install test in our CI
    https://github.com/microsoft/qlib/blob/70c84cbc77e52bea67e4528d6f79e2e462a4ffa6/.github/workflows/test.yml#L39

    It suceeded in all kinds of platforms. You can check the commands it uses.

    liu100286
    @liu100286
    Hello, where can I set the number of trading stocks as a multiple of 100?
    Jagadish Kumar E 🇮🇳
    @javajk_twitter

    Hello, I have two sets of datasets in csv format. One is the historical dataset and another one is the delta or current days shares OHLC values. When i run convert the historical dataset in csv format to bin format everything goes fine. But when I try to update the current days data in csv format to bin format i get following error "NaT is not in list" For the first time to convert from CSV format to bin format i use following command

    dump_bin.py dump_all --csv_path=nf.csv --qlib_dir=C:/data/.qlib/qlib_data/in_data --include_fields open,close,high,low,factor,tottrdqty,tottrdval

    For the second time to convert from CSV format to bin format i use following command

    dump_bin.py dump_update --csv_path=nf.csv --qlib_dir=C:/data/.qlib/qlib_data/in_data --include_fields open,close,high,low,factor,tottrdqty,tottrdval

    Is the command and arguments passed is correct or not, please let me know.

    Following is the error I get when i run the above command to convert the current days data to bin format

    dump bin errors: {'ABB': 'concurrent.futures.process._RemoteTraceback: \n"""\nTraceback (most recent call last):\n File "D:\Anaconda\lib\concurrent\futures\process.py", line 239, in _process_worker\n r = call_item.fn(call_item.args, *call_item.kwargs)\n File "E:\qlib-jk\qlib\scripts\dump_bin.py", line 250, in _dump_bin\n self._data_to_bin(df, calendar_list, features_dir)\n File "E:\qlib-jk\qlib\scripts\dump_bin.py", line 220, in _data_to_bin\n date_index = self.get_datetime_index(_df, calendar_list)\n File "E:\qlib-jk\qlib\scripts\dump_bin.py", line 211, in get_datetime_index\n return calendar_list.index(df.index.min())\nValueError: NaT is not in list\n"""\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "dump_bin.py", line 467, in _dump_features\n _future.result()\n File "D:\Anaconda\lib\concurrent\futures\_base.py", line 432, in result\n return self.get_result()\n File "D:\Anaconda\lib\concurrent\futures\_base.py", line 388, in get_result\n raise self._exception\nValueError: NaT is not in list\n'}

    2 replies
    you-n-g
    @you-n-g

    Hello, where can I set the number of trading stocks as a multiple of 100?

    Just for marking this question as solved in microsoft/qlib#395

    Jagadish Kumar E 🇮🇳
    @javajk_twitter
    Hello, I am trying on custom data for Indian market. I have converted NSE data to bin format. Now I wanted to run few existing forecasting models. I see that in all the yaml files, where we are initializing the configurations we have two arguments market and benchmark. What value should I provide. (i.e. the default value is market: &market us & benchmark: &benchmark ACC) For the Market i can give it as US, for benchmark what is the value to be given, I gave the NSE stock name ACC, I am getting following error "The benchmark ['ACC'] does not exist. Please provide the right benchmark"
    vs_99
    @vs_99:matrix.org
    [m]
    Hey, is there a way to create ensemble models of all the available models in qlib?
    Also, how can I edit the data api so as to have the code take open price and volume data and predict open price data?
    you-n-g
    @you-n-g
    @javajk_twitter Could you please send us more details about your error(e.g. the backpack information, the folder structure of your data)?
    you-n-g
    @you-n-g
    @vs_99:matrix.org Hi, Qlib supports simple interfaces to ensemble previous trained models.
    You can read the code here https://github.com/microsoft/qlib/blob/main/qlib/model/ens/ensemble.py
    Guodong Xu
    @docularxu

    Does anybody notice this error in the 'main' branch, when running "examples/workflow_by_code.ipynb" using jupyter-notebook. The error message is: "---------------------------------------------------------------------------
    TypeError Traceback (most recent call last)

    <ipython-input-7-e6a7b5f4da00> in <module>
    26 # backtest and analysis
    27 with R.start(experiment_name="backtest_analysis"):
    ---> 28 recorder = R.get_recorder(rid, experiment_name="train_model")
    29 model = recorder.load_object("trained_model")
    30

    TypeError: get_recorder() takes 1 positional argument but 2 positional arguments (and 1 keyword-only argument) were given"

    4 replies
    I didn't see such error on 29/May/2021. it happens only recently.
    w.z.
    @w.z.:matrix.org
    [m]
    does Qlib support quantitative analysis on cryptocurrencies ?
    you-n-g
    @you-n-g

    does Qlib support quantitative analysis on cryptocurrencies ?

    @w.z.:matrix.org You may be interested in this issue microsoft/qlib#387
    I guess you are interested in high-frequency trading.

    This PR in development may be helpful to you in the future
    microsoft/qlib#438

    panshuaiyin
    @panshuaiyin
    csv文件内的数据形如下图
    image.png
    dump时使用的代码如下
    python scripts/dump_bin.py dump_all --csv_path 我储存csv格式的数据的路径 --qlib_dir 我储存qlib格式的数据的路径 --symbol_field_name stock_code --date_field_name date --include_fields open,high,low,close,volume,money,factor,vwap,change
    Pengrong Zhu
    @zhupr
    @panshuaiyin
    Add the freq parameter: --freq month
    python scripts/dump_bin.py dump_all --csv_path 我储存csv格式的数据的路径 --qlib_dir 我储存qlib格式的数据的路径 --symbol_field_name stock_code --date_field_name date --include_fields open,high,low,close,volume,money,factor,vwap,change --freq month
    panshuaiyin
    @panshuaiyin
    截止到模型训练部分的代码都没问题了,但在回测时产生报错,代码和报错如下图
    image.png
    image.png
    image.png
    image.png
    请问我是不是应该在哪个地方增加一个参数“freq=‘month’”
    Pengrong Zhu
    @zhupr

    @panshuaiyin
    回测部分某些函数还不支持freq参数,之后会修复这个问题;
    这个问题可以通过下面的方法修复:

    • 在qlib数据calendars目录中,把month.txt 复制一份 重命名为 day.txt

    你可以先用上面的方法临时跑回测,等这个问题修复后再用正常方法。

    你有兴趣提PR修复这个问题吗

    panshuaiyin
    @panshuaiyin
    请问一下在哪提交PR
    Pengrong Zhu
    @zhupr
    fork qlib的代码到自己的github:https://github.com/microsoft/qlib
    然后基于这个代码进行修改,把改好的代码push到自己的github,然后从自己的仓库向https://github.com/microsoft/qlib提 PR
    panshuaiyin
    @panshuaiyin
    如果是让我修改代码的话,对我来说有点困难,我的工程能力还不太行,抱歉了
    panshuaiyin
    @panshuaiyin
    我把month.txt复制一份重命名为day.txt后,重新运行回测代码,但又出现下面这个报错了
    image.png
    看起来似乎是找不到benchmark,但我之前在日频数据上回测时没有这个问题