Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Morris Tai
    @morristai
    Hi all, I appreciate qlib's *.bin design for high efficiency, but what if I want to peak one of the .bin file(i.e. use pandas to show the raw data)? Thanks.
    you-n-g
    @you-n-g
    @morristai For example:
    if you want to read the data in ~/.qlib/qlib_data/cn_data/features/sh600000/close.day.bin
    You can use following code
    D.features(['SH600000'], ['$close'])
    Morris Tai
    @morristai
    @you-n-g I see, thanks for the quick response.
    kun liu
    @iexpos
    Besides official indexs, How can I create a custom combination?
    you-n-g
    @you-n-g
    The stock universe is stored in folders like ~/.qlib/qlib_data/cn_data/instruments/
    You can create your own universe with similar format.
    @iexpos
    kun liu
    @iexpos
    it's great, thank you! @you-n-g
    akushonkamen
    @akushonkamen
    How do i run backtest on examples though
    the documents are kind of confusing...
    I tried to run "qrun workflow_config_alstm_Alpha158.yaml" from qlib-main/examples/benchmarks
    and it bumps that no such directory
    under what environment should i execute qrun? under cmd or python?
    thanks
    and when i use qrun under python, it says that qrun is not defined, even after i imported qlib
    you-n-g
    @you-n-g
    @akushonkamen We are trying to make a more user-friendly demo . Please check the latest updates on
    microsoft/qlib#117
    akushonkamen
    @akushonkamen
    thanksw
    you-n-g
    @you-n-g
    There is a more user-friendly demo on the index page now https://github.com/microsoft/qlib#quick-start
    SchrodingersCot
    @SchrodingersCot
    Hi. After running a backtest with qrun, how can I check my positions and trades during the backtest period?
    you-n-g
    @you-n-g
    @SchrodingersCot https://github.com/microsoft/qlib/blob/main/qlib/workflow/record_temp.py#L241
    Please the position is saved by recorder
    you-n-g
    @you-n-g
    If you are using mlrun backend, you can check positions_normal.pkl in mlflow storage
    SchrodingersCot
    @SchrodingersCot
    Got it working. Thanks for the quick response and keep up the great work!
    SchrodingersCot
    @SchrodingersCot
    Hello. Is there any support for live trading? I imagined that, after having trained and backtested a model, I could use said model to generate trading signals from live data. However it does not seem that the qlib recorder supports this functionality.
    Also, for instance in the light gbm baseline model, there is a line "clf.save_model('lgbm_model.mdl')". Where is this model saved? I could not find it.
    you-n-g
    @you-n-g
    The framework is designed to support live data. But the workflow are not implemented now.
    We considered the requirements of live data when designing components like datahandler
    The model are saved on the disk (If you didn't change the configuration, the folder name will be mlflow)
    You can load the model by https://qlib.readthedocs.io/en/latest/component/recorder.html or read the files direclty.
    monkeycc
    @monkeycc
    I want to update the data, data_ Collector, I don't know how to import, what impact does it have on the original data
    Pengrong Zhu
    @zhupr
    @monkeycc Hi,
    If you need to use the latest data of yahoo finance, you can download and dump:
    download data: https://github.com/microsoft/qlib/tree/main/scripts/data_collector/yahoo
    dump data: https://qlib.readthedocs.io/en/latest/component/data.html#converting-csv-format-into-qlib-format
    When downloading data and dump data, the target directory should not overwrite the "origin data" directory, it will not affect the "origin data"
    lucasli0121
    @lucasli0121
    How get real time stock data instead of downloading from 'http://fintech.msra.cn/stock_data/downloads'
    you-n-g
    @you-n-g
    @lucasli0121 We released related API for updating data.
    You can use them for your real time data.
    https://github.com/microsoft/qlib/blob/090b68e44ea081c6802e91daa2821ba480490e6a/scripts/dump_bin.py#L225
    yaduns
    @yaduns
    It is version 0.0.2.dev20 with pip install? Something seems wrong.
    you-n-g
    @you-n-g
    @yaduns
    Could you give more details about the error?
    Thanks
    yaduns
    @yaduns
    @you-n-g the error is corrected. But other problem like this https://pastebin.com/zbrgwWh0
    Dingsu Wang
    @Derek-Wds
    @yaduns Hi, could you please try to install pyqlib without tsinghua mirror of pip, or you could download the pyqlib package from pypi website and install it. Changing mirror sometimes will cause the error of distribution of package is not found due to the untrusted host of your machine.
    In terms of install from source, could you provide the version of cython and gcc on your machine? Thanks 😊
    phamanhtu17295
    @phamanhtu17295
    Hello, I am having trouble with the data. I cannot open the bin file to see it structure
    change.day.bin
    i want to modify my data to fit into qlib but I can not see what is inside.
    you-n-g
    @you-n-g
    @phamanhtu17295 https://qlib.readthedocs.io/en/latest/component/data.html#converting-csv-format-into-qlib-format
    Here are the docs which could help you to convert your data into Qlib Format
    Zhichong Fang
    @fzc621
    Hi, I am a little confused about the label formula. The label formula of the example in the https://qlib.readthedocs.io/en/latest/component/report.html#id2 is Ref($close, -2)/Ref($close, -1)-1 (also used in Alpha158 and Alpha360), which is inconsistent with "the label is formulated as Ref($close, -1)/$close - 1" in the note. Thanks.
    The formula Ref($close, -1)/$close - 1 was used a lot in https://qlib.readthedocs.io/en/latest/reference/api.html . May I know the purpose of the difference?
    bxdd
    @bxdd
    @fzc621 Hi, Ref($close, -1)/$close - 1 means the change from T to T+1, which can be used in backtest.
    In model training, we use the label Ref($close, -2)/Ref($close, -1)-1that means the change from T+1 to T+2 rather than Ref($close, -1)/$close - 1, of which the reason is that when you get T day close price of a china stock, you can buy it in T+1 day and sell it in T+2 day.
    Zhichong Fang
    @fzc621
    @bxdd Thanks!
    BigW
    @BigW
    Hi - how can I control the prediction horizon in qlib, for example how can I evaluate the LightGBM model to classify and predict returns over 3 or 5 days in the future on a rolling basis? Is there a way to control that in the config or .yaml file?
    BigW
    @BigW
    Hi - this may be more of a future feature request but has anyone integrated this TA library to fast track technical features/indicators? https://twopirllc.github.io/pandas-ta/#indicators-by-category
    you-n-g
    @you-n-g
    @BigW You can control the prediction horizon in Qlib by changing the label
    For example, you can change ge label in the config.
    https://github.com/microsoft/qlib/blob/main/examples/benchmarks/LSTM/workflow_config_lstm_Alpha158.yaml#L32

    @BigW
    The expression engine of Qlib provides similar problem like https://twopirllc.github.io/pandas-ta/#indicators-by-category
    Users are encouraged to implement technical indicators with it.

    But Qlib doesn't force users to use it. If you prefer pandas-ta, you can implement your technical indicates in the datahandler after retrieving the raw data.

    yyll008
    @yyll008
    TypeError: Can only swap levels on a hierarchical axis. When I use the new fund-data，dataset = init_instance_by_config(task["dataset"])