I'm trying to reproduce the same results by a vw model but I'm not getting consistent results.
I'm using cats_pdf
and this is the command I'm using to get predictions :
vw --daemon --initial_regressor model.pkl --epsilon 0.2 --chain_hash --quiet --testonly --port 8888 --num_children 10
then I pass the context via netcat
If I repeat the process n times the result of the trials are not the same
2 replies
2 replies
Hi All,
I am very new to vowpalwabbit and open source contribution. I wanna set up vowpal in my local to better understand reduction code in case of contextual bandits. Is there any guide available to get started with development in vowpal wabbit.
PS : I tried to set it up in visual studio but it's getting some missing package issue. Thought of asking here before spending much time fixing them 😅
5 replies
and the response for an action bandit took)?
Hi Folks,
I am using vowpalwabbit in contextual bandit settings. But I am struck with a strange issue where vowpalwabbit just generating same PMF irrespective of context. Ideally it should generate different PMFs for action selection based on different context. Here is the sample data I am using.
shared |Context t1=a_c t2:5 t3=a_b t4:2 t5:10
|Action arm=a1
|Action arm=a2
|Action arm=a3
|Action arm=a4
0:-5:0.09 | Action arm=a5
|Action arm=a6
|Action arm=a7
|Action arm=a8
|Action arm=a9
|Action arm=a10
|Action arm=a11
I initialized my vowpalwabbit with following setting.
--cb_explore_adf --cb_type mtr --epsilon 0.05
Here is the action distribution irrespective of context in data.
The first image is action distribution after learning and the second is original distribution of action in data..
I am wondering what could possibly be the reason for this behaviour?
1 reply
Hi! I'm getting started with contextual bandits and working on developing one where the action list changes. I just wanted to clarify something..
Looking at the example in the tutorial if I have..
shared |User user=Tom time_of_day=morning
|Action article=politics
|Action article=sports
|Action article=music
|Action article=food
This will return an action based on those four, correct?
Now if I have another item to predict and I predict using
shared |User user=Aly time_of_day=morning
|Action article=music
|Action article=entertainment
This will return an action based on those two options. Is that right? So I could theoretically expand this list of articles? Can my action list expand so it contains |Action article=X and |Action advertisement=Y (aka two different action types?)
2 replies
One other question for a newbee...
I'm making a new model and as an example this is what I'm working with for my first prediction
shared |User tz=America/Los_Angeles plat=ios loh=12
|Action time=0
|Action time=1
|Action time=2
|Action time=3
|Action time=4
|Action time=5
|Action time=6
|Action time=7
When I see the model output from the prediction it looks like..
finished run
number of examples = 1
weighted example sum = 1.000000
weighted label sum = 0.000000
average loss = n.a.
total feature number = 120
Is total feature number =120 because I have a string in tz? I believe I need to transform my variables into OHE so if I don't do that does it turn it into text that is represented with 120 features? TY in advance!
3 replies
1 reply
2 replies
--cb_type {dr, ips, mtr} should always be set to mtr. I'm curious if there is a reason why this is recommended?
I am currently using the Vowpal Wabbit package in order to simulate a Contextual Bandit. I had a couple of questions regarding the usage of the library:
I have multiple contexts/categories where the actions are intersecting. For example, lets say I have jerseys of Team A, Team B and Team C. These jerseys come in sizes S, M and L. Based on past demand, I want to recommend a size of jersey to produce.
Contexts - Team A, Team B, Team C Actions - S, M and L
Each context has the same set of actions to choose from. I want Vowpal Wabbit to understand that each context is different, and create separate distributions of the action space. Vowpal Wabbit is utilizing the same distribution/pmf for the actions across all contexts.
So if, Team A is the context - The distribution is [0.1, 0.8, 0.1] after several runs. Team B also has the same distribution [0.1, 0.8, 0.1] even though VW has not seen this as an input, ideally I would want it to start from [0.33,0.33,0.33]
Is there a way I can utilize VW to differentiate contexts and give them separate distributions?
I am simulating the Contextual Bandit with Vowpal Wabbit with the following settings - "--cb_explore_adf --save_resume --quiet --epsilon 0.1"
I was also wondering if there was a way to access/view the underlying learnt policy? Where are the different distributions or learnt policies stored?
Thanks
Hello, does CATS support Offline Policy Evaluation? it seems like the values I am getting are the same as the average loss. Disclosure, I am trying to evaluate on the results of a deterministic policy for which i fixed pdf=1.0 so that could be the issue?
I would also be curious to know if there's a straightforward way to do OPE with CATS. Anyone has any suggestions how this can be done?
/home/platform/aakash/vowpal_wabbit3/vowpalwabbit/model_merger/src/main.cc: In function 'void logger_output_func(void*, VW::io::log_level, const string&)':
/home/platform/aakash/vowpal_wabbit3/vowpalwabbit/model_merger/src/main.cc:50:38: error: no matching function for call to 'std::basic_string<char>::erase(__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>::const_iterator)'
50 | newline_stripped_message.cend());
| ^
In file included from /opt/rh/devtoolset-9/root/usr/include/c++/9/string:55,
from /opt/rh/devtoolset-9/root/usr/include/c++/9/stdexcept:39,
from /opt/rh/devtoolset-9/root/usr/include/c++/9/array:39,
from /home/platform/aakash/vowpal_wabbit3/vowpalwabbit/common/include/vw/common/vw_exception.h:7,
from /home/platform/aakash/vowpal_wabbit3/vowpalwabbit/model_merger/src/main.cc:5:
/opt/rh/devtoolset-9/root/usr/include/c++/9/bits/basic_string.h:4698:7: note: candidate: 'std::basic_string<_CharT, _Traits, _Alloc>& std::basic_string<_CharT, _Traits, _Alloc>::erase(std::basic_string<_CharT, _Traits, _Alloc>::size_type, std::basic_string<_CharT, _Traits, _Alloc>::size_type) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]'
4698 | erase(size_type __pos = 0, size_type __n = npos)
| ^~~~~
/opt/rh/devtoolset-9/root/usr/include/c++/9/bits/basic_string.h:4698:23: note: no known conversion for argument 1 from '__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >' to 'std::basic_string<char>::size_type' {aka 'long unsigned int'}
4698 | erase(size_type __pos = 0, size_type __n = npos)
| ~~~~~~~~~~^~~~~~~~~
/opt/rh/devtoolset-9/root/usr/include/c++/9/bits/basic_string.h:4714:7: note: candidate: 'std::basic_string<_CharT, _Traits, _Alloc>::iterator std::basic_string<_CharT, _Traits, _Alloc>::erase(std::basic_string<_CharT, _Traits, _Alloc>::iterator) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::basic_string<_CharT, _Traits, _Alloc>::iterator = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >; typename _Alloc::rebind<_CharT>::other::pointer = char*]'
4714 | erase(iterator __position)
| ^~~~~
/opt/rh/devtoolset-9/root/usr/include/c++/9/bits/basic_string.h:4714:7: note: candidate expects 1 argument, 2 provided
/opt/rh/devtoolset-9/root/usr/include/c++/9/bits/basic_string.h:4734:7: note: candidate: 'std::basic_string<_CharT, _Traits, _Alloc>::iterator std::basic_string<_CharT, _Traits, _Alloc>::erase(std::basic_string<_CharT, _Traits, _Alloc>::iterator, std::basic_string<_CharT, _Traits, _Alloc>::iterator) [with _CharT = char; _Traits = std::char_traits<char>; _Alloc = std::allocator<char>; std::basic_string<_CharT, _Traits, _Alloc>::iterator = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >; typename _Alloc::rebind<_CharT>::other::pointer = char*]'
4734 | erase(iterator __first, iterator __last);
| ^~~~~
/opt/rh/devtoolset-9/root/usr/include/c++/9/bits/basic_string.h:4734:40: note: no known conversion for argument 2 from '__normal_iterator<const char*,[...]>' to '__normal_iterator<char*,[...]>'
4734 | erase(iterator __first, iterator __last);
| ~~~~~~~~~^~~~~~
1 reply
Command line basic .
pip install vowpalwabbit command
Hi @gautham_x:matrix.org since you installed python vw you should follow the python basic tutorial: https://vowpalwabbit.org/docs/vowpal_wabbit/python/latest/tutorials/python_first_steps.html
If you want to try command line vw, you need to build from source following this wiki page: https://github.com/VowpalWabbit/vowpal_wabbit/wiki/Building After building from source successfully you can follow the command line basic tutorial
@cheng-tan: I build it right now from source using option 1. But still not working.
-DVW_INSTALL=ON
Is that mistake
1 reply
1 reply
6 replies