These are chat archives for bayespy/bayespy

12th
Oct 2017
Jaakko Luttinen
@jluttine
Oct 12 2017 05:06
rodrigogadea_twitter: sorry, missed your first question. let me see..
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:06
@jluttine thanks :)
Jaakko Luttinen
@jluttine
Oct 12 2017 05:09
of course, the cluster assignment probabilities are roughly proportional to the mixture component likelihoods. (roughly, because the prior has a contribution too)
right?
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:11
yup, they should be assigned to highest component likelihood
Jaakko Luttinen
@jluttine
Oct 12 2017 05:12
you get probabilities for the assignments, not just which one is the highest one
hmm..
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:12
another way, although more complicated should be calculating the mahalanobis distance to the means of the components with their covariance matrices
the way I found is taking the argmax of the posterior moments of Z (the categorical variable for labels)
Jaakko Luttinen
@jluttine
Oct 12 2017 05:14
there's at least this function: Y._message_to_parent(0) that you can try. if Y is the variable from a mixture. 0 is the index of the first parent which is the cluster assignment variable. the mixture variable just sends the raw mixture component likelihoods to that parent
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:14
but the problem is how to classify a new observation
mmm... give me a min, let me try it...
Jaakko Luttinen
@jluttine
Oct 12 2017 05:17
Y should be a variable created as Y = Mixture(.....)
then Y._message_to_parent(0) probably gives what you want
if i now understood correctly
but this isn't really related to classifyin new observations
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:18
(sorry to flood and cluter)

In [2]: Y
Out[2]: <BayesianNetworkNode: [BN: bn4] - Y>

In [3]: Y_eo
Out[3]: <bayespy.inference.vmp.nodes.mixture.Mixture at 0x7f2a0d523a20>

In [4]: Y_eo._message_to_parent(0)
Out[4]:
[array([[ -2.00823407e+10, -4.96166651e+01, -5.56754086e+01, ...,
-2.00823407e+10, -2.00823407e+10, -1.66615308e+00],
[ -2.00704953e+10, -8.49287472e+01, -9.87792342e+01, ...,
-2.00704953e+10, -2.00704953e+10, -1.80070523e+00],
[ -2.00869318e+10, -3.96570506e+01, -4.62358618e+01, ...,
-2.00869318e+10, -2.00869318e+10, -2.60444381e+00],
...,
[ -2.00509588e+10, -1.76285661e+02, -1.15495974e+03, ...,
-2.00509588e+10, -2.00509588e+10, -1.99574162e+00],
[ -2.00502797e+10, -1.82478382e+02, -1.88824174e+03, ...,
-2.00502797e+10, -2.00502797e+10, -2.51803686e+00],
[ -2.00469922e+10, -2.01707495e+02, -1.64842560e+03, ...,
-2.00469922e+10, -2.00469922e+10, -2.39807638e+00]])]

Jaakko Luttinen
@jluttine
Oct 12 2017 05:20
but if you want (posterior) probabilities that a sample belongs to a particular, then the cluster assignment variable gives you that information.
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:22
aha, but how? it what I can't find
Jaakko Luttinen
@jluttine
Oct 12 2017 05:23
from that django-ai page:
Z
The belonging to a cluster.
not sure how you can access Z in django-ai framework
but that's the variables you're interested in
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:24
I do Z.get_moments() to get the estimation of the probability of each category for each observation, am I right?
Jaakko Luttinen
@jluttine
Oct 12 2017 05:24
Z.get_moments()[0] gives you the posterior probabilities for each variable belonging to each custer
yes
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:25
right, I getting it right them
but how can I get that for a new observation?
Jaakko Luttinen
@jluttine
Oct 12 2017 05:25
there's at least two ways
not sure i understand exactly. why can't you use all the observations you have when you run your inference and you'll get posterior for them all? is it that you want to learn the cluster means and precisions with some data and then keep them fixed, and when you get new observations just give cluster assignment probabilities without learning the cluster distributions?
Jaakko Luttinen
@jluttine
Oct 12 2017 05:31
or do you want to learn everything but sequentially as data comes in?
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:33
it should be predict the label
although it could be also update the model
sequentially
Jaakko Luttinen
@jluttine
Oct 12 2017 05:36
i don't know about django-ai, but in bayespy you can just create new nodes for new variables: Z1/Z2/Z3/.. for cluster assignments and Y1/Y2/Y3/.. for the observations, and just keep the same cluster mean and precision variables
those Zs and Ys can contain any number of variables so you don't need to create a node for one but a set of any size.
or you can just create the original model from scratch when you get new data
and learn with all data again. not very efficient though
you can also do sequential learning where you use data to learn the cluster distribution parameters (mean and precision), then use that posterior as a prior for next data batch and forget the previous data batch. this way you forge the information from previous data to the mean and precision parameters and then forget that data. this doesn't give you the true posterior but
something probably close to it. but this is probably quite efficient sequential learning
probably some other ways too, so it depends on what you want
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:43
aha, then it would be something like
Z_new = Categorical()
Y_new = Mixture(Z_new, Gaussian, mu_old, Lambda_old)
Q = VB(Z_new, Y_new, mu_old...)
Q.update()
Z_new.get_moments()
sorry but my internet connection is working like hell
Jaakko Luttinen
@jluttine
Oct 12 2017 05:49
rodrigogadea_twitter: yes. just add Z_old, Y_old to Q = VB(...) as well. all nodes there because they are all in the same connected graphical model
if you don't want to update all variables, just for instance Z_new, then you can Z_new.update() instead Q.update()
(or Q.update(Z_new))
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 05:51
cool, let me try it... :)
Jaakko Luttinen
@jluttine
Oct 12 2017 05:52
and you can have plates in Z_new if you want it to contain more than one variable, for instance if you're learning sequentially using mini-batches
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 06:05
Yay!
It works!
bn4 = BayesianNetwork.objects.get(name="bn4")
Y = bn4.nodes.get(name="Y")
y = Y.get_data()
Y_eo = bn4.get_engine_object()["Y"]
alpha_eo = bn4.get_engine_object()["alpha"]
Z_eo = bn4.get_engine_object()["alpha"]
mu_eo = bn4.get_engine_object()["mu"]
Lambda_eo = bn4.get_engine_object()["Lambda"]
Q = bp.inference.VB(Y_eo, alpha_eo, Z_eo, mu_eo, Lambda_eo)
Y_eo.observe(y)
Q.update(repeat=1000)
##
Z_new = bp.nodes.Categorical(alpha_eo)
Z_new.initialize_from_random()
Y_new = bp.nodes.Mixture(Z_new, bp.nodes.Gaussian, mu_eo, Lambda_eo)
Y_new.observe([30.2, 29.8])
Q_0 = bp.inference.VB(Z_new, Y_new, Y_eo, alpha_eo, Z_eo, mu_eo, Lambda_eo)
Q_0.update(Z_new)
print(Z_new.get_moments())
(although I'm seeing now that Z_eo is pointing to alpha, but nevermind...)
Jaakko Luttinen
@jluttine
Oct 12 2017 06:10
ok, nice!
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 06:10
Iteration 1: loglike=-1.098049e+03 (0.003 seconds)
[array([  0.00000000e+000,   5.14238102e-187,   0.00000000e+000,
         0.00000000e+000,   0.00000000e+000,   1.53892421e-221,
         0.00000000e+000,   8.68501324e-151,   0.00000000e+000,
         1.00000000e+000])]
Now I have to see how to wrap it
Is there a way of sending the new nodes to the existing Q?
I mean, without redefining it?
Jaakko Luttinen
@jluttine
Oct 12 2017 06:57
you don't even have to use VB object. it's just a convenience class. you can just use nodes directly, like Z.update(). you can create a new VB object from old one like: Q_new = VB(new_node1, new_node2, *Q.model)
you can also try adding a node to existing Q like: Q.model.append(new_node) but that's not recommended as you bypass some checkings so you may accidentally do something wrong. but in principle that works, perhaps
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 06:58
Oh, that's cool :)
Rodrigo Gadea
@rodrigogadea_twitter
Oct 12 2017 07:32
thanks a lot for the help and your time
Jaakko Luttinen
@jluttine
Oct 12 2017 07:37
no problem!