Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 11:34
    michaelosthege commented #5936
  • 11:33

    michaelosthege on main

    Add numpyro install to building… (compare)

  • 11:33
    michaelosthege closed #5936
  • 11:33
    michaelosthege closed #5935
  • 11:23
    isms commented #5936
  • 11:21
    michaelosthege review_requested #5940
  • 11:18
    michaelosthege milestoned #5938
  • 11:18
    michaelosthege milestoned #5933
  • 11:18
    michaelosthege milestoned #5929
  • 11:18
    michaelosthege milestoned #5928
  • 11:18
    michaelosthege milestoned #5920
  • 11:18
    michaelosthege milestoned #5919
  • 11:18
    michaelosthege milestoned #5917
  • 11:18
    michaelosthege milestoned #5916
  • 11:16
    larryshamalama commented #5793
  • 11:16
    michaelosthege milestoned #5378
  • 11:16
    michaelosthege demilestoned #5378
  • 11:16
    michaelosthege demilestoned #5291
  • 11:16
    michaelosthege milestoned #5291
  • 11:15
    michaelosthege milestoned #5334
Peter St. John
@pstjohn
:thumbsup: Thanks for the link!
Peter St. John
@pstjohn
aside from cost_part_grad_scale, are there any other arguments that can be used to tune the new ADVI optimizers?
Thomas Wiecki
@twiecki
@ferrine ^
Peter St. John
@pstjohn
Having better luck with the old ADVI adamax optimizer:
with pymc_model_minibatch:
    approx = pm.fit(n=50000, method='advi')

with pymc_model_minibatch:
    v_params = pm.advi(n=50000)

plt.semilogy(inference.hist)
plt.semilogy(-v_params.elbo_vals)
plt.ylabel('ELBO')
plt.xlabel('iteration')
elbo_example.png
Thomas Wiecki
@twiecki
can you upgarde to master and try again? pymc-devs/pymc3#2203 should help
you might have to set obj_optimizer=pm.adagrad_window, not quite sure
Peter St. John
@pstjohn
the obj_optimizer=pm.adagrad_window did it, I was on the most recent master (fab6938).
Thomas Wiecki
@twiecki
ok, we should probably set that as the default
Peter St. John
@pstjohn
elbo_example_2.png
¯\(ツ)/¯, not sure how universal those results would be.
Thanks for the help!
Thomas Wiecki
@twiecki
I've seen that a bunch before
Junpeng Lao
@junpenglao
agree with setting adagrad_window as default. At least it match the prior performance.
Maxim Kochurov
@ferrine
Authors also mension that it works when distributions are not very different there
Maxim Kochurov
@ferrine
There was something wrong with my gitter. ^ message is about Cost part grad scale. One previous message did not appear. I said that this coef is for fine tuning purposes and should be zero near optima.
Peter St. John
@pstjohn
Does pm.generator work with masked array values?
mhashemi0873
@mhashemi0873
Hello, I have a conceptual question! Using ADVI, I fit some data time series in order to estimate the model parameter values. Here is my problem: For different runs, with same ELBO value, I obtain different posterior distributions for parameters. This happens because I use advi? if I use NUTS, this should not happen or there are some other reasons such correlation between parameters?
Maxim Kochurov
@ferrine
Generators and minibatches can't work with masked.
Different ADVI solutions can happen because of local optimums. Did you try different optimization setups?
mhashemi0873
@mhashemi0873
@ferrine Thanks for the hint. Yes sounds there are many local optimums. But for different optimums I should obtain different ELBO values no? my problem is by same values for ELBO, I obtain different parameter values.
Maxim Kochurov
@ferrine
Whithout having a look at the model it is diffucult to say what else can go wrong.
I had sime experience with TS. Initial testval matters
But there was the case when you have some unobserved in time
Florf
@omenrust_twitter
Hi, I'm new to PYMC3 and I was wondering if someone could help me figure out why it's running so slow; even before any sampling has started. My model has ~4000 groups (each with their own observations and parameters) and I'm trying to model the groups as a mixture of 2 populations. I have a loop where I create lists with the individual parameters and Likelihoods for all the groups. But even this construction is very slow.
Maxim Kochurov
@ferrine
hi
I think vectorization can help
Florf
@omenrust_twitter
This is a snippet of the code if that helps:https://pastebin.com/D6e7VL73
You can create grouped priors in vector notation
Florf
@omenrust_twitter
So instead of looping for each ~4000 group, I can create priors with shape=~4000 you mean?
Maxim Kochurov
@ferrine
yes and pass appropriate hyperparams to them
logp is computed elemwise so hyper parameters should match one to one in rows (shape[0])
then for likelihood you can do the thing as in notebook radon_est = a[county_idx] + b[county_idx] * data.floor.values
concat all data and remember group id
Maxim Kochurov
@ferrine
I see you don't use hyperparams that depend on group. So this case is right the same as Hierarchical GLM in implementation
 Z = pm.Categorical(‘Z’,  tt.stack([1.0-w1,w1]), shape=n_groups)
 MU = pm.Normal(‘mu’, H1_mu[Z], H1_precision[Z], shape=n_groups)
 ALPHA = pm.Gamma('alpha', alpha=H1_alpha[Z], beta=H1_beta[Z], shape=n_groups)
loop can be rewritten in this way
Florf
@omenrust_twitter
Thanks! Not ignoring you by the way, was reading the notebook :) it helps alot. I think I can get it from here
Also, is there a way to specify starting values? For pymc2 I think it was value=xxx but doesn't seem to work for v3
Junpeng Lao
@junpenglao
you can provide starting value by adding testval=.5 etc, for example MU = pm.Normal(‘mu’, H1_mu[Z], H1_precision[Z], shape=n_groups, testval=np.ones(n_groups))
Florf
@omenrust_twitter
thanks!
Nikos Koudounas
@aplamhden
Hello,I would like to know if sample_ppc() is working with the new ADVI interface?
Maxim Kochurov
@ferrine
Yes, you should create a variational trace with pm.sample_approx
Nikos Koudounas
@aplamhden
I create my trace with this: trace=approx.sample(500) and then ppc = pm.sample_ppc(trace, model=basic_model, samples=500, progressbar=False) then i have an error : TypeError: object of type 'TensorVariable' has no len()
Maxim Kochurov
@ferrine
What is full traceback?
Nikos Koudounas
@aplamhden
what u mean with full traceback?
Maxim Kochurov
@ferrine
just copy-paste of full error
with functions calls