pymc-devs/pymc

PyMC version 3

Junpeng Lao

@junpenglao

maybe the best way to start is here: https://github.com/pymc-devs/pymc3/blob/master/docs/source/notebooks/bayesian_neural_network_opvi-advi.ipynb

Peter St. John

@pstjohn

:thumbsup: Thanks for the link!

Peter St. John

@pstjohn

aside from cost_part_grad_scale, are there any other arguments that can be used to tune the new ADVI optimizers?

Thomas Wiecki

@twiecki

@ferrine ^

Peter St. John

@pstjohn

Having better luck with the old ADVI adamax optimizer:

with pymc_model_minibatch:
    approx = pm.fit(n=50000, method='advi')

with pymc_model_minibatch:
    v_params = pm.advi(n=50000)

plt.semilogy(inference.hist)
plt.semilogy(-v_params.elbo_vals)
plt.ylabel('ELBO')
plt.xlabel('iteration')

Thomas Wiecki

@twiecki

can you upgarde to master and try again? pymc-devs/pymc3#2203 should help

you might have to set obj_optimizer=pm.adagrad_window, not quite sure

Peter St. John

@pstjohn

the obj_optimizer=pm.adagrad_window did it, I was on the most recent master (fab6938).

Thomas Wiecki

@twiecki

ok, we should probably set that as the default

Peter St. John

@pstjohn

¯\(ツ)/¯, not sure how universal those results would be.

Thanks for the help!

Thomas Wiecki

@twiecki

I've seen that a bunch before

Junpeng Lao

@junpenglao

agree with setting adagrad_window as default. At least it match the prior performance.

Maxim Kochurov

@ferrine

Authors also mension that it works when distributions are not very different there

Maxim Kochurov

@ferrine

There was something wrong with my gitter. ^ message is about Cost part grad scale. One previous message did not appear. I said that this coef is for fine tuning purposes and should be zero near optima.

Peter St. John

@pstjohn

Does pm.generator work with masked array values?

mhashemi0873

@mhashemi0873

Hello, I have a conceptual question! Using ADVI, I fit some data time series in order to estimate the model parameter values. Here is my problem: For different runs, with same ELBO value, I obtain different posterior distributions for parameters. This happens because I use advi? if I use NUTS, this should not happen or there are some other reasons such correlation between parameters?

Maxim Kochurov

@ferrine

Generators and minibatches can't work with masked.

Different ADVI solutions can happen because of local optimums. Did you try different optimization setups?

mhashemi0873

@mhashemi0873

@ferrine Thanks for the hint. Yes sounds there are many local optimums. But for different optimums I should obtain different ELBO values no? my problem is by same values for ELBO, I obtain different parameter values.

Maxim Kochurov

@ferrine

Whithout having a look at the model it is diffucult to say what else can go wrong.

I had sime experience with TS. Initial testval matters

But there was the case when you have some unobserved in time

Florf

@omenrust_twitter

Hi, I'm new to PYMC3 and I was wondering if someone could help me figure out why it's running so slow; even before any sampling has started. My model has ~4000 groups (each with their own observations and parameters) and I'm trying to model the groups as a mixture of 2 populations. I have a loop where I create lists with the individual parameters and Likelihoods for all the groups. But even this construction is very slow.

Maxim Kochurov

@ferrine

I think vectorization can help

Florf

@omenrust_twitter

This is a snippet of the code if that helps:https://pastebin.com/D6e7VL73

Maxim Kochurov

@ferrine

You can try this approach https://github.com/pymc-devs/pymc3/blob/master/docs/source/notebooks/GLM-hierarchical.ipynb

You can create grouped priors in vector notation

Florf

@omenrust_twitter

So instead of looping for each ~4000 group, I can create priors with shape=~4000 you mean?

Maxim Kochurov

@ferrine

yes and pass appropriate hyperparams to them

logp is computed elemwise so hyper parameters should match one to one in rows (shape[0])

then for likelihood you can do the thing as in notebook radon_est = a[county_idx] + b[county_idx] * data.floor.values

concat all data and remember group id

Maxim Kochurov

@ferrine

I see you don't use hyperparams that depend on group. So this case is right the same as Hierarchical GLM in implementation

 Z = pm.Categorical(‘Z’,  tt.stack([1.0-w1,w1]), shape=n_groups)
 MU = pm.Normal(‘mu’, H1_mu[Z], H1_precision[Z], shape=n_groups)
 ALPHA = pm.Gamma('alpha', alpha=H1_alpha[Z], beta=H1_beta[Z], shape=n_groups)

loop can be rewritten in this way

Florf

@omenrust_twitter

Thanks! Not ignoring you by the way, was reading the notebook :) it helps alot. I think I can get it from here

Also, is there a way to specify starting values? For pymc2 I think it was value=xxx but doesn't seem to work for v3

Junpeng Lao

@junpenglao

you can provide starting value by adding testval=.5 etc, for example MU = pm.Normal(‘mu’, H1_mu[Z], H1_precision[Z], shape=n_groups, testval=np.ones(n_groups))

Florf

@omenrust_twitter

thanks!

Nikos Koudounas

@aplamhden

Hello,I would like to know if sample_ppc() is working with the new ADVI interface?

Maxim Kochurov

@ferrine

Yes, you should create a variational trace with pm.sample_approx

Nikos Koudounas

@aplamhden

I create my trace with this: trace=approx.sample(500) and then ppc = pm.sample_ppc(trace, model=basic_model, samples=500, progressbar=False) then i have an error : TypeError: object of type 'TensorVariable' has no len()

Maxim Kochurov

@ferrine

What is full traceback?

Nikos Koudounas

@aplamhden

what u mean with full traceback?

Maxim Kochurov

@ferrine

just copy-paste of full error

with functions calls

Where communities thrive

People

Repo info

Activity