These are chat archives for inveniosoftware/invenio

27th
Apr 2016
Lars Holm Nielsen
@lnielsen
Apr 27 2016 07:10
@Panos512 It’s a little bit tricky to add after_record_indexdue to the bulk processing - i.e. you would have to call the after_record_index only after the ES bulk request succeeded - i.e. you’ll have to keep track of records which succeeeded the bulk indexing and which did not. Also, if you plan processing these immediately, you won’t actually be sure that the record is available in the ES index at that point in time….
Samuele Kaplun
@kaplun
Apr 27 2016 07:11
I believe what @Panos512 really needs is a sort of after_before_record_index signal. I.e. after the before_record_index has enhanced the JSON and populated with plenty of new value, we can intercept that and do further actions.
In other words shall we officialise the after_record_enhance signal?
Lars Holm Nielsen
@lnielsen
Apr 27 2016 07:12
but then you can just do that from the before_record_index?
Jiri Kuncar
@jirikuncar
Apr 27 2016 07:12
@kaplun you can fire your own signal in your enhancer.
You can also write a task that checks list of record IDs since last run and queries ES.
Samuele Kaplun
@kaplun
Apr 27 2016 07:14
@lnielsen : but you wouldn't be sure about the order (it could be received before the record is enhance). @jirikuncar yep, probably it's like that. But what if e.g. we make this push to orcid functionality as independent Invenio module. In that case we would need to assume that overlay managers are firing a custom signal not declared.
To be clear: if push-to-orcid was a receiver for after_record_enhance which is not fired anywhere but only in specific overlays, then users of such module would be surprised for it to not work if they don't have any enhancer. In that case they would need to create a stub enhancer just for the sake of firing the after_record_enhance.
Lars Holm Nielsen
@lnielsen
Apr 27 2016 07:16
@kaplun Yes you would still only have one receiver, but inside that receiver you could ship off a celery task to push the record to orcid
Jiri Kuncar
@jirikuncar
Apr 27 2016 07:16
@kaplun your problem is caused by concept of using "enhanced" record just for indexing and not storing it in DB.
Lars Holm Nielsen
@lnielsen
Apr 27 2016 07:17
for an ORCID module, it’s likely best to separate the API which take care of pushing some record representaiton vs. from where this API is called…..
Samuele Kaplun
@kaplun
Apr 27 2016 07:18
@jirikuncar this is required by the fact that you don't want to present to cataloguers fields that are derived automatically for the sake of indexing or formatting, and that they should not touch. In the DB you would store only the real information, not denormalized values (e.g. name variations for authors, citing records information etc.)
Jiri Kuncar
@jirikuncar
Apr 27 2016 07:19
@kaplun this is a matter of serializers that can filter such fields. There are more advantages in keeping DB and ES as close as possible.
Samuele Kaplun
@kaplun
Apr 27 2016 07:32
@jirikuncar thanks! We're thinking about these
Esteban J. G. Gabancho
@egabancho
Apr 27 2016 07:33

hey guys, I am creating our db but apparently the records_metadata_version and transaction tables are not created
We have SQLAlchemy-Continuum==1.2.4 installed and this is our config for the DB

SQLALCHEMY_DATABASE_URI = os.environ.get(
    "SQLALCHEMY_DATABASE_URI",
    "postgresql+psycopg2://localhost/cds")
SQLALCHEMY_ECHO = False
SQLALCHEMY_TRACK_MODIFICATIONS = True

any ideas?

Lars Holm Nielsen
@lnielsen
Apr 27 2016 07:35
i had the same problem - perhaps try update oauth2server ?
Esteban J. G. Gabancho
@egabancho
Apr 27 2016 07:37
lets see!
Esteban J. G. Gabancho
@egabancho
Apr 27 2016 07:47
nope, actullay we don’t have invenio-oauth2server as a requirement
Lars Holm Nielsen
@lnielsen
Apr 27 2016 07:48
and you’re installing CDS with pip install -e .[postgresql] right? :-)
Esteban J. G. Gabancho
@egabancho
Apr 27 2016 07:49
yes :relaxed:
BTW I think I've updated all the packages to their latest release
Jiri Kuncar
@jirikuncar
Apr 27 2016 07:55
@egabancho try something like this in your overlay:
$ pip install requirements-builder pip-tools
$ requirements-builder --level=pypi setup.py > top-requirements.txt
$ pip-compile top-requirements.txt > all-requirements.txt
$ pip install -r all-requirements.txt
$ grep SQLAlchemy-Continuum all-requirements.txt
SQLAlchemy-Continuum==1.2.4
Esteban J. G. Gabancho
@egabancho
Apr 27 2016 07:56

I did this before

$ pip freeze | grep SQLAl
Flask-SQLAlchemy==2.1
SQLAlchemy==1.0.12
SQLAlchemy-Continuum==1.2.4
SQLAlchemy-Utils==0.32.4

I’ll try this now

so SQLAlchemy-Continuum is not in all-requirements.txt :worried:
Jiri Kuncar
@jirikuncar
Apr 27 2016 08:02
do you have invenio-records['postgresql'] in your requirements?
Esteban J. G. Gabancho
@egabancho
Apr 27 2016 08:02
I have invenio-db[postgresql,versioning], should I add also this other one?
Øystein Blixhavn
@blixhavn
Apr 27 2016 10:32
What would be the best way of adding JS/SASS dependencies to a module that are already included in e.g. Invenio-theme? I want the module to work on its own, but also I want to avoid loading anything twice
Lars Holm Nielsen
@lnielsen
Apr 27 2016 11:30
Right now there’s no nice way of extending existing invenio bundles…..to make your own bundle, and include the invenio bundle in your bundle…...
Jacopo Notarstefano
@jacquerie
Apr 27 2016 12:42
What was the reason to avoid using pytest-runner and manually building the PytestCMD? Is it still valid?
Lars Holm Nielsen
@lnielsen
Apr 27 2016 13:00
@jacquerie No reason……pytest-runner was just not usable when we started as far as I remember
best is just to use the pytest-runner
Jacopo Notarstefano
@jacquerie
Apr 27 2016 13:06
Nice, thanks, it’s always nice getting rid of boilerplate :+1:
vancheese
@vancheese
Apr 27 2016 14:04
Hi people, Is this the right place to ask a simple question re installing via vagrant and the guide on http://invenio.readthedocs.org/en/latest/installation/installation-quick.html
Tibor Simko
@tiborsimko
Apr 27 2016 15:07
@vancheese Yes, please go ahead.
xzacharb
@xzacharb
Apr 27 2016 21:23
Hi all, I´ve installed invenio maint-2.0 with invenio-demostite. After install i´ve in the Deposit section only one document type "article". How can I add another one ? For example Video? Thank you for your help.
Tibor Simko
@tiborsimko
Apr 27 2016 22:20
@xzacharb In Invenio 2.0, adding a new deposit workflow involves writing some Python code, which should be one in your instance overlay. You can check and emulate the "article" example from the demo site: https://github.com/inveniosoftware/invenio-demosite/blob/maint-2.0/invenio_demosite/modules/deposit/workflows/article.py