Hello guys. We have one plugging which assumes to submit tasks to celery and retrieve their outputs. I have a feedback from one of our developers, that beer garden is dealing only with sync tasks and this makes a problem to integrate this solution with the beer garden. Do we understand the limitation properly? How to solve this?
Anyway, some update from my side. We think to use result.get(timeout=300)in celery. This turns the call into sync one (http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html#keeping-results)
It's been a real long time since I came back to gitter. I updated my installation of beer garden with the latest version, (this time in a virtual environment) but it seems the old systems I wrote no longer works. The error I am receiving is "Backend request timed out" when the system starts up.
This is on a CentOS 7.5, on Python 3.7. Thanks!
I attended the Open Space talk at Pycon and was inspired to give Beer Garden a go, especially with the intention of spinning up quick plugins with Docker to help get some of our plugins accessibility across our organization. We've had luck with the docker-compose you've provided on the site, along with the example plugins compose. The issue I'm having is creating a plugin from scratch. I've attempted to follow the docs provided, but I think I'm a little lost. I have the generic Hello World plugin in the my_plugin-1.0.0.0.tar.gz saved into the folder referenced in the compose. I've composed up and when I go into the container, I see it there and untar/compress it.... That's about as far as I can get. I try to rescan the system directory... but it's not going so well. So at the risk of looking foolish in this chat, I thought I would pose the question and see what I get.
.tar.gz. You've transferred the .tar.gz to the correct folder in the compose, exploded the file, and then tried to rescan but to no avail
docker-compose logs -f bartender
I found the issue. Seems I didn't read enough :)
TLDR: I was missing some required fields in the beer.conf
After I ran the docker-compose logs (thanks @loganasherjones ) and saw the error messages, I went back and read the entry for the required fields below in the tutorial. Might want to consider updating the "Configure your plugin" portion on the plugin local guide. For a noob like myself, it can be misleading. I was under the impression all I needed was the PLUGIN_ENTRY.
That all said, I have my first plugin running. Now the real fun begins.
Thanks again!
SystemClient and pass in the username/password you want to use. It will automatically take care of making the correct auth requests for you.
- Get a token by using your username/password
- Use the token in the
POSTon/requests
SystemClient I highly recommend that instead. That way you can just avoid all the headache other than just passing username/password.
Trying out BG for the time. The docker-compose gives me the following error for brew-view container's logs.
brew-view_1 | 2020-03-28 21:03:49,535 - bg_utils.mongo.util - WARNING - No roles found: creating convenience roles
brew-view_1 | 2020-03-28 21:03:49,540 - bg_utils.mongo.util - WARNING - Creating missing admin user...
brew-view_1 | 2020-03-28 21:03:49,540 - bg_utils.mongo.util - INFO - Creating username "admin" with password "password"
brew-view_1 | 2020-03-28 21:03:49,960 - tornado.application - ERROR - Exception in callback functools.partial(<function wrap.<locals>.null_wrapper at 0x7fd32810d598>, <Future finished exception=NotUniqueError('Tried to save duplicate unique keys (E11000 duplicate key error collection: beer_garden.principal index: unique_index dup key: { : "admin" })',)>)
brew-view_1 | Traceback (most recent call last):
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/mongoengine/document.py", line 378, in save
brew-view_1 | object_id = self._save_create(doc, force_insert, write_concern)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/mongoengine/document.py", line 434, in _save_create
brew-view_1 | object_id = collection.save(doc, write_concern)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 3158, in save
brew-view_1 | to_save, True, check_keys, manipulate, write_concern)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 612, in _insert
brew-view_1 | bypass_doc_val, session)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 600, in _insert_one
brew-view_1 | acknowledged, _insert_command, session)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1491, in _retryable_write
brew-view_1 | return self._retry_with_session(retryable, func, s, None)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1384, in _retry_with_session
brew-view_1 | return func(session, sock_info, retryable)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 597, in _insert_command
brew-view_1 | _check_write_command_response(result)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/pymongo/helpers.py", line 221, in _check_write_command_response
brew-view_1 | _raise_last_write_error(write_errors)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/pymongo/helpers.py", line 202, in _raise_last_write_error
brew-view_1 | raise DuplicateKeyError(error.get("errmsg"), 11000, error)
brew-view_1 | pymongo.errors.DuplicateKeyError: E11000 duplicate key error collection: beer_garden.principal index: unique_index dup key: { : "admin" }
brew-view_1 |
brew-view_1 | During handling of the above exception, another exception occurred:
brew-view_1 |
brew-view_1 | Traceback (most recent call last):
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/tornado/ioloop.py", line 758, in _run_callback
brew-view_1 | ret = callback()
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/tornado/stack_context.py", line 300, in null_wrapper
brew-view_1 | return fn(*args, kwargs)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/tornado/ioloop.py", line 779, in _discard_future_result
brew-view_1 | future.result()
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
brew-view_1 | yielded = self.gen.throw(*exc_info)
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/brew_view/init.py", line 106, in startup
brew-view_1 | partial(setup_database, config), "Unable to connect to mongo, is it started?"
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
brew-view_1 | value = future.result()
brew-view_1 | File "/usr/local/lib/python3.6/site-packages/tornado/gen.py", line 326, in wrapper
brew-view_1 |
Questions about running beer-garden/bartender and plugins in different Python versions.
From the documentation, It looks like bartender install requirement is 2.7 or 3.4. What controls the version used? The default "/usr/bin/python" version.
Looking at Plugin_Runner.py, bartender spawns plugins in a equivalent python version using sys.executable. Because of 3rd party libraries, it looks like i'm limited to python2.7 for the plugin. Would you consider a enhancement to allow the "setup_args" to specify a python version like setuptools? For example 'python_requires' : '~=2.7'.
latest-python2 tag.
@keith.ancowitz_gitlab & @loganasherjones
I agree with Logan. Right now the python environment that Bartender uses is based on the version that it is running. This is why we support both libraries. If you plugins are not using 3.4, then I the interim I would suggest installing the 2.7 docker image as Logan stated.
It would be interesting to see what would happen if we update the subprocess command to use the local environment or a user specified configuration (python version). @hazmat345 you have been working in this area for our next version. The biggest concern I have so far is making sure the users are aware that they are running something in a different version of python. That way they don't get into weird scenarios where they are debugging the wrong version.
Another option is to run your plugins are Remote Plugins. These you would have to manage independent from what is located in the plugin directory, but this gives you the most control over the python environment. This will work if you have a 3.4 Bartender running and 2.7 Remote Plugins.