These are chat archives for allegro/ralph

11th
Jan 2019
ahaupt
@ahaupt
Jan 11 07:35
@vi4m
ahaupt
@ahaupt
Jan 11 10:22
Serious question: what's the "maturity status" of the Ralph3 API? I just ask as the search API seems to have issues, either. Searches always seem to return all objects
curl -s http://ralph-demo.allegro.tech/api/networks/'?vlan__exact=2' -H 'Authorization: Token cdccc11590df5628ad245ca4db06224f82d2913e'

Or:

curl -s http://ralph-demo.allegro.tech/api/networks/'?vlan=2' -H 'Authorization: Token cdccc11590df5628ad245ca4db06224f82d2913e'

They all return all objects. Or is the search just wrong?

Roman Prykhodchenko
@romcheg
Jan 11 10:57
@ahaupt we heavily rely on Ralph API internally
So I would say it is pretty mature
The API does not consider filters by any property by default
If you go to http://your.ralph.host.com/api/networks/ with your browser and press the OPTIONS key you wil see a human-readable description of the endpoint
at the bottom of that there is this:
Screen Shot 2019-01-11 at 12.02.44.png
Roman Prykhodchenko
@romcheg
Jan 11 11:03
which means you cannot use vlans for filtering
Since adding new filters does not break the backward compatibility of the API we usually add them on demand rather than overloading the db by possible joins and slowing down the API by allowing to filter by every possible field
therefore, if you need to filter by vlans, that's pretty easy to add :)
ahaupt
@ahaupt
Jan 11 11:35
@romcheg Ah, ok - learned something again! :)
Roman Prykhodchenko
@romcheg
Jan 11 11:37
Ralph stores a relatively big number of different kinds of objects
The API it provides for those kinds, however, is usually simple CRUD endpoints
it's hard to predict what fields or fielters API users will need in the future but it's usually easier to add what's necessary on request without breaking the world for anyone
Roman Prykhodchenko
@romcheg
Jan 11 11:43
@ahaupt that said
if you add 'vlan' to this list https://github.com/allegro/ralph/blob/ng/src/ralph/networks/admin.py#L119 it should work both in the UI and in the API, since for networks filteres are shared
engelant
@engelant
Jan 11 12:32
With the risk of sounding like a comlete noob, I installed ralph on ubuntu 18.04 with the provided package and the nginx configuration, but I ran into some issues. One of them is, I don't get that top menu bar. Another was ralphctl didn't care much about my database.conf, which I solved by adding "source /etc/ralph/conf.d/database.conf
export $(cut -d= -f1 /etc/ralph/conf.d/database.conf)" to my .profile. How much did I screw up so it doesn't work out of the box?
Roman Prykhodchenko
@romcheg
Jan 11 12:33
@engelant lemme grab myself a coffee and I'll be back to you :)
engelant
@engelant
Jan 11 12:35
@romcheg sure, thanks for taking the time
engelant
@engelant
Jan 11 12:40
ralphctl sitetree_resync_apps - my bad, didn't do that
Roman Prykhodchenko
@romcheg
Jan 11 12:44
sitetree_resync_apps that's right
there are a few things users have to do on their own
run migrations, create a super user and resync the menu
re-settings
ahaupt
@ahaupt
Jan 11 12:45
@vi4m
[nomos127] ~ % curl -svv -X PATCH -H 'Content-Type: application/json' -H 'Authorization: Token xxx' -d '{"gateway": 26 }' http://ralph:8000/api/networks/415/
*   Trying 141.34.252.16...
* TCP_NODELAY set
* Connected to ralph (141.34.252.16) port 8000 (#0)
> PATCH /api/networks/415/ HTTP/1.1
> Host: ralph:8000
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: application/json
> Authorization: Token xxx
> Content-Length: 16
> 
* upload completely sent off: 16 out of 16 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Fri, 11 Jan 2019 12:43:20 GMT
< Server: WSGIServer/0.2 CPython/3.4.2
< X-Frame-Options: SAMEORIGIN
< Content-Language: en
< Allow: GET, PUT, PATCH, DELETE, HEAD, OPTIONS
< Content-Type: application/json
< Vary: Accept, Accept-Language, Cookie
< 
* Closing connection 0
{"id":415,"url":"http://ralph:8000/api/networks/415/","ui_url":"http://ralph:8000/networks/network/415/","name":"VLAN 3","created":"2019-01-11T08:29:50","modified":"2019-01-11T13:43:20.826541","address":"141.34.3.0/24","remarks":"Windows Desktops","vlan":3,"dhcp_broadcast":true,"reserved_from_beginning":10,"reserved_from_end":0,"lft":1,"rght":2,"tree_id":2,"level":0,"parent":null,"gateway":null,"network_environment":null,"kind":null,"service_env":null,"dns_servers_group":null,"terminators":[],"racks":[]}%
Roman Prykhodchenko
@romcheg
Jan 11 12:46
@engelant I think there is a but in ralphctl
engelant
@engelant
Jan 11 12:47
are there any further docs on ralphctl?
Roman Prykhodchenko
@romcheg
Jan 11 12:47
for f in /etc/conf.d/*.conf; do source $f; done
not really -- it's just a wrapper on ralph that reads the settings, that's it
engelant
@engelant
Jan 11 12:52
hmm, your suggestion for ralphctl seems much cleaner, mind If I use it to quickly create a PR?
Roman Prykhodchenko
@romcheg
Jan 11 12:52
Sure
I mean, sure -- I don't mind :)
We will merge this asap
engelant
@engelant
Jan 11 12:54
Sure thing, againt the ng or master branch?
Roman Prykhodchenko
@romcheg
Jan 11 12:54
ng
engelant
@engelant
Jan 11 13:03
PR is out
Roman Prykhodchenko
@romcheg
Jan 11 13:03
thanks!
engelant
@engelant
Jan 11 13:03
no no no, thank you
Roman Prykhodchenko
@romcheg
Jan 11 13:16
Commented on it
ahaupt
@ahaupt
Jan 11 13:42

@vi4m Ah, according to the API ( /api/networks/ ) gateway is marked as "readonly"!

"gateway": {
                "type": "nested object",
                "required": false,
                "read_only": true,
                "label": "Gateway",
[...]

I wonder how the web interface is dealing with that ...

engelant
@engelant
Jan 11 13:45
how often does the bintray build for debian packages run?
Roman Prykhodchenko
@romcheg
Jan 11 13:52
as often as required :)
we don't run it automatically
However, we have this deploy wednesdey here when we check whether there are some unreleased changes since the last week
I will make a release that includes your fix later today for sure -- that's important
Roman Prykhodchenko
@romcheg
Jan 11 14:22
@engelant the 20190111.1 release that includes your patch is been built
Fabian Krack
@onibox
Jan 11 14:22
@romcheg Are you using Ubuntu or the Docker image internally?
Roman Prykhodchenko
@romcheg
Jan 11 14:22
yup
I will upload a completely re-done image early next week but it also uses ubuntu:bionic as the base image
TL;DR we are using a bare ubuntu:bionic image and installing the deb package there
Roman Prykhodchenko
@romcheg
Jan 11 14:28
Then we add a few tools that allow to actually use that image in contenerised environments and that's it
we wanted to have Ralph built the same way for the package and for the image to reduce maintenance cost for the team
otherwise we have neither time nor resources to support the package -- that's why it used to be unusable for a long time
Fabian Krack
@onibox
Jan 11 15:09
So using the Docker Image would be the recommended way in the future? I'm currently looking into building an RPM package.
Roman Prykhodchenko
@romcheg
Jan 11 15:09
No -- we won't have a recommended way
Fabian Krack
@onibox
Jan 11 15:09
:-)
Roman Prykhodchenko
@romcheg
Jan 11 15:10
Especcially now, since the image itself uses the package
There are two issues with building RPM packages
engelant
@engelant
Jan 11 15:10
I like that, docker creates quiet an overhead on ProxmoxVE
Roman Prykhodchenko
@romcheg
Jan 11 15:11
RHEL 7 does not support python 3 which is required in Ralph and RHEL 8 is not out yet, is it?
Fabian Krack
@onibox
Jan 11 15:12
RHEL 8 in currently in beta state, will be either based on Fedora 29 or 30.
Roman Prykhodchenko
@romcheg
Jan 11 15:12
the second issue is that Ralph is extremely sensitive to versions of its dependencies
Fabian Krack
@onibox
Jan 11 15:12
RHEL 7 can use EPEL for installing python 3.
Roman Prykhodchenko
@romcheg
Jan 11 15:13
so we resolved the issue for Ubuntu by using dh-virtualenv
that tool installs Ralph and its dependencies into a virtualenv and then places that virtualenv to the package and adds necessary links
for RPM you will probably need to do that on your own
Fabian Krack
@onibox
Jan 11 15:14
For RHEL 7 there is allegro/ralph#3093.
Yes, i'm currently struggling with the fact that the path of the build directory is stored at multiple places when installing the RPM.
Roman Prykhodchenko
@romcheg
Jan 11 15:15
re-EPEL -- is it possible to use it without losing the support?
Fabian Krack
@onibox
Jan 11 15:16
Red Hat Support? I don't know (and i don't care) because we are using Scientific Linux ;-)
Roman Prykhodchenko
@romcheg
Jan 11 15:17
I'll clarify that with RH folks
So to summarise -- we as a maintenance team will provide the necessary support for contributors who will work on creating an RPM package. It is better to make the image work with the upcoming RHEL8 that supplies python 3 so the users don't have to use EPEL
As soon as there are ready to use tools for uilding the package we will inegrate them with our tests and the CI so RPM will not be a second-class citizen
Roman Prykhodchenko
@romcheg
Jan 11 15:23
That said -- I have to rush home to start my weekend :)
Fabian Krack
@onibox
Jan 11 15:24
Sure, have a nice weekend!
Adam
@tretos53
Jan 11 15:36

Can anyone help with below error while upgrading Ralf ?

administrator@localhost:~$sudo apt-get upgrade ralph-core
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
ralph-core is already the newest version.
0 to upgrade, 0 to newly install, 0 to remove and 0 not to upgrade.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y

Setting up ralph-core (3.0.0-snapshot-20181116-8965) ...
Traceback (most recent call last):
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/base/base.py", line 130, in ensure_connection
self.connect()
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/transaction_hooks/mixin.py", line 75, in connect
super(TransactionHooksDatabaseWrapperMixin, self).connect(a, kw)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/base/base.py", line 119, in connect
self.connection = self.get_new_connection(conn_params)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/mysql/base.py", line 276, in get_new_connection
conn = Database.connect(
conn_params)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/MySQLdb/init.py", line 85, in Connect
return Connection(
args, kwargs)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/MySQLdb/connections.py", line 204, in init
super(Connection, self).init(*args,
kwargs2)
_mysql_exceptions.OperationalError: (1045, "Access denied for user 'ralph_ng'@'localhost' (using password: YES)")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/ralph/ralph-core/bin/ralph", line 11, in <module>
sys.exit(prod())
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/ralph/main.py", line 29, in prod
main('ralph.settings.prod')
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/ralph/main.py", line 14, in main
execute_from_command_line(sys.argv)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/core/management/init.py", line 354, in execute_from_command_line
utility.execute()
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/core/management/init.py", line 346, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/core/management/base.py", line 394, in run_from_argv
self.execute(args, cmd_options)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/core/management/base.py", line 444, in execute
self.check()
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/core/management/base.py", line 482, in check
include_deployment_checks=include_deployment_checks,
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/core/checks/registry.py", line 72, in run_checks
new_errors = check(app_configs=app_configs)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/core/checks/model_checks.py", line 28, in check_all_models
errors.extend(model.check(
kwargs))
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/models/base.py", line 1205, in check
errors.extend(cls._check_fields(kwargs))
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/models/base.py", line 1282, in _check_fields
errors.extend(field.check(
kwargs))
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/models/fields/init.py", line 934, in check
errors = super(AutoField, self).check(kwargs)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/models/fields/init.py", line 207, in check
errors.extend(self._check_backend_specific_checks(
kwargs))
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/models/fields/init.py", line 306, in _check_backend_specific_checks
return connection.validation.check_field(self, *
kwargs)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/mysql/validation.py", line 18, in check_field
field_type = field.db_type(connection)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/models/fields/init.py", line 614, in db_type
return connection.data_types[self.get_internal_type()] % data
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/init.py", line 36, in getattr
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/utils/functional.py", line 59, in get
res = instance.dict[self.name] = self.func(instance)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/mysql/base.py", line 196, in data_types
if self.features.supports_microsecond_precision:
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/utils/functional.py", line 59, in get
res = instance.dict[self.name] = self.func(instance)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/mysql/features.py", line 52, in supports_microsecond_precision
return self.connection.mysql_version >= (5, 6, 4) and Database.version_info >= (1, 2, 5)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/utils/functional.py", line 59, in get
res = instance.dict[self.name] = self.func(instance)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/mysql/base.py", line 371, in mysql_version
with self.temporary_connection():
File "/usr/lib/python3.4/contextlib.py", line 59, in enter
return next(self.gen)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/base/
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/base/base.py", line 130, in ensure_connection
self.connect()
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/utils.py", line 98, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/base/base.py", line 130, in ensure_connection
self.connect()
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/transaction_hooks/mixin.py", line 75, in connect
super(TransactionHooksDatabaseWrapperMixin, self).connect(a, kw)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/base/base.py", line 119, in connect
self.connection = self.get_new_connection(conn_params)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/django/db/backends/mysql/base.py", line 276, in get_new_connection
conn = Database.connect(
conn_params)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/MySQLdb/init.py", line 85, in Connect
return Connection(
args, kwargs)
File "/opt/ralph/ralph-core/lib/python3.4/site-packages/MySQLdb/connections.py", line 204, in init
super(Connection, self).init(*args,
kwargs2)
django.db.utils.OperationalError: (1045, "Access denied for user 'ralph_ng'@'localhost' (using password: YES)")
dpkg: error processing package ralph-core (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
ralph-core
E: Sub-process /usr/bin/dpkg returned an error code (1)

Errors I see

_mysql_exceptions.OperationalError: (1045, "Access denied for user 'ralph_ng'@'localhost' (using password: YES)")

django.db.utils.OperationalError: (1045, "Access denied for user 'ralph_ng'@'localhost' (using password: YES)")
dpkg: error processing package ralph-core (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
ralph-core
E: Sub-process /usr/bin/dpkg returned an error code (1)

engelant
@engelant
Jan 11 16:16
so apt dist-upgrade just overwrote my settings (at least database.conf), any reason for the package not checking the existance of the file?
Roman Prykhodchenko
@romcheg
Jan 11 18:57
@engelant yes, there is a reason -- it's probably a bug :)
IIRC @szymi- was working on that particular thing in the package
@szymi- could you please take a look when you're back?
Fabian Krack
@onibox
Jan 11 21:17
@romcheg Just installed Ralph on CentOS 7 with plain python 3.4 from EPEL instead of fiddling around with virtualenv, works like a charm. Also had a look into Fedora 29 for RHEL 8 preview. Fedora 29 provides python 3.7. Does Ralph also run with this version already?
Fabian Krack
@onibox
Jan 11 22:34
Will provide a PR for creating an RPM.