Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Tobias Oberstein
    @oberstet
    @BertHooyman sure, you can find a complete runnable example using WAMP-Ticket authentication here https://github.com/crossbario/crossbar-examples/tree/master/authentication/ticket/static
    I just ran it to make sure it works. it does;) see ^ screenshot. left is crossbar, right is the python client authenticating successfully and the trying a couple of wamp actions it is allowed - and some it isn't
    there is also a javascript based client included .. that looks like this:
    Bildschirmfoto von 2019-03-17 12-28-00.png
    BertHooyman
    @BertHooyman
    thanks.
    is the v2 config file stuff documented somewhere?
    jberends
    @jberends
    hi ppls... I just installed autobahn 19.3.1, however flatbuffers is not installed using pip install.
    Tobias Oberstein
    @oberstet
    @jberends yeah, sorry .. missed some import guards, and our CI runs with all dependencies installed. anyways, fix is coming crossbario/autobahn-python#1142
    Tobias Oberstein
    @oberstet
    fixed in v19.3.2: https://pypi.org/project/autobahn/19.3.2/ via crossbario/autobahn-python#1142
    (cpy372_3) oberstet@intel-nuci7:~$ pip install autobahn==19.3.2
    Collecting autobahn==19.3.2
      Downloading https://files.pythonhosted.org/packages/ea/65/e474985b604f91e15b6d07a83ab99650f3e43791c9711f72ad32a29b24e2/autobahn-19.3.2-py2.py3-none-any.whl (387kB)
        100% |████████████████████████████████| 389kB 3.3MB/s 
    Requirement already satisfied: txaio>=18.8.1 in ./cpy372_3/lib/python3.7/site-packages (from autobahn==19.3.2) (18.8.1)
    Requirement already satisfied: six>=1.11.0 in ./cpy372_3/lib/python3.7/site-packages (from autobahn==19.3.2) (1.12.0)
    Installing collected packages: autobahn
    Successfully installed autobahn-19.3.2
    You are using pip version 18.1, however version 19.0.3 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    (cpy372_3) oberstet@intel-nuci7:~$ python
    Python 3.7.2 (default, Feb 25 2019, 08:26:41) 
    [GCC 7.3.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import txaio
    >>> txaio.use_asyncio()
    >>> from autobahn.wamp import protocol, auth
    >>>
    and flatbuffers is installed (only) as a dependency when doing pip install "autobahn[all]==19.3.2"
    jberends
    @jberends
    @oberstet - Thanks! that is nice to see. We see that on our side its resolved indeed. We were using a package which was dependent on autobahn. In CI run failed to install due to the missing dependency of flatbuffers. Great responsiveness!
    Tobias Oberstein
    @oberstet
    great:) thanks for reporting back!
    Mike Lehan
    @M1ke

    Hi @oberstet ; thanks for a great product. I am currently attempting to build a docker image based on the python:3-slim image and then building crossbar inside of it. I used the example from the Dockerfile in the crossbar github repo.

    I have been using this Dockerfile for a few weeks now and it has always built correctly. Today when building I am getting errors during compliation of Python vmprof wheel. This is weird because it's changed since last build but there's been no release of crossbar or vmprof from what I can see. Vmprof distribute a wheel as well so I am unsure why it's building it from source.

    The only thing I can see is that the python:3-slim image was modified yesterday. Has your build system continued working as expected?

    BertHooyman
    @BertHooyman
    How unique is a session ID created by crossbar? More specifically, are session ID's re-used when a router is restarted or is something more unique generated?
    Can I consider a sessionId as a key in long-term storage of events etc.?
    Thanks,
    Tobias Oberstein
    @oberstet
    @BertHooyman session IDs in crossbar are simply random integers from [0, 2**53] and are considered ephemeral, not long-term globally unique

    53 bits of randomness is not enough to "ensure" long-term globally unique IDs, so I would not consider session ID a solution to what you want.

    of course you can generate client side ID yourself for a new session (eg create new UUID for the session in onJoin) - but a router assigned globally unique (in time and space) ID would be probably be even more useful.

    we could add that trivially in crossbar, and we would have existing ways in the WAMP protocol to transport that ID during the opening handshake.

    if you are interested in that, best would be file an issue on crossbario/crossbar repo and describe your goals and use case ..
    Tobias Oberstein
    @oberstet
    @M1ke rgd docker images: we still need to publish those. our efforts right now are focused on finishing the automation for crossbarfx though
    if you need it now, I think adding apt install libunwind-dev should make it work
    rgd wheels: we are building everything from source, because we support multiple CPU archs
    Tobias Oberstein
    @oberstet
    @M1ke ok, there was indeed more "fallout" from the recent additions of some dependencies, but new images are now released (and those are built exactly from the Dockerfiles in the repo .. so it should work for you as well when you build yourself):
    Tobias Oberstein
    @oberstet
    oberstet@intel-nuci7:~/scm/crossbario/crossbar/docker$ make version_amd64_cpy3
    docker run \
        --rm --entrypoint=/usr/local/bin/crossbar -it \
        crossbario/crossbar:cpy3 version
    
    
        :::::::::::::::::
              :::::          _____                      __
        :::::   :   :::::   / ___/____ ___   ___  ___  / /  ___ _ ____
        :::::::   :::::::  / /__ / __// _ \ (_-< (_-< / _ \/ _ `// __/
        :::::   :   :::::  \___//_/   \___//___//___//_.__/\_,_//_/
              :::::
        :::::::::::::::::   Crossbar v19.3.5
    
        Copyright (c) 2013-2019 Crossbar.io Technologies GmbH, licensed under AGPL 3.0.
    
     Crossbar.io        : 19.3.5
       txaio            : 18.8.1
       Autobahn         : 19.3.3
         UTF8 Validator : autobahn
         XOR Masker     : autobahn
         JSON Codec     : stdlib
         MsgPack Codec  : msgpack-0.6.1
         CBOR Codec     : cbor-1.0.0
         UBJSON Codec   : ubjson-0.12.0
         FlatBuffers    : flatbuffers-1.10
       Twisted          : 18.9.0-EPollReactor
       LMDB             : 0.94/lmdb-0.9.22
       Python           : 3.7.2/CPython
     Frozen executable  : no
     Operating system   : Linux-4.15.0-46-generic-x86_64-with-debian-9.8
     Host machine       : x86_64
     Release key        : RWTugc8VlQqH/Fr5WgYYPCRo6CqsJuXOlLN0ZcEQpM8JXbj3g8KN6Aww
    
    oberstet@intel-nuci7:~/scm/crossbario/crossbar/docker$ make version_amd64_pypy3
    docker run \
        --rm --entrypoint=/usr/local/bin/crossbar -it \
        crossbario/crossbar:pypy3 version
    /usr/local/site-packages/cryptography/hazmat/bindings/openssl/binding.py:163: CryptographyDeprecationWarning: OpenSSL version 1.0.1 is no longer supported by the OpenSSL project, please upgrade. A future version of cryptography will drop support for it.
      utils.CryptographyDeprecationWarning
    
    
        :::::::::::::::::
              :::::          _____                      __
        :::::   :   :::::   / ___/____ ___   ___  ___  / /  ___ _ ____
        :::::::   :::::::  / /__ / __// _ \ (_-< (_-< / _ \/ _ `// __/
        :::::   :   :::::  \___//_/   \___//___//___//_.__/\_,_//_/
              :::::
        :::::::::::::::::   Crossbar v19.3.5
    
        Copyright (c) 2013-2019 Crossbar.io Technologies GmbH, licensed under AGPL 3.0.
    
     Crossbar.io        : 19.3.5
       txaio            : 18.8.1
       Autobahn         : 19.3.3
         UTF8 Validator : autobahn
         XOR Masker     : autobahn
         JSON Codec     : stdlib
         MsgPack Codec  : umsgpack-2.5.1
         CBOR Codec     : cbor-1.0.0
         UBJSON Codec   : ubjson-0.12.0
         FlatBuffers    : flatbuffers-1.10
       Twisted          : 18.9.0-EPollReactor
       LMDB             : 0.94/lmdb-0.9.22
       Python           : 3.5.3/PyPy-7.0.0
     Frozen executable  : no
     Operating system   : Linux-4.15.0-46-generic-x86_64-with-debian-8.11
     Host machine       : x86_64
     Release key        : RWTugc8VlQqH/Fr5WgYYPCRo6CqsJuXOlLN0ZcEQpM8JXbj3g8KN6Aww
    
    oberstet@intel-nuci7:~/scm/crossbario/crossbar/docker$
    Mike Lehan
    @M1ke
    @oberstet thanks for getting back to me; I'll try a rebuild and check the libunwind dev package if needed
    Mike Lehan
    @M1ke
    Yeah adding those extra packages now allows it to compile, thanks. Think I might add a version specifier to the pip build to prevent this happening again
    Mike Lehan
    @M1ke
    Does anybody know how when using a worker process and outputting content from the process to stdout, how you can cause the Twisted logger inside Crossbar to output newlines? Outputting "\n" to stdout would print this in a regular shell but Twisted just prints the raw character.
    BertHooyman
    @BertHooyman
    When using WAMP-CRA with salted passwords, what is the recommended administrative procedure to ensure the config file has the correct salted secret?On another note, is it a recommended practice to use WAMP-CRA with salted passwords combined with dynamic authentication?
    Ilyes Bouchlaghem
    @Ibouch

    Hi, I'm a little confused, I created a web application with Flask-SocketIO, but recently, with a peak of users, the server crashes and does not support a lot of simultaneous connections.
    I read a lot of documentation to have a lighter and more efficient library implementing the websocket protocol, but I found nothing relevant. I turned to the tools pub / sub (Crossbar.io, Redis, Zeromq).

    I have a lot of questions:

    1 - What is the best tool to integrate it into a Flask server?

    2 - With Publish-subscribe, is it possible to have a "room" system as provided by SocketIO? I'm learning the URI model for topics and I think it's possible, but I'd like a confirmation.

    3 - Is it possible to have a flask session context when defining the handler function of an event, typically I receive an event from a browser client, and I check within its flask session if it is to authenticate, if it has a specific attribute etc. I think that this is not possible because the request is received and then sent by the WAMP router, but here again I would like confirmation.

    Thank you in advance !

    Tobias Oberstein
    @oberstet

    @BertHooyman

    When using WAMP-CRA with salted passwords, what is the recommended administrative procedure to ensure the config file has the correct salted secret?

    edit the config file? I don't get the question ..

    On another note, is it a recommended practice to use WAMP-CRA with salted passwords combined with dynamic authentication?

    have a backend database driven dynamic authenticator with the per-user salt stored along the user DB record.

    sidenote: if you are worried about an attacker stealing your whole credentials database, then you might consider using WAMP-SCRAM rather than WAMP-CRA. I'll dump a couple of links (sorry, I am short in time):

    ultimately lead to the design of WAMP-SCRAM:

    which you can find in the spec here:

    and which is implemented in AutobahnPython (thus works for WAMP clients) and Crossbar.io

    Tobias Oberstein
    @oberstet
    @Ibouch sorry to say, but running into issues with long running connections (WebSocket) on a blocking, threaded server is expected. Flask is based on WSGI, and that is "deeply flawed" for that kind of use. It was never designed for that, and is unfixable.

    let me dump a couple of links to material that are relevant in this context (sorry, I have little time .. can't explain everything):

    ultimately, the "best" is using Klein, a Flask rewrite that is natively async:

    Tobias Oberstein
    @oberstet

    2 - With Publish-subscribe, is it possible to have a "room" system as provided by SocketIO? I'm learning the URI model for topics and I think it's possible, but I'd like a confirmation.

    yes, exactly. using 1 room == 1 topic works perfectly. you can have all those "chat room" stuff needed: authorization (because topics allow you to control that), presence (Crossbar.io has a WAMP based meta API that allows you to dynamically query the sessions subscribed to some topic -- the "room"), etc

    Ilyes Bouchlaghem
    @Ibouch

    Thank you for your reply @oberstet !

    Indeed I intend to completely rewrite my application, at first to change it from python2 to python3, and also to use an asynchronous web server, I think https://vibora.io/ which is very flask friendly and has better performance than Aiohttp and Sanic due to the implementation of its important features in Cython.

    Which brings me to a question, you recommended me to switch to an asynchronous web server, and I seem to have seen in the crossbar.io documentation that the router also embed an http server? If so, is it asynchronous?

    In case I choose to have all the websocket implementation by Autobahn and therefore to use the router crossbar.io as http server, what performance can I expect compared to the tools I mentioned? (Vibora, Sanic, Aiohhtp ..)
    Thank you again for your help !

    Tobias Oberstein
    @oberstet

    yeah, Crossbar.io includes a web server, and it is async

    performance: here is some older benchmark on a 40 core machine, here serving static stuff (so performance with dynamic web stuff will be not as high obviously):

    • over 627990 HTTP requests/s at 360 us avg latency
    • over 12.6 GB/s HTTP reply traffic

    https://github.com/crossbario/crossbar-examples/tree/master/benchmark/web

    we've also started to add more benchmarks recently, and will expand this over the year.

    eg WAMP routing performance on a single CPU core: ~46k routed RPCs/sec

    https://crossbario.com/docs/benchmarks/rpc_roundtrip/brummer1.html
    https://crossbario.com/docs/crossbarfx/scaling/benchmarks.html

    Tobias Oberstein
    @oberstet
    all the libs you mention (sanic, aiohttp, uvloop, ..) ultimately boil down to the same syscalls (epoll_wait) so they don't have any inherent advantage. unless you do kernel-bypass networking or use the latest linux kernel shit (io_submit) there isn't an upside. fwiw, with crossbar.io and autobahn, we'll likely eventually support running natively on io_submit - which plays in a different league
    • cython: this is slower than pypy (which we recommend for crossbar.io) - we measured that
    • at the websocket level, the cpu burner is XOR masking and UTF8 validation, and we have vectorized native code upcoming for that in autobahn (NVX: https://github.com/crossbario/autobahn-python/tree/master/autobahn/nvx)
    • handwritten vectorized code blows away anything written in regular C or C++ obviously .. anyways, this is really only relevant if you push GB/s traffic
    Tobias Oberstein
    @oberstet

    vibora: "the fastest web server" ;) rendering HTML is not what a modern application design should be based on. that's just feeding browsers, and even there, only relevant if you don't use single-page app (SPA) frontends - which IMO is the right way.

    an application should be composed of microservices, and some browser SPA is just that: another microservice.

    so what's actually needed for a modern application is a microservice middleware, not a webserver

    Ilyes Bouchlaghem
    @Ibouch

    Very precise and complete answers, thank you @oberstet !

    Indeed for the project I want to do, I am very far from this degree of optimization and clearly even in terms of traffic (a community of players) I expect a few thousand simultaneous connections, at most.

    Let's say that I really discovered the web side and networks in python, and that I was really looking for objective facts to differentiate all existing libraries while ensuring that I will never live again server interruptions :)

    Thank you also for insisting on the blocking mode of operation of WSGI and the problems that arise from it. This allowed me to understand the importance of an asynchronous server in realizing a real-time application.

    Tobias Oberstein
    @oberstet

    sure, np!

    to sum up, my best advice: first think about the overall arch / approach you wanna have: "classic web" (pushing HTML .. async is just a "detail" - though relevant in practice) vs "all microservices" (eg HTML frontends are SPAs == microservices with a human sitting in front, a native mobile frontend is just another non-HTML/non-SPA microservice etc etc), then go on from there ..

    Ilyes Bouchlaghem
    @Ibouch
    Exactly, precisely I learn about microservices architecture, but from what I understand, because each 'service' is isolated, it forces the exchange of data via an internal API (which must also develop) . I don't know if this can correspond for any type of project, the concept of session is still very practical. If I realize that my project can match I would do it, there is a lot of profit to work like that
    BertHooyman
    @BertHooyman
    Further on "what is the recommended administrative procedure to ensure the config file has the correct salted secret?
    edit the config file? I don't get the question .."
    In reading earlier questions on the same subject, it seems that the salted password is only used during the transmission of the credentials, not during the storage, correct? That's a bit surprising to me but OK. I was mislead by Unix /etc/passwd where the stored password is salted.
    BertHooyman
    @BertHooyman
    " if you are worried about an attacker stealing your whole credentials database, then you might consider using WAMP-SCRAM rather than WAMP-CRA."
    Thanks - I'll go that way.
    Thank you for helping me.
    someApprentice
    @someApprentice

    Hi, I trying to implement dynamic authorization and I don't get one thing. How to pass to authorize method extra data from the client?

    For example right now my dynamic authentication and authorization prints this data

    2019-04-01T19:42:03+0700 [Router       4180] WAMP-Anonymous dynamic authenticator invoked: realm='realm1', authid='public'
    2019-04-01T19:42:03+0700 [Router       4180] {'authextra': {'Bearer token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1dWlkIjoiMTAzZjFmY2ItMTQ4YS00OGEyLTkxZmUtZDlmZGMxM2M1ZTA5IiwiZW1haWwiOiJib2JAY3J5cHRlci5jb20iLCJuYW1lIjoiQm9iIiwiaGFzaCI6IiQyYiQxMyR3ZFNad1dRNXdHcjNjNFJZLjlJYndlMXVDN2lZWERWNkJaRTh6R28yQ1RqYy5HaFpRWmNBYSIsImlhdCI6MTU1MzM0NDc2Mn0.ybW-wjOSQGJ5je8Z_9ijpL5QEedxrZWQsJoDdu7XXhY'},
    2019-04-01T19:42:03+0700 [Router       4180]  'authmethod': 'anonymous',
    2019-04-01T19:42:03+0700 [Router       4180]  'session': 7891285622896062,
    2019-04-01T19:42:03+0700 [Router       4180]  'transport': {'cbtid': None,
    2019-04-01T19:42:03+0700 [Router       4180]                'http_headers_received': {'cache-control': 'no-cache',
    2019-04-01T19:42:03+0700 [Router       4180]                                          'connection': 'Upgrade',
    2019-04-01T19:42:03+0700 [Router       4180]                                          'host': 'localhost:8080',
    2019-04-01T19:42:03+0700 [Router       4180]                                          'pragma': 'no-cache',
    2019-04-01T19:42:03+0700 [Router       4180]                                          'sec-websocket-key': '9WGp4+D1nm5iMivcwpwjKw==',
    2019-04-01T19:42:03+0700 [Router       4180]                                          'sec-websocket-protocol': 'wamp.2.cbor,wamp.2.json',
    2019-04-01T19:42:03+0700 [Router       4180]                                          'sec-websocket-version': '13',
    2019-04-01T19:42:03+0700 [Router       4180]                                          'upgrade': 'WebSocket',
    2019-04-01T19:42:03+0700 [Router       4180]                                          'user-agent': 'AutobahnPython/19.2.1'},
    2019-04-01T19:42:03+0700 [Router       4180]                'http_headers_sent': {},
    2019-04-01T19:42:03+0700 [Router       4180]                'peer': 'tcp4:127.0.0.1:50903',
    2019-04-01T19:42:03+0700 [Router       4180]                'protocol': 'wamp.2.cbor',
    2019-04-01T19:42:03+0700 [Router       4180]                'type': 'websocket',
    2019-04-01T19:42:03+0700 [Router       4180]                'websocket_extensions_in_use': []}}
    2019-04-01T19:42:03+0700 [Router       4180] authorize: session={'session': 7891285622896062, 'authid': 'public', 'authrole': 'user', 'authmethod': 'anonymous', 'authprovider': 'dynamic', 'authextra': {'x_cb_node_id': None, 'x_cb_peer': 'tcp4:127.0.0.1:50903', 'x_cb_pid': 4180}, 'transport': {'type': 'websocket', 'protocol': 'wamp.2.cbor', 'peer': 'tcp4:127.0.0.1:50903', 'http_headers_received': {'user-agent': 'AutobahnPython/19.2.1', 'host': 'localhost:8080', 'upgrade': 'WebSocket', 'connection': 'Upgrade', 'pragma': 'no-cache', 'cache-control': 'no-cache', 'sec-websocket-key': '9WGp4+D1nm5iMivcwpwjKw==', 'sec-websocket-protocol': 'wamp.2.cbor,wamp.2.json', 'sec-websocket-version': '13'}, 'http_headers_sent': {}, 'websocket_extensions_in_use': [], 'cbtid': None}}, uri=message.to.bob, action=subscribe, options={}

    And I have no idea how to authorize the client with this authorize session data? For authentication I have an authextra with Bearer token but how I can do something like this with authorization? How can I pass extra data to the authorization function?

    someApprentice
    @someApprentice
    The problem is that I want to allow for the server to publish and for the client only subscribe
    George.UA
    @gendalf

    Hi All
    Just now turned on logging for some function, and detect that crossbar is ignoring "cache" option from dynamic authorization!!.
    So on every call and every publish I have much more traffic and double load on server. Authenticator function called before every call/publish!
    Authenticator answer :

    { "allow": true, "cache": true, "disclose": true }

    Version:

    Crossbar.io        : 19.3.5
       txaio            : 18.8.1
       Autobahn         : 19.3.3
         UTF8 Validator : wsaccel-0.6.2
         XOR Masker     : wsaccel-0.6.2
         JSON Codec     : stdlib
         MsgPack Codec  : msgpack-0.6.1
         CBOR Codec     : cbor-1.0.0
         UBJSON Codec   : ubjson-0.12.0
         FlatBuffers    : flatbuffers-1.10
       Twisted          : 18.9.0-EPollReactor
       LMDB             : 0.94/lmdb-0.9.22
       Python           : 3.7.2/CPython

    Has anyone encountered such a problem?

    someApprentice
    @someApprentice

    Hi, I trying to implement dynamic authorization and I don't get one thing. How to pass to authorize method extra data from the client?

    For example right now my dynamic authentication and authorization prints this data

    And I have no idea how to authorize the client with this authorize session data? For authentication I have an authextra with Bearer token but how I can do something like this with authorization? How can I pass extra data to the authorization function?

    Question is resolved. I didn't realize that I have to set the extra property for a principals data which returned by authenticate method.

    Alexander Goedde
    @goeddea

    Hi!

    We're moving discussions about Crossbar.io and the Autobahn libraries to https://forum.crossbar.io.

    Gitter is great for quick, real-time interactions – but we're not always here to answer questions. Rather than you posting here and not receiving an immediate answer in most cases, we think it makes more sense to concentrate on using a forum.

    This also has the advantage that old questions and answers remain accessible and easily searchable.

    This channel will be deleted in the coming days.

    See you on the forums!

    Alex

    Ilyes Bouchlaghem
    @Ibouch
    Why not make a Discord server?
    Alexander Goedde
    @goeddea
    @Ibouch - The forum (i.e. gitter) is not the problem. This works fine. The issue is that the model (instant messaging) and the requirements (a forum where async answers are the norm, allow searching of old support questions) don't match for us at the moment. For discussions there's also always the IRC channel (freenode/autobahn, where meejah and Omer are regularly present)