Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Phil D
    @philodavies

    Hey Emily, PNDA.io is another open source in the Linux Foundation family.
    Documentation regarding creating a PNDA cluster can be found at PNDA.

    My warning to you is that PNDA is quite a large project, i.e. the smallest cluster (pico) requires 65GiB of RAM and 20 CPU threads. The pico build is primarily used for development and learning purposes.

    Emily (Linh) Cao
    @emlcao_twitter
    thanks @philodavies for a quick response
    I'm setting up with MariaDB and the cron script seems not to be working
    Phil D
    @philodavies
    I've only setup the aio and ui docker containers for snas so I wouldn't know how to fix the problem with MariDB at the moment. I just happen to have been working on PNDA for quite a while and figured I could provide you a quick answer
    Emily (Linh) Cao
    @emlcao_twitter
    I see. Do you integrate SNAS with PNDA, @philodavies ?
    dgedia
    @dgedia
    Hey @philodavies , I am running aio snas docker and have noticed that when I peer with public routeviews data, mysql database keeps growing in size which uses significant memory as well as storage. Does SNAS periodically delete old data from database to restore the storage capacity?
    Phil D
    @philodavies
    @dgedia I think the deletion of older data is up to the user. I believe the goal is to save all the data to be able to do an analasys on it in the future. If you only need to analyze over short periods of time, creating a script to delete any data in the database that are over a certain age should be pretty straightforward.
    @emlcao_twitter I am not yet on the stage of integrating SNAS with PNDA, but it is the goal
    dgedia
    @dgedia
    @philodavies I see. Is there a way to change the default fetching interval of 15 minutes (from routeviews). I know there is a conf file in src/etc/openbmp-mrt2bmp.yml that has parameter "timestamp_interval_limit:20". However, not sure whether this is the one I need to modify to change the fetching time interval between two queries?
    Phil D
    @philodavies
    No idea, sorry
    Emily (Linh) Cao
    @emlcao_twitter
    @dgedia you can config how much memory that MariaDB is using by looking at the config file at /etc/mysql/my.cnf inside docker image
    they'll use 80% of RAM memory by default
    Tim Evens
    @TimEvens
    @emlcao_twitter, @dgedia, you can also use the -e MEM=<GB value> to limit MySQL
    the -e MEM=<gb value> works on the AIO and Mysql containers.
    MySQL will be configured automatically for 80% of the value of MEM
    Emily (Linh) Cao
    @emlcao_twitter
    Hi @TimEvens , I still have questions about MariaDB cron script!
    dgedia
    @dgedia
    Hi, @philodavies I've peered with oregon routeviews server. However, when I go to security analysis tab, I don't see any RPKI/IRR information. I also tried setting the flag when running docker command for AIO by appending -e ENABLE_RPKI=1. Is there something I am missing?
    Phil D
    @philodavies
    Do you know if the routeview server has peers with IRR or RPKI data? IRR data was available by default to me in the AIO container.
    Phil D
    @philodavies
    A question for anyone that would know.. The AS Analysis section of the UI is left blank for me, and I have BMP data being sent for all my peers. How does this section get populated? Do I need to be a transit AS or is there a specific configuration that is required to start receiving data in that section?
    Phil D
    @philodavies
    @dgedia Did you do anything with the ARIN TAL as shown at this link?
    dgedia
    @dgedia
    @philodavies Yes, the oregon routeviews server has rpki data but, it is not appearing in the security analysis tab. I placed the ARIN TAL in /var/openbmp/config/rpki/tal/ directory as well.
    @philodavies AS Analysis section appears to be blank for me as well.
    Phil D
    @philodavies
    Hmm, I'm going through the configurations and scripts to see what might be causing the RPKI data to be left out.
    In regards to the AS analysis hopefully @TimEvens can guide us in the right direction. It seems that there are certain factors that lead to an AS being shown in those windows, but I'm not sure what conditions need to be met for an AS to be shown.
    dgedia
    @dgedia
    ok. In my case, mysql tables "rpki_validator" and "rpki_history_stats" are populated with the data. However, table "gen_prefix_validation" doesn't have any data.
    dgedia
    @dgedia
    Hi @philodavies , where you able to figure out why RPKI Data isn't showing up? thanks
    Phil D
    @philodavies
    I have not been able to figure out why just yet. I've been busy on other projects this week and may not be able to get back to this for another week on top of that.
    Hopefully Tim will be back to respond to these questions while I work on other things. If not, I will dig some more to try and understand what is missing or why it isn't displaying the rpki data.
    Tim Evens
    @TimEvens
    @emlcao_twitter , @philodavies , @dgedia , FYI - I'm in the process of migrating MySQL/MariaDB to PostgreSQL/TimescaleDB. As part of this migration, I'm updating the cron scripts (whois) and RPKI. The above issues should be resolved with the postgres version. I'm migrating this because of the following:
    • MySQL/MariaDB has no real implementation for time series, TimescaleDB addresses this.
    • MySQL/MariaDB is very hard to recover when disk space runs out, Postgres addresses this.
    • MySQL/MariaDB does not support array, uuid/hash, and inet data types natively. Postgres does.
    • For this type of routing data and time series, Postgres/TimescaleDB outperforms InfluxDB, ElastichSearch, Cassandra, Mongo, and MySQL/MariaDB.
    Tim Evens
    @TimEvens
    @/all , check out https://gitter.im/snas/alerts. This room has live internet monitoring alerts.
    kjeojfeofjgeofkjodfioejo
    @jkldgoefgkljefogeg
    How does bmp monitoring station detect liveness of BMP speaker?
    kjeojfeofjgeofkjodfioejo
    @jkldgoefgkljefogeg
    It seems tcp keepalive is necessary, but not mentioned in RFC7854
    ayalaalex
    @ayalaalex
    It is controlled by TCP. Depending on the implementation of the active party BMP Client, you maybe have tcp keepalives. But that is not dependent on BMP. With BMP the active party sends request to establish connection, then it just starts sending messages until either party closes the connection. I did notice though that if active party shuts the port or simply crashes before it can send TCP RST to openBMP, openBMP won't know that it has gone down. There may be some cleanup or configuration on openBMP that I haven't played with. I saw something about heartbeat but I never got it to work. So to answer your question, BMP as an application doesn't have keepalives, but an implementation of BMP client might make use tcp keepalives.
    Tim Evens
    @TimEvens
    The TCP stack implements tcp keepalives. The application requests to enable them. TCP keepalive settings, such as idle timeout, interval, ... can be set with sysctl or /proc. The openbmp collector does enable keepalives but you have to tune that via host system settings. TCP keepalives are unidirectional. This means it should be enabled on both sides of the connection. IOS XR supports this in 6.3 and later
    kjeojfeofjgeofkjodfioejo
    @jkldgoefgkljefogeg
    in the case where TCP socket not getting properly torn down, the session might stay down while collector still thinks the socket is open. imo it's worth a note in the RFC. Some router BMP implementation doesn't support keepalive and things might break when router is configured to run BMP passive
    dgedia
    @dgedia
    I had a quick question if you know by chance. I am not able to peer with more than one route-views server at the same time from the same VM. Have you guys experienced the same? To peer with multiple route-views server does it require separate VMs?
    Tim Evens
    @TimEvens
    You need to edit the openbmpd.conf file and set enable_pat to true. This will allow the same source address to be used for more than one router connection.
    #pat_enabled value is a boolean:
    #    false (the default) - MD5 of (connection source address, collector hash)
    #
    #    true                - MD5 of one of the following:
    #
    #                 If INIT_BGP_ID (type=65531) is present:
    #                (bgp_router_id, collector_hash)  
    #
    #                        If INIT doesn't include the BGP_ID, then:
    #                (name, collector_hash)
    #
    #                 If INIT doesn't include either bgp_id or name, then it uses:
    #                (connection source address, collector hash)
    pat_enabled: false
    Kris Lloyd
    @KrisLloyd

    Hey guys! I have SNAS and PNDA both deployed in Openstack, and I'm not quite sure how I should get them working together. I read up on the 'integrating openBMP' steps in the docs on http://pnda.io/guide , but that just left me confused. Anyone ever get this up and running? Appreciate any steps you can provide to point me in the right direction. Both services are up and running with no errors, but I don't know how to link them. PNDA is an Openstack Pico deployment.

    Thanks for your help!
    -K

    Tim Evens
    @TimEvens
    Hey @KrisLloyd , PNDA integration only stores/archives the raw parsed messages. you could do the same thing with a cheap instance in elastic, influx, or file. The app that was presented was an in-memory iPython script that could only handle about three peers. What use-case are you looking to address? I’m confident you can solve your use case(s) with the new postgres backend. Btw, I will be creating a new repo for grafana that interacts with postgres. It would be great if you and others contribute to the dashboards so everyone can benefit from the visualizations. Check out http://demo-rv.snas.io:3000 for an example of the latest dashboards. Regarding alerting... check out gitter.im/snas/alerts. This is an example showing how you can alert intelligently realtime.
    @jkldgoefgkljefogeg , I agree this does need to be documented. We have talked about doing a new draft on implementations, both senders and receivers. The network setup/design also is something that needs to be documented. For example, where to monitor and which knobs to enable to meet the desired use-cases.
    Phil D
    @philodavies
    Hey @TimEvens, the desired use-case of the SNAS/PNDA integration is to have an analytics engine similar to the example video which is essentially a recap of the features you implemented in the PNDA example at NANOG70.
    Tim Evens
    @TimEvens

    PostgreSQL with TimescaleDB, RPKI, and IRR integration is available now via openbmp/postgres container

    The openbmp/collector and openbmp/kafka containers should be used wtih the openbmp/postgres containers to
    provide an end-to-end BGP monitoring and analytics.

    Grafana is the preferred visualization tool. Grafana enables collabration with dashboards and plugins without requiring
    much development experience. We encourge everyone to contribute new or updated dashboards and plugins via
    github pull requests. You can start using Grafana today with openbmp/postgres container by following the
    OpenBMP Grafana instructions.

    NOTE:

    At this time there is full support for IPv4, IPv6 unicast and labeled unicast address families.
    Soon to come will be L3VPN, Link-State, and EVPN.

    Please Join the chat on Gitter
    to collaborate with others interactively.

    edoboker
    @edoboker
    Hi all,
    I've been trying to get started with SNAS.io on my laptop as the beginning of a large network PoC, currently getting stuck after running the aio container and the ui container (just copy-paste the commands from the guide in the project's website). When I try to log in to the webpage (using any username and password) I get an error saying "authentication service is unreachable". Any thoughts? has anyone encountered that?
    Tim Evens
    @TimEvens
    @edoboker , is your laptop running Linux? The problem with Mac and windows is that docker has to run a Linux vm. This introduces some tricky network setup for container to container connectivity. Which address families are you going to poc and how many routers/peers/nlris?
    edoboker
    @edoboker
    I'm running on Ubuntu 18.04
    Everything configured to localhost, of course (that's the default settings in the website)
    Victor Liu
    @packerliu
    I have successfully enabled SNAS-aio and SNAS-UI, but still working on postgres part with collector. @edoboker , to hookup postgres with Grafana you need to use docker0 or host IP instead of localhost
    Lorenzo
    @lollo93_gitlab
    Hi everybody, I'm trying to build a custom kafka consumer for parsing the raw data (BGP Updates mainly). Is there any link or doc I can read to binary parse your header? I can read without any problem the internal message (BGP Update) but there is a header of about 127 bytes I'm not able to understand
    Tim Evens
    @TimEvens

    @lollo93_gitlab , you mean you are reading the "bmp_raw" topic? The message structure is documented Message Bus API. Jump to the bottom of that doc, or search for BMP RAW Data.

    The message has two parts... First part is the header and the second part is the RAW binary BMP message (actual BMP message, headers and all). The two parts are split by a double newline \n\n.. BMP (and BGP) route monitor messages do not convey the router that the message came from. You'll need to parse the HEADER field R_HASH_ID for that. The R_HASH_ID links the the hash id in the openbmp.parsed.routers topic.

    Lorenzo
    @lollo93_gitlab
    Hi @TimEvens , yes the bmp_raw topic. I'm reading the message in binary so the double newline how can I find it? which charset/encoding are you using?
    Tim Evens
    @TimEvens
    UTF-8, but you should be using the kafka byte deserializer