Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    SuperQ
    @SuperQ:matrix.org
    [m]
    Yea
    Thibault "bui" Koechlin
    @buixor
    don't know how much you looked into crowdsec yet, but in your case you're likely to want to have multiple agents and one local api
    SuperQ
    @SuperQ:matrix.org
    [m]
    I just started reading the docs yesterday
    Thibault "bui" Koechlin
    @buixor
    beefing up one machine and attempting to have one instance dealing with all the logs doesn't sound like a good idea
    SuperQ
    @SuperQ:matrix.org
    [m]
    I'm exploring options for better global rate controls / WAF-style request scoring
    Yes, we run everything on Kubernetes, so sharding is necessary/easy
    Multiple clusters, regions
    Although, it would probably be good enough to only run the traffic control per region
    Thibault "bui" Koechlin
    @buixor
    yes, except if you have some cdn on top of it ?
    SuperQ
    @SuperQ:matrix.org
    [m]
    Yes
    Thibault "bui" Koechlin
    @buixor
    there is currently a bouncer for cloudflare, and we are working on the fastly one. any chance you use either of those two ? :p
    SuperQ
    @SuperQ:matrix.org
    [m]
    Yes
    I guess I should read up on how the bouncer integration works.
    Thibault "bui" Koechlin
    @buixor
    basically : the bouncers use the local api (to which the agents pushes alerts that are converted into decisions) to know which ip/ranges/country/whatever to apply decisions on (or to delete existing decisions)
    I'll let you read tho, and don't hesitate if things are unclear, documentation is a never ending task :D
    SuperQ
    @SuperQ:matrix.org
    [m]
    Oh, yea, I know
    I also am a contributor to an open source project.
    Sébastien Aperghis-Tramoni
    @maddingue
    Hello, I am trying to setup a Crowsed POC at work, and I feel a bit lost
    a first (minor) thing is that, unless I'm completely misreading, the crowdsec_service.acquisition_dir attribute in the main config.yaml doesn't seem to be honored, while crowdsec_service.acquisition_path is
    Thibault "bui" Koechlin
    @buixor
    Hello @maddingue :) It "should" be honored (looking at the code), it's going to check for *.yaml files within that directory
    Sébastien Aperghis-Tramoni
    @maddingue
    oh, only *.yaml and not *.yml?
    Thibault "bui" Koechlin
    @buixor
    yes ... looking back at it, it might not be the brightest idea :)
    Sébastien Aperghis-Tramoni
    @maddingue
    indeed, I confirm that this works
    Thibault "bui" Koechlin
    @buixor
    opened an issue for this :)
    Sébastien Aperghis-Tramoni
    @maddingue
    thanks!
    that's probably because of Ansible, but I mostly use .yml, although I don't mind naming these files .yaml
    Thibault "bui" Koechlin
    @buixor
    yes, but you're right and this might be misleading
    Sébastien Aperghis-Tramoni
    @maddingue
    The second problem I have is less trivial, and probably me missing something. I've installed Crowdsec using the Debian package, version 1.2.1, on a test server with nginx. I'm trying to "attack" the server by requesting with curl a URL that should trigger an alert. However nothing happens. Removing the corresponding network from the whitelist doesn't change anything.
    XY: what I'm trying to do is test & understand how Crowdsec processes such alerts
    Thibault "bui" Koechlin
    @buixor
    by default it's going to drop events from private IPs (see. https://hub.crowdsec.net/author/crowdsecurity/configurations/whitelists) but you can remove it with cscli parsers remove & reloading crowdsec
    if you want to see what's parsed or not, cscli metrics is going to expose some prometheus metrics in CLI, and should give you hints on what is going on :)
    (and/or, last but not least, cscli explain might help you understand what happens to a given log line or set of log lines !)
    Sébastien Aperghis-Tramoni
    @maddingue
    oh, I just realized with cscli parsers list that it apparently didn't like me directly editing hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml; trying what you said
    Thibault "bui" Koechlin
    @buixor
    oh actually you can edit the parser directly if you wish, should work too, but you should explicitely reload crowdsec tho
    Sébastien Aperghis-Tramoni
    @maddingue
    that's what I previous did, but cscli parsers list showed this:
     crowdsecurity/whitelists        ⚠️  enabled,tainted  ?        /etc/crowdsec/parsers/s02-enrich/whitelists.yaml
    Thibault "bui" Koechlin
    @buixor
    yep, it's fine, it's just telling you that you modified the upstream version and it's not going to be updated by cscli upgrade or such
    but if you modified it to remove the ip, reloaded and it doesn't detect things, something might be wrong
    does cscli metrics seems to show that your nginx logs are parsed at least ?
    Sébastien Aperghis-Tramoni
    @maddingue
    yes, now it seems to have successfully parsed the nginx logs
    Thibault "bui" Koechlin
    @buixor
    great, do you see the scenarios being triggered ?
    Sébastien Aperghis-Tramoni
    @maddingue
    oh, didn't know that command. It indeed explains well what it saw in the nginx log. The scenarios are correctly triggered:
        ├ Scenarios
            ├ 🟢 crowdsecurity/http-crawl-non_statics
            ├ 🟢 crowdsecurity/http-path-traversal-probing
            └ 🟢 crowdsecurity/http-probing
    Thibault "bui" Koechlin
    @buixor
    great :)
    Klaus Agnoletti
    @klausagnoletti_twitter
    Hey, I'm head of community at CrowdSec! I would really like to spend 30 mins of your time to understand how you're using CrowdSec if you have not yet talked to me about it :-) Please DM me if you want to participate. It would be a great help!
    Arto Pastinen
    @apassi
    Hi, is it possible to send alert, altaugh the postoverflow whitelists has discarded the overflow ?
    Thibault "bui" Koechlin
    @buixor
    Hello @apassi ! No, the alert won't be sent if the overflow was whitelisted, but depending on what you want to achieve there might be some other ways : simulation flag on specific scenarios, or profiles so that a given scenario only trigger alerting but no decision :)
    Arto Pastinen
    @apassi
    @buixor , ah, the simulation flag should do it, thanks. I like to whitelist my public IP, but still send the alert if the overflow happends. What i like to be able to do, is to run OS command with expr, (which results my public ip), but in current implementation i will run it on separate process and write to file, which i can read with expr.
    Thibault "bui" Koechlin
    @buixor
    oh, having a expr helper to execute a command might help ... but it might as well pose some security questions :)
    please let me know how it goes !