Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • May 15 10:19
    mikeruss1 commented #307
  • May 15 10:18
    mikeruss1 commented #307
  • May 15 10:13
    mikeruss1 commented #307
  • May 15 05:12
    Maikuolan commented #307
  • May 14 18:01
    mikeruss1 commented #307
  • May 14 14:13
    Maikuolan edited #307
  • May 14 10:52
    mikeruss1 opened #307
  • May 13 18:29
    mikeruss1 commented #302
  • May 13 13:51
    Maikuolan commented #302
  • May 13 13:47
    Maikuolan commented #302
  • May 13 13:46
    Maikuolan commented #302
  • May 13 13:14
    eurobank commented #305
  • May 13 13:10
    mikeruss1 commented #305
  • May 13 12:55
    eurobank commented #305
  • May 13 12:50
    737simpilot commented #305
  • May 13 12:50
    737simpilot commented #305
  • May 13 12:49
    737simpilot commented #305
  • May 13 12:45
    737simpilot commented #305
  • May 12 15:31

    Maikuolan on v3

    L10N patch. Restore "logged in" notice. Re… Accounts page patch. Changelog… and 3 more (compare)

  • May 12 10:10
    mikeruss1 commented #304
Aaron
@737simpilot

So, lets try this:

Test:
 Logic: "Any"
 Reason: "Test"
 Suppress output template: true
 Block:
  If matches:
   Profile:
- "Cloud service"

That'll do it for my intended purpose?

Does capitalization matter for Profile names?

Aaron
@737simpilot
Oh, one more thing. Does it matter where in the execution chain of the yaml this rule is placed? Remember my whitelisted VPN IP I had to place at the very bottom? I'm wondering if a rule like this above needs to be at the very top? Though, you may need to see what I have in the auxiliary yaml file thus far. Kinda not wanting to share that since it gives potential ammunition and Intel. to a would-be hacker. Though, I might be a little "too" paranoid. It's just my website is associated to this project and other metadata out there. Knowing how a potentially really good hacker would think, I'd do some OSINT. Probably don't have much to worry about though. But I don't like being owned ("pwned") and tea bagged if I can prevent it, and being a little paranoid never hurt anyone as it pertains to the computer centric ethos if you will. LOL! Anyway...
Aaron
@737simpilot

I'm a freaking nerd, man. HAHAHA

img

Caleb Mazalevskis
@Maikuolan
lol
Caleb Mazalevskis
@Maikuolan

That'll do it for my intended purpose?

I should probably clarify there: The "Cloud service" message gets populated into the "Why blocked" field (i.e., I don't actually specifically "profile" for "cloud service" currently). So, to catch requests using that particular message, you'll probably want the "Why blocked" rather than the "Profile" fields in the auxiliary rule. Also, given that "short messages" (messages like "Cloud service", "Generic", "Invalid IP", "Malware", "Spam risk", etc, which get populated into the "Why blocked" field) can be clustered together within a single request, when matching against such "short messages", you'll probably want to add wildcards at the beginning and end of the match, too, just in case there's more than one unrelated signatures triggering for the request in question (i.e., "Cloud service"). Given the use of wildcards, you'll probably also want to use the Windows-style wildcards option for the use. So, all in all, something like this would be best:

Test:
 Method: "WinEx"
 Logic: "Any"
 Reason: "Test"
 Suppress output template: true
 Block:
  If matches:
   WhyReason:
    - "*Cloud service*"
(Not tested yet at my end mind you, but should work properly, in theory).

Does capitalization matter for Profile names?

I don't think so. Let me just quickly double-check my code right now though to make sure about that. One sec.

Ah.. Nah, yeah; It's case-sensitive. '^.^
So, capitalisation will matter, lol.
But since your use in the above matches what's in the L10N data.. I don't anticipate any problems there regardless.
Though, for those situations where you really, really need to use case-insensitive matching for a condition, you'll want to use regular expressions (seeing as you can indicate with a regex flag whether the match should be case sensitive or insensitive).
Caleb Mazalevskis
@Maikuolan

So, if using regular expressions, for the above example, you could use something like this, for example:

~^.*Cloud service.*$~i

(That "i" at the end tells PHP that the regular expression is case-insensitive).

Or, for a more complete example:
Test:
 Method: "RexEx"
 Logic: "Any"
 Reason: "Test"
 Suppress output template: true
 Block:
  If matches:
   WhyReason:
    - "~^.*Cloud service.*$~i"
But yeah. Both "direct string comparison" and "Windows-style wildcards" alike = Sensitive. But, regular expressions = You can specify it yourself whether it should be sensitive or insensitive (based on the presence or absence of that "i" flag).
Caleb Mazalevskis
@Maikuolan
..I wonder though. Should I maybe change that, so that "short messages" can be "profiled"? Might make things easier, maybe.
Aaron
@737simpilot

..I wonder though. Should I maybe change that, so that "short messages" can be "profiled"? Might make things easier, maybe.

Sounds reasonable.

Caleb Mazalevskis
@Maikuolan
..and done. :-)
The "shorthand" words used by signatures will now also be populated to profiles.
(Which means you could now do this, if you wanted):
Test:
 Logic: "Any"
 Reason: "Test"
 Suppress output template: true
 Block:
  If matches:
   Profile:
    - "Cloud"
Aaron
@737simpilot

Thanks.

Just to be sure, are these the shorthand words?

ReasonMessage_Attacks:
ReasonMessage_BadIP: 
ReasonMessage_Banned:
ReasonMessage_Bogon: 
ReasonMessage_Cloud: 
ReasonMessage_Generic:
ReasonMessage_Legal: 
ReasonMessage_Malware:
ReasonMessage_Proxy: 
ReasonMessage_Spam:

Or these?

Short_Attacks: "Attacks"
Short_BadIP: "Invalid IP"
Short_Banned: "Banned"
Short_Bogon: "Bogon IP"
Short_Cloud: "Cloud service"
Short_Generic: "Generic"
Short_Legal: "Legal"
Short_Malware: "Malware"
Short_Proxy: "Proxy"
Short_RL: "Rate limited"
Short_Spam: "Spam risk"

I ask because in your example you simply used "Cloud" instead of "Cloud service". It must be from the first group. i.e "BadIP" instead of "Invalid IP".

And we're clear that capitalization DOES matter? I guess that's the way the code is.
Caleb Mazalevskis
@Maikuolan
From just slightly above here: https://github.com/CIDRAM/Docs/blob/master/readme.en.md#71-tags
The available shorthand words are:

- Attacks
- Bogon
- Cloud
- Generic
- Legal
- Malware
- Proxy
- Spam
So basically, those "shorthand" words are used internally by CIDRAM as cues for which L10N data it should be pulling when generating messages for various block reasons. Those particular internally-used "shorthand" words are what I've set it to use for profiling now. :-)
(The reason I've opted to use those instead of something from the L10N data is so that our auxiliary rules will still work the same regardless of the preferred language specified for an installation, i.e., if we change our settings from English to Chinese or Spanish or whatever, our rules should still behave the same way).
Also, sorry for the delayed reply. Busy past few days.
Aaron
@737simpilot

Thanks. I wasn't aware that the documentation got updated.

Kinda a stupid question I guess: How can one download a file from a repository? I always have to go to raw, copy to Notepad ++ and then save. It seems there should be a better way and I'm not seeing the option. A download option shows up for some repositories on Github, but not all. I just want to update my readme file.

Caleb Mazalevskis
@Maikuolan
Could do it somewhat relatively easily via Git bash (command line), but not sure about when it comes to the GitHub UI.
Probably would be forced to just download a tarball or zip for the entire archive, find the file in question, extract, and do as you will with it.
Aaron
@737simpilot

Probably would be forced to just download a tarball or zip for the entire archive, find the file in question, extract, and do as you will with it.

That's what I thought.

And downloaded.

Another question, or rather your opinion. I uploaded my public key to my Repo. and archived it. Whatda think of me doing that? I saw some other website for public keys and user verification for things, but I don't remember it anymore. Actually saw a Github code Dev. use that service actually. I'm reluctant to add the email address though. I'm hoping it serves as a way for me to say, "yeah, this public key can be used with this email I give you to help provide authenticity."

Caleb Mazalevskis
@Maikuolan
Difficult to say. I publicly list my own public key online, but not alongside my email address directly. Seeing as emails are tied to Git commits anyway, plus there are GPG lookup directories available, I figure anyone wanting to actually use my public key to contact me should be smart enough to already know how to find the address connected to it, so explicitly listing the address shouldn't be necessary. But I don't know what others would think about that.
Aaron
@737simpilot

plus there are GPG lookup directories available

I've read about a year or two ago they did away with one due to some flaw they couldn't figure out how to fix due to the way it works. Like being a victim to a DDoS on an email server without being able to use a reverse proxy like Cloudflare et al. Though, a quick Google search for this Info. I remember reading turns up nothing and instead I see some other lookup websites. LOL Go figure.

Question: I see in the update notes that you're now using Github discussions. Should we just abandon this Gitter channel or what?

Caleb Mazalevskis
@Maikuolan

I've already deleted phpMussel's Gitter channel, so that one'll 404 now. (Nobody had used phpMussel's Gitter channel in over a year and a half now anyway, so I didn't feel much need to bother informing anyone about it beforehand).

I was planning to eventually (hopefully very soon, but pending my available time, which hasn't been too great recently) do the same here for CIDRAM's Gitter channel, but because I haven't yet "formalised" exactly how we'll be using GitHub Discussions for CIDRAM (I had that idea about sharing auxiliary rules and maybe setting up a code library there, but I haven't quite figured out exactly how that's supposed to work just yet), because I also haven't properly told everyone my plans yet, and also because CIDRAM's Gitter channel is still actively used at this time (albeit by just you, me, and mikeruss; and 3 people isn't really many people; the other two people here haven't posted anything since as long as I can actually remember offhandedly), I've delayed deleting CIDRAM's Gitter channel, for now.

(Note: Using the discussions feature the exact same way we use Gitter is totally fine, and I'm cool with that. General chat or whatever is fine as far as I'm concerned. I don't plan to be rigid or strict about it or anything like that. But, by "formalised", I mean I want to think about those ideas first, just in case I realise something doesn't work, or how to make it work properly, or in case I discover we still need the Gitter channel after having already deleted it or whatever; and also, when we're ready, dropping a message somewhere along of lines of, "Hey everyone! Maikuolan here. Just to let you know, we're planning to delete the Gitter channel in X days from now. You can all just use the Discussions feature from now on for that stuff"). But, we can use it now, too, if we want. Whichever is preferable for the moment (albeit noting that stuff here at Gitter won't be staying around for too long, most likely).

mikeruss1
@mikeruss1
whatever works best for you is fine, dont want to be looking in 2 places, so just shout when ready to move
Aaron
@737simpilot

(Note: Using the discussions feature the exact same way we use Gitter is totally fine, and I'm cool with that. General chat or whatever is fine as far as I'm concerned. I don't plan to be rigid or strict about it or anything like that.

Sounds good, thanks.

mikeruss1
@mikeruss1
is v3 regarded as stable, and assuming we are coming from v2 is there a conversion process? Like config.ini, custom_mods, auxiliary rules ?
Caleb Mazalevskis
@Maikuolan
Not yet stable (still "alpha", since there are still a number of backwards-incompatible changes I plan to make for v3, which could cause breaks between individual commits/updates yet). When it reaches "beta" though, I'll make an announcement. :-)
Stable enough that you could safely run a local test copy though, if just for the sake of checking things out and providing feedback. I'd wait a little bit yet before deploying it to an actual website though.
mikeruss1
@mikeruss1
OK thanks, think I will wait for beta.
mikeruss1
@mikeruss1
could you please remind me where we are with bypassing search engines like Duckduckbot who use MSAZ? I have the mcmathan module turned off for reasons I cant remember ! Came across this ...
which might be the basis of a useful replacement?
I am blocking Duckduckbot coming from a verified source on MSAZ
Aaron
@737simpilot

The verification.yaml file located in the vault has the IPs that are whitelisted. And those IPs currently are only 14 in total and come from the source here: https://help.duckduckgo.com/duckduckgo-help-pages/results/duckduckbot/

ALL of DuckDuckGo's current IPs are from the Azure cloud except two which are from Amazon AWS. None of DuckDuckGo's PTRs show up except from the AWS assigned IPs. Apparently, the closure is a UA to IP. So maybe the UA doesn't match? The UA pattern is currently this:

~duckduck(?:go-favicons-)?bot~

Anything else would be blocked. Maybe DuckDuckGo has a new UA?

For posterity, this people are idiots! 100% confidence!?

I wish Mozilla's HSTS pre-load list efforts were this simple with 14 or so IPs and a unique UA...
This message was deleted
Aaron
@737simpilot
OH! I have the following auxiliary rules just in case. I see now I have to edit the IPs.
DuckDuckGo Profile:
 Logic: "Any"
 Profile:
  If matches:
   IPAddr:
    - "20.191.45.212"
    - "23.21.227.69"
    - "40.88.21.235"
    - "50.16.241.113"
    - "50.16.241.114"
    - "50.16.241.117"
    - "50.16.247.234"
    - "52.5.190.19"
    - "52.204.97.54"
    - "54.197.234.188"
    - "54.208.100.253"
    - "54.208.102.37"
    - "107.21.1.8"
DuckDuckGo:
 Method: "WinEx"
 Logic: "Any"
 Whitelist:
  If matches:
   UA:
    - "*DuckDuckGo-*"
   Profile:
    - "DuckDuckGo Profile"

I also have this in my htaccess:

#
#Check that the request is for /robots.txt
RewriteCond %{REQUEST_URI} ^/robots.txt
#Check that the request matches an existing file
RewriteCond %{REQUEST_FILENAME} -f
# Check that the user agent does not contain google etc
RewriteCond %{HTTP_USER_AGENT} !google
RewriteCond %{HTTP_USER_AGENT} !yahoo
RewriteCond %{HTTP_USER_AGENT} !bing
RewriteCond %{HTTP_USER_AGENT} !duckduckgo
# If all conditions above are met, then deny access to this request
RewriteRule ^ - [F,L]

(Will work in Litespeed and probably Apache).
My Cloudflare rules further expand upon this. It seems htaccess can be bypassed and I've tried figuring out how that is done, but couldn't find a resource on it. I guess this is where a php.ini file shines.

Aaron
@737simpilot
A fake UA can be used, but CIDRAM has IP to UA verification for bots, it's just that a .TXT file isn't protected unless a type of PHP "router" is used I think is what it's called.
mikeruss1
@mikeruss1
thanks Aaron