Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Apr 12 17:21
    chrisinmtown edited #254
  • Apr 12 17:20
    chrisinmtown edited #254
  • Apr 12 12:08
    adulau commented #261
  • Apr 12 12:08

    adulau on main

    new: [doc] Add a contributing g… chg: [doc] Add back ticks Update CONTRIBUTING.md and 5 more (compare)

  • Apr 12 12:08
    adulau closed #261
  • Apr 12 07:44
    PROTechThor labeled #260
  • Apr 12 07:39
    PROTechThor synchronize #261
  • Apr 12 07:38
    PROTechThor opened #261
  • Apr 07 09:39
    chrisinmtown commented #260
  • Apr 07 07:37
    mokaddem commented #260
  • Apr 06 17:38
    chrisinmtown edited #260
  • Apr 06 17:29
    chrisinmtown edited #260
  • Apr 06 17:25
    chrisinmtown edited #260
  • Apr 06 16:57
    chrisinmtown opened #260
  • Apr 04 06:41
    adulau commented #258
  • Apr 04 06:41
    adulau closed #249
  • Apr 04 06:41

    adulau on main

    chg: [Automation] Add separate … Merge pull request #258 from Wa… (compare)

  • Apr 04 06:41
    adulau closed #258
  • Apr 02 09:10
    PROTechThor labeled #249
  • Apr 02 09:09
    PROTechThor labeled #245
Levi
@levitannin
Hello MISP peeps. I'm looking for any good resources on utilizing the REST client and/or connecting MISP to RSA NetWitness. I'm newer to this platform (and the community) so if anyone has any information please let me know :)
Chris Lott
@chrisInMtown_twitter
@levitannin if you don't know about it already, you might look at PyMISP which lets you build your own custom REST Client to use the MISP REST interface
Chris Lott
@chrisInMtown_twitter
hmm at the risk of interrupting dinner, or a post-exercise adult beverage, I'd like to ask a question: Can Cake cache (copy to Redis) object attributes
andras
@andras:matrix.circl.lu
[m]
Atm no, but I mean it can do anything we code it to do 😂
But not sure i understood the question
Chris Lott
@chrisInMtown_twitter
we would like overlap (correlation) analysis for object attributes also. Today we only get that for plain attributes
andras
@andras:matrix.circl.lu
[m]
Oh wow really? That’s a bug
It should be for everything
You mean via feed/server caching?
Chris Lott
@chrisInMtown_twitter
um let me rephrase please. I'm not saying it cannot. I'm checking if it CAN
andras
@andras:matrix.circl.lu
[m]
Generally it should ignore objects altogether and fetch all attribute values
Meaning it flattens the event first
Object attributes and normal ones alike
Chris Lott
@chrisInMtown_twitter
none of our feeds processed by supplied MISP/Cake/delta ingest use object-attributes. Only our custom feeds; and MISP/Cake cannot fetch/cache those
andras
@andras:matrix.circl.lu
[m]
It should be able to cache those too
Unless i completely missed the point
Chris Lott
@chrisInMtown_twitter
that's why I say "MISP/Cake cannot cache"
The problem is the ETL task of consuming the premium data feed and munging it into events and attributes;
andras
@andras:matrix.circl.lu
[m]
Yeah depending on the format that might need a fair bit of glue
Chris Lott
@chrisInMtown_twitter
hmm questioning my assumptions here .. I believe that "cache" is an action that copies from the source (the remote server) to the local redis cache.
andras
@andras:matrix.circl.lu
[m]
It’s even dumber than that
Chris Lott
@chrisInMtown_twitter
If however "cache" can operate as copying from local MISP database of events, obj & attributes to local Redis memcache, then that would be very interesting
andras
@andras:matrix.circl.lu
[m]
It loops the remote through the ingestion that you would normally use for a fetch operation
Extracts the values from the derived attributes
Hashes them
And throws them into Redis
When it comes to misp format feeds/misp servers
It does something different
Chris Lott
@chrisInMtown_twitter
Thanks for the details. but for our in-house-developed ingestion scripts that copy remote to local database, I don't know how to get that data into the redis cache/analysis chain
andras
@andras:matrix.circl.lu
[m]
What format are they in?
Chris Lott
@chrisInMtown_twitter
some of our in-house ingestion scripts create object attributes
oh boy, format. Some XML, some JSON
andras
@andras:matrix.circl.lu
[m]
That should be fine
If you convert them to misp format already
You’re half way there
Generate a misp feed out of them
Chris Lott
@chrisInMtown_twitter
hmm starting to question everything we're doing :/
andras
@andras:matrix.circl.lu
[m]
Hehe
Chris Lott
@chrisInMtown_twitter
we use Python & PyMISP to massage the data and inject straight to events & attributes & objects
andras
@andras:matrix.circl.lu
[m]
So basically what we do normally
If we have a source in a format misp can’t cope with
Chris Lott
@chrisInMtown_twitter
I think you're suggesting, if we consume a premium feed and transform it to a MISP-friendly format, then we could use the MISP ingester, which would also allow us to use the MISP cacher. Did I understand correctly?
andras
@andras:matrix.circl.lu
[m]
We generate a misp feed out of it with a simple pymisp script + our own conversion logic
Indeed
What you said
That way we can interact with the feed’s content as if it was a misp instance
Chris Lott
@chrisInMtown_twitter
Thanks for clarifying. I will take this under advisement. It is not at all clear to me if this will let us cope with all the special situations; for example in some cases we delete the old event before ingesting a new version of it.
andras
@andras:matrix.circl.lu
[m]
Not sure. Think you folks have some pretty specific use cases that might not fit. If you see any obvious limitations holler maybe we have some ideas
Chris Lott
@chrisInMtown_twitter
well, the use cases are invariably driven by provider data oddities, the need to avoid duplicates, etc.
andras
@andras:matrix.circl.lu
[m]
Yeah duplicate management can be tough especially for sources that don’t contain unique identifiers