Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Nov 30 13:08

    cvandeplas on main

    chg: [feeds] added inherit from… (compare)

  • Nov 29 15:33
    besendorf opened #273
  • Oct 26 09:39

    adulau on main

    chg: [types and categories] ssh… Merge branch 'main' of github.c… (compare)

  • Oct 13 20:04
    adulau commented #272
  • Oct 13 20:04

    adulau on main

    Update README.md Fix typo Merge pull request #272 from cl… (compare)

  • Oct 13 20:04
    adulau closed #272
  • Oct 13 08:34
    cliodhna-lynch opened #272
  • Oct 01 17:55
    adulau commented #270
  • Oct 01 17:55

    adulau on main

    Fix typos Merge pull request #270 from ga… (compare)

  • Oct 01 17:55
    adulau closed #270
  • Oct 01 12:15
    Wachizungu opened #271
  • Oct 01 09:47
    garanews opened #270
  • Aug 20 06:34

    cvandeplas on main

    chg: [types] updated types and … (compare)

  • Jun 13 22:09
    Wachizungu opened #269
  • Jun 01 19:08
    Wachizungu opened #268
  • May 09 08:38
    adulau commented #267
  • May 09 08:38
    adulau closed #266
  • May 09 08:38

    adulau on main

    chg: [warninglists] Restructure… Merge pull request #267 from Wa… (compare)

  • May 09 08:38
    adulau closed #267
  • May 08 18:29
    Wachizungu opened #267
msc_xyz
@msc_xyz:matrix.org
[m]
I've switched to redis for php session and added innodb_buffer_pool_size to the mariadb config file (section mysqld). This already speeds up everything much
could you suggest some other variables for tuning the database?
eCrimeLabs
@eCrimeLabs

@msc_xyz:matrix.org

[mysqld]
bind-address=127.0.0.1
innodb_buffer_pool_instances=6
query_cache_size=2048M
innodb_buffer_pool_size=6G
max_allowed_packet=300M
innodb_log_file_size=256M

This did some good for me, could potentially be optimized even more :)

2 replies
github-germ
@github-germ
Hello... can anyone educate me on the source of the data in redis 'misp:cidr_cache_list' ?
github-germ
@github-germ
OK, figured that out. thx.
meltedpenguin
@meltedpenguin
@eCrimeLabs Could you provide some information where these settings are? I am running Ubuntu 20.04.
@msc_xyz:matrix.org Where did you add the buffer_pool size?
3 replies
msc_xyz
@msc_xyz:matrix.org
[m]
Is it normal that I must use the admin account to trigger the fetchFromAllFeeds API endpoint? I got permissions denied with the org-admin account...
lucatrabalza
@lucatrabalza
hello, every day i fetch feed from misp comunity, when misp dowload feeds dowload all feeds and remove existing from misp db or just add new feeds ? Cause i have like 10 gb of events every day..
BinksJar
@BinksJar007_twitter
Hello, I am having issue configuring a sync from MISP Instance A to MISP Instance B - when I add the server into MISP and test the connection I get a 403 error I have checked permission and I am able to see the API Key being used in MISP however I still receive 403. Can anyone please advise ?
Daniel Jaraud
@fojac
Hello, I'm having an issue on a fresh MISP installation on a Debian LxC with the INSTALL.sh script downloaded today from https://raw.githubusercontent.com/MISP/MISP/2.4/INSTALL/INSTALL.sh . The installation starts smootlhy (I tried to install only MISP Core) and nearly everything is downloaded and installed. However, the installer stalls at "Cloning into faup... resolving deltas: 100%" and never gets further. No error displayed. I've tried multiple times, without success. Has anyone encountered this? Any hint on how to proceed? Thanks!
Daniel Jaraud
@fojac
FYI, workaround: relaunching the script (without deleting the files) did the trick...
fl0x2208
@fl0x2208
So I am using mail to misp and it works - but if there is virustotal - it adds as network activity rather then external URL - how can I fix that?
Sorry external analysis
riccardosl
@riccardosl
Hello everyone, do you know where successful and failed LOCAL login events are logged in MISP file system?
cybgit
@cybgit
Correlations: Does anyone know if you can query the correlations exclusions index by API (pymisp etc) ?
5 replies
Michael
@ag-michael
where are the logs for the prio worker? how do I find out why it keeps dying and backlogging a ton of jobs?
Andras Iklody
@iglocska
it should be together with the other logs
MISP/app/tmp/logs
iirc
resque-worker-error.log
and resque-worker-[date].log
Michael
@ag-michael
I see some redis error abou tit loading the dataset in memory. I ran FLUSHALL with redis-cli , redis was taking up 8GB ram again :/
Michael
@ag-michael
is there a better way to publish without using jobs/workers? can I directly set a column in mysql?
and tyvm @iglocska
andras
@andras:matrix.circl.lu
[m]
directly setting the column will just cause issues and not have the intended effect
it is "possible" to disable all background processing
1 reply
but your requests will take a fair bit longer
would be better to figure out what's causing it
I remember you had that weird issue with your redis memory usage growing - we couldn't reproduce that, though we should dig deeper with you and perhaps set up a debug session to figure out what's going on there
Michael
@ag-michael
I have a lot of headache from the whole publication process. When you have events constantly coming in where you expect them to be published by default, it isn't working well
andras
@andras:matrix.circl.lu
[m]
but it will be SLOW at times
Michael
@ag-michael
when the prio workers fails, I get 300k+ jobs in the queue in a few hours
andras
@andras:matrix.circl.lu
[m]
woaaaah
300k jobs? What the hell
Michael
@ag-michael
because I have a script that auto-publishes unpublished stuff (and it does not attempt to publish twice, even if the past attempt failed)
and I'm only publishing so the attributes are searchable outside of MISP , to make use of them operationally ,when publishing fails all those events are missed until it catches up again
strangely, after flusing redis, I now have all works with 0 jobs in their queue but endless pages of jobs in "queue"
i'll just flush all jobs and hope clearing redis made things better
I think part of it maybe that 16G ram isn't cutting it, but when redis is using 8GB ... maybe like you said there is a deeper issue, i'm just not familiar with redis to poke around and easily figure it out
it's gotten so bad, I have a script that looks at 500 errors in apache/misp logs and restarts apache every 10min (sort of improves things..until it doesn't :P )
Michael
@ag-michael
Usually, I can get something out of the logs i can use to file a bug report, but I haven't been able to for these issues.. I'll keep trying /tyvm
Michael
@ag-michael
workers aren't dead yet, but still 170K+ jobs and only increasing lol. 5 prio workers. I think a big part of the problem is how publishing has to send emails,update zmq,etc... maybe there is a setting I missed that would disable all that since I don't need those features?
eCrimeLabs
@eCrimeLabs
When enabling Advanced auth keys, is there a way to create a "Service account" and then create the API key through the API to avoid having to login as the user ?
Asking for a friend ;)
adulau
@adulau:matrix.circl.lu
[m]
Virtual MISP Summit 0x06 - Thursday 21st October 2021.
Registration is now open.
Do you want to present or show how you use MISP, the call-for-papers is also open. #ThreatIntel #OpenSource #CTI See you there!
https://misp-project.org/misp-summit/
cybgit
@cybgit
Nice one. I've signed up.
lucatrabalza
@lucatrabalza
hello , somebody that knows if can I integrate misp with Ibm Qradar?
Matthew Keay
@matthewkeay_twitter
Am I doing something stupid in that when I freetext import an IP IOC it shows correlations but when I actually import them; those correlations don't show on the event? (I can spam a screenshot/example event if nobody minds?)
Matthew Keay
@matthewkeay_twitter
(on misppriv, for context sorry)