Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • May 13 12:23
    MatteoManzoni opened #64
  • Apr 22 19:14
    aouinizied closed #61
  • Apr 22 19:14
    aouinizied commented #61
  • Apr 22 14:51
    aouinizied commented #61
  • Apr 22 11:02
    SoerenBusse opened #61
  • Apr 13 14:49
    winterqz closed #60
  • Apr 13 14:49
    winterqz commented #60
  • Apr 13 14:34
    aouinizied commented #60
  • Apr 13 14:34
    aouinizied closed #48
  • Apr 13 00:17
    aouinizied commented #48
  • Apr 08 12:06
    aouinizied assigned #60
  • Apr 08 12:06
    aouinizied labeled #60
  • Apr 08 12:05
    aouinizied closed #46
  • Apr 08 12:05
    aouinizied commented #46
  • Apr 08 12:04
    aouinizied closed #41
  • Apr 07 13:16
    winterqz commented #60
  • Apr 07 12:53
    aouinizied commented #60
  • Mar 30 09:31
    winterqz opened #60
  • Mar 26 00:22
    zambery closed #59
  • Mar 26 00:22
    zambery commented #59
Zied Aouini
@aouinizied
to avoid overhead induced by per packet callbacks
Wait I will give you more details
See that line
it's the main loop
and here is The None we receive when there is no packets
we use this cycles to clean cache and stuff
Simone Errico
@Lyonid_gitlab
Clear
Zied Aouini
@aouinizied
And even with PF_RING DPDK
or other high speed appliance
you will have such behavior
it's active polling
Simone Errico
@Lyonid_gitlab
Yea, from that point of view that PF_RING or anything doesn't make any difference
Clearly
See this article
It explain clearly why we use active polling
you can add usleep(1)
if you want
but that will results in more drops
Simone Errico
@Lyonid_gitlab
I think you couldn't have been clearer even if you wanted :) I'll keep on working on it and report back from time to time. You have been of great help, thanks
Zied Aouini
@aouinizied
You are welcome!
Zied Aouini
@aouinizied
@Lyonid_gitlab just a small precision, libpcap is already using af_packetv3
Zied Aouini
@aouinizied
so for future plans, it will be the first option that we will evaluate
Simone Errico
@Lyonid_gitlab
Hi @aouinizied, here I am again. Is there a way to monitor performances at runtime? Like, if I want to check packet drop percentage every n minutes, is there some built-in function I'm missing?
Zied Aouini
@aouinizied
@Lyonid_gitlab
Zied Aouini
@aouinizied
@Lyonid_gitlab You must upgrade to version 6.1.1 where we implemented this feature for live capture.
  • performance_report=N will print every N seconds performance metrics (https://github.com/nfstream/nfstream/blob/master/assets/PERFORMANCE_REPORT.md).
  • To limit drops, you must set n_meters to 0 (default value). Note that processes will consume CPU only when packets are there (No busy-wait anymore, so fixed your previous issue).
  • NFStream now supports AF_PACKETv3 + FANOUT on Linux machines.
  • Before considering more powerful hardware, make sure to test with Pypy3 instead of CPython as it is faster.
Simone Errico
@Lyonid_gitlab
@aouinizied Thanks.
Neul Do
@doitez2
Hello Sir, Can I Run nfstream code in colab.research.google.com for extraction pcap?, i have lil bit problem for importing my pcap and dataset
Zied Aouini
@aouinizied
@doitez2 Yes nfstream can be installed on Colab without issues. Which problem you have for pcap import?
Neul Do
@doitez2

It's Say Like This :
Process Process-2:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(self._args, *self._kwargs)
File "/usr/local/lib/python3.6/dist-packages/nfstream/meter.py", line 185, in meter_workflow
set_affinity(root_idx+1)
File "/usr/local/lib/python3.6/dist-packages/nfstream/utils.py", line 103, in set_affinity
psutil.Process().cpu_affinity(list(temp[mask]))
IndexError: list index out of range

I Have Tried Pcap From NFStream and got same error

Zied Aouini
@aouinizied
@doitez2 Please upgrade to nfstream last version (6.2.0). It should fix the issue you are reporting.
George Cox
@gjvc
hello i'm investigating the use of nfstream for some analysis of multicast data. I will need to be able to extract the sequence number out of each multicast packet (i know how to decode sequence number, given a byte buffer), but could someone point me in the direction of how i might write the "plugin" for this? I think it will be a plugin to the NDPI side of things. thank you very much! :-)
Zied Aouini
@aouinizied
@gjvc Yes there is also some plugins already implemented by the community: MDNS, DHCP, ..
feel free to open a PR with your plugin (once done) if you think it is possible to share it with the community.
the bytes buffer starting from IP header is available on the NFPAcket that NFStream will pass to your Plugin
Zied Aouini
@aouinizied
and Yes possible too in nDPI side, so both options are valid
you can prototype in a fast way as a Plugin for NFstream, once you are sure everything is working as you want
you can pass it to nDPI side
and it will be automatically integrated in NFStream
second option is interesting if you care about performances
but as you will run it with PyPy and your Plugin is pure Python, I think it's too early to think about that
George Cox
@gjvc
@aouinizied ahhhh lovely. I will use the what's in the plugin directory as a starting point. thank you very much!
Lyonid
@Lyonid
image.png
Hi! Do you have any idea why this is happening?
All other attributes work fine. But from documentation flow object should have src_mac attribute
Lyonid
@Lyonid
Upgrading nfstream package fixed this. Sorry for disturbance