cea-sec/ivre

IVRE: a network recon framework

Pierre Lalet

@p-l-

The scan results will be merged in the view (after running ivre db2view).

Pierre Lalet

@p-l-

Also, if you can provide a sample of other protocols you would like to see supported in IVRE, I will try to add them.

Senanfurkan

@Senanfurkan

Sure, I can provide... I would love to. I think it is fine not to handle zmap results, since zgrab only took open port banners.

Scanning entire internet with nmap and masscan is a pain. Masscan is good but I think it is not reliable much when you compare with the zmap

Pierre Lalet

@p-l-

I am far from agreeing with your opinion regarding Masscan!But anyway supporting Zgrab(2) output is a good thing for IVRE since it is very handy in some situations.

Senanfurkan

@Senanfurkan

I am not experienced much in that area, it's just my personal project and I'm still trying to learn. I watched one of zmap developers presentation and he showed a research chart about scan reliability between zmap, nmap and masscan. Zmap was on top about reliability/speed efficiency.

Pierre Lalet

@p-l-

Yeah I have no doubt a Zmap dev can prove Zmap is better ;-). Anyway I have no idea about how Zmap and Masscan differ on the reliability (since they use the same concepts, I don't see why there might be a huge gap). What I do know however, is that Masscan is able to collect banners and have a (limited) interaction with the service during the SYN scan without the need for an external program (Zgrab2) or even a "real" connection.

On the other hand, Zgrab2 is able to collect much more data / have more complex interactions with the services.

You may want to read this part of the doc: https://doc.ivre.rocks/en/latest/overview/faq.html#scanning-the-internet-is-slow

Senanfurkan

@Senanfurkan

Thank you for your answer, I've read all the docs. But couldn't understand the commands actually. I have got a 6000 random ip for test from several companies (uber, apple, microsoft etc) I scanned the ips by using zgrab2 http -p 80 -f ips.txt -o http.json --debug But I am unable to import the data to ivre using scan2db such as;

ivre scan2db --merge -s http.json 
0 results imported.

Pierre Lalet

@p-l-

I don't think you need to use --merge, but that should work anyway.

Do you have already inserted this file? Because you cannot import a file twice.

If you can share the file http.json, I will have a look.

Senanfurkan

@Senanfurkan

I scanned the ip's once before that is why I used --merge, I just tried with different ip addresses now. Result was the same 0 results imported I have 2 different output as .json file also couldn't imported.

Here is the file: https://send.firefox.com/download/57ae58f48a740d53/#6msomx4ndlF_yABIC6-wtw

Pierre Lalet

@p-l-

I was not clear in my question: had you, prior to this insertion, already added the very same file? ivre scan2db first hashes each file and will not insert already inserted files.

Also, which version of ivre are you using? I had no problem inserting this file with the current devel version

Senanfurkan

@Senanfurkan

Nope, I didn't added the same file. I added another file which consist of Uber's ASN ip addresses. I used ivre runscans command to scan Uber ASN. I import the scan results by using command ivre scan2db and ivre db2view which are worked without problem.

In this example;
This spesific file which I sent you consist of Paypal's ASN ip addresses. I scanned the ip addresses by using zgrab2 and tried to import. When I first time add it, output was 0 results imported

I am using IVRE Version 0.9.14

Pierre Lalet

@p-l-

OK zgrab2 support is recent. You should switch to the current dev version or wait for the next release (should be soon now)

Erwan

@Erwan14536010_twitter

Dr. Di Prodi

@robomotic

Hello guys! Just got access to the demo!

very exciting I like this project very much.

I am wondering if somebody has worked on indexing banners

basically like Shodan does

in nmap could be replicated by running this

https://nmap.org/nsedoc/scripts/banner.html

and then storing the output in ElasticSearch

Dr. Di Prodi

@robomotic

wait I am an idiot I just noticed I can click on the last octet and it displays the banner ahahah!

Pierre Lalet

@p-l-

@robomotic clicking the last octet adds a filter, which has a side consequence: the result that interests you is the only one, and it displays entirely (instead of only the summary)

but you can get the detailed view without adding a filter: just "long-click" in a result.

Dr. Di Prodi

@robomotic

great it does work nicely, maybe should add some icon to make it more intuitive

do you also have a demo API account I can use ?

https://doc.ivre.rocks/en/latest/dev/web-api.html

do I just need Basic authentication to use the Web API (e.g. same credentials as the demo) ?

Pierre Lalet

@p-l-

Yep, you can use the same credentials as in the demo. Also, you need to force a "Referer: https://ivre.rocks/demo"

(sorry I missed your message for some reason)

Max Davitt

@themaxdavitt

Hi everyone - I was wondering how I should go about labeling/tagging results as I go through them after importing. I saw there was an attempt or two at a label implementation open in the PRs, but they were from years ago. What's the status on this feature?

Pierre Lalet

@p-l-

Hi! It is not implemented. For now, we use the categories that are set at import time, or in a database shell. Hope this helps!

Max Davitt

@themaxdavitt

Gotcha, thanks! I feel like this could be a helpful feature if finished, I'd consider setting it as a goal or milestone for a future update if I were you. :) Have a nice day and stay safe!

Flyrabby

@Flyrabby

may i get more details about the scheme?

Pierre Lalet

@p-l-

What do you mean exactly?

If you have not already, maybe you could have a look at this page: https://doc.ivre.rocks/en/latest/overview/principles.html

Flyrabby

@Flyrabby

thank you for your fast reply. i mean the scheme version 14

the data format in the mongodb

and the page you providing for me is also very helpful

Pierre Lalet

@p-l-

This part is indeed not well documented. Basically, the document format in both nmap and view purposes is close to the structured output of Nmap (XML produced by -oX). You can see what changes from Nmap output to the current format by reading the code of the migration methods (they should be documented) in ivre/db/__init__.py and ivre/db/mongo.py.

Flyrabby

@Flyrabby

thank you very much!Then I want to know when ivre had supported to zeek scan

I didnt find this function last year.🙃

Pierre Lalet

@p-l-

I'm not sure what you mean. IVRE has been supporting intelligence collection with Zeek (Bro) since the beginning. Turning those results into Nmap-like results if fairly recent. No idea when exactly, you will have to git log if it really matters.

Flyrabby

@Flyrabby

It doesnt matter.I appreciate you and your work

Pierre Lalet

@p-l-

Thanks :-)

Where communities thrive

People

Repo info

Activity