Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Alex Walker
    @amlwwalker
    My project is open source, you can see my application here: https://github.com/DecodedCo/Mustard/blob/master/app/controllers/ProxyControllers.go
    on line 86 I am trying to allow connections that are SSL, that I havent been able to intercept (Im not installing a certificate on the client), through to the requested site

    So when someone loads a website, the page may be loaded over HTTPS and there is nothing I can do about it, but they might happen to load an HTTP page that I can inject into, I would like to allow anything through over HTTPS and only inject code into a page that is HTTP (or demonstrate how a picture loaded over HTTP is vulnerable etc etc. But otherwise the persons experience is exactly how it would be if there was no attacker (smooth as possible for the client)

    My app that I linked to above is the beginnings of a simple web based gui for controlling a MITM proxy

    also is there a way to know when the current page is the / page so that if I inject into it, I dont inject into all HTML files, just that one
    Alexandre Bourget
    @abourget
    ok.. so on HandleConnect calls.. you might want to check the port the user is requesting..
    and if the port is 443.. then you'd know they probably want to connect via HTTPS.. then let it go through.
    but if it's port 80 or some other port.. you then return goproxy.MITM .. and then handle the query in HandleRequestFunc...
    FORWARD will not let you inspect the content.. it will merely close its eyes..
    so if you MITM an HTTP connection.. there's no SSL certs involved..
    it's possible some people use CONNECT even to go through port 80 ...
    Alexandre Bourget
    @abourget
    @amlwwalker ^^
    regarding the /.. see in my code up here ^^ .. there ctx.Req.URL.Path and such.. you can check the Host, the Path and all.. it's a plain Go http.Request
    once in the HandleRequestFunc section though.. you won't have the path in the HandleConnect callback.. as the Request there will be the CONNECT call..
    Alex Walker
    @amlwwalker

    @abourget so I am doing a goproxy.FORWARD:

     proxy.HandleConnectFunc( func(ctx *goproxy.ProxyCtx) goproxy.Next {
                // Potentially best to REJECT so that MITM is not detected. detection of MITM could cause suspicion
                if ctx.SNIHost() != "" {
                    //log.Printintln(" *** HTTPS Connection: ", ctx.SNIHost())
                    //return goproxy.MITM // This is failing...
                    return goproxy.NEXT
                }
                return goproxy.FORWARD
            })

    But it feels like going inside this function is causing the SSL Error. Or should I just ALWAYS return a FORWARD?

    Nick Ma
    @nma
    Hi, just a quick question
    I'm trying to setup a transparent proxy in front of a docker instance
    for network / call / request metrics
    I set up
    iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to 8888
    but I don't see any requests being proxied to goproxy, though for squid this is the accepted solution https://hub.docker.com/r/jpetazzo/squid-in-a-can/
    is goproxy the right tool for the job?
    shawokou123
    @shawokou123
    hi
    Elazar Leibovich
    @elazarl
    @nma I believe that it is possible to do with goproxy
    @nma search for transparent proxy in the goproxy-dev mailinglist
    Alexandre Bourget
    @abourget
    @elazarl so, what do we do with my fork ?
    Elazar Leibovich
    @elazarl
    @abourget I'll start the design ASAP
    I really hope that we'll have some shared doc for you guys to review by next Tuesday
    Almog Baku
    @AlmogBaku
    hi @all
    Almog Baku
    @AlmogBaku

    just wished to share with you with my stress test results compared to squid :smile: :
    Squid proxy:

    ubuntu@client:~$ ab -c 700 -n 40000 -X squid.proxy:3128 http://tapuz.co.il/
    This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
    Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
    Licensed to The Apache Software Foundation, http://www.apache.org/
    Benchmarking tapuz.co.il [through squid.proxy:3128] (be patient)
    Completed 4000 requests
    Completed 8000 requests
    Completed 12000 requests
    Completed 16000 requests
    Completed 20000 requests
    Completed 24000 requests
    Completed 28000 requests
    Completed 32000 requests
    Completed 36000 requests
    Completed 40000 requests
    Finished 40000 requests
    
    Server Software:        
    Server Hostname:        tapuz.co.il
    Server Port:            80
    Document Path:          /
    Document Length:        233 bytes
    Concurrency Level:      700
    Time taken for tests:   20.550 seconds
    Complete requests:      40000
    Failed requests:        0
    Non-2xx responses:      40000
    Total transferred:      26200000 bytes
    HTML transferred:       9320000 bytes
    Requests per second:    1946.48 [#/sec] (mean)
    Time per request:       359.623 [ms] (mean)
    Time per request:       0.514 [ms] (mean, across all concurrent requests)
    Transfer rate:          1245.07 [Kbytes/sec] received
    Connection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0   45  19.7     45     122
    Processing:   131  312  64.1    302    1058
    Waiting:      131  312  64.0    301    1058
    Total:        147  357  68.3    348    1089
    Percentage of the requests served within a certain time (ms)
      50%    348
      66%    372
      75%    390
      80%    402
      90%    441
      95%    484
      98%    534
      99%    571
     100%   1089 (longest request)

    Goproxy:

    ubuntu@client:~$ ab -c 700 -n 40000 -X go.proxy:8080 http://tapuz.co.il/
    This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
    Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
    Licensed to The Apache Software Foundation, http://www.apache.org/
    Benchmarking tapuz.co.il [through go.proxy:8080] (be patient)
    Completed 4000 requests
    Completed 8000 requests
    Completed 12000 requests
    Completed 16000 requests
    Completed 20000 requests
    Completed 24000 requests
    Completed 28000 requests
    Completed 32000 requests
    Completed 36000 requests
    Completed 40000 requests
    Finished 40000 requests
    
    Server Software:        
    Server Hostname:        tapuz.co.il
    Server Port:            80
    Document Path:          /
    Document Length:        55 bytes
    Concurrency Level:      700
    Time taken for tests:   9.727 seconds
    Complete requests:      40000
    Failed requests:        34668
       (Connect: 0, Receive: 0, Length: 34668, Exceptions: 0)
    Non-2xx responses:      40000
    Total transferred:      16361249 bytes
    HTML transferred:       6819754 bytes
    Requests per second:    4112.45 [#/sec] (mean)
    Time per request:       170.215 [ms] (mean)
    Time per request:       0.243 [ms] (mean, across all concurrent requests)
    Transfer rate:          1642.69 [Kbytes/sec] received
    Connection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0   22  78.5     15    1037
    Processing:    10  145 106.1    163     665
    Waiting:       10  145 106.2    163     665
    Total:         17  167 132.4    180    1557
    Percentage of the requests served within a certain time (ms)
      50%    180
      66%    214
      75%    238
      80%    253
      90%    294
      95%    338
      98%    411
      99%    475
     100%   1557 (longest request)
    tested on the same ec2 flavor
    cc: @elazarl
    Elazar Leibovich
    @elazarl

    Thanks @AlmogBaku , Thanks. I'm glad the longest request is only 50% longer than squid.

    I'd consider using https://github.com/giltene/wrk2 for a better latency analysis by Gil Tenne

    I'll try to mention it in the wiki
    Almog Baku
    @AlmogBaku
    @elazarl on the previous test it was faster... BUT unfortunately it seems like my test was wrong

    the goproxy had lots of missed requests.. im seeing from the console this:

    2016/03/09 12:31:51 [086] WARN: Error copying to client: read tcp 172.31.25.102:8080->198.27.114.146:34600: read tcp 172.31.25.102:49891->204.79.197.200:80: read: connection reset by peer
    2016/03/09 12:31:52 [085] WARN: Error copying to client: read tcp 172.31.25.102:8080->198.27.114.146:34594: read tcp 172.31.25.102:49890->204.79.197.200:80: read: connection reset by peer

    and

    2016/03/09 12:32:29 [118] WARN: Error copying to client: read tcp 172.31.25.102:8080->158.69.118.48:50948: write tcp 172.31.25.102:8080->158.69.118.48:50948: use of closed network connection
    2016/03/09 12:32:29 [117] WARN: Error copying to client: read tcp 172.31.25.102:8080->158.69.118.48:50646: write tcp 172.31.25.102:8080->158.69.118.48:50646: use of closed network connection
    and
    2016/03/09 13:12:01 http: Accept error: accept tcp [::]:8080: accept4: too many open files; retrying in 5ms
    2016/03/09 13:12:01 http: Accept error: accept tcp [::]:8080: accept4: too many open files; retrying in 5ms
    2016/03/09 13:12:01 http: Accept error: accept tcp [::]:8080: accept4: too many open files; retrying in 10ms
    2016/03/09 13:12:02 http: Accept error: accept tcp [::]:8080: accept4: too many open files; retrying in 5ms
    2016/03/09 13:12:02 http: Accept error: accept tcp [::]:8080: accept4: too many open files; retrying in 5ms

    I executed the goproxy from root, and exectue ulimit -n 10240 before :|

    open files while the stress test is around 1024:

    root@ip-172-31-25-102:/home/ubuntu# lsof -p 3946|wc -l
    1024

    open files after is 13:
    root@ip-172-31-25-102:/home/ubuntu# lsof -p 3946|wc -l 13

    limit:

    root@ip-172-31-25-102:/home/ubuntu# ulimit -n
    10240
    Almog Baku
    @AlmogBaku
    ahhhh... i need to run the ulimt from the same tty of the goproxy!
    now it worked.. and the results are still better than squid.. but i still had few errors of the tcp write...
    untu@ip-172-31-25-103:~$ ab -c 700 -n 40000 -X go.proxy:8080 http://tapuz.co.il/
    This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
    Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
    Licensed to The Apache Software Foundation, http://www.apache.org/
    
    Benchmarking tapuz.co.il [through go.proxy:8080] (be patient)
    Completed 4000 requests
    Completed 8000 requests
    Completed 12000 requests
    Completed 16000 requests
    Completed 20000 requests
    Completed 24000 requests
    Completed 28000 requests
    Completed 32000 requests
    Completed 36000 requests
    Completed 40000 requests
    Finished 40000 requests
    
    
    Server Software:        
    Server Hostname:        tapuz.co.il
    Server Port:            80
    
    Document Path:          /
    Document Length:        233 bytes
    
    Concurrency Level:      700
    Time taken for tests:   19.765 seconds
    Complete requests:      40000
    Failed requests:        0
    Non-2xx responses:      40000
    Total transferred:      20600000 bytes
    HTML transferred:       9320000 bytes
    Requests per second:    2023.76 [#/sec] (mean)
    Time per request:       345.892 [ms] (mean)
    Time per request:       0.494 [ms] (mean, across all concurrent requests)
    Transfer rate:          1017.81 [Kbytes/sec] received
    
    Connection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0   16  15.1     12      70
    Processing:   107  325 124.5    299    1100
    Waiting:      107  325 124.5    299    1100
    Total:        107  341 128.0    315    1117
    
    Percentage of the requests served within a certain time (ms)
      50%    315
      66%    386
      75%    429
      80%    457
      90%    520
      95%    574
      98%    636
      99%    681
     100%   1117 (longest request)
    only those three errors
    root@ip-172-31-25-102:/home/ubuntu# ./goproxy-basic 
    2016/03/09 13:28:29 [180] WARN: Error copying to client: read tcp 172.31.25.102:8080->188.68.224.51:32895: write tcp 172.31.25.102:8080->188.68.224.51:32895: use of closed network connection
    2016/03/09 13:29:34 [073] WARN: Error copying to client: read tcp 172.31.25.102:48632->212.74.50.1:443: write tcp 172.31.25.102:48632->212.74.50.1:443: use of closed network connection
    2016/03/09 13:31:08 [081] WARN: Error copying to client: read tcp 172.31.25.102:8080->188.68.224.51:43394: write tcp 172.31.25.102:8080->188.68.224.51:43394: use of closed network connection
    Almog Baku
    @AlmogBaku
    @elazarl wrk2 doesn't support proxy out of the box...
    Almog Baku
    @AlmogBaku
    @elazarl do you have any idea regarding: elazarl/goproxy#149 ?
    Almog Baku
    @AlmogBaku
    see golang/go#15015
    Alex Scotton
    @Illizian
    Hi, please excuse my ignorance. This is my first foray into Golang. I am trying to write a custom ReqCondition func. I unfortunately lack the knowledge to understand why it doesn't work.
    Here is the prototype func:
    func isBlacklisted() goproxy.ReqCondition {
      return goproxy.ReqConditionFunc(func(req *http.Request, ctx *goproxy.ProxyCtx) bool {
        return true;
      })
    }
    and I am including thusly:
    proxy.OnRequest(isBlacklisted()).HandleConnect(goproxy.AlwaysReject)
    the isBlacklisted func is being called but the ReqConditionFunc is not, I have also tried that without the wrapper
    Alex Scotton
    @Illizian
    scrap that! the issue this whole time was my use of goproxy.AlwaysReject , Pretty sure I'm doing it wrong :P as I'm already useing HandleConnect with an AlwaysMitM function, can you only have one HandleConnect statement?
    ZiYuan
    @Bpazy
    Hi, I have a problem:
    func handler(r *http.Request, ctx *goproxy.ProxyCtx) (*http.Request, *http.Response) {
        temp := make([]byte, 1024)
        r.Body.Read(temp) // How to seek to start?
        return r, nil
    }