Gor is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data. It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.
Looking at the rate limiting options in gor, it appears that there are two choices. 1) % of captured traffic 2) max reqs per second. Would you be interested in a PR to add a third of just reqs per second. This would be similar to max reqs per second which according to https://github.com/buger/gor/wiki/Rate-limiting does
staging.server will not get more than X requests per second but will instead do staging.server will always get X requests per second. This would make testing much easier as we can specify a request rate rather than hunt around for it using the proxy of %
@Christopher-Bradshaw I guess it may require gathering some stats about existing traffic, to know current RPS, and based on that skip requests in more stable way.
So yes, i would love to see such PR, just briefly explain how do you want to implement it
I have some future plans to add "proxy" mode, but this will mean putting Gor to your traffic pipeline. Do you think it is acceptable for your purpose?
What we need now is something we hope to not keep for a long time, some king of patch due to time constraints
The loosing of packets/requests is less acceptable for us. If you asking about the "pipeline" as general it sounds like an acceptable solution.
Thank you for your answer!
The loosing of packets/requests is less acceptable for us. If you asking about the "pipeline" as general it sounds like an acceptable solution.
Thank you for your answer!
We use gor on our production servers to store requests to later be used for replay testing. These are on servers getting > 1 million requests per day. Not quite what you are asking as we are not forwarding to other servers, just to file but it has performed without errors for us
Hey, guys! This is probably a stupid question, but if I write request log using
--input-raw :80 --output-file file.log --output-file-append and then replay it to 2 separate instances using --input-file file.log --output-http , is it guaranteed that both instances will receive the exact same requests? Probably worth mentioning that the instance I'm getting the input from is under heavy load, that's why I'm writing the results to a log first.
It doesn't really matter to me whether I'll capture all of the requests. But it does matter that the other two instances I'm replaying requests to will receive the exact same ones.
@pegasd hello! Missed your message. Totally get what you mean, and if you replay from the file, both servers receive exactly same requests. Worth noticing, that it does not mean that they receive exactly same streams. So for example if in original traffic you had first user making 10 requests (in same tcp session), and second user making 2 requests (in same tcp session), while replaying GoReplay community version does not guarantee that this will be 2 TCP sessions, most likely it will be like 5 sessions with 2 requests per each. GoReplay PRO handle it, and replay them with exactly same number of TCP sessions.
Do i have to launch a server beside gor or gor can do this alone ?
buger/goreplay#532
@marcinkuzminski when you reply from file you can set replay rate in percentage, like this: --input-file "requests.gor|2000%", so it will be 20x more speed. Note that it is not concurrency, but faster replay. If you really need concurrency, I would recommend just adding as many --input-file arguments as you want.
@buger, what exactly means faster replay ? I
sorry this isn't clear for me ;(
@marcinkuzminski see this https://github.com/buger/goreplay/wiki/Saving-and-Replaying-from-file#performance-testing
So basically if you had 2 requests in your file and time period between them is 100ms, if you set 2000% replay speed (20x), GoReplay will short this time by 20x as well to 5ms, in other words will replay faster
Greetings! I'm working on creating some middleware to record http responses (particularly image files). I'm trying to follow along on the docs and the proto spec, but I've been noticing some odd behavior with the response bodies. It looks like they might be output as framed chunks?
Besides the
\r\n\r\n delimiter between headers and body. The body appears to be separated out in chunks delimited by \r\n and some extra data
Are there any more details or specs around that formatting?
@mjuarez regarding chunks you see, it is so called HTTP chunked encoding. GoReplay can transform it to one single chunk for you if you pass
—prettify-http option. Hope it helps!
Anybody has same experience？
I wonder if its because different accept encoding headers
Can you try to validate this theory?
What is
gcount for example?
@buger Is it possible to allow multiple urls in single command. https://github.com/buger/goreplay/wiki/Request-filtering . Wiki only shows single urls