These are chat archives for request/request

Aug 2017
Felipe Sabino
Aug 26 2017 00:18
@tcurdt I suppose it is because content-length is the total number of octets, not characters.
Octets consist of eight bits and your body is a string, which javascript encoded using UTF-16, but you are probably receiving the data using UTF-8, a multi-byte encoding. In the end it is very possible that there is a mismatch of bytes needed to represent the string and the number of characters the string actually consist of.
Here are some references that would explain it much better:
Torsten Curdt
Aug 26 2017 01:57
@felipesabino this could indeed be true if we are talking about some kind of string - but it's an image and the buffer should just be just bytes.
Felipe Sabino
Aug 26 2017 12:30

@tcurdt Oh, I looked deeper and it is a weird behavior but it at least documented…

Here is the README part that is important:

encoding - encoding to be used on setEncoding of response data. If null, the body is returned as a Buffer. Anything else (including the default value of undefined) will be passed as the encoding parameter to toString() (meaning this is effectively utf8 by default). (Note: if you expect binary data, you should set encoding: null.)

So what I said earlier about strings still holds up and the problem is that request encoding is comparing strictly to null here and as your params are empty, encoding is actually undefined, so request is handling your image not as a Buffer, but as an string by doing response.body.toString(self.encoding)…

So if you just send null for you encoding

request(url, {encoding: null}, function...

you will end up with a Buffer instead of a string for you body, and the length will be correct

Check this example with no encoding provided where body.length is different from content-length header:
And this one where I explicitly set encoding: null and the values are the same:

(had to use another image url for the example as yours was returning 403 to me on runkit :S)

Torsten Curdt
Aug 26 2017 18:53
@felipesabino thanks for clearing this up.