These are chat archives for IndySockets/Indy

16th
Nov 2017
Ludwig Behm
@lbehm
Nov 16 2017 22:21 UTC

Hy, I'm using 10.1 (Berlin) and have a MemoryLeak somewhere in TIdCustomHTTPServer. Just as an example:

class PACKAGE THTTPServer : public TIdCustomHTTPServer
{
public:
__fastcall THTTPServer(TComponent *Owner) : TIdCustomHTTPServer(Owner) {
    ServerSoftware = "MyWebServer";
}
__fastcall ~THTTPServer() {}
}
void __fastcall DoCommandGet(TIdContext *Thread, TIdHTTPRequestInfo *RequestInfo, TIdHTTPResponseInfo *ResponseInfo) {
    ResponseInfo->ResponseNo = 200;
    ResponseInfo->ResponseText = "OK";
}

Now, doing some minimal load tests increase memory usage continuously. Any hints what is missing or where to begin debugging?

Remy Lebeau
@rlebeau
Nov 16 2017 22:35 UTC
@lbehm there is no memory leak in that server code. You are misdiagnosing things. And HOW are you diagnosing the "leak"? Task Manager? That doesn't report leaks correctly, as it doesn't know how allocated memory is being used. The RTL's memory manager doesn't release freed memory back to the OS, it caches it for reuse. That is likely what you are seeing. Only the memory manager knows what constitutes an actual leak. Are you using ReportMemoryLeaksOnShutdown=true? Does it report anything? If you suspect a real leak, but no leak is being reported, your app might be fragmenting memory instead, which is a different issue
Ludwig Behm
@lbehm
Nov 16 2017 22:39 UTC
So if it's reusing freed memory I shouldn't see a permanent increase of memory usage in task manager, right?
Can I even turn ReportMemoryLeaksOnShutdown on in C++ packages/apps?
Remy Lebeau
@rlebeau
Nov 16 2017 22:52 UTC
@lbehm fragmentation would show up as increasing memory, since cached memory is not being reused effectively. I'm not saying you ARE fragmenting, just that it is a possibility. And ReportMemoryLeaksOnShutdown may or may not work in C++, it depends on the version of the memory manager used.
Ludwig Behm
@lbehm
Nov 16 2017 22:54 UTC
@rlebeau thanks for the first tips. Currently I'm thankful for every hint.
Any tips on how to detect fragmentation?
Remy Lebeau
@rlebeau
Nov 16 2017 23:05 UTC
@lbehm that is very hard. You have to really study closely how your code (re)uses memory. Fragmentation happens when gaps between consecutive memory blocks are not filled in, requiring more blocks be used. So things like allocation size and alignment make a big difference. Grouping like-sized, like-aligned items together fills memory more effectively. And that is exactly what memory managers like FastMM (Delphi's default) actually do to avoid fragmenting
Ludwig Behm
@lbehm
Nov 16 2017 23:38 UTC
@rlebeau I can't imagine that it's my code that is creating many objects filling up the memory... I can see the memory increase only when the load test runs (~100 req/s). And in my code doesn't create objects.
Are you sure the code above is "enough"? No specials like scheduler/etc necessary?
I mean, I have this behavior on different systems - and no one else does?
Remy Lebeau
@rlebeau
Nov 16 2017 23:52 UTC
@lbehm Indy servers are multi-threaded. During those 100 requests per second, how many unique TCP connections are you using? 1 connection? 100 connections? How many unique threads are running to service those connections? 1 thread? 100 threads? Are you using Indy's default thread scheduler, or the thread pool scheduler? Multi-threaded memory usage is hard to troubleshoot. But ultimately, the memory manager is serialized, and shouldn't fragment under normal conditions. So really, unless you have an actual leak report from the memory manage, you are flying blind trying to figure things out. And again, ignore what Task Manager tells you, it is not reliable info.