Click here to Skip to main content
15,121,414 members
Articles / Web Development
Technical Blog
Posted 17 Mar 2015

Tagged as

Stats

10.2K views
7 bookmarked

Java: A Memory Leak Caused by Dynamic Creation of log4j Loggers

Rate me:
Please Sign up or sign in to vote.
5.00/5 (3 votes)
17 Mar 2015CPOL2 min read
Java: A Memory Leak Caused by Dynamic Creation of log4j Loggers

At the company I work for, we had a situation where a highly loaded server that was handling several thousands requests per second consumed memory increasingly, and after about 30 days, it would become unusable and required a restart. By looking at our monitoring tools, we concluded it was clearly a memory leak, and we figured it must be an easy one to detect as memory exhibited an almost perfect linear grow. The first thing we did was we took a heap dump and looked at most frequent instances and to our shock over 30 GB of memory was occupied by log4j loggers!

Hunting the Memory Leak

We started to try to isolate the problem and found a possible red flag - for every client that connected, we generated a new logger containing class name and IP address of the client. It was very easy to set up an experiment to test the hypothesis. I created a minimal piece of code that created a lot of dynamically named loggers and attached a profiler to it. For this simple case, Java VisualVM, which comes with Java is more than enough.

Java
for( int i = 0; i < 100000; i++) {
    Logger.getLogger("Logger - " + i);
}

jvisualvm can be found in the bin folder of your jdk. I run the test code from IDE and made it stop after creating the loggers. After that, I opened jvisualvm, found my process at Application list and took a heap dump (can be done by right clicking on process and selecting Heap Dump). After the dump is generated, I opened ‘Classes’ tab. Here is how it looks like:

_config.yml

We can see that there is more than 100,000 instances of log4j classes, each holding strings that can really add up to size and eat heap memory. The solution was very simple, we just replaced creating a new logger with a static one. This doesn’t mean you should be conservative and use one logger per application - it is still a good idea to create loggers named by logical parts of your application of even named by class names as soon as you don’t create new loggers uncontrollably. Share your favourite tools and methods for hunting memory leaks in comments and subscribe for more interesting debugging adventures!

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Ivan Korhner
Software Developer
Yugoslavia Yugoslavia
I am a software developer from Serbia with a passion in backend programming and technologies.

Comments and Discussions

 
QuestionMemory Leak Vs Memory Bloat Pin
VC Sekhar Parepalli16-Jun-15 11:16
MemberVC Sekhar Parepalli16-Jun-15 11:16 
Sorry am a beginner and trying to understand in what way this is considered a Memory Leak (failure to release discarded memory) and could we call this as a Memory Bloat (memory gets eaten up unnecessarily) instead ?
QuestionInitial Logger Initialization Pin
rancidvess19-Mar-15 0:31
professionalrancidvess19-Mar-15 0:31 
AnswerRe: Initial Logger Initialization Pin
Ivan Korhner19-Mar-15 10:46
professionalIvan Korhner19-Mar-15 10:46 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.