Q:

Debugging the "Too many files open" issue

The application i am working on suddenly crashed with 
java.io.IOException: ... Too many open files

As i understand the issue it means that files are opened but not closed. 
Stacktrace of course happens after the fact and can only help understand before what event error occurred.
What would be an intelligent way to search your code base to find this issue which only seems to occur when app is under high stress load.

A:

use  lsof -p pid to check what cause leak of file references;
use ulimit -n to see the limit of opened file references of a single process;
check any IO resources in your project,are they released in time?,Note  that,File,Process,Socket(and Http connections) are all IO resources.
sometimes, too many threads will cause this problem too.

A:

I think the best way to use a tool specifically designed for the purpose, such as this one:

This little Java agent is a tool that keeps track of where/when/who opened files in your JVM. You can have the agent trace these operations to find out about the access pattern or handle leaks, and dump the list of currently open files and where/when/who opened them.
In addition, upon "too many open files" exception, this agent will dump the list, allowing you to find out where a large number of file descriptors are in use.

I seem to remember YourKit also having some facilities around this, but can't find any specific information at the moment.

A:

What OS?  If it's linux/mac, there is information under /proc that should help. On Windows, use the Process Explorer.
As far as searching the code base, perhaps look for code that catches or raises IOException - I think I/O methods that already catch/raise this have a high likelihood of needing a close() call.

