I'm building an application with lots of components, a lot of which are third-party so I only know what I can get from their documentation.From time to time, by pure luck, I find out one of these components was pretty heavy and had an option to cache stuff.I can't stop thinking that if I weren't lucky, one of these components would end up killing the performance of my app and I wouldn't have a clue what to do. I would end up adding more hardware like an idiot.So how do you find these kind of bottlenecks? I mean I wouldn't have a nice report saying this part is slow, like it happens with the database.For example, today I found that a serializer I was using was heavy and wasn't being cached. Are there repeatable steps I could have followed to discover that the serializer was slowing things down?
How to find bottlenecks in an application?
performance;optimization;profiling
There are two paths to follow.Blindly follow hunches.Get an application profiler.Let's look at the pros & cons.Blindly follow hunchesThis is:Low capital cost (hunches are free)  Exceptionally easy to instrument (there isn't any instrumentation) Relies heavily upon your programming experience to see problems   Not terribly accurate at finding issues  Great at creating rabbit holes that consume amazing amounts of time but generally don't generate much measurable improvement.  (which assumes you're measuring things)Application ProfilingGenerally has an acquisition cost; and the really good profilers can get expensiveProvides measurements of application performance.  The better ones isolate methods thatConsume the most amount of time  Have the most amount of calls  Require you to be able to instrument your application or at least manually step through use cases that are of concern.Can flood you with too much information, so you need to understand what to look forDepending upon how large your application, you can get away with following blind hunches and relying upon wall-clock time or throwing some DateTime calls in your code to keep track of how long execution takes.  I have personally taken some applications with 24+ hour runtimes and brought them down to 15 minutes and 1 - 2 hours using this approach.  But they were small applications and I could understand all of the code involved.   On the other hand, I have used profilers a couple of times with really large applications.  You need to be able to recreate the use cases that are of most concern, and that can be a pain.  Manually stepping through long processes can be problematic.  But I have used these tools to track background start-up processes and have been able to generate significant performance improvements for those applications.To find some profilers, just search for your_language + profiler and you will likely have quite a few options to pick from.Follow up to comments:Mason Wheeler correctly points out that some good profilers can be inexpensive.  The functionality provided is most likely the biggest driver for cost.  In some ways, it's like any other software product - there is a correlation between cost and features.There can be a half-way point as ChocoDeveloper pointed out.  You can build your own profiler, but this can be a significant investment of time.  And you run the risks of the blind path.  If you just know the problem is in MyModule.Foo(), then you can be oblivious to the problems in Module2.Bar().  But if you're aware of that risk, you can make sure you profile everything.  You can create a lighter-weight profiler this way by being very specific about what's measured and logged.  Some of the better profilers provide this ability too though.Profilers can be run on production code.  It's generally not done as they do have a performance impact (as Doc Brown points out).  But if you can't recreate the performance issues anywhere else except production, then you run the profiler in production.  You have to understand the risks and potential issues it will create, but it can be done.My initial response didn't give a lot of credit to building your own profiler.  In my experience, I have found it less expensive to go with an already existing profiler than to build my own.  Profilers aren't necessarily super, super complicated, but they are another project that you're taking on and will need to make sure is working correctly before you can trust the results.  For the organizations I have worked for, building a profiler was tangential to our core competencies so it didn't make business sense for us to do so.Have a look at the Wikipedia article on Program Optimization if you haven't already.  Here are a few choice quotes cited in that article.Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you have proven that's where the bottleneck is.  Rob PikeThe First Rule of Program Optimization: Don't do it. The Second Rule of Program Optimization (for experts only!): Don't do it yet.  Michael A. JacksonWe should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified  Donald Knuth