1. A method, in a data processing system, for utilizing algorithm components based on categories in a question answering system, the method comprising: capturing a history of performance and correctness metrics for identifying efficiency of respective algorithms for finding answers to questions in respective question categories in a question answering system; determining sets of algorithms to use for respective question categories according to efficiency and correctness analysis; for a given input question, determining a question category of the given input question; and executing a set of algorithms corresponding to the question category of the given input question that meet an efficiency threshold to contribute to finding a correct answer for the given input question, wherein capturing the history of performance and correctness metrics comprises: receiving training data comprising a plurality of questions, associated context, and corresponding question categories; executing a plurality of algorithms to find at least one answer for each of the plurality of questions; recording outputs of the plurality of algorithms; determining an amount of influence the outputs of the plurality of algorithms contribute to the at least one answer of each of the plurality of questions.