The use of immense memories for database systems: A pragmatic approach
Abstract We examine the cost and performance advantages of immense main memories and of the use of extended storage units for a wide range of database applications. Most database systems are required to guarantee a close to absolute degree of data integrity. This requirement is a serious limitation on the usage of main memories as the primary repository of data because of the vulnerability of the content to errors, particularly, software related errors. Current research often neglects this point and focuses instead on the technological issues of the silicon memories content volatility. Transaction processing performance is mostly evaluated on the basis of throughput (i.e., transaction per second) provided that the average response time does not exceed 1 s. Most update intensive applications can be classified as transaction processing and therefore driven by throughput considerations. We will show that large memories have little effect on throughput. Given this point of view, there is very little (if any) performance advantage in using main memories for transaction processing. Consequently, the potential of the immense memory system is for improving the performance of those (non-updating) applications that execute complex queries. The coexistence of query applications with the updating transactions is a realistic view of most business systems as well as many engineering and scientific applications. By taking these facts into consideration, a new vision of in-memory databases is created. In this approach, hardware architecture changes are shown to be less important than the use of proper algorithms and software systems.