Q:

How can I improve performance of string processing with less memory?

I'm implementing this in Java.
Symbol file     Store data file

1\item1         10\storename1
10\item20       15\storename6
11\item6        15\storename9
15\item14       1\storename250
5\item5         1\storename15

The user will search store names using wildcards like storename?
My job is to search the store names and produce a full string using symbol data. For example:
item20-storename1
item14-storename6
item14-storename9  
My approach is:

reading the store data file line by line
if any line contains matching search string (like storename?), I will push that line to an intermediate store result file
I will also copy the itemno of a matching storename into an arraylist (like 10,15)
when this arraylist size%100==0 then I will remove duplicate item no's using hashset,  reducing arraylist size significantly
when arraylist size >1000

sort that list using Collections.sort(itemno_arraylist)
open symbol file & start reading line by line
for each line Collections.binarySearch(itemno_arraylist,itmeno)
if matching then push result to an intermediate symbol result file

continue with step1 until EOF of store data file

...
After all of this I would combine two result files (symbol result file & store result file) to present actual strings list.
This approach is working but it is consuming more CPU time and main memory.
I want to know a better solution with reduced CPU time (currently 2 min) & memory (currently 80MB). There are many collection classes available in Java. Which one would give a more efficient solution for this kind of huge string processing problem?
If you have any thoughts on this kind of string processing problems that too in Java would be great and helpful.
Note: Both files would be nearly a million lines long.

A:

Replace the two flat files with an embedded database (there's plenty of them, I used SQLite and Db4O in the past): problem solved.

