Theoretical situation:One trillion foobars are stored in a text file (no fancy databases). Each foobar must have some business logic executed on it. A set of 1 trillion will not fit in memory so the data layer cannot return a big set to the business layer. Instead they are to be streamed in 1 foobar at a time, and have business logic execute on 1 foobar at a time.  The stream must be closed when finished.In order for the stream to be closed the business layer must close the stream (a data operation detail), thus violating the separation of concerns. Is it possible to incrementally process data without violating layers?
Processing a stream. Must layers be violated?
layers
Good question.Here is a C#-ish code that lets you have it both ways. The trick here is lazy evaluation (yield).// Lazy producer that will auto-close the file.// The OS and the disk do the caching for you.public static IEnumerable<FooBar> GetFoobars(string fileName, long maxNumber = 1000000000000){    using(File file = open(fileName)) // pseudocode    {        FooBar nextFooBar;        do        {            nextFoobar = new Foobar(file.ReadByte()); // or something like that.            yield return nextFoobar;        }        while(nextFooBar != null || /* reach max */); // pseudocode    }}// Consumer ... stupid example. Oh well ...public static int Consume(){    int result = (from foo in GetFoobars(foo.txt).AsParallel()                 where foo.Depth % 10 == 0).Count();    return result;}The GetFoobars(...) function know which file to open, how many thingies to read and how to close the stream. You just pull on the results as you need them. Concerns are separated. What seems to be the problem? You could probably do the whole thing as a one-liner ... maybe not, but close.P.S. I recommend going through SICP video lectures.