As I understand it, programs that support streaming I/O can work with files more than the size of the memory.  Is this correct?Is there a way to know which programs support streaming I/O?
How to know which programs support streaming I/O
linux;files;io
if a program supports streaming I/O it can work with files more than the size of the memory, is this correctUsually yes, but not necessarily. Imagine a program, like uniq but more sophisticated, that counts up the number of occurrences of each unique line it gets on its input. Unlike uniq it outputs a running total of the number of occurrences seen so far. Therefore it counts as streaming because it produces output incrementally is it receives input. But if every line of the input is unique or close to it, it has to remember each different line in a data structure to maintain the running counts, and it will not actually be able to handle a file that is larger than memory.Also is there a way to know which programs support streaming I/O ?Not unless it's documented or unless you can verify it yourself, no, because it depends on what the program does.You can do something close, though: you can test whether the program needs a seekable input file. Some programs accept either a pipe or a regular file (or socket or something else) as input. Others can only accept a regular file. If a program insist on a regular, seekable, file, then there's a good chance it does not do any kind of streaming. If it allows a pipe as input then there's a good chance that it does. But there are exceptions. For example, sort allows a pipe as its input but it doesn't do streaming: it only outputs anything after the end of the input.