WishI want to run a command repeatedly in response to lines piped to it:firehose | expensive-commandHowever, I'm getting a lot of lines, and the command is resource-intensive.  I want the input for the command to be filtered, such that it only runs at most once every x seconds:firehose | interval 1 second | expensive-commandThe interval command should not just be a filter.  It should send the most recently received line at the end of its cooldown period if it has one, rather than just block everything arriving during the cooldown.How can I do this?Attemptepoch () { date +%s --date=$* }interval () {    INTERVAL=$*    LAST_RUN_AT=0    WHEN_TO_RUN=0    while read LINE; do        if (( $(epoch now) >= $WHEN_TO_RUN )) then            echo $LINE            WHEN_TO_RUN=$(epoch now + $INTERVAL)        fi    done}alias firehose='(print 1\n2\n3 ; sleep 2 ; print 4\n)'alias expensive-command='cat'firehose | interval 1 second | expensive-commandThat mostly works, but has the problem that it can't delay passing lines on until laterit can only decide to pass them on immediately, or drop them.What happens:14The throttle receives the 1, and passes it on, then goes on cooldown.  The 1 and 3 arrive during the cooldown, so they are discarded completely.  The cooldown finishes before 4 arrives, so it is passed on.What I wished would happen:134After receiving the 1, the throttle should go on cooldown for 1 second.  Then it should receive the 2, and file it for later because it's still on cooldown.  Then it receives the 3, which replaces the 2 filed for later.  The throttle then comes off cooldown, at which point it should immediately send 3.  Finally, the 4 arrives as that round is back off cooldown, so it is sent immediately.If zsh had closures, I'd launch a subshell that sleeps for $INTERVAL, then echoes the last received LINE, but alas, zsh has no closures.
How do I throttle how frequently lines of output are passed on?
shell script;zsh
I did it!Here's my interval script (also on github):#!/usr/bin/env zsh# Lets a line pass only once every $1 seconds.  If multiple lines arrive during# the cooldown interval, only the latest is passed on when the cooldown ends.INTERVAL=$1CHILD_PID=BUFFER=$(mktemp)CAN_PRINT_IMMEDIATELY=1CAN_START_SUBPROCESS=1# Reset state when child process returnschild-return () {    CAN_START_SUBPROCESS=1    CAN_PRINT_IMMEDIATELY=1}trap child-return CHLD# Clean up when quittingcleanup () {    kill -TERM $CHILD_PID &> /dev/null    rm $BUFFER    exit}trap cleanup TERM INT QUITwhile read LINE; do    # If we're just starting, just print immediately    if [[ -n $CAN_PRINT_IMMEDIATELY ]]; then        echo $LINE        CAN_PRINT_IMMEDIATELY=    else        # Otherwise, store the line for later        echo $LINE > $BUFFER        # And spawn a subprocess to handle it one interval later, unless one is        # already running.  With the SIGCHLD trap, the state variables will        # reset when it exits.        if [[ -n $CAN_START_SUBPROCESS ]]; then            CAN_START_SUBPROCESS=            (                sleep $INTERVAL                tail -n1 $BUFFER            ) &            CHILD_PID=$!        fi    fidone# Once we exhaust stdin, wait for the last child process to finish, if any.if [[ -n $CHILD_PID ]]; then    wait $CHILD_PID &> /dev/null    cleanupfiI observed that the loop reading lines cannot always be in charge of printing them, because the program sometimes needs to print lines asynchronously (when none are being received, sometimes even long after stdin has ended).  Hence the child process.Here's it working, with the input also tee >(sed)'d aside to observe the timing:That matches my earlier diagram: