Q:

Best practice to expand a list (efficiency) in python

I'm working with large data sets. I'm trying to use the NumPy library where I can or python features to process the data sets in an efficient way (e.g. LC).
First I find the relevant indexes:
dt_temp_idx = np.where(dt_diff > dt_temp_th)

Then I want to create a mask containing for each index a sequence starting from the index to a stop value, I tried:
mask_dt_temp = [np.arange(idx, idx+dt_temp_step) for idx in dt_temp_idx]

and:
  mask_dt_temp = [idxs for idx in dt_temp_idx for idxs in np.arange(idx, idx+dt_temp_step)]

but it gives me the exception:
The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Example input:
indexes = [0, 100, 1000]

Example output with stop values after 10 integers from each indexes:
list = [0, 1, ..., 10, 100, 101, ..., 110, 1000, 1001, ..., 1010]

1) How can I solve it?
2) Is it the best practice to do it?

A:

Using masks (boolean arrays) are efficient being memory-efficient and performant too. We will make use of SciPy's binary-dilation to extend the thresholded mask.
Here's a step-by-step setup and solution run-
In [42]: # Random data setup
    ...: np.random.seed(0)
    ...: dt_diff = np.random.rand(20)
    ...: dt_temp_th = 0.9

In [43]: # Get mask of threshold crossings
    ...: mask = dt_diff > dt_temp_th

In [44]: mask
Out[44]: 
array([False, False, False, False, False, False, False, False,  True,
       False, False, False, False,  True, False, False, False, False,
       False, False])

In [45]: W = 3 # window size for extension (edit it according to your use-case)

In [46]: from scipy.ndimage.morphology import binary_dilation

In [47]: extm = binary_dilation(mask, np.ones(W, dtype=bool), origin=-(W//2))

In [48]: mask
Out[48]: 
array([False, False, False, False, False, False, False, False,  True,
       False, False, False, False,  True, False, False, False, False,
       False, False])

In [49]: extm
Out[49]: 
array([False, False, False, False, False, False, False, False,  True,
        True,  True, False, False,  True,  True,  True, False, False,
       False, False])

Compare mask against extm to see how the extension takes place.
As, we can see the thresholded mask is extended by window-size W on the right side, as is the expected output mask extm. This can be use to mask out those in the input array : dt_diff[~extm] to simulate the deleting/dropping of the elements from the input following boolean-indexing or inversely dt_diff[extm] to simulate selecting those.
Alternatives with NumPy based functions
Alternative #1
extm = np.convolve(mask, np.ones(W, dtype=int))[:len(dt_diff)]>0

Alternative #2
idx = np.flatnonzero(mask)
ext_idx = (idx[:,None]+ np.arange(W)).ravel()

ext_mask = np.ones(len(dt_diff), dtype=bool)
ext_mask[ext_idx[ext_idx<len(dt_diff)]] = False
 
# Get filtered o/p
out = dt_diff[ext_mask]

