ndices, which contains the subset of the data I may be interested in but keeping a logical index around is not really feasible, either in MATLAB or on disk.
Further, to the comparison of logical vs. numerical indexing (on a smaller footprint example)
x = rand(2 * 1e9, 1); %16GB of data
tic; l = x > 0.5; toc
Elapsed time is 4.771542 seconds.
tic; i = find(l); toc
Elapsed time is 29.321260 seconds.
>> tic; y=x(l); toc
Elapsed time is 49.782177 seconds.
>> tic; z=x(i); toc
Elapsed time is 51.138457 seconds.
As you can see, there is very little in it for a large set, since most of the computation is likely to be memory bound. I have to say that I’m disappointed with the implementation of the numeric subscript. It is not surprising that a logical index must grow the resulting array, since it doesn’t know in advance how many elements may be in the result. However, to see reallocations on my system monitors during the final subscript call seems unnecessary, since I know *exactly* how many elements there’ll be. Memory reallocation would actually be the reason why, for large datasets, I’d expect the numerical index to outperform the logical index.
On a similar note, here are two features I’d like to see in MATLAB:
1) Don’t construct the logical array if you don’t need to: For example,
i = find(logical expression);
b = logical expression;
i = find(b);
% b no longer used after here
There seems no reason why you have to do construct the temporary logical array first and then follow the computation by another pass over the logical array. This is slow and wastes an unnecessary amount of memory for a temporary.
2) Again, suppose I have a large dataset (200+ GB in size). I want to perform a computation on a subset of it (say 50GB). I have to make a copy to use the data in the computation, which is a problem, as I’m already riding hard against the amount of available memory. If I have a list of the indices already, I’d like to use an “indirect” array – an object that holds both indices and the original data and allows me to subscript it as a regular array. To preempt objections: a) Yes, I could write a class to do it but that’s not very fast. b) The object could be read only, so that you wouldn’t have to worry about multiple numerical indices with the same value.
Thank you very much for Method #4.
I’ve been trying hard to find a way for vectorizing relational matrix manipulation in Matlab.
Maybe there is an even simpler way that I am not aware of, but your #4 works.
I used it with
indices = sub2ind(size(A),rnotA+1,cnotA+1);
Note +1. I basically needed to manipulate the i+1,j+1 neighbors of elements satisfying a condition without relying on a for loop.
i want to calculate numbers of columns in a matrix on matlab, will some1 kindly help me,,,?
Perhaps the function size will help you out, especially with 2 inputs ( http://www.mathworks.com/access/helpdesk/help/techdoc/ref/size.html ).
hi i would like to ask something…I want to remove an especific NaN element from a matrix, I do not know how to do it. All the solutions that I’ve read explain how to remove rows where the NaN exists, but not specific elements..Could you please help me? I appreciate your help
If you have a vector, you can remove NaNs like so:
a(isnan(a)) = ;
But if you have a matrix, you aren’t guaranteed that the same number of NaNs are in each row/column so you may not end up with a rectangular array.
i have a question about removing elements from an array based on their indices, not their values. please consider the following code snippet showing 3 methods for doing such an operation:
n=100; % square nxn matrices n2=n^2; A0=rand(n,n,n); % generate matrices for i=1:n % for each matrix % method 1 of dropping a matrix A1=A0; A1(:,:,i)=; % method 2 of dropping a matrix A2=A0; A2(1:n2)=; % method 3 of dropping a matrix A3=A0(:,:,2:n); % mean computation for comparison purposes mean(A0,3); end
note that upon running profiler, method 3 takes about half the time of method 1, and about 1/3 of the time of method 2. but the mean computation takes far less time than any!
this quite surp