I have 1 million rows with 20 attributes to do hierarchical clustering. When I want to build a distance matrix on this data by dist() in R, it says that it needs 5 TB memory. I have these approaches:Reduce the number of rows by samplingChange the method of clustering?Now, do you suggest another approach?And I have an idea, I think if I reduce the accuracy of the values and then doing group by, then I can remove duplicated rows and have a new column with the count of duplicates for each row. Is there any R package that can do hierarchical clustering with these data?group by: count number of duplicated rows and add a column that say how many times this row was duplicated in source.
Clustering big data by reducing data accuracy?
r;clustering;bigdata