The method of grouping data objects into a hierarchy clusters is called hierarchy clustering. It is useful for data summarization and data visualization. The hierarchy clustering can be done using agglomerative and divisive method.
The agglomerative method starts with individual objects as clusters, which are iteratively merged to form larger clusters. The divisive method initially lets all the given objects form one cluster, which they iteratively split into smaller clusters.
Hierarchical clustering methods can encounter difficulties as the methods will neither undo what was done previously, nor perform object swapping between clusters. It may lead to low-quality clusters if merge or split decisions are not well chosen.
Hierarchical clustering methods can be categorized into algorithmic methods, probabilistic methods, and Bayesian methods.
The agglomerative and divisive are algorithmic methods. An agglomerative method requires at most n iterations.
A tree structure called a dendrogram is commonly used to represent the process of hierarchical clustering. It shows how objects are grouped together in an agglomerative method or partitioned in a divisive method step-by-step.
The measurement of the distance between two clusters, where each cluster is generally a set of objects is the core need in algorithmic methods. The widely used distance measures are Minimum distance, Maximum distance, Mean distance and Average distance. They are also known as linkage measures.
When an algorithm uses the minimum distance to measure the distance between clusters, it is called a nearest-neighbor clustering algorithm.
An agglomerative hierarchical clustering algorithm that uses the minimum distance measure is also called a minimal spanning tree algorithm.
When an algorithm uses the maximum distance to measure the distance between clusters, it is sometimes called a farthest-neighbor clustering algorithm.
The minimum and maximum measures tend to be overly sensitive to outliers or noisy data. The use of mean or average distance is a compromise between the minimum and maximum distances and overcomes the outlier sensitivity problem. The average distance can also handle categoric as well as numeric data.