IASC - d-dimensional Data Characterisation via Entropy Estimation

Statistics is a branch of science that is centred on data handling in order to proffer solutions to the many problems in the real world. As realisations of real world scenarios, these data are always assumed to have different probability laws with different supports from the sets of integers and real numbers to their different truncations under different dimensional spaces. Traditionally, data, in both univariate and multivariate spheres, have been characterised by the nature of the probability law (discrete or continuous) as well as the underlying probability law, with its characteristic parameter(s). Besides these, several other forms of data characterisations have been introduced in the literature. One of such is the entropy measure of a set of data. It measures the amount or degree of disorder, randomness or uncertainty in a system. On the other hand, it can be seen as a measure of rate of information generation in a system. Although entropy is measured for different probability distributions, several estimators of it have been developed in the literature at different variable dimensions such that in situations where the distributional assumption of the dataset is not important, entropy measure can be a good method of data characterisation. Such cases arise for instance, with the advent of high dimensional and high complexity data such as in signal processing. One of the fundamental measures of entropy is the Shannon entropy (Shannon, 1948), which is called Shannon differential entropy when the dataset is continuous. Unfortunately, due to the presence of its several estimators, data analysts are most of the times at cross roads as to which of them will be most appropriate for their datasets.