What is Directed Information?

Directed Information is an information-theoretic quantity defined for a pair of jointly distributed sequences, which is often a natural measure of the extent to which one sequence is relevant for causal inference on the other. It first appeared in the context of feedback communications, and was subsequently found useful in identifying and measuring causal relevance in neurological, biological and financial data. The well-known measure --Granger Causality-- is one special case, as it is the manifestation of directed information under certain (Gaussian and linear) model assumptions.

What Can Our Software Do?

Our software is a MATLAB package that can calculate the directed information and mutual information between any two input sequences. It uses the universal sequential probability assignment induced by Context-Tree Weighting Method, and has desirable convergence properties. For details about how it works, please take a look at our paper "Universal estimation of directed information", IEEE Transactions on Information Theory 59, no. 10 (2013): 6220-6242. For examples of this software, please read Section V of the paper.