Independent Component Analysis - Binary Independent Component Analysis

Binary Independent Component Analysis

A special variant of ICA is Binary ICA in which both signal sources and monitors are in binary form and observations from monitors are disjunctive mixtures of binary independent sources. The problem was shown to have applications in many domains including medical diagnosis, multi-cluster assignment, network tomography and internet resource management.

Let be the set of binary variables from monitors and be the set of binary variables from sources. Source-monitor connections are represented by the (unknown) mixing matrix, where indicates that signal from the i-th source can be observed by the j-th monitor. The system works as follows: at any time, if a source is active and it is connected to the monitor then the monitor will observe some activity . Formally we have:

where is Boolean AND and is Boolean OR. Note that noise is not explicitly modeled, rather, can be treated as independent sources.

The above problem can be heuristically solved by assuming variables are continuous and running FastICA on binary observation data to get the mixing matrix (real values), then apply round number techniques on to obtain the binary values. This approach has been shown to produce a highly inaccurate result.

Another method is to use dynamic programming: recursively breaking the observation matrix into its sub-matrices and run the inference algorithm on these sub-matrices. The key observation which leads to this algorithm is the sub-matrix of where corresponds to the unbiased observation matrix of hidden components that do not have connection to the -th monitor. Experimental results from show that this approach is accurate under moderate noise levels.