Ternary sparse coding

Georgios Exarchakis, Marc Henniges, Julian Eggert, Jörg Lücke

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

7 Citations (SciVal)


We study a novel sparse coding model with discrete and symmetric prior distribution. Instead of using continuous latent variables distributed according to heavy tail distributions, the latent variables of our approach are discrete. In contrast to approaches using binary latents, we use latents with three states (-1, 0, and 1) following a symmetric and zero-mean distribution. While using discrete latents, the model thus maintains important properties of standard sparse coding models and of its recent variants. To efficiently train the parameters of our probabilistic generative model, we apply a truncated variational EM approach (Expectation Truncation). The resulting learning algorithm infers all model parameters including the variance of data noise and data sparsity. In numerical experiments on artificial data, we show that the algorithm efficiently recovers the generating parameters, and we find that the applied variational approach helps in avoiding local optima. Using experiments on natural image patches, we demonstrate large-scale applicability of the approach and study the obtained Gabor-like basis functions.

Original languageEnglish
Title of host publicationLatent Variable Analysis and Signal Separation - 10th International Conference, LVA/ICA 2012, Proceedings
EditorsF. Theis, A. Chichoki, A. Yeredor, M. Zibulevsky
Place of PublicationBerlin, Germany
Number of pages9
ISBN (Electronic)978364285516
Publication statusPublished - 31 Dec 2012
Event10th International Conference on Latent Variable Analysis and Signal Separation, LVA/ICA 2012 - Tel Aviv, Israel
Duration: 12 Mar 201215 Mar 2012

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume7191 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference10th International Conference on Latent Variable Analysis and Signal Separation, LVA/ICA 2012
CityTel Aviv

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Ternary sparse coding'. Together they form a unique fingerprint.

Cite this