Sequential Latent Dirichlet Allocation: Discover underlying topic structures within a document

Lan Du*, Wray Buntine, Huidong Jinr

*Corresponding author for this work

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    34 Citations (Scopus)

    Abstract

    Understanding how topics within a document evolve over its structure is an interesting and important problem. In this paper, we address this problem by presenting a novel variant of Latent Dirichlet Allocation (LDA): Sequential LDA (SeqLDA). This variant directly considers the underlying sequential structure, i.e., a document consists of multiple segments (e.g., chapters, paragraphs), each of which is correlated to its previous and subsequent segments. In our model, a document and its segments are modelled as random mixtures of the same set of latent topics, each of which is a distribution over words; and the topic distribution of each segment depends on that of its previous segment, the one for first segment will depend on the document topic distribution. The progressive dependency is captured by using the nested two-parameter Poisson Dirichlet process (PDP). We develop an efficient collapsed Gibbs sampling algorithm to sample from the posterior of the PDP. Our experimental results on patent documents show that by taking into account the sequential structure within a document, our SeqLDA model has a higher fidelity over LDA in terms of perplexity (a standard measure of dictionary-based compressibility). The SeqLDA model also yields a nicer sequential topic structure than LDA, as we show in experiments on books such as Melville's "The Whale".

    Original languageEnglish
    Title of host publicationProceedings - 10th IEEE International Conference on Data Mining, ICDM 2010
    Pages148-157
    Number of pages10
    DOIs
    Publication statusPublished - 2010
    Event10th IEEE International Conference on Data Mining, ICDM 2010 - Sydney, NSW, Australia
    Duration: 14 Dec 201017 Dec 2010

    Publication series

    NameProceedings - IEEE International Conference on Data Mining, ICDM
    ISSN (Print)1550-4786

    Conference

    Conference10th IEEE International Conference on Data Mining, ICDM 2010
    Country/TerritoryAustralia
    CitySydney, NSW
    Period14/12/1017/12/10

    Fingerprint

    Dive into the research topics of 'Sequential Latent Dirichlet Allocation: Discover underlying topic structures within a document'. Together they form a unique fingerprint.

    Cite this