Adversarial Autoencoder and Multi-Task Semi-Supervised Learning for Multi-stage Process

Andre Mendes, Julian Togelius, Leandro dos Santos Coelho

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    In selection processes, decisions follow a sequence of stages. Early stages have more applicants and general information, while later stages have fewer applicants but specific data. This is represented by a dual funnel structure, in which the sample size decreases from one stage to the other while the information increases. Training classifiers for this case is challenging. In the early stages, the information may not contain distinct patterns to learn, causing underfitting. In later stages, applicants have been filtered out and the small sample can cause overfitting. We redesign the multi-stage problem to address both cases by combining adversarial autoencoders (AAE) and multi-task semi-supervised learning (MTSSL) to train an end-to-end neural network for all stages together. The AAE learns the representation of the data and performs data imputation in missing values. The generated dataset is fed to an MTSSL mechanism that trains all stages together, encouraging related tasks to contribute to each other using a temporal regularization structure. Using real-world data, we show that our approach outperforms other state-of-the-art methods with a gain of 4x over the standard case and a 12% improvement over the second-best method.

    Original languageEnglish (US)
    Title of host publicationAdvances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings
    EditorsHady W. Lauw, Ee-Peng Lim, Raymond Chi-Wing Wong, Alexandros Ntoulas, See-Kiong Ng, Sinno Jialin Pan
    PublisherSpringer
    Pages3-16
    Number of pages14
    ISBN (Print)9783030474355
    DOIs
    StatePublished - 2020
    Event24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020 - Singapore, Singapore
    Duration: May 11 2020May 14 2020

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume12085 LNAI
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020
    Country/TerritorySingapore
    CitySingapore
    Period5/11/205/14/20

    Keywords

    • Autoencoder
    • Multi-stage
    • Multi-task

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • General Computer Science

    Fingerprint

    Dive into the research topics of 'Adversarial Autoencoder and Multi-Task Semi-Supervised Learning for Multi-stage Process'. Together they form a unique fingerprint.

    Cite this