http://www.tu-ilmenau.de

Logo TU Ilmenau


Ansprechpartner

Prof. Dr. Kai-Uwe Sattler

Head

Telefon +49 3677 69 4577

E-Mail senden

INHALTE

Transactional Stream Processing on Non-Volatile Memory

Project Description

Transactional Stream Processing is a project of the Databases and Information Systems Group at Technische Universität Ilmenau. The project is funded by the German Research Foundation (DFG) and belongs to the Priority Program on Scalable Data Management for Future Hardware (SPP 2037).

Emerging application domains such as IoT, healthcare or various types of monitoring systems gave rise to the wide-ranging utilization of the stream processing model. More and more systems additionally require to work with shared states (possibly persistent), such as historical data or metadata, besides the continuous streams of data. Since multiple instances (streams or ad-hoc queries) are simultaneously accessing these states, which can be represented as sequences of inserts or updates, certain transactional characteristics are required. This issue is addressed by the paradigm of Transactional Stream Processing to provide guarantees including ACID properties for both streaming and traditional transactions.

Along with this research, the group examines various memory and storage technologies to efficiently provide the required guarantees. In particular we consider non-volatile memory (NVRAM), a byte-addressable and persistent storage technology with latencies close to DRAM, as a perfect candidate to realize transactional stream processing. Hence, different data structures optimally suited for transactional requirements and NVRAM characteristics are researched and developed.

Furthermore, an additional topic about manycore processors has arisen from modern hardware advancement. In comparison to regular multicore CPUs, a manycore CPU utilizes a much higher amount of cores and threads for increased parallelism without offloading penalties like GPUs or Co-processors and without network delays in multicore CPU clustered systems. High bandwidth memory (HBM) therefore becomes more and more interesting to overcome the memory bandwidth bottleneck, which is a common problem in manycore GPU and CPU architectures caused by massive parallel memory accesses. This leads to new challenges and opportunities for query optimization and operator design that we want to address in this project.

 

Team

  • Professor Dr.-Ing. Kai-Uwe Sattler
    Technische Universität Ilmenau
    Institut für Praktische Informatik und Medieninformatik
    Fachgebiet Datenbanken und Informationssysteme
PhD students
  • M.Sc. Philipp Götze
    Technische Universität Ilmenau
    Institut für Praktische Informatik und Medieninformatik
    Fachgebiet Datenbanken und Informationssysteme
  • M.Sc. Constantin Pohl
    Technische Universität Ilmenau
    Institut für Praktische Informatik und Medieninformatik
    Fachgebiet Datenbanken und Informationssysteme

 

Funding-Notice

This project is funded by the DFG under grant no: SA782-28

Publications

2019

  • Philipp Goetze, Kai-Uwe Sattler
    Snapshot Isolation for Transactional Stream Processing
    Proceedings of the 22th International Conference on Extending Database Technology, EDBT 2019, March, Pages 650--653, 2019, Publisher OpenProceedings.org, Lisbon, Portugal

  • Philipp Goetze, Constantin Pohl, Kai-Uwe Sattler
    Query Planning for Transactional Stream Processing on Heterogeneous Hardware: Opportunities and Limitations
    Datenbanksysteme für Business, Technologie und Web (BTW 2019), 18. Fachtagung des GI-Fachbereichs Datenbanken und Informationssysteme (DBIS), Workshopband, March, Pages 71--80, LNI, 2019, Publisher GI, Rostock, Germany

2018

  • Constantin Pohl, Kai-Uwe Sattler
    Joins in a Heterogeneous Memory Hierarchy: Exploiting High-Bandwidth Memory
    DaMoN Workshop, 2018, Houston, Texas, USA

  • Philipp Goetze, Stephan Baumann, Kai-Uwe Sattler
    An NVM-aware Storage Layout for Analytical Workloads
    ICDE HardBD & Active Workshop, 2018, Paris, France

2017 

  • Constantin Pohl, Philipp Goetze, Kai-Uwe Sattler
    A Cost Model for Data Stream Processing on Modern Hardware
    ADMS Workshop, September, 2017, Munich, Germany

  • Constantin Pohl
    A Hardware-Oblivious Optimizer for Data Stream Processing
    VLDB PhD Workshop, August, 2017, Munich, Germany

  • Constantin Pohl
    Exploiting Manycore Architectures for Parallel Data Stream Processing
    Proceedings of the 29th GI-Workshop Grundlagen von Datenbanken (GvD 2017), June, Pages 66-71, 2017, Blankenburg (Harz), Germany