Zero-shot cost models for parallel stream processing. - In: Proceedings of the sixth International Workshop on Exploiting Artificial Intelligence Techniques for Data Management, (2023), 5, insges. 5 S.
This paper addresses the challenge of predicting the level of parallelism in distributed stream processing (DSP) systems, which are essential to deal with different high workload requirements of various industries such as e-commerce, online gaming, etc., where DSP systems are extensively used. Existing DSP systems rely on either manual tuning of parallelism degree or workload-driven learned models for tuning parallelism, which is either not efficient or can lead to costly operator migrations and downtime when there are workload drifts. Thus, we argue for a learned model that can autonomously decide on the right parallelism degree while generalizing across workloads and meeting the current demands of DSP applications. We propose a novel approach that leverages zero-shot cost models to predict parallelism degree while generalizing across unseen streaming workloads out-of-the-box. To reduce training effort, we propose a rule-based strategy that selects parallelism degree and meaningful transferable features related to query workload and hardware that influences the parallelism decisions. We demonstrate the effectiveness of our strategy by evaluating it with different amount of training queries and show that it achieves lower costs for parallel continuous query processing.
https://doi.org/10.1145/3593078.3593934
Realizing LTI models by identifying characteristic parameters using least squares optimization*. - In: 2023 European Control Conference (ECC), (2023), S. 1-6
This paper considers the realization of discrete-time linear time-invariant dynamical systems using input-output data. Starting from a generalized state-space representation that accounts for static offsets, a state-independent system representation is derived using the Cayley-Hamilton theorem and characteristic parameters are introduced to describe the system dynamics in an alternative way. Given input-output data, we present two formulations to address model deviations and to identify characteristic parameters by minimizing considered error terms in a least squares sense. The applicability of the proposed subspace identification method is demonstrated with physical data of the identification database DaISy.
https://doi.org/10.23919/ECC57647.2023.10178224
Distributed observer-based time-varying formation control under switching topologies. - In: 2023 European Control Conference (ECC), (2023), S. 1-6
This paper proposes the distributed observer-based control approach to achieve time-varying formation of second-order autonomous unmanned systems (AUSs) under switching topologies. It is assumed that each AUS has access only to the positions of its neighbouring AUS agents. An observer is then designed to estimate the velocity of neighbouring AUS agents. Furthermore, a distributed control for the AUSs is designed using information of position and estimated velocity. A common Lyapunov function is employed in order to establish sufficient conditions for the stability of closed-loop systems. In order to address the effect of switching topologies, a dwell-time condition is has been considered. Moreover, the observer-based time-varying formation controller satisfies the separation principle. Finally, simulation results are presented to illustrate the effectiveness of the proposed approach.
https://doi.org/10.23919/ECC57647.2023.10178289
Competence oriented study in engineering education - examples from the practicing programme. - In: Engineering for a changing world, (2023), 4.2.068, S. 1-20
The interdisciplinary and agile processing of projects in teams increasingly characterizes the engineer’s work. Problem-solving skills, creativity, entrepreneurship, and initiative, as well as the ability to engage in dialog and conflict resolution, are relevant competencies for this. All engineering students at TU Ilmenau can work on complex interdisciplinary projects in teams (practicING projects) right from the start of their studies. Participants in these practicING projects can also experience significant steps, aspects, and system engineering methods for demand-oriented products. The paper describes the motivation, the learning goals and methodology of practicING projects from the perspective of the supervising teachers and the participating students. Two examples illustrate the potential of the practicING concept: the projects "Wind turbine model with digital twin" and "CrossLab/ experimental ball drop test environment".
https://doi.org/10.22032/dbt.58899
Optimal design of a photovaltaic station using Markov and energy price modelling. - In: Engineering for a changing world, (2023), 4.2.083, S. 1-14
This paper addresses the optimization of photovoltaic (PV) systems to increase their efficiency. The study introduces a new pricing model that considers the current price of PV inverters. In addition, Markov modeling is used in a new optimization framework to determine the optimal configuration, considering the number of PV modules and inverters, operational constraints, and failure events of PV inverters up to 100 kW. A case study with six real PV inverters confirms the effectiveness of the proposed framework. It calculates the average daily hours of rated power generation considering geographic location, temperature, and solar irradiance using real data from a real PV system. The study identifies both local and global optimal solutions for PV inverters (15 kW to 100 kW), while minimizing the effective levelized cost of energy. The results of the study have important implications for future assessments of PV module failures and repairs.
https://doi.org/10.22032/dbt.58910
Robust adaptive tracking control for highly dynamic nanoprecision motion systems. - In: Engineering for a changing world, (2023), 1.2.121, S. 1-2
https://doi.org/10.22032/dbt.58700
Temporal graph processing in modern memory hierarchies. - In: Advances in databases and information systems, (2023), S. 103-106
Updates in graph DBMS lead to structural changes in the graph over time with different intermediate states. These intermediate states in a DBMS and the time when the actions to the actual data take place can be processed using temporal DBMSs. Most DBMSs built their temporal features based on their non-temporal processing and storage without considering the memory hierarchy of the underlying system. This leads to slower temporal processing and poor storage utilization. In this paper, we propose a storage and processing strategy for (bi-) temporal graphs using temporal materialized views (TMV) while exploiting the memory hierarchy of a modern system. Further, we show a solution to the query containment problem for certain types of temporal graph queries. Finally, we evaluate the overhead and performance of the presented approach. The results show that using TMV reduces the runtime of temporal graph queries while using less memory.
https://doi.org/10.1007/978-3-031-42914-9_8
Extracting provenance of machine learning experiment pipeline artifacts. - In: Advances in databases and information systems, (2023), S. 238-251
Experiment management systems (EMSs), such as MLflow, are increasingly used to streamline the collection and management of machine learning (ML) artifacts in iterative and exploratory ML experiment workflows. However, EMSs typically suffer from limited provenance capabilities rendering it hard to analyze the provenance of ML artifacts and gain knowledge for improving experiment pipelines. In this paper, we propose a comprehensive provenance model compliant with the W3C PROV standard, which captures the provenance of ML experiment pipelines and their artifacts related to Git and MLflow activities. Moreover, we present the tool MLflow2PROV that extracts provenance graphs according to our model from existing projects enabling collected pipeline provenance information to be queried, analyzed, and further processed.
https://doi.org/10.1007/978-3-031-42914-9_17
Interactive data cleaning for real-time streaming applications. - In: HILDA '23, (2023), 13, insges. 3 S.
The importance of data cleaning systems has continuously grown in recent years. Especially for real-time streaming applications, it is crucial, to identify and possibly remove anomalies in the data on the fly before further processing. The main challenge however lies in the construction of an appropriate data cleaning pipeline, which is complicated by the dynamic nature of streaming applications. To simplify this process and help data scientists to explore and understand the incoming data, we propose an interactive data cleaning system for streaming applications. In this paper, we list requirements for such a system and present our implementation to overcome the stated issues. Our demonstration shows, how a data cleaning pipeline can be interactively created, executed, and monitored at runtime. We also present several different tools, such as the automated advisor and the adaptive visualizer, that engage the user in the data cleaning process and help them understand the behavior of the pipeline.
https://doi.org/10.1145/3597465.3605229
Traveling back in time: a visual debugger for stream processing applications. - In: 2023 IEEE 39th International Conference on Data Engineering, (2023), S. 3647-3650
Stream processing takes on an important role as a hot topic of our time. More and more applications generate large amounts of heterogeneous data that need to be processed in real-time. However, the dynamic and high frequent nature of stream processing applications complicates the debugging process since the constant flow of data can not be slowed down, paused, or reverted to previous states to analyze the execution step-by-step. In this demonstration, we present StreamVizzard’s visual and interactive pipeline debugger that allows reverting the pipeline state to any arbitrary point in the past to review or repeat critical parts of the pipeline step by step. During this process, our extensive visualizer allows to explore the processed data and statistics of each operator to retrace and understand the data flow and behavior of the pipeline.
https://doi.org/10.1109/ICDE55515.2023.00289