Publikationen

A combined convolutional autoencoder-recurrent neural network machine learning model is presented to directly analyze and forecast the dynamics and low-order statistics of the local convective heat flux field in a two-dimensional turbulent Rayleigh-Bénard convection flow at Prandtl number Pr=7 and Rayleigh number Ra=10^7. Two recurrent neural networks are applied for the temporal advancement of turbulent heat transfer data in the reduced latent data space, an echo state network, and a recurrent gated unit. Thereby, our work exploits the modular combination of three different machine learning algorithms to build a fully data-driven and reduced model for the dynamics of the turbulent heat transfer in a complex thermally driven flow. The convolutional autoencoder with 12 hidden layers is able to reduce the dimensionality of the turbulence data to about 0.2% of their original size. Our results indicate a fairly good accuracy in the first- and second-order statistics of the convective heat flux. The algorithm is also able to reproduce the intermittent plume-mixing dynamics at the upper edges of the thermal boundary layers with some deviations. The same holds for the probability density function of the local convective heat flux with differences in the far tails. Furthermore, we demonstrate the noise resilience of the framework. This suggests that the present model might be applicable as a reduced dynamical model that delivers transport fluxes and their variations to coarse grids of larger-scale computational models, such as global circulation models for atmosphere and ocean.

Relocalization inside pre-built maps provides a big benefit in the course of today’s autonomous driving tasks where the map can be considered as an additional sensor for refining the estimated current pose of the vehicle. Due to potentially large drifts in the initial pose guess as well as maps containing unfiltered dynamic and temporal static objects (e.g. parking cars), traditional methods like ICP tend to fail and show high computation times. We propose a novel and fast relocalization method for accurate pose estimation inside a pre-built map based on 3D point clouds. The method is robust against inaccurate initialization caused by low performance GPS systems and tolerates the presence of unfiltered objects by specifically learning to extract significant features from current scans and adjacent map sections. More specifically, we introduce a novel distance-based matching loss enabling us to simultaneously extract important information from raw point clouds and aggregating inner- and inter-cloud context by utilizing self- and cross-attention inside a Graph Neural Network. We evaluate StickyLocalization’s (SL) performance through an extensive series of experiments using two benchmark datasets in terms of Relocalization on NuScenes and Loop Closing using KITTI’s Odometry dataset. We found that SL outperforms state-of-the art point cloud registration and relocalization methods in terms of transformation errors and runtime.

Poaceae represent one of the largest plant families in the world. Many species are of great economic importance as food and forage plants while others represent important weeds in agriculture. Although a large number of studies currently address the question of how plants can be best recognized on images, there is a lack of studies evaluating specific approaches for uniform species groups considered difficult to identify because they lack obvious visual characteristics. Poaceae represent an example of such a species group, especially when they are non-flowering. Here we present the results from an experiment to automatically identify Poaceae species based on images depicting six well-defined perspectives. One perspective shows the inflorescence while the others show vegetative parts of the plant such as the collar region with the ligule, adaxial and abaxial side of the leaf and culm nodes. For each species we collected 80 observations, each representing a series of six images taken with a smartphone camera. We extract feature representations from the images using five different convolutional neural networks (CNN) trained on objects from different domains and classify them using four state-of-the art classification algorithms. We combine these perspectives via score level fusion. In order to evaluate the potential of identifying non-flowering Poaceae we separately compared perspective combinations either comprising inflorescences or not. We find that for a fusion of all six perspectives, using the best combination of feature extraction CNN and classifier, an accuracy of 96.1% can be achieved. Without the inflorescence, the overall accuracy is still as high as 90.3%. In all but one case the perspective conveying the most information about the species (excluding inflorescence) is the ligule in frontal view. Our results show that even species considered very difficult to identify can achieve high accuracies in automatic identification as long as images depicting suitable perspectives are available. We suggest that our approach could be transferred to other difficult-to-distinguish species groups in order to identify the most relevant perspectives.

Due to the continuous digitalization of our society, distributed and web-based applications become omnipresent and making them more secure gains paramount relevance. Deep learning (DL) and its representation learning approach are increasingly been proposed for program code analysis potentially providing a powerful means in making software systems less vulnerable. This systematic literature review (SLR) is aiming for a thorough analysis and comparison of 32 primary studies on DL-based vulnerability analysis of program code. We found a rich variety of proposed analysis approaches, code embeddings and network topologies. We discuss these techniques and alternatives in detail. By compiling commonalities and differences in the approaches, we identify the current state of research in this area and discuss future directions. We also provide an overview of publicly available datasets in order to foster a stronger benchmarking of approaches. This SLR provides an overview and starting point for researchers interested in deep vulnerability analysis on program code.

In recent years, deep neural networks (DNNs) have accomplished impressive success in various applications, including autonomous driving perception tasks. However, current deep neural networks are easily deceived by adversarial attacks. This vulnerability raises significant - concerns, particularly in safety-critical applications. As a result, research into attacking and defending DNNs has gained much coverage. In this work, detailed adversarial attacks are applied on a diverse multi-task visual perception deep network across distance estimation, semantic segmentation, - motion detection, and object detection. The experiments consider both white and black box attacks for targeted and un-targeted cases, while attacking a task and inspecting the effect on all others, in addition to inspecting the effect of applying a simple defense method. We conclude this paper - by comparing and discussing the experimental results, proposing insights and future work. The visualizations of the attacks are available at https://youtu.be/6AixN90budY.

FisheyeDistanceNet [1] proposed a self-supervised monocular depth estimation method for fisheye cameras with a large field of view (> 180˚). To achieve scale-invariant depth estimation, FisheyeDistanceNet supervises depth map predictions over multiple scales during training. To overcome this bottleneck, we incorporate self-attention layers and robust loss function [2] to FisheyeDistanceNet. A general adaptive robust loss function helps obtain sharp depth maps without a need to train over multiple scales and allows us to learn hyperparameters in loss function to aid in better optimization in terms of convergence speed and accuracy. We also ablate the importance of Instance Normalization over Batch Normalization in the network architecture. Finally, we generalize the network to be invariant to camera views by training multiple perspectives using front, rear, and side cameras. Proposed algorithm improvements, FisheyeDistanceNet++, result in 30% relative improvement in RMSE while reducing the training time by 25% on the WoodScape dataset. We also obtain state-of-the-art results on the KITTI dataset, in comparison to other self-supervised monocular methods.

3D object detection based on LiDAR point clouds is a crucial module in autonomous driving particularly for long range sensing. Most of the research is focused on achieving higher accuracy and these models are not optimized for deployment on embedded systems from the perspective of latency and power efficiency. For high speed driving scenarios, latency is a crucial parameter as it provides more time to react to dangerous situations. Typically a voxel or point-cloud based 3D convolution approach is utilized for this module. Firstly, they are inefficient on embedded platforms as they are not suitable for efficient parallelization. Secondly, they have a variable runtime due to level of sparsity of the scene which is against the determinism needed in a safety system. In this work, we aim to develop a very low latency algorithm with fixed runtime. We propose a novel semantic segmentation architecture as a single unified model for object center detection using key points, box predictions and orientation prediction using binned classification in a simpler Bird's Eye View (BEV) 2D representation. The proposed architecture can be trivially extended to include semantic segmentation classes like road without any additional computation. The proposed model has a latency of 4 ms on the embedded Nvidia Xavier platform. The model is 5X faster than other top accuracy models with a minimal accuracy degradation of 2% in Average Precision at IoU = 0.5 on KITTI dataset.

Deep learning algorithms classify plant species with high accuracy, and smartphone applications leverage this technology to enable users to identify plant species in the field. The question we address here is whether such crowd-sourced data contain substantial macroecological information. In particular, we aim to understand if we can detect known environmental gradients shaping plant co-occurrences. In this study we analysed 1 million data points collected through the use of the mobile app Flora Incognita between 2018 and 2019 in Germany and compared them with Florkart, containing plant occurrence data collected by more than 5000 floristic experts over a 70-year period. The direct comparison of the two data sets reveals that the crowd-sourced data particularly undersample areas of low population density. However, using nonlinear dimensionality reduction we were able to uncover macroecological patterns in both data sets that correspond well to each other. Mean annual temperature, temperature seasonality and wind dynamics as well as soil water content and soil texture represent the most important gradients shaping species composition in both data collections. Our analysis describes one way of how automated species identification could soon enable near real-time monitoring of macroecological patterns and their changes, but also discusses biases that must be carefully considered before crowd-sourced biodiversity data can effectively guide conservation measures.

Assisting and automating software engineering tasks is a state-of-the-art way to support stakeholders of development projects. A common assistance function of IDEs is the auto-completion of source code. Assistance functions, such as auto-completion, are almost entirely missing in modeling tools though auto-completion in general gains continuously more importance in software development. We analyze a user’s performed editing operations in order to anticipate modeling activities and to recommend appropriate auto-completions for them. Editing operations are captured as events and modeling activities are defined as complex event patterns, facilitating the matching by complex-event-processing. The approach provides adapted auto-completions reactively upon each editing operation of the user. We implemented the RapMOD prototype as add-in for the modeling tool Sparx Enterprise Architect™ . A controlled user experiment with 37 participants performing modeling tasks demonstrated the approach's potential to reduce modeling effort significantly. Users having auto-completions available for a modeling scenario performed the task 27 percent faster, needed to perform 56 percent less actions, and perceived the task 29 percent less difficult.

Defocus particle tracking (DPT) has gained increasing importance for its use to determine particle trajectories in all three dimensions with a single-camera system, as typical for a standard microscope, the workhorse of todays ongoing biomedical revolution. DPT methods derive the depth coordinates of particle images from the different defocusing patterns that they show when observed in a volume much larger than the respective depth of field. Therefore it has become common for state-of-the-art methods to apply image recognition techniques. Two of the most commonly and widely used DPT approaches are the application of (astigmatism) particle image model functions (MF methods) and the normalized cross-correlations between measured particle images and reference templates (CC methods). Though still young in the field, the use of neural networks (NN methods) is expected to play a significant role in future and more complex defocus tracking applications. To assess the different strengths of such defocus tracking approaches, we present in this work a general and objective assessment of their performances when applied to synthetic and experimental images of different degrees of astigmatism, noise levels, and particle image overlapping. We show that MF methods work very well in low-concentration cases, while CC methods are more robust and provide better performance in cases of larger particle concentration and thus stronger particle image overlap. The tested NN methods generally showed the lowest performance, however, in comparison to the MF and CC methods, they are yet in an early stage and have still great potential to develop within the field of DPT.

Surround View fisheye cameras are commonly deployed in automated driving for 360˚ near-field sensing around the vehicle. This work presents a multi-task visual perception network on unrectified fisheye images to enable the vehicle to sense its surrounding environment. It consists of six primary tasks necessary for an autonomous driving system: depth estimation, visual odometry, semantic segmentation, motion segmentation, object detection, and lens soiling detection. We demonstrate that the jointly trained model performs better than the respective single task versions. Our multi-task model has a shared encoder providing a significant computational advantage and has synergized decoders where tasks support each other. We propose a novel camera geometry based adaptation mechanism to encode the fisheye distortion model both at training and inference. This was crucial to enable training on the WoodScape dataset, comprised of data from different parts of the world collected by 12 different cameras mounted on three different cars with different intrinsics and viewpoints. Given that bounding boxes is not a good representation for distorted fisheye images, we also extend object detection to use a polygon with non-uniformly sampled vertices. We additionally evaluate our model on standard automotive datasets, namely KITTI and Cityscapes. We obtain the state-of-the-art results on KITTI for depth estimation and pose estimation tasks and competitive performance on the other tasks. We perform extensive ablation studies on various architecture choices and task weighting methodologies. A short video at https://youtu.be/xbSjZ5OfPes provides qualitative results.

Many applications in chemistry, biology and medicine use microfluidic devices to separate, detect and analyze samples on a miniaturized size-level. Fluid flows evolving in channels of only several tens to hundreds of micrometers in size are often of a 3D nature, affecting the tailored transport of cells and particles. To analyze flow phenomena and local distributions of particles within those channels, astigmatic particle tracking velocimetry (APTV) has become a valuable tool, on condition that basic requirements like low optical aberrations and particles with a very narrow size distribution are fulfilled. Making use of the progress made in the field of machine vision, deep neural networks may help to overcome these limiting requirements, opening new fields of applications for APTV and allowing them to be used by nonexpert users. To qualify the use of a cascaded deep convolutional neural network (CNN) for particle detection and position regression, a detailed investigation was carried out starting from artificial particle images with known ground truth to real flow measurements inside a microchannel, using particles with uni- and bimodal size distributions. In the case of monodisperse particles, the mean absolute error and standard deviation of particle depth-position of less than and about 1 [my]m were determined, employing the deep neural network and the classical evaluation method based on the minimum Euclidean distance approach. While these values apply to all particle size distributions using the neural network, they continuously increase towards the margins of the measurement volume of about one order of magnitude for the classical method, if nonmonodisperse particles are used. Nevertheless, limiting the depth of measurement volume in between the two focal points of APTV, reliable flow measurements with low uncertainty are also possible with the classical evaluation method and polydisperse tracer particles. The results of the flow measurements presented herein confirm this finding. The source code of the deep neural network used here is available on https://github.com/SECSY-Group/DNN-APTV.

Die Nachvollziehbarkeit von Anforderungen ist wichtiges Qualitätsmerkmal der Softwareentwicklung. Für eine Vielzahl von Softwareentwicklungsaktivitäten ist die Nachvollziehbarkeit von Anforderungen eine notwenige Voraussetzung. Dazu gehören unter anderem die Analyse funktionaler Sicherheit, die Einflussanalyse, die Analyse des Abdeckungsgrades oder die Compliance. Für die Entwicklung sicherheitskritischer Softwaresysteme ist dieses Qualitätsmerkmal von besonderer Bedeutung. Daher wird dieses von entsprechenden Richtlinien zur Entwicklung sicherheitskritischer Software explizit vorgeschrieben. Obwohl die Relevanz der Nachvollziehbarkeit in Softwareprojekten allgemein bekannt ist, findet nur in wenigen Fällen eine systematische Planung zur Erreichung dieses Qualitätsmerkmals Anwendung. Häufig wird Nachvollziehbarkeit erst nachträglich umgesetzt. Daraus resultieren oft unvollständige Implementierungen der Nachvollziehbarkeit, die trotzdem als Grundlage für schwerwiegende Entscheidungen herangezogen werden. Aus diesem Grunde sollten die entsprechenden Implementierungen einer eingehenden Prüfung unterzogen werden, besonders im Rahmen der Entwicklung sicherheitskritischer Systeme. Dazu ist jedoch eine Vielzahl von Herausforderungen zu meistern. Zum einen hängt die Nachvollziehbarkeit von den projektspezifischen Zielen ab. Bei sicherheitskritischen Systemen müssen oft Vorgaben aus Richtlinien erfüllt werden. Auch die Nutzung der Nachvollziehbarkeit ist sehr stark von den jeweiligen Zielen abhängig. In dieser Arbeit wird ein Ansatz zur systematischen Prüfung von Softwareprojekten im Hinblick auf deren Nachvollziehbarkeit der Anforderungen vorgeschlagen. Eine notwendige Voraussetzung für den Prüfansatz ist die präzise Planung und Definition der Nachvollziehbarkeit von Anforderungen in einem Softwareprojekt. Daher wird im Rahmen dieser Arbeit ein entsprechender Planungsansatz präsentiert. Weiterhin wird ein analytisches Modell zur systematischen Prüfung der Nachvollziehbarkeit in Softwareprojekten präsentiert. Dieses Modell umfasst eine vollständige Klassifikation möglicher Fehlertypen. Außerdem werden Kriterien zur systematischen Erkennung dieser Fehler vorgeschlagen. Die Ergebnisse einer Expertenbefragung bestätigen die Vollständigkeit des analytischen Prüfmodells. Zudem wurde der vorgeschlagene Ansatz zur systematischen Prüfung der Nachvollziehbarkeit von Anforderungen in zwei Studien evaluiert. Dabei konnte der Nutzen des Ansatzes für die Entwicklung von sicherheitskritischer und nicht sicherheitskritischer Software nachgewiesen werden.