Dropout is NOT all you need to prevent gradient leakage. - In: 37th AAAI Conference on Artificial Intelligence (AAAI-23), (2023), S. 9733-9741
Gradient inversion attacks on federated learning systems reconstruct client training data from exchanged gradient information. To defend against such attacks, a variety of defense mechanisms were proposed. However, they usually lead to an unacceptable trade-off between privacy and model utility. Recent observations suggest that dropout could mitigate gradient leakage and improve model utility if added to neural networks. Unfortunately, this phenomenon has not been systematically researched yet. In this work, we thoroughly analyze the effect of dropout on iterative gradient inversion attacks. We find that state of the art attacks are not able to reconstruct the client data due to the stochasticity induced by dropout during model training. Nonetheless, we argue that dropout does not offer reliable protection if the dropout induced stochasticity is adequately modeled during attack optimization. Consequently, we propose a novel Dropout Inversion Attack (DIA) that jointly optimizes for client data and dropout masks to approximate the stochastic client model. We conduct an extensive systematic evaluation of our attack on four seminal model architectures and three image classification datasets of increasing complexity. We find that our proposed attack bypasses the protection seemingly induced by dropout and reconstructs client data with high fidelity. Our work demonstrates that privacy inducing changes to model architectures alone cannot be assumed to reliably protect from gradient leakage and therefore should be combined with complementary defense mechanisms.
Model-based data generation for the evaluation of functional reliability and resilience of distributed machine learning systems against abnormal cases. - In: Engineering for a changing world, (2023), 5.3.128, S. 1-6
Future production technologies will comprise a multitude of systems whose core functionality is closely related to machine-learned models. Such systems require reliable components to ensure the safety of workers and their trust in the systems. The evaluation of the functional reliability and resilience of systems based on machine-learned models is generally challenging. For this purpose, appropriate test data must be available, which also includes abnormal cases. These abnormal cases can be unexpected usage scenarios, erroneous inputs, accidents during operation or even the failure of certain subcomponents. In this work, approaches to the model-based generation of an arbitrary abundance of data representing such abnormal cases are explored. Such computer-based generation requires domain-specific approaches, especially with respect to the nature and distribution of the data, protocols used, or domain-specific communication structures. In previous work, we found that different use cases impose different requirements on synthetic data, and the requirements in turn imply different generation methods . Based on this, various use cases are identified and different methods for computer-based generation of realistic data, as well as for the quality assessment of such data, are explored. Ultimately we explore the use of Federated Learning (FL) to address data privacy and security challenges in Industrial Control Systems. FL enables local model training while keeping sensitive information decentralized and private to their owners. In detail, we investigate whether FL can benefit clients with limited knowledge by leveraging collaboratively trained models that aggregate client-specific knowledge distributions. We found that in such scenarios federated training results in a significant increase in classification accuracy by 31.3% compared to isolated local training. Furthermore, as we introduce Differential Privacy, the resulting model achieves on par accuracy of 99.62% to an idealized case where data is independent and identically distributed across clients.
Data-driven estimation of scalar quantities from planar velocity measurements by deep learning applied to temperature in thermal convection. - In: Experiments in fluids, ISSN 1432-1114, Bd. 64 (2023), 12, 191, S. 1-18
The measurement of the transport of scalar quantities within flows is oftentimes laborious, difficult or even unfeasible. On the other hand, velocity measurement techniques are very advanced and give high-resolution, high-fidelity experimental data. Hence, we explore the capabilities of a deep learning model to predict the scalar quantity, in our case temperature, from measured velocity data. Our method is purely data-driven and based on the u-net architecture and, therefore, well-suited for planar experimental data. We demonstrate the applicability of the u-net on experimental temperature and velocity data, measured in large aspect ratio Rayleigh-Bénard convection at Pr = 7.1 and Ra = 2 x 10^5, 4 x 10^5, 7 x 10^5. We conduct a hyper-parameter optimization and ablation study to ensure appropriate training convergence and test different architectural variations for the u-net. We test two application scenarios that are of interest to experimentalists. One, in which the u-net is trained with data of the same experimental run and one in which the u-net is trained on data of different Ra. Our analysis shows that the u-net can predict temperature fields similar to the measurement data and preserves typical spatial structure sizes. Moreover, the analysis of the heat transfer associated with the temperature showed good agreement when the u-net is trained with data of the same experimental run. The relative difference between measured and reconstructed local heat transfer of the system characterized by the Nusselt number Nu is between 0.3 and 14.1% depending on Ra. We conclude that deep learning has the potential to supplement measurements and can partially alleviate the expense of additional measurement of the scalar quantity.
Bridging the gap: how to adopt opportunistic plant observations for phenology monitoring. - In: Frontiers in plant science, ISSN 1664-462X, Bd. 14 (2023), 1150956, S. 1-13
Plant phenology plays a vital role in assessing climate change. To monitor this, individual plants are traditionally visited and observed by trained volunteers organized in national or international networks - in Germany, for example, by the German Weather Service, DWD. However, their number of observers is continuously decreasing. In this study, we explore the feasibility of using opportunistically captured plant observations, collected via the plant identification app Flora Incognita to determine the onset of flowering and, based on that, create interpolation maps comparable to those of the DWD. Therefore, the opportunistic observations of 17 species collected in 2020 and 2021 were assigned to “Flora Incognita stations” based on location and altitude in order to mimic the network of stations forming the data basis for the interpolation conducted by the DWD. From the distribution of observations, the percentile representing onset of flowering date was calculated using a parametric bootstrapping approach and then interpolated following the same process as applied by the DWD. Our results show that for frequently observed, herbaceous and conspicuous species, the patterns of onset of flowering were similar and comparable between both data sources. We argue that a prominent flowering stage is crucial for accurately determining the onset of flowering from opportunistic plant observations, and we discuss additional factors, such as species distribution, location bias and societal events contributing to the differences among species and phenology data. In conclusion, our study demonstrates that the phenological monitoring of certain species can benefit from incorporating opportunistic plant observations. Furthermore, we highlight the potential to expand the taxonomic range of monitored species for phenological stage assessment through opportunistic plant observation data.
Erfahrungen bei der Integration des Autograding-Systems CodeOcean in die universitäre Programmierausbildung. - In: Proceedings of the Sixth Workshop "Automatische Bewertung von Programmieraufgaben" (ABP 2023), (2023), S. 67-74
Eine effektive und effiziente universitäre Programmierausbildung erfordert zunehmend den Einsatz automatisierter Bewertungssysteme. Im Rahmen des Projekts examING erprobt das Teilprojekt AutoPING den Einsatz des quelloffenen Autograding-Systems CodeOcean für übergreifende Lehrangebote und Prüfungen an der TU Ilmenau mit dem Ziel, selbstgesteuertes und kompetenzorientiertes Lernen zu ermöglichen und zu fördern. Der Beitrag gibt einen Überblick über erste Projekterfahrungen bei der Adaption didaktischer Szenarien in der Programmierausbildung hin zu testgetriebener Softwareentwicklung sowie der Generierung von Feedback. Es werden wesentliche Erkenntnisse aus Sicht der Studierenden und Lehrenden erörtert, Herausforderungen und Lösungsansätze zur Integration und Erweiterung von CodeOcean für neue Anwendungsfelder diskutiert sowie zukünftige Perspektiven eröffnet.
FS3change: a scalable method for change pattern mining. - In: IEEE transactions on software engineering, ISSN 1939-3520, Bd. 49 (2023), 6, S. 3616-3629
Mining change patterns can give unique understanding on the evolution of dynamically changing systems like social relation graphs, weblinks, hardware descriptions and models. A more recent focus is source code change pattern mining that may qualitatively justify expected or uncover unexpected patterns. These patterns then offer a basis, e.g., for program language evolution or auto-completion support. We present a change pattern mining method that greatly expands the limits of input data and pattern complexity, over existing methods. We propose scalability solutions on conceptual and algorithmic level, thereby evolving the state-of-the-art sampling-based frequent subgraph mining method FS3, resulting in 75% reduction in memory consumption and a speedup of 6500 for a large scale dataset. Patterns can have 100,000 s of occurrences for which manual review is impossible and may lead to misinterpretation. We propose the novel content track approach for interactively exploring pattern contents in context, based on marginal distributions. We evaluate our approach by mining 1,000 open source projects contributing a total of 558 million changes and 2 billion contextual connections among them, thereby, demonstrating its scalability. A manual interpretation of 19 patterns shows sensible mined patterns allowing to deduct implications for language design and demonstrating the soundness of the approach.
Artificial intelligence in nature conservation :
Künstliche Intelligenz im Naturschutz. - In: Natur und Landschaft, ISSN 0028-0615, Bd. 98 (2023), 6/7, S. 304-311
SW: Naturschutz ; maschinelles Lernen ; künstliche Intelligenz (KI) ; automatische Artenerkennung ; Vorhersagemodelle ; nachvollziehbare KI ; Reproduzierbarkeit
Discovering unknown response patterns in progress test data to improve the estimation of student performance. - In: BMC medical education, ISSN 1472-6920, Bd. 23 (2023), 1, 193, S. 1-12
Background: The Progress Test Medizin (PTM) is a 200-question formative test that is administered to approximately 11,000 students at medical universities (Germany, Austria, Switzerland) each term. Students receive feedback on their knowledge (development) mostly in comparison to their own cohort. In this study, we use the data of the PTM to find groups with similar response patterns. Methods: We performed k-means clustering with a dataset of 5,444 students, selected cluster number k = 5, and answers as features. Subsequently, the data was passed to XGBoost with the cluster assignment as target enabling the identification of cluster-relevant questions for each cluster with SHAP. Clusters were examined by total scores, response patterns, and confidence level. Relevant questions were evaluated for difficulty index, discriminatory index, and competence levels. Results: Three of the five clusters can be seen as “performance” clusters: cluster 0 (n = 761) consisted predominantly of students close to graduation. Relevant questions tend to be difficult, but students answered confidently and correctly. Students in cluster 1 (n = 1,357) were advanced, cluster 3 (n = 1,453) consisted mainly of beginners. Relevant questions for these clusters were rather easy. The number of guessed answers increased. There were two “drop-out” clusters: students in cluster 2 (n = 384) dropped out of the test about halfway through after initially performing well; cluster 4 (n = 1,489) included students from the first semesters as well as “non-serious” students both with mostly incorrect guesses or no answers. Conclusion: Clusters placed performance in the context of participating universities. Relevant questions served as good cluster separators and further supported our “performance” cluster groupings.
The HAInich: a multidisciplinary vision data-set for a better understanding of the forest ecosystem. - In: Scientific data, ISSN 2052-4463, Bd. 10 (2023), 1, 168, S. 1-11
We present a multidisciplinary forest ecosystem 3D perception dataset. The dataset was collected in the Hainich-Dün region in central Germany, which includes two dedicated areas, which are part of the Biodiversity Exploratories - a long term research platform for comparative and experimental biodiversity and ecosystem research. The dataset combines several disciplines, including computer science and robotics, biology, bio-geochemistry, and forestry science. We present results for common 3D perception tasks, including classification, depth estimation, localization, and path planning. We combine the full suite of modern perception sensors, including high-resolution fisheye cameras, 3D dense LiDAR, differential GPS, and an inertial measurement unit, with ecological metadata of the area, including stand age, diameter, exact 3D position, and species. The dataset consists of three hand held measurement series taken from sensors mounted on a UAV during each of three seasons: winter, spring, and early summer. This enables new research opportunities and paves the way for testing forest environment 3D perception tasks and mission set automation for robotics.
A systematic comparison of deep learning methods for EEG time series analysis. - In: Frontiers in neuroinformatics, ISSN 1662-5196, Bd. 17 (2023), 1067095, S. 01-17
Analyzing time series data like EEG or MEG is challenging due to noisy, high-dimensional, and patient-specific signals. Deep learning methods have been demonstrated to be superior in analyzing time series data compared to shallow learning methods which utilize handcrafted and often subjective features. Especially, recurrent deep neural networks (RNN) are considered suitable to analyze such continuous data. However, previous studies show that they are computationally expensive and difficult to train. In contrast, feed-forward networks (FFN) have previously mostly been considered in combination with hand-crafted and problem-specific feature extractions, such as short time Fourier and discrete wavelet transform. A sought-after are easily applicable methods that efficiently analyze raw data to remove the need for problem-specific adaptations. In this work, we systematically compare RNN and FFN topologies as well as advanced architectural concepts on multiple datasets with the same data preprocessing pipeline. We examine the behavior of those approaches to provide an update and guideline for researchers who deal with automated analysis of EEG time series data. To ensure that the results are meaningful, it is important to compare the presented approaches while keeping the same experimental setup, which to our knowledge was never done before. This paper is a first step toward a fairer comparison of different methodologies with EEG time series data. Our results indicate that a recurrent LSTM architecture with attention performs best on less complex tasks, while the temporal convolutional network (TCN) outperforms all the recurrent architectures on the most complex dataset yielding a 8.61% accuracy improvement. In general, we found the attention mechanism to substantially improve classification results of RNNs. Toward a light-weight and online learning-ready approach, we found extreme learning machines (ELM) to yield comparable results for the less complex tasks.
Particle detection and size recognition based on defocused particle images: a comparison of a deterministic algorithm and a deep neural network. - In: Experiments in fluids, ISSN 1432-1114, Bd. 64 (2023), 2, 21, S. 1-16
The systematic manipulation of components of multimodal particle solutions is a key for the design of modern industrial products and pharmaceuticals with highly customized properties. In order to optimize innovative particle separation devices on microfluidic scales, a particle size recognition with simultaneous volumetric position determination is essential. In the present study, the astigmatism particle tracking velocimetry is extended by a deterministic algorithm and a deep neural network (DNN) to include size classification of particles of multimodal size distribution. Without any adaptation of the existing measurement setup, a reliable classification of bimodal particle solutions in the size range of 1.14 μm–5.03 μm is demonstrated with a precision of up to 99.9 %. Concurrently, the high detection rate of the particles, suspended in a laminar fluid flow, is quantified by a recall of 99.0 %. By extracting particle images from the experimentally acquired images and placing them on a synthetic background, semi-synthetic images with consistent ground truth are generated. These contain labeled overlapping particle images that are correctly detected and classified by the DNN. The study is complemented by employing the presented algorithms for simultaneous size recognition of up to four particle species with a particle diameter in between 1.14 μm and 5.03 μm. With the very high precision of up to 99.3 % at a recall of 94.8 %, the applicability to classify multimodal particle mixtures even in dense solutions is confirmed. The present contribution thus paves the way for quantitative evaluation of microfluidic separation and mixing processes.
Parameterizing echo state networks for multi-step time series prediction. - In: Neurocomputing, ISSN 1872-8286, Bd. 522 (2023), S. 214-228
Prediction of multi-dimensional time-series data, which may represent such diverse phenomena as climate changes or financial markets, remains a challenging task in view of inherent nonlinearities and non-periodic behavior In contrast to other recurrent neural networks, echo state networks (ESNs) are attractive for (online) learning due to lower requirements w.r.t.training data and computational power. However, the randomly-generated reservoir renders the choice of suitable hyper-parameters as an open research topic. We systematically derive and exemplarily demonstrate design guidelines for the hyper-parameter optimization of ESNs. For the evaluation, we focus on the prediction of chaotic time series, an especially challenging problem in machine learning. Our findings demonstrate the power of a hyper-parameter-tuned ESN when auto-regressively predicting time series over several hundred steps. We found that ESNs’ performance improved by 85.1%-99.8% over an already wisely chosen default parameter initialization. In addition, the fluctuation range is considerably reduced such that significantly worse performance becomes very unlikely across random reservoir seeds. Moreover, we report individual findings per hyper-parameter partly contradicting common knowledge to further, help researchers when training new models.