From explainable AI to explainable simulation: using machine learning and XAI to understand system robustness. - In: ACM SIGSIM-PADS 2023, (2023), S. 96-106
Evaluating robustness is an important goal in simulation-based analysis. Robustness is achieved when the controllable factors of a system are adjusted in such a way that any possible variance in uncontrollable factors (noise) has minimal impact on the variance of the desired output. The optimization of system robustness using simulation is a dedicated and well-established research direction. However, once a simulation model is available, there is a lot of potential to learn more about the inherent relationships in the system, especially regarding its robustness. Data farming offers the possibility to explore large design spaces using smart experiment design, high performance computing, automated analysis, and interactive visualization. Sophisticated machine learning methods excel at recognizing and modelling the relation between large amounts of simulation input and output data. However, investigating and analyzing this modelled relationship can be very difficult, since most modern machine learning methods like neural networks or random forests are opaque black boxes. Explainable Artificial Intelligence (XAI) can help to peak into this black box, helping us to explore and learn about relations between simulation input and output. In this paper, we introduce a concept for using Data Farming, machine learning and XAI to investigate and understand system robustness of a given simulation model.
https://doi.org/10.1145/3573900.3591114
Explainable AI for data farming output analysis: a use case for knowledge generation through black-box classifiers. - In: 2022 Winter Simulation Conference (WSC), (2022), S. 1152-1163
Data farming combines large-scale simulation experiments with high performance computing and sophisticated big data analysis methods. The portfolio of analysis methods for those large amounts of simulation data still yields potential to further development, and new methods emerge frequently. Among the most interesting are methods of explainable artificial intelligence (XAI). Those methods enable the use of black-box-classifiers for data farming output analysis, which has been shown in a previous paper. In this paper, we apply the concept for XAI-based data farming analysis on a complex, real world case study to investigate the suitability of such concept in a real world application, and we also elaborate on which black-box classifiers are actually the most suitable for large-scale simulation data that accumulates in a data farming project.
https://doi.org/10.1109/WSC57314.2022.10015304
Development of an integrated solution for data farming and knowledge discovery in simulation data. - In: Simulation Notes Europe, ISSN 2164-5353, Bd. 32 (2022), 3, S. 121-126
Simulation is an established methodology for planning and evaluating manufacturing and logistics systems. In contrast to classical simulation studies, the method of knowledge discovery in simulation data uses a simulation model as a data generator (data farming). Subsequently, hidden, previously unknown and potentially useful cause-effect relationships can be uncovered on the generated data using data mining and visual analytics methods. So far, however, there was a lack of integrated, easy-to-use software solutions for the application of the data farming in operational practice. This paper presents such an integrated solution, which allows generating experiment designs, implements a method to distribute the necessary experiment runs, and provides the user with tools to analyze and visualize the result data.
https://dx.doi.org/10.11128/sne.32.tn.10611
Simulationsbasiertes Deep Reinforcement Learning für Modulare Produktionssysteme. - In: ASIM SST 2022 Proceedings Langbeiträge, (2022), S. 65-72
Modulare Produktionssysteme sollen die traditionelle Fließbandproduktion in der Automobilindustrie ablösen. Die Idee dabei ist, dass sich hochgradig individualisierte Produkte dynamisch und autonom durch ein System flexibler Arbeitsstationen bewegen können. Dieser Ansatz stellt hohe Anforderungen an die Planung und Organisation solcher Systeme. Da jedes Produkt seinen Weg durch das System frei und individuell bestimmen kann, kann die Implementierung von Regeln und Heuristiken, die die Flexibilität des Systems zur Leistungssteigerung ausnutzen, in diesem dynamischen Umfeld schwierig sein. Transportaufgaben werden in der Regel von fahrerlosen Transportsystemen (FTS) ausgeführt. Daher bietet die Integration von KI-basierten Steuerungslogiken eine vielversprechende Alternative zu manuell implementierten Entscheidungsregeln für den Betrieb der FTS. In diesem Beitrag wird ein Ansatz für den Einsatz von Reinforcement Learning (RL) in Kombination mit Simulation vorgestellt, um FTS in modularen Produktionssystemen zu steuern. Darüber hinaus werden Untersuchungen zu dessen Flexibilität und Skalierbarkeit durchgeführt.
https://dx.doi.org/10.11128/arep.20.a2007
A method using generative adversarial networks for robustness optimization. - In: ACM transactions on modeling and computer simulation, ISSN 1558-1195, Bd. 32 (2022), 2, S. 12:1-12:22
The evaluation of robustness is an important goal within simulation-based analysis, especially in production and logistics systems. Robustness refers to setting controllable factors of a system in such a way that variance in the uncontrollable factors (noise) has minimal effect on a given output. In this paper, we present an approach for optimizing robustness based on deep generative models, a special method of deep learning. We propose a method consisting of two Generative Adversarial Networks (GANs) to generate optimized experiment plans for the decision factors and the noise factors in a competitive, turn-based game. In a case study, the proposed method is tested and compared to traditional methods for robustness analysis including Taguchi method and Response Surface Method.
https://doi.org/10.1145/3503511
Automation within the process of knowledge discovery in simulation data : characterization of the result data
Automatisierung im Prozess der Wissensentdeckung in Simulationsdaten : Charakterisierung der Ergebnisdaten. - In: Simulation in Produktion und Logistik 2021, (2021), S. 367-376
Literaturangaben
The traditional application of simulation in production and logistics is usually aimed at changing certain parameters in order to answer clearly defined objectives or questions. In contrast to this approach, the method of knowledge discovery in simulation data (KDS) uses a simulation model as a data generator (data farming). Subsequently using data mining methods, hidden, previously unknown and potentially useful cause-effect relationships can be uncovered. So far, however, there is a lack of guidelines and automatization-tools for non-experts or novices in KDS, which leads to a more difficult use in industrial applications and prevents a broader utilization. This paper presents a concept for automating the first step of the KDS, which is the process of characterization of the result data, using meta learning and validates it on small case study.
Data farming output analysis using explainable AI. - In: 2021 Winter Simulation Conference (WSC), (2021), insges. 12 S.
Data Farming combines large-scale simulation experiments with high performance computing and sophisticated big data analysis methods. The portfolio of analysis methods for those large amounts of simulation data still yields potential to further development, and new methods emerge frequently. Especially the application of machine learning and artificial intelligence is difficult, since a lot of those methods are very good at approximating data for prediction, but less at actually revealing their underlying model of rules. To overcome the lack of comprehensibility of such black-box algorithms, a discipline called explainable artificial intelligence (XAI) has gained a lot of traction and has become very popular recently. This paper shows how to extend the portfolio of Data Farming output analysis methods using XAI.
https://doi.org/10.1109/WSC52266.2021.9715470
Development of an integrated solution for data farming and knowledge discovery in simulation data :
Entwicklung einer integrierten Lösung für das Data Farming und die Wissensentdeckung in Simulationsdaten. - In: Simulation in Produktion und Logistik 2021, (2021), S. 377-386
Literaturangaben
Simulation is an established methodology for planning and evaluating manufacturing and logistics systems. In contrast to classical simulation studies, the method of knowledge discovery in simulation data uses a simulation model as a data generator (data farming). Subsequently, hidden, previously unknown and potentially useful cause-effect relationships can be uncovered on the generated data using data mining and visual analytics methods. So far, however, there is a lack of integrated, easy-to-use software solutions for the application of the data farming in operational practice. This paper presents such an integrated solution, which allows for generating experiment designs, implements a method to distribute the necessary experiment runs, and provides the user with tools to analyze and visualize the result data.
Simulation-based deep reinforcement learning for modular production systems. - In: 2020 Winter Simulation Conference (WSC), (2020), S. 1596-1607
Modular production systems aim to supersede the traditional line production in the automobile industry. The idea here is that highly customized products can move dynamically and autonomously through a system of flexible workstations without fixed production cycles. This approach has challenging demands regarding planning and organization of such systems. Since each product can define its way through the system freely and individually, implementing rules and heuristics that leverage the flexibility in the system in order to increase performance can be difficult in this dynamic environment. Transport tasks are usually carried out by automated guided vehicles (AGVs). Therefore, integration of AI-based control logics offer a promising alternative to manually implemented decision rules for operating the AGVs. This paper presents an approach for using reinforcement learning (RL) in combination with simulation in order to control AGVs in modular production systems. We present a case study and compare our approach to heuristic rules.
https://doi.org/10.1109/WSC48552.2020.9384089
Knowledge discovery in simulation data. - In: ACM transactions on modeling and computer simulation, ISSN 1558-1195, Bd. 30 (2020), 4, S. 24:1-24:25
This article provides a comprehensive and in-depth overview of our work on knowledge discovery in simulations. Application-wise, we focus on manufacturing simulations. Specifically, we propose and discuss a methodology for designing, executing, and analyzing large-scale simulation experiments with a broad coverage of possible system behavior targeted at generating knowledge about the system. Based on the concept of data farming, we suggest a two-phase process which starts with a data generation phase, in which a smart experiment design is used to set up and efficiently execute a large number of simulation experiments. In the second phase, the knowledge discovery phase, data mining and visually aided analysis methods are applied on the gathered simulation input and output data. This article gives insights into this knowledge discovery phase by discussing different machine learning approaches and their suitability for different manufacturing simulation problems. With this, we provide guidelines on how to conduct knowledge discovery studies within the manufacturing simulation context. We also introduce different case studies, both academic and applied, and use them to validate our methodology.
https://doi.org/10.1145/3391299