Publikationen

Ikegami, Yukino; Tsuruta, Setsuo; Kutics, Andrea; Damiani, Ernesto; Knauf, Rainer
Fast ML-based next-word prediction for hybrid languages. - In: Internet of things and cyber-physical systems, ISSN 2667-3452, Bd. 25 (2024), 101064, S. 1-15

Smartphone users are beyond two billion worldwide. Heavy users of the texting application rely on input prediction to reduce typing effort. In languages based on the Roman alphabet, many techniques are available. However, Japanese text is based on multiple character sets such as Kanji (Chinese-like word symbols), Hiragana and Katakana syllable sets. For its time/labor intensive input, next word prediction is crucial. It is still an open challenge. To tackle this, a hybrid language model is proposed. It integrates a Recurrent Neural Network (RNN) with an n-gram model. RNNs are powerful models for learning long sequences for next word prediction. N-gram models are best at current word completion. Our RNN language model (RNN-LM) predicts the next words. According the “price” of the performance gain paid by a higher time complexity, our model best deploys on a client-server architecture. Heavily-loaded RNN-LM deploys on the server while the n-gram model on the client. Our RNN-LM consists of an input layer equipped with word embedding, an output layer, and hidden layers connected with LSTMs (Long Short-Term Memories). Training is done via BPTT (Back Propagation Through Time). For robust training, BPTT is elaborated by learning rate refinement and gradient norm scaling. To avoid overfitting, the dropout technique is applied except for LSTM. Our novel model is compact (2 LSTMs, 650 units per layer), indeed. Due to synergetic elaboration, it shows 10 % lower perplexity than Zaremba's excellent conventional models in our Japanese text prediction experiment. Our model has been incorporated into IME (Input Method Editor) we call Flick. On the Japanese text input experiment, Flick outperforms Mozc (Google Japanese Input) by 16 % in time and 34 % in the number of keystrokes.

https://doi.org/10.1016/j.iot.2024.101064

Arnold, Oksana; Franke, Ronny; Jantke, Klaus P.; Knauf, Rainer; Schramm, Tanja; Wache, Hans-Holger
Thinking and chatting deontically - novel support of communication for learning and training with time travel prevention games. - In: Creative approaches to technology-enhanced learning for the workplace and higher education, (2023), S. 25-37

The authors’ key area of application is training for the prevention of accidents in the process technology industries. They run a professional training center with own 3D virtual environments. At TLIC 2021, four of the present authors delivered a contribution advocating planning of human training experiences as dynamically as managing some severely disturbed technical system back into a normal operation - such as an out of control chemical reactor - and enabling human trainees who failed to complete a risky task - thereby possibly ruining a (fortunately only virtual) technical installation - to virtually travel back in time to make good the damage. At TLIC 2022, they introduced cascades of gradually more intricate categories of time travel games. With every step from one category to the next, the deployed AI gets more powerful and effective in providing adaptive guidance of human trainees. The most advanced time travel games are those that allow for the dynamic modification of events experienced in the virtual past. In this way, the game system evolves over time and adapts to the needs of human trainees with emphasis on guidance for trainees who fail repeatedly. The extended team of authors presents a novel perspective at time travel prevention games that leads to a more human-centered adaptive guidance. Training is seen through the lens of deontic modal logic. The focus is on undesired events such as explosions, fire, health hazards due to toxic vapors, and the like. The game system’s AI is reasoning about necessity and possibility of such events. It offers to human trainees/players helpful chats about modalities of decisive events of training.

https://doi.org/10.1007/978-3-031-41637-8_3

Bottoni, Brooke; Moolenaar, Yasmine; Hevia, Anthony; Anchor, Thomas; Benko, Kyle; Knauf, Rainer; Jantke, Klaus P.; Gonzalez, Avelino J.; Wu, Annie
Character depth and sentence diversification in automated narrative generation. - In: Proceedings of the Thirty-third International Florida Artificial Intelligence Research Society Conference, (2020), S. 21-26

This paper describes and discusses methods for improving character depth and sentence diversification in automated storytelling systems. The fAIble III system that is the subject of this paper addresses a major limitation of its immediate predecessor (fAIble II) in that the characters in its stories seemed to act in a vacuum, without any apparent reasons for their choices or emotions. This is accomplished through generating character backstories. fAIble III also addresses the diversity of generated sentences with a pattern recognition system that removes many of the awkward and repetitive sentences that drew negative comments in the testing of fAIble I and II. Lastly, stories generated by fAIble II and fAIble III are compared and empirical test results are presented.

Batarseh, Feras A.; Gonzalez, Avelino J.; Knauf, Rainer
Context-assisted test cases reduction for cloud validation. - In: SSRN eLibrary, ISSN 1556-5068, (2020), insges. 14 S.
Last revised: 4 May 2020

Cloud computing is currently receiving much attention from the industry, government, and academia. It has changed the way computation is performed and how services are delivered to customers. Most importantly, cloud services change the way we design software, handle data, and perform testing. In cloud computing, testing is delivered as a service (TaaS). Case testing is one of the most common validation approaches for software. However, executing test cases on a software system could be expensive and time consuming. Therefore, test case reduction is performed to minimize the number of test cases to be executed on the system. In this paper, we introduce a validation method called Context-Assisted Test Case Reduction (CATCR) for systems that are deployed on the cloud. In CATCR, test cases are reduced based on the context of the validation process. The results of previous test cases are used to select the next set of test cases while the validation process is ongoing. The minimized set of test cases needs to have effective coverage of the entire system. To evaluate CATCR, an experimental evaluation is performed through Amazon's Cloud and a Java validation tool. Experimental results are recorded and presented.

https://dx.doi.org/10.2139/ssrn.3570307

Kurashige, Kentarou; Tsuruta, Setsuo; Sakurai, Eriko; Sakurai, Yoshitaka; Knauf, Rainer; Damiani, Ernesto; Kutics, Andrea
Robotized counselor evaluation using linguistic detection of feeling polarity change. - In: 2019 IEEE Symposium Series on Computational Intelligence, (2019), S. 962-967

In Japan, many people suffer from bad conditions of their mental health with an increasing tendency that should be alarmed. For this reason, a counseling robot was created in our former work and enhanced more and more. Our former quantitative evaluation based on the utterance amount. Here, we provide a new solution on evaluating this system. We introduce an evaluation polarity of the utterance contents by using a polarity dictionary This evaluates the feeling polarity of utterances by positive through negative degree on a scale from -1 (negative) to +1 (positive). Combined with generally used conventional impression evaluation by a questionnaire, it provides a much better way to evaluate the performance of our Counseling Agent Robot.

https://doi.org/10.1109/SSCI44817.2019.9002842

Horii, Tsubasa; Sakurai, Yoshitaka; Sakurai, Eriko; Tsuruta, Setsuo; Knauf, Rainer; Damiani, Ernesto; Kutics, Andrea
More general evaluation of a client-centered counseling agent. - In: 2019 IEEE World Congress on Services, (2019), S. 190-196

A lot of people in Japan suffer from bad conditions of their mental health with an increasing tendency, in particular jobseekers and elderly persons. The classical way to solve their problem is to consult a counselor, who treats these people in a way to become aware of the core of their problem and to solve it. However, the number of well qualified counselors is limited. For this purpose, we developed a VCA (Virtual Counseling Agent) as a further evolution of a formerly developed CRECA (Context Respectful Counseling Agent). CRECA. CRECA had a text interface. To much more imitate a human counselor, VCA has an image avatar and a voice conversation using the Google Cloud audio API. Further, VCA is made independent of counseling content fields. Thus, it is more generalized than CRECA or ELIZA in order for every people to easily use everywhere in every situation. Here, VCA, CRECA, and ELIZA are comparatively evaluated by questionnaire after the use of 10 college students having career problems as well as elderly persons struggling with modern IT. As a result, VCA significantly exceeded the average value of ELIZA along with the significant difference at the level of 5%. Moreover, average of the evaluation value is not worse than CRECA. Compared to CRECA, VCA does not limit the content and field of consultation. Especially, elderly persons struggling with modern IT could not use CRECA that have only text interface. It can be used generally at any time in everyday natural conversation. It can easily be used by elderly people and the digital divided due to voice conversion.

https://doi.org/10.1109/SERVICES.2019.00052

Sakurai, Yoshitaka; Ikegami, Yukino; Sakai, Motoki; Fujikawa, Hiroshi; Tsuruta, Setsuo; Gonzalez, Avelino J.; Sakurai, Eriko; Damiani, Ernesto; Kutics, Andrea; Knauf, Rainer; Frati, Fulvio
VICA, a visual counseling agent for emotional distress. - In: Journal of ambient intelligence and humanized computing, ISSN 1868-5145, Bd. 10 (2019), 12, S. 4993-5005

We present VICA, a Visual Counseling Agent designed to create an engaging multimedia face-to-face interaction. VICA is a human-friendly agent equipped with high performance voice conversation designed to help psychologically stressed users, to offload their emotional burden. Such users specifically include non-computer-savvy elderly persons or clients. Our agent builds replies exploiting interlocutor's utterances expressing such as wishes, obstacles, emotions, etc. Statements asking for confirmation, details, emotional summary, or relations among such expressions are added to the utterances. We claim that VICA is suitable for positive counseling scenarios where multimedia specifically high-performance voice communication is instrumental for even the old or digital divided users to continue dialogue towards their self-awareness. To prove this claim, VICA's effect is evaluated with respect to a previous text-based counseling agent CRECA and ELIZA including its successors. An experiment involving 14 subjects shows VICA effects as follows: (i) the dialogue continuation (CPS: Conversation-turns Per Session) of VICA for the older half (age > 40) substantially improved 53% to CRECA and 71% to ELIZA. (ii) VICA's capability to foster peace of mind and other positive feelings was assessed with a very high score of 5 or 6 mostly, out of 7 stages of the Likert scale, again by the older. Compared on average, such capability of VICA for the older is 5.14 while CRECA (all subjects are young students, age < 25) is 4.50, ELIZA is 3.50, and the best of ELIZA's successors for the older (> 25) is 4.41.

https://doi.org/10.1007/s12652-019-01180-x

Kurashige, Kentarou; Tsuruta, Setsuo; Sakurai, Eriko; Sakurai, Yoshitaka; Knauf, Rainer; Damiani, Ernesto; Kutics, Andrea
Counseling robot implementation and evaluation. - In: SMC 2018, (2018), S. 1716-1722

A lot of IT personnel have psychological distress and counselors to help them are lack in number. Therefore, we proposed a counseling agent (CA) called CRECA (context respectful counseling agent), which listens to clients and promotes their reflection context respectfully namely in a context preserving way. This agent is now enhanced using a body language called "unazuki" in Japanese, a kind of nodding to greatly promote dialogue, often accompanying "un-un" (meaning "exactly") of Japanese onomatopoeia. This body language significantly helps represent empathy or entire approval. Our agent is enhanced with such dialog promotion nodding robot to continue the conversation naturally or context respectfully towards clients' further reflection. To realize it, the robot nods twice at each end of dialog sentence input by clients. Here, we introduce a robot that behaves human-like by an appropriate nodding behavior. The motivation for such a more human-like robot was the extension of application fields from IT workers' counselling to people, who suffer from more social problems such as financial debt, or anxiety of victory or defeat. For such applications, it is important that the agent behaves as much as possible human-like. Here, we present an enhanced experimental evaluation. The quantitative evaluation is based on the utterance amounts of a test group of individuals. These amount with and without the nodding feature are compared. Additionally, the robots with and without nodding are compared according several subjective feelings by the evaluation subjects.

https://doi.org/10.1109/SMC.2018.00297

Sakurai, Yoshitaka; Ikegami, Yukino; Sakai, Motoki; Fujikawa, Hiroshi; Tsuruta, Setsuo; Gonzalez, Avelino J.; Sakurai, Eriko; Damiani, Ernesto; Kutics, Andrea; Knauf, Rainer
Visual counseling agent and its evaluation. - In: Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence (SSCI 2018), (2018), S. 2286-2291

There are significantly many people including IT workers, old persons, job seekers, etc., who suffer from (psychological) distress in their work or life. Compared with this, there are only a limited number of human counselors who can provide mental care to such many distress persons. Initially for psychologically distressed IT workers, we developed a counseling agent (CA) called a context respectful counseling agent (CRECA), which serves IT workers just like a human counsellor. In a context respectful/ preserving way to provide unconditional positive regard, CRECA replies prompting to dig or clarify problems as well as paraphrases or processes mirroring. In this paper, to save more persons including even elderly people other than IT workers, CRECA is enhanced to realize Virtual Counseling Agent (VCA) as follows: 1) making more human-friendly by an image avatar with voice conversation, 2) letting confirmation replies such as the relationship among wishes, obstacles, etc. as well as with past sayings in client’s utterances, especially for the forgetful/ elderly persons. For successful voice conversation, Google cloud speech API is not only exploited but its error processing is elaborated in our VCA system. The evaluation by 14 subjects showed the effect of this new VCA as follows: 1) the dialogue continuation length became twice as long as that of ELIZA, 2) positive feeling by selfawareness became high score 5 or 6 in 7 stages of Likert scale for all the elderly 7 persons.

https://doi.org/10.1109/SSCI.2018.8628880

Knauf, Rainer;
Knowledge engineering of system refinement what we learnt from software engineering. - In: Synergies Between Knowledge Engineering and Software Engineering, (2018), S. 93-105

Formal methods are a usual means to avoid errors or bugs in the development, adjustment and maintenance of both software and knowledge bases. This chapter provides a formal method to refine a knowledge base based on insides about its correctness derived from its use in practice. The objective of this refinement technique is to overcome particular invalidities revealed by the application of a case-oriented validation technology, i.e. it is some kind of "learning by examples". Approaches from AI or Data Mining to solve such problems are often not useful for a system refinement that aims at is an appropriate modeling of the domain knowledge in way humans would express that, too. Moreover, they often lead to a knowledge base which is difficult to to interpret, because it is too far from a natural way to express domain knowledge. The refinement process presented here is characterized by (1) using human expertise that also is a product of the validation technique and (2) keeping as much as possible of the original humanmade knowledge base. At least the second principle is pretty much adopted from Software Engineering. This chapter provides a brief introduction to AI rule base refinement approaches so far as well as an introduction to a validation and refinement framework for rulebased systems. It also states some basic principles for system refinement, which are adopted from Software Engineering. The next section introduces a refinement approach based on these principles. Moreover, it considers this approach from the perspective of the principles. Finally, some more general conclusions for the development, employment, and refinement of complex systems are drawn. The developed technology covers five steps : (1) test case generation, (2) test case experimentation, (3) evaluation, (4) validity assessment, and (5) system refinement. These steps can be performed iteratively, where the process can be conducted again after the improvements have been made.

https://doi.org/10.1007/978-3-319-64161-4_5

Kurashige, Kentarou; Tsuruta, Setsuo; Sakurai, Eriko; Sakurai, Yoshitaka; Knauf, Rainer; Damiani, Ernesto
Design of counseling robot for production by 3D printer. - In: 13th International Conference on Signal-Image Technology and Internet-Based Systems, (2017), S. 56-62

Nowadays, a lot of IT personnel have psychological distress. Meanwhile, counselors to help them are lack in number. To solve the problem, we proposed a counseling agent (CA) called CRECA (context respectful counseling agent). CRECA listens to clients and promotes their reflection context respectfully, namely in a context preserving way. This agent can be enhanced using a body language called "unazuki" in Japanese, a kind of "nodding" to greatly promote dialogue, often accompanying "un-un" (meaning "exactly") of Japanese onomatopoeia. This body language is expected to significantly help represent empathy or entire approval. In this paper, the agent is integrated with such a "unazuki" or "dialog promotion nodding" robot to continue the conversation naturally or context respectfully towards clients further reflection. To realize such "unazuki", the robot nods twice at each end of dialog sentence input by clients. Here, we introduce our newly developed robot that behaves human-like by an appropriate nodding behavior. The main motivation for developing a more human-like robot was the extension of application fields from IT workers' counselling to people, who suffer from more social problems such as financial debt, or anxiety of victory or defeat. For such applications, it is often very important that the agent behaves as much as possible human-like. Finally, we present the experimental evaluation results that proves such nodding is effective in counseling.

https://doi.org/10.1109/SITIS.2017.20

Kubota, Yoshihiko; Tsuruta, Setsuo; Muranushi, Takayuki; Hada-Maranushi, Yuko; Knauf, Rainer; Sakurai, Yoshitaka; Damiani, Ernesto
Extending the SVM integration with case based restarting GA to predict solar flare. - In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), ISBN 978-1-5386-1645-1, (2017), S. 1994-1999

The world and human life can be had serious impact by Unusual high solar flare. In particular, serious problems may be caused by destroy satellites, damage electric power plants due to it. Therefore, it is desirable to predict sun flare peaks. However, such prediction is difficult and often inaccurate. In previous work, we used a Support Vector Machine (SVM) as a machine learning technique to predict the sun flare intensity based on data of the past. Later, we extended this SVM by a Case Based Genetic Algorithm, because such prediction turned out to be a combinatorial challenge of patterns. Since GAs are known for a tendency to fall into local optima with no further improvement, we developed a strategy of increasing the mutation rate and/or introduced restarts, whenever such stagnation occurred. The focus of the present paper is a further extension by a technology called UFCORIN along with experimental work to estimate optimal parameters.

https://doi.org/10.1109/SMC.2017.8122911

Kurashige, Kentarou; Tsuruta, Setsuo; Sakurai, Eriko; Sakurai, Yoshitaka; Knauf, Rainer; Damiani, Ernesto
Context respectful counseling agent integrated with robot nodding for dialog promotion. - In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), ISBN 978-1-5386-1645-1, (2017), S. 1540-1545

Nowadays, a lot of IT personnel have psychological distress. Meanwhile, counselors to help them are lack in number. To solve the problem, we proposed a counseling agent (CA) called CRECA (context respectful counseling agent). CRECA listens to clients and promotes their reflection context respectfully namely in a context preserving way. This agent can be enhanced using a body language called "unazuki" in Japanese, a kind of "nodding" to greatly promote dialogue, often accompanying "un-un" (meaning "exactly") of Japanese onomatopoeia. This body language is expected to significantly help represent empathy or entire approval. In this paper, the agent is integrated with such a "unazuki" or "dialog promotion nodding" robot to continue the conversation naturally or context respectfully towards clients' further reflection. To realize such "unazuki", the robot nods twice at each end of dialog sentence input by clients. The experimental evaluation proves such nodding is effective in counseling.

https://doi.org/10.1109/SMC.2017.8122833

Shinozaki, Tetsuo; Yamamoto, Yukiko; Tsuruta, Setsuo; Sakurai, Yoshitaka; Damiani, Ernesto; Knauf, Rainer
Goal aware context respectful counseling agent. - In: 12th International Conference on Signal Image Technology & Internet-Based Systems : SITIS 2016 : 28 November-1 December 2016 : Naples, Italy : proceedings, ISBN 978-1-5090-5698-9, (2016), S. 252-257

A lot of IT workers suffer from stress in doing their work and there are a few counselors to help them. To cope with this problem, an enhanced context respectful counseling agent is proposed. This counseling agent extracts emotional words from clients' utterances throughout their dialogue to detect their changes and provides clients with such changes as dialogue summary. If no change is detected, it replies with clients' utterances' paraphrases followed by context-respectful prompts to dig/narrow problems. Especially to continue the conversation towards clients' further reflection, our newly enhanced agent restricts or relates the topics with eventual and emotional words in dialogue sentences. Namely, it asks still context-respectful but self-disclosure type questions to understand the distressed client more correctly or more sincerely. For supporting the clients' awareness of a method to achieve the goal or wish lingualized in conversation, the newly proposed counseling agent interacts by context-respectfully asking mental difficulties to achieve wish or goal. This promotes clients' reflection to obtain a broader view and thus conversation time for problem solving can be shorten. This introduces clients' more speedy awareness and enables the decrease of clients' mental and physical burden. Owing to this enhancement, clients keeping and deepening reflection on themselves without boring in conversation, reach more problem clarification and self-awareness, which enables them to solve their more difficult problems.

https://doi.org/10.1109/SITIS.2016.48

Yamamoto, Yukiko; Kawabe, Takashi; Tsuruta, Setsuo; Damiani, Ernesto; Yoshitaka, Atsuo; Mizuno, Yoshiyuki; Sakurai, Yoshitaka; Knauf, Rainer
Enhanced IoT-aware online shopping system. - In: 12th International Conference on Signal Image Technology & Internet-Based Systems : SITIS 2016 : 28 November-1 December 2016 : Naples, Italy : proceedings, ISBN 978-1-5090-5698-9, (2016), S. 31-35

In online business, it is important to construct sale web pages offering attractive services for popular products in order to improve page access as well as purchase rates. Moreover, online shop owners need to hold various types of sales frequently throughout the year to keep customers coming back. Also, online shopping systems have to adapt to its circumstances such as customers' needs and the surrounding economic situations. To cope with this, a self-organized IoT aware system is proposed. Here, awareness is achieved by monitoring / analyzing the data of user's behavior such as gazing, listening, smelling etc. The focus here is the analysis of such as users' eye gaze, listening /smelling activity etc. The aim is to derive insights about products the user is interested in or not and to adapt the system accordingly. This is achieved by presenting product information including sales associates' presence, their voice, and product's smell or even other products with similar attributes, by hiding products with less interesting attributes and by building a user's interest model through integrating all the positive preferences.

https://doi.org/10.1109/SITIS.2016.14

Yamamoto, Yukiko; Itoh, Daichi; Tsuruta, Setsuo; Muranushi, Takayuki; Hada-Muranushi, Yuko; Kobashi, Syoji; Mizuno, Yoshiyuki; Knauf, Rainer
SVM integrated case based restarting GA for further improving solar flare prediction. - In: 2016 IEEE International Conference on Systems, Man, and Cybernetics, (2016), S. 004716-004723

Solar activity has various influences on the global environment. Specifically, it may have serious impacts on the Earth such as satellite damage, etc. and power plant failures causing more serious disaster. For a precise forecast of larger scale solar flares causing serious disaster, it is important to improve the space weather forecast, a daily forecast of the solar flare. In our work so far, a machine-learning algorithm called Support Vector Machine (SVM) was used. We extended this technology by integrating Case Based Genetic Algorithm (CBGA) for a more precise forecast. It was shown experimentally that triple mutation rate on the slowdown of evolution in our CBGA improves considerably (e.g. another 5%) more than original mutation rate in the True Skill Statistics TSS. For further obtaining the optimality towards more imbalanced data analysis applicable to the recognition of serious disaster or medical disease, Restart CBGA is proposed with its expected effect. Here GA integrating SVM is restarted using highly optimized but diversified solutions in the case base as initial individuals. Further this restart CBGA is repetitively and evolutionary performed, evolving and maintaining the case base by the result of each (restarted) GA.

https://doi.org/10.1109/SMC.2016.7844976

Yamamoto, Yukiko; Tsuruta, Setsuo; Muranushi, Takayuki; Hada-Muranushi, Yuko; Kobashi, Syoji; Mizuno, Yoshiyuki; Knauf, Rainer
Solar flare prediction by SVM integrated GA. - In: 2016 IEEE Congress on Evolutionary Computation (CEC), ISBN 978-1-5090-0623-6, (2016), S. 4127-4134

Solar flare has various influences on the global environment, in particular on the magnetic storm and the likelihood of natural disasters. Specifically, it may have serious impacts on the Earth such as failure of satellite communication and navigation (GPS), satellite damage, increased radiation exposure to astronauts, geomagnetic storm and aurora, and power plant failures causing more serious disaster. For a precise forecast of larger scale solar flares causing serious disaster, it is important to improve the space weather forecast, which is basically a daily forecast of the solar flare. In the work so far, a machine-learning algorithm called Support Vector Machine (SVM) was used to forecast the space weather. We extend this technology by integrating Genetic Algorithm (GA) elaborately combined with Case Based Reasoning for more precise forecast or imbalanced data classification. Finally, basic evaluation of this architectural idea called CBGALO shows it is promising in improving solar flare prediction.

http://dx.doi.org/10.1109/CEC.2016.7744314

Yamamoto, Yukiko; Itoh, Daichi; Tsuruta, Setsuo; Muranushi, Takayuki; Hada-Muranushi, Yuko; Kobashi, Syoji; Mizuno, Yoshiyuki; Knauf, Rainer
Solar flare prediction by SVM integrated CBGA with dynamic mutation rate. - In: 2016 World Automation Congress (WAC), ISBN 978-1-889335-51-3, (2016), insges. 7 S.

http://dx.doi.org/10.1109/WAC.2016.7583029

Yamamoto, Yukiko; Shinozaki, Tetsuo; Tsuruta, Setsuo; Kurashige, Kentarou; Knauf, Rainer
Nodding behavioral context respectful counseling agent. - In: 2016 World Automation Congress (WAC), ISBN 978-1-889335-51-3, (2016), insges. 6 S.

http://dx.doi.org/10.1109/WAC.2016.7583027

Yamamoto, Yukiko; Kawabe, Takashi; Kobayashi, Yuuki; Tsuruta, Setsuo; Sakurai, Yoshitaka; Knauf, Rainer
A refined case based genetic algorithm for intelligent route optimization. - In: 11th International Conference on Signal-Image Technology and Internet-Based Systems, ISBN 978-1-4673-9721-6, (2015), S. 698-704

http://dx.doi.org/10.1109/SITIS.2015.36

Yamamoto, Yukiko; Kawabe, Takashi; Tsuruta, Setsuo; Damiani, Ernesto; Yoshitaka, Atsuo; Mizuno, Yoshiyuki; Knauf, Rainer
Towards self-organizing Internet of Things - aware systems for online sales. - In: 11th International Conference on Signal-Image Technology and Internet-Based Systems, ISBN 978-1-4673-9721-6, (2015), S. 208-215

http://dx.doi.org/10.1109/SITIS.2015.85

Kawabe, Takashi; Namihira, Yoshimi; Suzuki, Kouta; Nara, Munehiro; Yamamoto, Yukiko; Tsuruta, Setsuo; Knauf, Rainer
A part-of-speech based sentiment classification method considering subject-predicate relation. - In: Big data analytics for human-centric systems, ISBN 978-1-4799-8697-2, (2015), S. 999-1004

Based on the topic and opinion classification, a tweet credibility analysis method is proposed to detect false information or rumors spreading on Twitter on and after the Great East Japan Earthquake. The credibility is assessed by calculating the ratio of the same opinions to all opinions about a topic identified by topic models generated using Latent Dirichlet Allocation. To identify an opinion (positive or negative) expressed in a tweet, a sentiment analysis is performed using a semantic orientation dictionary. However, the accuracy is a problem to identify the few false tweets. The accuracy of the originally proposed method was susceptible since the sentiment opinion of most tweets was identified negative by the baseline (namely Takamura's) semantic orientation dictionary. Furthermore, specialty namely expertise of users was not considered. To cope with these problems, a method for extracting sentiment orientations of words and phrases was proposed considering user's specialty / expertise degree or mark to each of the same / opposite opinion tweets. The effects of both improvements are proven by experiments using a large number of real tweets. Namely, in these experiments rumor tweets were detected more accurately.

http://dx.doi.org/10.1109/SMC.2015.181

Fujikawa, Hiroshi; Yamaki, Hirofumi; Yamamoto, Yukiko; Tsuruta, Setsuo; Knauf, Rainer; Damiani, Ernesto
Evaluation of method for multiplexing communication routes to avoid intentional barriers. - In: Big data analytics for human-centric systems, ISBN 978-1-4799-8697-2, (2015), S. 815-820

It is common to operate an IT system where client computers in offices in a country access cloud computers in another country via the Internet. However, in some countries including China, network communication is often shut down by governmental bodies, in addition to network outage caused by network attacks. In case of such intentional interruptions, users need countermeasures to avoid them. Here, we propose a method to form bypass routes which consist of application-level gateways and intelligent routers placed at offices where client computers run, to select bypass routes based on the Internet status. A method for applying asymmetric criteria to decide whether to apply bypass routes is proposed for robust operation of Internetbased applications. Differential values of network latency are used for detecting intentional barriers through monitoring and analyzing the huge amount of temporal data, and absolute values to determine their ends. Such temporal data analysis knowledge is verified by a network simulator.

http://dx.doi.org/10.1109/SMC.2015.151

Kawabe, Takashi; Kobayashi, Yuuta; Tsuruta, Setsuo; Sakurai, Yoshitaka; Knauf, Rainer
Case based human oriented delivery route optimization. - In: IEEE Congress on Evolutionary Computation (CEC), 2015, ISBN 978-1-4799-7493-1, (2015), S. 2368-2375

Delivery route optimization is a well-known NPcomplete problem based on the Traveling Salesman Problem (TSP) involving 20-2000 cities though human oriented factors make the problem more complex. Despite of NP-completeness, the scheduling should be solved every time within interactive response time and below expert level error or local optimality, considering human oriented factors including personal, social, and cultural factors. To cope with this, Cases and NI (Nearest Insertion) are introduced into a Genetic Algorithm (GA), based on the insight that real problems are similar to previous ones. A solution can be derived from former solutions, considering human oriented factors as follows: (1) retrieving the most similar cases, (2) modifying them by removing and adding locations by NI, and (3) further optimizing them by a GA using only NI operations. This cannot only diminish the costs to compute new solutions from scratch but also inherit many parts of previous routes to respect human factors. Experimental evaluation revealed remarkable results. Though the most effective TSP solving method LKH needed more than 3 seconds, the proposed method yielded results within 3% of the worst error rate and in less than 3 seconds. Furthermore, the proposed method is able to inherit most of the delivery routes, while LKH leads to significant changes.

http://dx.doi.org/10.1109/CEC.2015.7257178

Kawabe, Takashi; Suzuki, Masaki; Matsumaru, Taro; Yamamoto, Yukiko; Tsuruta, Setsuo; Sakurai, Yoshitaka; Knauf, Rainer
Distributed GAs with case-based initial populations for real-time solution of combinatorial problems. - In: 2014 IEEE Symposium on Evolving and Autonomous Learning Systems (EALS), ISBN 978-1-4799-4494-1, (2014), S. 95-101

http://dx.doi.org/10.1109/EALS.2014.7009509

Suzuki, Masaki; Tsuruta, Setsuo; Knauf, Rainer; Sakurai, Yoshitaka
Knowledge acquisition issues for intelligent route optimization by evolutionary computation. - In: IEEE Congress on Evolutionary Computation (CEC), 2014, ISBN 978-1-4799-1486-9, (2014), S. 3252-3257

The paper introduces a Knowledge Acquisition and Maintenance concept for a Case Based Approximation method to solve large scale Traveling Salesman Problems in a short time (around 3 seconds) with an error rate below 3 %. This method is based on the insight, that most solutions are very similar to solutions that have been created before. Thus, in many cases a solution can be derived from former solutions by (1) selecting a most similar TSP from a library of former TSP solutions, (2) removing the locations that are not part of the current TSP and (3) adding the missing locations of the current TSP by mutation, namely Nearest Insertion (NI). This way of creating solutions by Case Based Reasoning (CBR) avoids the computational costs to create new solutions from scratch.

http://dx.doi.org/10.1109/CEC.2014.6900415

Knauf, Rainer; Yamamoto, Yukiko; Sakurai, Yoshitaka; Kinshuk
Optimizing university curricula through correlation analysis. - In: International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), 2013, ISBN 978-1-4799-3212-2, (2013), S. 324-329

In this paper, we introduce a refined Educational Data Mining approach, which refrains from explicit learner modeling along with an evaluation concept. We use a Data Mining technology, which models students' learning characteristics by considering real data instead of deriving their characteristics explicitly. It aims at mining course characteristics similarities of former students' study traces and utilizing them to optimize curricula of current students based to their performance traits revealed by their educational history. This refined technology generates suggestions of personalized curricula. The technology includes an adaptation mechanism, which compares recent data with historical data to ensure that the similarity of mined characteristics follow the dynamic changes affecting curriculum (e.g., revision of course contents and materials, and changes in teachers, etc.). Finally, the paper shows some pre-validation results and approaches for a final validation.

http://dx.doi.org/10.1109/SITIS.2013.60

Suzuki, Masaki; Motomura, Takaaki; Tsuruta, Setsuo; Sakurai, Yoshitaka; Knauf, Rainer
An approach to consider diversity issues from a semantic point of view. - In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2013, ISBN 978-1-4799-0650-5, (2013), S. 1696-1701

In this paper, we discuss a semantic and application-driven approach to estimate diversity respectively similarity in Genetic Algorithms (GA) based on a relative distance. This diversity metric can used to decide, whether or not a new individual meets a requested degree of diversity. Furthermore, the trade-off between several versions of the metric and their computational complexity is discussed. Finally, the application of this metric and a formerly developed Backtrack- and Restart GA to solve the Travelling Salesman Problem under certain real time requirements is introduced along with experimental evaluation.

https://doi.org/10.1109/SMC.2013.292

Sakurai, Yoshitaka; Knauf, Rainer; Kawabe, Takashi; Tsuruta, Setsuo
Adaptive kansei search method using user's subjective criterion deviation. - In: Intelligent computer vision and image processing, ISBN 978-1-4666-3906-5, (2013), S. 14-26

Takada, Kohei; Miyazawa, Yuta; Yamamoto, Yukiko; Imada, Yosuke; Tsuruta, Setsuo; Knauf, Rainer
Curriculum optimization by correlation analysis and its validation. - In: Human-Computer Interaction and Knowledge Discovery in Complex, Unstructured, Big Data, (2013), S. 311-318

The paper introduces a refined Educational Data Mining approach, which refrains from explicit learner modeling along with an evaluation concept. The technology is a "lazy" Data Mining technology, which models students' learning characteristics by considering real data instead of deriving ("guessing") their characteristics explicitly. It aims at mining course characteristics similarities of former students' study traces and utilizing them to optimize curricula of current students based to their performance traits revealed by their educational history. This (compared to a former publication) refined technology generates suggestions of personalized curricula. The technology is supplemented by an adaptation mechanism, which compares recent data with historical data to ensure that the similarity of mined characteristics follow the dynamic changes affecting curriculum (e.g., revision of course contents and materials, and changes in teachers, etc.). Finally, the paper derives some refinement ideas for the evaluation method.

http://dx.doi.org/10.1007/978-3-642-39146-0_28

Knauf, Rainer; Kinshuk; Takada, Kouhei; Sakurai, Yoshitaka; Kawabe, Takashi; Tsuruta, Setsuo
Personalized and adaptive curriculum optimization based on a performance correlation analysis. - In: Eighth International Conference on Signal Image Technology and Internet Based Systems (SITIS), 2012, ISBN 978-0-7695-4911-8, (2012), S. 655-660

http://dx.doi.org/10.1109/SITIS.2012.99

Sakurai, Yoshitaka; Takada, Kouhei; Tsuruta, Setsuo; Knauf, Rainer
A case study on using data mining for university curricula. - In: IEEE 12th International Conference on Advanced Learning Technologies (ICALT), 2012, ISBN 978-0-7695-4702-2, (2012), S. 3-4

In former work, the authors developed a modeling system for university learning processes, which aims at evaluating and refining university curricula to reach an optimum of learning success in terms of best possible best possible grade point average (GPA). This is performed by applying an Educational Data Mining (EDM) technology to former students curricula and their degree of success (GPA) and thus, uncovering golden didactic knowledge for successful education. After a short introduction to this technology, we discuss the result of a practical application and draw conclusions. In particular, we could not obtain sufficient data to establish this kind of learner profiles. Therefore, we shifted strategy from an "eager" strategy of holding an explicit model towards a "lazy" strategy of mining with data, which is really available, holds empirically, and is not a result of "guesses" about the students' general characteristics. In particular, we utilize the educational history of the students and vocational ambitions for student modeling.

http://dx.doi.org/10.1109/ICALT.2012.212

Jantke, Klaus P.; Knauf, Rainer
Taxonomic concepts for storyboarding digital games for learning in context. - In: Proceedings of 4th International Conference on Computer Supported Education, ISBN 978-989-8565-07-5, (2012), S. 401-409

The design and employment of digital games for serious purposes such as learning has several prerequisites. Designing a game that affects human players effectively requires the anticipation of particular human game playing experiences. Recent digital games taxonomies provide the WHAT and storyboarding is the technology for determining the HOW of planning the manifold of potential affective experiences in digital game playing. Game-based learning needs storyboarding and storyboarding needs concepts of digital games taxonomies. The appropriate consolidation of taxonomies and storyboarding results in explicit media didactics in context.

akurai, Yoshitaka; Tsuruta, Setsuo; Knauf, Rainer
Success chances estimation of university curricula based on educational history, self-estimated intellectual traits and vocational ambitions. - In: 11th IEEE International Conference on Advanced Learning Technologies (ICALT), 2011, ISBN 978-1-61284-209-7, (2011), S. 476-478

http://dx.doi.org/10.1109/ICALT.2011.148

Sakurai, Yoshitaka; Knauf, Rainer; Kawabe, Takashi; Tsuruta, Setsuo
Adaptive kansei search method using user's subjective criterion deviation. - In: International journal of computer vision and image processing, ISSN 2155-6997, Bd. 1 (2011), 1, S. 14-26

Sensibility-vectors (kansei-vectors) are useful for retrieving objects like pictures, music, perfumes, and apparels on the Internet. The sensibility-vector is an array of values, each indicating a degree of feeling or impression represented as sensibility word or kansei word. However, even such an approach leaves a gap between user's subjective sensibility (image, feeling) value and the corresponding one stored in the database. This paper proposes a search method to automatically and adaptively decrease such gaps by estimating a subjective criterion deviation (SCD) of the user's search histories and fuzzy modeling. Conventional methods need tests and questionnaires beforehand to infer user's individual sensibility to his or her instinct or impression. The proposed method automatically decreases such gaps without users' burden caused by such conventional methods as requiring questionnaires. Moreover, this method reflects the dynamic changes in user's preferences. Namely, this method does not need to know user's preferences beforehand with questionnaires. An experiment was conducted by building and using a perfume search system. Experimental data results showed that the proposed method is effective.

Sakurai, Yoshitaka; Takada, Kouhei; Tsukamoto, Natsuki; Onoyama, Takashi; Knauf, Rainer; Tsuruta, Setsuo
A simple optimization method based on Backtrack and GA for delivery schedule. - In: IEEE Congress on Evolutionary Computation (CEC), 2011, ISBN 978-1-4244-7834-7, (2011), S. 2790-2797

http://dx.doi.org/10.1109/CEC.2011.5949968

Knauf, Rainer; Sakurai, Yoshitaka; Tsuruta, Setsuo
A data mining method for the optimization of learning processes. - In: Proceeding of the IRAST International Congress on Computer Applications and Computational Science (CACS 2010), ISBN 978-981-086846-8, (2010), S. 77-80

Sakurai, Yoshitaka; Kinshuk; Takada, Kouhei; Kawabe, Takashi; Knauf, Rainer; Tsuruta, Setsuo
Evaluation of enriched cyberspace for adaptive support of remote collaboration. - In: Sixth International Conference on Signal-Image Technology and Internet Based Systems (SITIS), 2010, ISBN 978-1-4244-9527-6, (2010), S. 225-232

http://dx.doi.org/10.1109/SITIS.2010.46

Sakurai, Yoshitaka; Kinshuk; Graf, Sabine; Zarypolla, Ardah; Takada, Kouhei; Knauf, Rainer; Tsuruta, Setsuo
Proposal and evaluation of adaptive multimedia utilization method for reliable web based collaboration. - In: Fifth International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), 2009, ISBN 978-1-4244-5740-3, (2010), S. 385-392

Due to the geographical distribution, different cognitive capacity, and different domain competency of workers or learners, many misunderstandings can occur during Web based remote collaboration, leading to inefficient discussions and undesired results. - To overcome these problems, ag Multimedia adaptive utilization framework for dependable web based collaboration is proposed and evaluated. This assesses situations of remote users through fusing information of multiple biological sensors and the related contexts such as user profiles. Transmitting and using such information, the system adaptively supports Web based remote collaboration by stressing, warning, and presenting keywords/ summaries in multimedia. - Then, effects of adaptive keyword/summary presentation are evaluated as to cognitive profiles of users such as short term memory capacity and associative learning skills when misunderstanding possibilities are estimated on the Web. These evaluations totally demonstrate the feasibility and usefulness of the proposed method.

http://dx.doi.org/10.1109/SITIS.2009.67

Knauf, Rainer; Sakurai, Yoshitaka; Tsuruta, Setsuo; Takada, Kouhei
Empirical evaluation of a data mining method for success chance estimation of University Curricula. - In: IEEE International Conference on Systems, Man and Cybernetics (SMC), 2010, ISBN 978-1-4244-6586-6, (2010), S. 1127-1133

The paper deals with modeling, processing, evaluating and refining processes with humans involved like (not only, but also e-) learning. A formerly developed concept called storyboarding has been applied at a university to model the various ways to study at this university. Along with this storyboard, we developed a data mining technology to estimate success chances of curricula. Here, we introduce a validation method for this technology and its results. Further, we discuss chances to improve these results by implementing a formerly introduced learner profiling concept that represents the students' individual properties, talents and preferences for personalized data mining.

http://dx.doi.org/10.1109/ICSMC.2010.5642356

Knauf, Rainer; Sakurai, Yoshitaka; Takada, Kohei; Tsuruta, Setsuo
Personalizing learning processes by data mining. - In: IEEE 10th International Conference on Advanced Learning Technologies (ICALT), 2010, (2010), S. 488-492

A modeling approach for learning processes is utilized to process, evaluate and refine them. A formerly-developed concept called storyboarding has been applied at Tokyo Denki University (TDU) to model the various curricula for students to progress in their studies. Along with this particular storyboard, we developed a data mining technology to estimate chances for success for the students following each curricular path. This paper introduces a concept of personalized data mining by learner profiling. The learner profile represents the students individual properties, talents and preferences constructed through mining personal log data

http://dx.doi.org/10.1109/ICALT.2010.141

Batarseh, Feras; Gonzalez, Avelino J.; Knauf, Rainer;
Validation of knowledge-based systems through CommonKADS. - In: Crossing borders within the ABC, (2010), S. 577-582

This paper defines a method that can be used for validating knowledge-based systems (KBS) throughout their entire lifecycle. Method's name is MAVERICK. It stands for Method for Automated Validation Embedded into the Reusable and Incremental CommonKADS. The lack of suitable, rigorous and general validation methods has become a serious obstacle to user acceptance of KBS for critical applications. In spite of recent significant advances in validation of KBS, it still remains an open problem. The ideas presented in this paper are based on the concept that validation should be performed in a structured and guided manner, integrated within a knowledge-based systems' lifecycle development method.. We define an incremental validation method for KBS based on extracting test cases from CommonKADS. Furthermore, we introduce our method for reducing the number of test cases and thus reducing validation's effort and cost.

http://www.db-thueringen.de/servlets/DocumentServlet?id=17166

Sakurai, Yoshitaka; Kinshuk; Takada, Kouhei; Knauf, Rainer; Tsuruta, Setsuo
Enriched cyberspace through adaptive multimedia utilization for dependable remote collaboration. - In: IEEE 24th International Conference on Advanced Information Networking and Applications workshops (WAINA), 2010, ISBN 978-1-4244-6701-3, (2010), S. 61-67

Due to the geographical distribution, different cognitive capacity, and different domain competency of workers or learners, many misunderstandings can occur during distributed remote collaboration, leading to inefficient discussions and undesired results. To make remote collaboration more efficient and dependable, enriching cyberspace through adaptively utilizing is proposed and evaluated. This assesses situations of remote users through information fusion of multiple biological sensors and the related contexts such as user profiles. Transmitting and using such information, the system adaptively supports the distributed remote collaboration by stressing, warning, and presenting keywords/summaries in multimedia. Effects of presenting keywords/summaries adaptively depending on situations and cognitive profiles of remote members are evaluated as to the decrease of not-/misunderstanding possibilities during the explanation on the Cyberspace. The evaluation demonstrates the feasibility and usefulness of the proposed method.

http://dx.doi.org/10.1109/WAINA.2010.106

Knauf, Rainer; Sakurai, Yoshitaka; Tsuruta, Setsuo
Personalization in learning by knowledge engineering with didactic knowledge. - Online-Ressource (PDF-Datei: 9 S., 295,6 KB)Publ. entstand im Rahmen der Veranst.: 7th Conference on Computer Methods and Systems (CMS '09), November 26 - 27, 2009, Krakow, Poland

The paper proposes an approach to model, process, evaluate and refine learning processes. A formerly-developed concept to visualize learning paths called storyboarding has been applied at Tokyo Denki University (TDU) to model the various curricula for students to progress in their studies at this university. Along with this storyboard, we developed a data mining technology to estimate chances for success for the students following each curricular path. This paper introduces a concept (we call "personalized data mining") of learner profiling. This learner profile represents the students' individual properties, talents and preferences constructed through mining personal log data.

http://www.db-thueringen.de/servlets/DocumentServlet?id=15577

Knauf, Rainer; Sakurai, Yoshitaka; Tsuruta, Setsuo
Knowledge engineering technologies for learning processes. - Online-Ressource (PDF-Datei: 10 S., 295,6 KB)CD-ROM-Ausg.: Education and technology for a better world : 9th WCCE, IFIP World Conference on Computers in Education, Bento Gon¸calves, Brazil, 27 - 31 July, 2009 ; proceedings / [ed.: Ian Setwood ...] - [S.l.] : WCCE, 2009. - ISBN 978-3-901882-35-7, insges. 10 S.

Generally, learning systems suffer from a lack of an explicit and adaptable didactic design. Since e-learning systems are digital by their very nature, their introduction rises the issue of modeling the didactic design in a way that implies the chance to apply AI Techniques. A previously introduced modeling approach called storyboarding is setting the stage to apply Knowledge Engineering Technologies to verify and validate the didactics learning processes. Moreover, didactics can be refined according to revealed weaknesses and proven excellence. Successful didactic patterns can be explored by applying Mining techniques to the various ways students went through the storyboard and their associated level of success.

http://www.db-thueringen.de/servlets/DocumentServlet?id=15579

Chiang, Feng-Kuang; Wuttke, Heinz-Dietrich; Knauf, Rainer; Sun, Chung-Shan; Tso, Tai-Cheng
Students' attitudes toward using innovative information technology for learning based on theory of planned behavior. - In: International journal of advanced corporate learning, ISSN 1867-5565, Bd. 2 (2009), 4, S. 9-14

The main purpose of this study was to explore how IT is integrated into teaching and learning in a German bachelor curriculum, and to explore the university students' attitude toward the use of integration innovative information technology for learning in Germany. The data were collected by document collection and questionnaires. Multiple Regression Analysis was used to determine the three variables of behavioral intention. The results show that both attitude and subject norm are good predictors of IT for students' learning intention. Finally, results indicated that the theory of reasoned action predicts the use of innovative IT for learning intention well, though its impact is greater than the impact of the theory of planned behavior.

http://dx.doi.org/10.3991/ijac.v2i4.969

Knauf, Rainer; Böck, Ronald; Sakurai, Yoshitaka; Tsuruta, Setsuo
Knowledge engineering with didactic knowledge : first steps towards an ultimate goal. - Online-Ressource (PDF-Datei: 2 S., 860,3 KB)Druckausg.: Proceedings of the Twenty-Second International Florida Artificial Intelligence Research Society Conference : 19 - 21 May 2009, Sanibel Island, Florida, USA / ed. by H. Chad Lane ... - Menlo Park, Calif. : AAAI Press, 2009. - ISBN 978-1-577-35419-2, S. 418-419

Generally, learning systems suffer from a lack of an explicit and adaptable didactic design. A previously introduced modeling approach called storyboarding is setting the stage to apply Knowledge Engineering Technologies to verify and validate the didactics behind a learning process. Moreover, didactics can be refined according to revealed weaknesses and proven excellence. Successful didactic patterns can be explored by applying mining techniques to the various ways students went through the storyboard and their associated level of success.

http://www.db-thueringen.de/servlets/DocumentServlet?id=13147

Knauf, Rainer; Böck, Ronald; Sakurai, Yoshitaka; Dohi, Shinichi; Tsuruta, Setsuo
Knowledge mining for supporting learning processes. - In: IEEE International Conference on Systems, Man and Cybernetics, 2008, (2008), S. 2615-2621

AI technologies for knowledge mining are commonly used in technical environments. Their application for social processes like learning processes, for example, is a quite a new challenge, which is characterized by having "humans in the loop". Humans' desires, preferences and decisions may be unpredictable and thus, not appropriate for modeling - at a first glance. However, in learning processes didactic variants can be anticipated and can become a subject of AI technologies. A semi-formal modeling approach called storyboarding, is outlined here. A storyboard represents various opportunities for composing a learning process according to individual circumstances, such as topical prerequisites (educational history), mental prerequisites (preferred learning styles, etc.), performance prerequisites (a requested success level in former learning activities, etc.), and personal aspects (needs, wishes, talents, aims). By storyboarding, various didactic variants can be validated by considering the average learning success associated with the different paths through a storyboard in a case study. Based on validation results, success chances can be derived for the different paths. Here, a concept and an implementation to pre-estimate success chances of intended (future) learning paths through a storyboard are introduced. They are based on a Data Mining technology, and construct a decision tree by analyzing former learners' paths and their degrees of success. Furthermore, this technology generates a supplement to a submitted path, which is optimal according to the success chances. This technology has been tested at a Japanese university, in which students had to compose their individual plan (subject sequences) in advance, and the technology helped them by predicting success chances and suggesting alternatives.

http://dx.doi.org/10.1109/ICSMC.2008.4811690

Chiang, Feng-Kuang; Wuttke, Heinz-Dietrich; Knauf, Rainer; Shih, Chun-Ming
Information technology for learning: a European students' perspective. - Ilmenau : Univ.-Bibliothek. - Online-Ressource (PDF-Datei: 6 S., 549,1 KB)Druckausg.: E-Learn 2008 : World Conference on E-Learning in Corporate, Government, Healthcare & Higher Education ; November 17 - 21, 2008, Las Vegas, Nevada, USA / AACE, Association for the Advancement of Computing in Education. Ed. by Curtis J. Bonk ... ; Vol. 4. - Chesapeake, VA : Assoc. for the Advancement of Computing in Education (AACE), 2008. - S. 2566-2571

This study was mainly focused on European university students, in particular on their thinking about IT for learning. Participators came from different European countries and took part in a summer school this year. A total of 21 international students participated in this summer school program. They came from different European countries. The data collection includes a questionnaire survey and interview data. The data were analyzed and common opinions have been extracted. Finally, the authors of the present paper provide six conclusions and refer to university education in European countries to implement an IT integrated instruction. Researchers hope to promote the instructional quality by a well-performed way of IT integration into instruction in high education.

http://www.db-thueringen.de/servlets/DocumentServlet?id=11910

Knauf, Rainer; Sakurai, Yoshitaka; Tsuruta, Setsuo
Applying knowledge engineering methods to didactic knowledge : first steps towards an ultimate goal. - In: IEEE International Joint Conference on Neural Networks, 2008, ISBN 978-1-4244-1820-6, (2008), S. 38-45

Generally, learning systems suffer from a lack of an explicit and adaptable didactic design. Since E-Learning systems are digital by their very nature, their introduction rises the issue of modeling the didactic design in a way that implies the chance to apply Knowledge Engineering Techniques (like Machine Learning and Data Mining). A modeling approach called storyboarding, is outlined here. Storyboarding is setting the stage to apply Knowledge Engineering Technologies to verify and validate the didactics behind a learning process. Moreover, didactics can be refined according to revealed weaknesses and proven excellence and successful didactic patterns can be inductively inferred by analyzing the particular knowledge processing and its alleged contribution to learning success.

http://dx.doi.org/10.1109/IJCNN.2008.4633764

Knauf, Rainer; Böck, Ronald; Sakurai, Yoshitaka; Tsuruta, Setsuo
A priori evaluation & refinement of curricula by data mining over storyboards. - 6 S. = 146,6 KB, TextDruckausg.: Proceedings of the Twenty-First International Florida Artificial Intelligence Research Society Conference : 15 - 17 May 2008 in Coconut Grove, Florida, USA / ed. by David C. Wilson ... - Menlo Park, Calif. : AAAI Press, 2008. - ISBN 978-1-577-35365-2, S. 335-340

In university studies, there is a flexible but complicated learning system of subject offers, enrollment rules for particular subject combinations, and prerequisites to meet for taking particular subjects, which need to be matched with students' needs and desires. Students need assistance in the jungle of such learning opportunities and limitations at today's universities. To face this problem, we employed our formerly developed storyboard concept and used it to develop, maintain, and evaluate curricula. Storyboarding is based on the idea of formally representing, processing, evaluating and refining didactic knowledge. This concept is utilized to supplement an educational system called Dynamic Learning Needs Reflection System (DLNRS) of the School of Information Environment of Tokyo Denki University, Japan. Didactic knowledge of DLNRS can be represented by storyboarding and used for supporting dynamic learning activities of students. Here, we introduce an additional benefit of storyboarding. By using data mining-like methods to evaluate storyboard paths, we are able to estimate success chances of storyboard paths. Based on this evaluation we will be able to rate planned (future) paths and thus, to prevent students from failing by non-appropriate curricula. Moreover, besides the evaluation, the estimation can be used for computer enforced suggestions to complete a path towards optimal success chances.

http://www.db-thueringen.de/servlets/DocumentServlet?id=10755

Knauf, Rainer; Böck, Ronald; Sakurai, Yoshitaka; Dohi, Shinichi; Tsuruta, Setsuo
Using storyboarding and data mining to estimate success chances of curricula. - 9 S. = 166,5 KB, TextPubl. entstand im Rahmen der Veranst.: Proceedings of IADIS International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2007), 7 - 9 December 2007, Algarve, Portugal / ed. by Kinshuk. - IADIS Press, 2007. - ISBN 978-972-8924-48-5, S. 235-242

In university studies, there is a flexible but complicated learning system of subject offers, enrollment rules for particular subject combinations, and prerequisites to meet for taking particular subjects, which need to be matched with students' needs and desires. Students need assistance in the jungle of such learning opportunities and limitations at today's universities. To face this problem, we employed our formerly developed storyboard concept and used it to develop, maintain, and evaluate curricula. Storyboarding is based on the idea of formally representing, processing, evaluating and refining didactic knowledge. This concept is utilized to supplement an educational system called Dynamic Learning Needs Reflection System (DLNRS) of the School of Information Environment of Tokyo Denki University, Japan. Concretely speaking, didactic knowledge of DLNRS can be represented by storyboarding and used for supporting dynamic learning activities of students. Here, we introduce an additional benefit of the storyboard concept. By using data mining - like methods to evaluate storyboard paths, we are able to estimate success chances of storyboard paths. Based on such an evaluation we will be able to rate planned (future) paths and thus, to prevent students from failing by non-appropriate curricula. Moreover, besides the evaluation, the estimation can be used for computer enforced suggestions to complete a path towards optimal success chances.

http://www.db-thueringen.de/servlets/DocumentServlet?id=9686

Knauf, Rainer;
Compiling experience into knowledge. - In: Proceedings of the Twentieth International Florida Artificial Intelligence Research Society conference, ISBN 978-1-577-35319-5, (2007), S. 550-551

Typical application fields of Knowledge Based Systems are a usually characterized by having human expertise as the only one source to specify their desired behavior. Their design, evaluation and refinement has to make effective use of this valuable source. After sketching the concept of collecting validation experience in a Validation Knowledge Base (VKB), the paper introduces an estimation of the significance of the cases collected in the VKB. A high significance signalizes that a VKB should not longer serve as a case-based source of external knowledge, but compiled into the Knowledge Base instead. A technology to compile well-selected cases into the Knowledge Base of the system under evaluation is shown.

Knauf, Rainer; Tsuruta, Setsuo; Sakurai, Yoshitaka
Toward knowledge engineering with didactic knowledge. - In: , (2007), S. 119-125

Learning systems suffer from a lack of an explicit and adaptable didactic design. A way to overcome such deficiencies is (semi-) formally representing the didactic design. A modeling approach, storyboarding, is outlined here. Storyboarding is setting the stage to apply Knowledge Engineering Technologies to verify, validate the didactics behind a learning process. As a vision, didactics can be refined according to revealed weaknesses and proven excellence. Furthermore, successful didactic patterns can be inductively inferred by analyzing the particular knowledge processing and its alleged contribution to learning success.

http://www.db-thueringen.de/servlets/DocumentServlet?id=8736

Knauf, Rainer; Tsuruta, Setsuo;
Toward reducing human involvement in validation of knowledge-based systems. - In: IEEE transactions on systems, man, and cybernetics, ISSN 1558-2426, Bd. 37 (2007), 1, S. 120-131

Human experts employed in validation exercises for knowledge-based systems often have limited time and availability. Furthermore, they often have different opinions from each other as well as from themselves over time. We address this situation by introducing the use of validation knowledge used in prior validation exercises for the same knowledge-based system. We present a Validation Knowledge Base (VKB) that is the collective best experience of several human experts. The VKB is constructed and maintained across various validation exercises, and its primary benefits are as follows: (a) more reliable validation results by incorporating external knowledge and (b) decreasing the experts' workload. We also present the concept of Validation Expert Software Agents (VESA) that represent a particular expert's knowledge. VESA is a software agent corresponding to a specific human expert. It models the validation knowledge and behavior of its human counterpart by analyzing similarities with the responses of other experts. After a learning period, it can be used to temporarily substitute for its corresponding human expert. We also describe experiments with a small prototype system to evaluate the usefulness of these concepts.

http://dx.doi.org/10.1109/TSMCA.2006.886365

Jantke, Klaus P.; Knauf, Rainer; Knauf, Rainer *1963-*;
Storyboarding for playful learning. - In: E-Learn 2006, (2006), S. 3174-3182

Currently, e-learning systems as well as learning environments are still suffering from a lack of explicit and adaptable didactic design - an issue of quality management. Students complain about insufficient adaptability of e-learning offers to the learners’ needs. This is especially the case in university teaching. University professors are not necessarily experts in the didactics of teaching their subject. As a way out of this dilemma the authors utilize their formerly developed storyboard concept. Storyboards are an explicit representation of a didactic design, which enjoys simplicity, clarity, visual appearance, and the chance for standardized development. Everybody may easily become a storyboard author - a step toward pedagogical engineering. By representing didactics explicitly, learners are supported to find their individually appropriate way. Besides this processing issue, storyboards have the potential to identify successful didactic patterns. This is, in fact, the vision of knowledge discovery in didactics. In the authors' experience, embedding topical content into game adventures appears to be a promising didactic mean when properly linked in a related didactic pattern. The success of the authors' approach is demonstrated by an application to one of the authors' real life courses.

Dohi, Shinichi; Knauf, RainerSakurai, Yoshitaka; Tsuruta, Setsuo
Managing academic education through dynamic storyboarding. - Online-Ressource (9 S. = 287,1 KB)CD-ROM-Ausg.: Proceedings of E-Learn 2006, World Conference on E-Learning in Corporate, Government, Healthcare & Higher Education : October 13. - 17., 2006, Waikiki Beach, Honolulu / T. Reeves & S. Yamashita (ed.). - Chesapeake, Va. : AACE, 2006. - ISBN 1-880094-60-6, S. 1611-1619

Complex long term learning activities may be exhausting, tiring and sometimes even frustrating. In high level education such as university studies, there is a system of offers, rules, requests and prerequisites, which need to be matched with students' needs and desires. University students need assistance in the jungle of opportunities and limitations at today's universities. Here, we employ our formerly developed storyboard concept to face this problem and introduce a storyboard to develop, maintain, and evaluate academic education. Storyboarding is based on the idea of formally representing, processing, evaluating and refining didactic knowledge. It is more powerful in managing education than general AI knowledge representations such as frames, because the syntax of storyboards is driven by the particular nature of didactic knowledge. The concept is a supplement to the educational system (called Dynamic Learning Needs Reflection System: DLNRS) of the School of Information Environment of Tokyo Denki University, Japan. Concretely speaking, the didactic knowledge of DLNRS can be represented by storyboard and used for supporting dynamic learning activities of students.

http://www.db-thueringen.de/servlets/DocumentServlet?id=7334

Harmon, Scott Y.; Knauf, Rainer; Barr, Valerie B.Hoffman, C. W. David;
Validation of human behavior representation. - Ilmenau : Univ.-Bibliothek. - 35 S. = 257,1 KB, TextPubl. entstand im Rahmen der Veranst.: Workshop on Foundations for Modeling and Simulation (M&S), Verification and Validation (V&V) in the 21st Century, 22. - 24. Oktober 2002, Laurel, Maryland (USA)

http://www.db-thueringen.de/servlets/DocumentServlet?id=6196

Knauf, Rainer; Tsuruta, Setsuo
A case-based approach to explore validation experience. - 7 S. = 230,7 KB, TextDruckausg.: Proceedings of the Nineteenth International Florida Artificial Intelligence Research Society Conference : [11 - 13 May 2006 in Melbourne Beach, Florida, USA] / ed. by Geoffrey C. J. Sutcliffe ... - Menlo Park, Calif. : AAAI Press, 2006. - ISBN 978-1-57735-261-7, S. 414-419

The success of Turing Test technologies for system validation depends on the quality of the human expertise behind the system. As an additional source of human experts' validation knowledge a Validation Knowledge base (VKB) and so called Validation Expert Software Agents (VESAs) revealed to be useful. Both concepts aim at using collective (VKB) and individual (VESA) experience gained in former validation sessions. However, a drawback of these concepts were their disability to provide a reply to cases, which have never been considered before. The paper proposes a case-based data mining approach to cluster the entries of VKB and VESA and derive a reply to unknown cases by considering a number of most similar known cases and coming to a "weighted majority" decision. The approach has been derived from the k Nearest-Neighbor approach.

http://www.db-thueringen.de/servlets/DocumentServlet?id=6195

Kelbassa, Hans-Werner; Knauf, Rainer
A process approach to rule base validation and refinement. - In: Knowledge engineering and software engineering, (2005), S. 25-36

Knauf, Rainer; Tsuruta, Setsuo;
Towards modeling human expertise : an empirical case study. - Ilmenau : Univ.-Bibliothek. - 6 S. = 145,4 KB, TextDruckausg.: Recent advances in artificial intelligence : proceedings of the eighteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS â05) : [May 15 - 17, 2005, Clearwater Beach, Florida] / ed. by Ingrid Russell ... - Menlo Park, Calif. : AAAI Press, 2005. - ISBN 1-57735-234-3. - S. 232-237

The success of Turing Test technologies for system validation depends on the quality of the human expertise behind the system. The authors developed models of collective and individual human expertise, which are shortly outlined here. The focus of the paper is an experimental work aimed at determining the quality of these models. The models have been used for both solving problem cases and rating (other agents') solutions to these cases. By comparing the models' solutions and ratings with those of the human original we derived assessments of their quality. An analysis revealed both the general usefulness and some particular weaknesses.

http://www.db-thueringen.de/servlets/DocumentServlet?id=4540

Knauf, Rainer; Tsuruta, Setsuo;
Overcoming human weaknesses in validation of knowledge-based systems. - In: Marktplatz Internet: von e-Learning bis e-Payment, (2005), S. 254-263

Human experts employed in validation exercises for knowledge-based systems often have limited time and availability. They often have different opinions from each other as well as from themselves over time. We address this situation by introducing the use of validation knowledge used in prior validation exercises for the same knowledge-based system. We present a Validation Knowledge Base (VKB) that is the collective best experience of several human expert validators. We also present the concept of Validation Expert Software Agents (VESA) that represent a particular expert's knowledge. VESA is a software agent corresponding to a specific human expert validator. It models the validation knowledge and behavior of its human counterpart by analyzing similarities with the responses of other experts. We also describe experiments with a small prototype system to evaluate the usefulness of these concepts.

Jantke, Klaus P.; Knauf, Rainer
Didactic design through storyboarding : standard concepts for standard tools. - In: Proceedings of the 4th International Symposium on Information and Communication Technologies, (2005), S. 20-25

The current state of affair in e-learning world-wide shows a reluctance to didactic˜design. Learners frequently complain and scientists discuss about insufficient adaptivity of e-learning offers to the learnersâ needs. Didactics is badly underestimated. - High quality didactic designs is seen as a crucial aspect of dissemination. E-learning content and services need to reach their audience properly. Learners with different prerequisites, with different needs, with different expectations and under varying context conditions have to be addressed appropriately. Didactic designs is seen as an issue of quality assurance in e-learning. - As well-known from quality management, high quality requirements and related measures towards quality assurance may turn out to be obstacles to dissemination, because quality may turn out to be expensive. The related answer are solutions frequently called quick and dirty. This does apply to e-learning as well. - The authors' own storyboard concept is introduced. Its reach goes far beyond the limits of current practices in e-learning systems and service development. The modeling concepts required are standard: annotated graphs. The software in use is standard as well: Visio. Emphasis is put on the investigation of how a suitable usage of the concepts allows for an expressive didactic design. - To sum up, the authors' intended contribution is twofold. First, they want to encourage didactic design through storyboarding in e-learning. Concepts are introduced and applications are demonstrated. Second, with the dissemination problem in mind, they want to show that concepts are crucial, but not tools. One can exploit advanced concepts toward sophisticated didactic design without an urgent need for costly software.

Knauf, Rainer; Tsuruta, Setsuo; Uehara, Kenichi; Onoyama, Takashi; Kurbad, Torsten
The power of experience : on the usefulness of validation knowledge. - 7 S. = 108,0 KB, TextDruckausg.: Proceedings of the seventeenth International Florida Artificial Intelligence Research Society conference : [Miami Beach, Florida, May 17 - 19, 2004] / ed. by Valerie Barr ... - Menlo Park, Calif. : AAAI Press, 2004, S. 337-342

Turing Test technologies are promising ways to validate AI systems which may have no alternative way to indicate validity. Human experts (validators) are often too expensive to involve. Furthermore, they often have different opinions from each other and from themselves over time. One way out of this situation is to employ a Validation Knowledge Base (VKB) which can be considered to be the collective experience of human expert panels. VKB is constructed and maintained across various validation sessions. Primary benefits are (1) decreasing the validators' workload and (2) refining the methodology itself. Additionally, there are some side effects that (1) improve the selection of an appropriate expert panel and (2) improve the identification of an optimal solution to a test cases. Furthermore, Validation Experts Software Agents (VESA) are introduced as an model of a particular expert's knowledge. VESA is a software agent corresponding to a human validator. It systematically models the validation knowledge and behavior of its human origin. After a learning period, it can be used to substitute the human exert.

http://www.db-thueringen.de/servlets/DocumentServlet?id=4537

Knauf, Rainer; Tsuruta, Setsuo; Ihara, Hirokazu; Gonzalez, Avelino J.; Kurbad, Torsten
Improving AI systems' dependability by utilizing historical knowledge. - In: Proceedings, (2004), S. 343-352
Parallel als Druckausg. erschienen

A Turing Test is a promising way to validate AI systems which usually have no way to proof correctness. However, human experts (validators) are often too busy to participate in it and sometimes have different opinions per person as well as per validation session. To cope with these and increase the validation dependability, a Validation Knowledge Base (VKB) in Turing Test - like validation is proposed. The VKB is constructed and maintained across various validation sessions. Primary benefits are (1) decreasing validators' workload, (2) refining the methodology itself, e.g. selecting dependable validators using VKB, and (3) increasing AI systems' dependabilities through dependable validation, e.g. support to identify optimal solutions. Finally, Validation Experts Software Agents (VESA) are introduced to further break limitations of human validator's dependability. Each VESA is a software agent corresponding to a particular human validator. This suggests the ability to systematically "construct" human-like validators by keeping personal validation knowledge per corresponding validator. This will bring a new dimension towards dependable AI systems.

http://dx.doi.org/10.1109/PRDC.2004.1276590

Knauf, Rainer; Tsuruta, Setsuo;
Models of collective and individual human expertise. - In: Session 4-10, (2004), S. 352-359

Turing Test technologies are promising ways to validate AI systems that may have no alternative way to indicate validity. Unfortunately, human experts are sometimes too expensive to be heavily involved in the Turing Test process. Furthermore, they often have different opinions from each other as well as from themselves over time. One way out of this situation is to employ a Validation Knowledge Base (VKB) that is the collective experience of human expert panels. VKB is constructed and maintained across various validation sessions. Furthermore, Validation Experts Software Agents (VESA) are introduced as a model of the individual experience of a particular human expert, which can serve as his/her substitute in case of the expert's unavailability. A Turing Test experiment with a small prototype system indicates the usefulness of these concepts to model the collective (VKB) and individual (VESA) validation expertise.

Kelbassa, Hans-Werner; Knauf, Rainer;
The rule retranslation problem and the validation interface. - 5 S. = 104,4 KB, TextDruckausg.: Proceedings of the sixteenth International Florida Artificial Intelligence Research Society conference : [St. Augustine] / ed. by Ingrid Russell ... - Menlo Park, Calif. : AAAI Press, 2003. - ISBN 1-57735-177-0. - S. 213-217

Current rule base maintenance is wasting refinement and inference performance. There are only few maintenance concepts, which enjoy both (1) formal rule refinement and (2) utilizing topical knowledge provided by experts within the refinement process. The current state of the art in rule base validation and refinement reveals that there is no generic validation interface and no optimal rule trace refinement. This paper characterizes two different retranslation approaches for reduced rule bases and proposes a two-step validation process, which combines a case-based approach with a rule trace validation approach.

http://www.db-thueringen.de/servlets/DocumentServlet?id=4530

Knauf, Rainer; Gonzalez, Avelino J.; Abel, Thomas
A framework for validation of rule-based systems. - In: IEEE transactions on systems, man, and cybernetics, ISSN 1941-0492, Bd. 32 (2002), 3, S. 281-295
Parallel als Druckausg. erschienen

http://dx.doi.org/10.1109/TSMCB.2002.999805

Knauf, Rainer; Philippow, Ilka; Gonzalez, Avelino J.; Jantke, Klaus P.
Towards validation of rule-based systems - the loop is closed. - Ilmenau : Univ.-Bibliothek. - 5 S. = 387,5 KB, TextDruckausg.: Proceedings of the thirteenth International Florida Artificial Intelligence Research Society Conference : [Orlando, Florida, 22 - 24 May 2000] (FLAIRS 2000) / ed. by Jim Etheredge and Bill Manaris. - Menlo Park, Calif : AAAI Press, 2000, S. 331-335

http://www.db-thueringen.de/servlets/DocumentServlet?id=4339

Knauf, Rainer; Philippow, Ilka; Gonzalez, Avelino J.; Jantke, Klaus P.
The control issue of AI systems considering the evolution of rule-based systems from a cybernetics point of view. - In: 45. Internationales Wissenschaftliches Kolloquium, (2000), S. 763-768

Gonzalez, Avelino J.; Murillo, Maureen; Knauf, Rainer;
Validating human behavior models. - In: 45. Internationales Wissenschaftliches Kolloquium, (2000), S. 751-756

Salecker, Dirk; Knauf, Rainer;
Validation manager : a tool for AI systems&softcy; evaluation by a turing test-like methodology. - 5 S. = 385 KB, TextDruckausg.: Proceedings of the twelfth International Forida AI Research Society Conference : Orlando, Florida, 3 - 5 May 1999 (FLAIRS 1999) / ed. by Amruth N. Kumar ... - Menlo Park, Calif. : AAAI Press, 1999, S. 531-535

http://www.db-thueringen.de/servlets/DocumentServlet?id=4335

Knauf, Rainer; Gonzalez, Avelino J.; Jantke, Klaus P.;
Validating rule-based systems : a complete methodology. - Ilmenau : Univ.-Bibliothek. - 6 S. = 821,9 KB, TextDruckausg.: IEEE SMC '99 conference proceedings / 1999 IEEE International Conference on Systems, Man, and Cybernetics : October 12- 15 1999, Tokyo. - Piscataway, NJ : IEEE Service Center, 1999, Vol. 5, S. 744-749

http://www.db-thueringen.de/servlets/DocumentServlet?id=4298

Michels, Jan-Eike; Abel, Thomas; Knauf, Rainer; Gonzalez, Avelino J.
Investigating the validity of a test case selection methodology for expert system validation. - Ilmenau : Univ.-Bibliothek. - 6 S. = 339,6 KB, TextDruckausg.: Proceedings of the Eleventh International Florida Artificial Intelligence Research Symposium Conference : Sanibel Island, Florida, 18 - 20 May 1998 (FLAIRS 1998) / ed. by Diane Cook. - Menlo Park, Calif. : AAAI Press, 1998, S. 462-466

http://www.db-thueringen.de/servlets/DocumentServlet?id=4301

Jantke, Klaus P.; Knauf, Rainer; Abel, Thomas
The TURING test approach to Validation. - In: Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, (1997), S. 35-45

Jantke, Klaus P.; Knauf, Rainer; Stephan, Andreas
Validation von Anwendungssoftware: von der Forschung zum Marktfaktor. - In: Tagungsbericht, (1997), S. 1-21

Knauf, Rainer; Philippow, Ilka; Gonzalez, Avelino J.
Towards an assessment of an AI system's validity by a turing test. - In: Proceedings of the Tenth International Florida Artificial Intelligence Research Symposium, (1997), S. 397-401

Knauf, Rainer; Jantke, Klaus P.; Abel, Thomas; Philippow, Ilka
Fundamentals of a TURING test approach to validation of AI systems. - In: [Vortragsreihen, (1997), S. 59-64

Herrmann, Jörg; Jantke, Klaus P.; Knauf, Rainer
Towards cost driven system validation. - In: [Vortragsreihen, (1997), S. 41-46

Abel, Thomas; Knauf, Rainer; Gonzalez, Avelino J.
Generation of a minimal set of test cases that is functionally equivalent to an exhaustive set, for use in knowledge-based system validation. - In: Proceedings of the Ninth Florida Artificial Intelligence Research Symposium, (1996), S. 280-284