Read an Excerpt
KBS First Prototype V&V Process Plan as a Way to Produce Reliable Requirements
Keywords: Knowledge Based Systems; Validation & Verification; Software Abstract: The Quality Assessment Process is in charge of assuring several
1. Introduction
The Quality Assessment Process is in charge of assuring several performances of the software Product and Processes. One of them is the Software Reliability, perhaps the closest concept to the more general of Quality [Gillies 92]. Then reliability is always associated to the lack of errors, and above all of critical consequences, but also to the precision in the system response. That is, speaking of reliability implies an idea of robustness of the system, of being hardly responsive to the environment. In all this, there are underlying concepts present in all the Software Quality Processes. However, this area does not only study the processes, Quality Assurance is always associated to the idea of Quality checking, tests and metrics, that is, setting when a product successfully complies with a series of predetermined requirements. Then the metrics are very linked to the products or subproducts derived from the software system.
One of the processes or rather concepts associated to the development of a software system as well as to the Quality Assessment are Validation and Verification [Cardenosa 94], that are in charge of methodising the early phases of the Quality and Reliability process during the development.
These processes also have to be integrated with the development processes of the product or software system and for this, they essentially depend on two factors. One is the development methodology that integrates the Validation and Verification activities (V&V on the following) in some moment of the development. The other factor is the type of product or subproduct that, after all, is determined by a phase of the life cycle of this one.
Among the software products lacking of a clear definition or with a product taxonomy that qualitatively differ, the Knowledge Based System (KBS on the following) development presents special difficulties when applying the V&V processes. KBS more generally means the systems that separate the knowledge from the processes that handle it (Knowledge Base and Inference Engine respectively). But contrary to the people who have identified these systems as systems whose knowledge is represented in some of its more frequent formalisms (production rules), it is necessary that the noalgorithmic knowledge comes from an expert or an experience-based knowledge body.
Then, these systems, increasingly present in applications in all the fields, whether integrated with other systems or "stand-alone", present a series of peculiarities that affect the knowledge they model. We can summarise them as follows: incompleteness, imprecision, apparent inconsistencies at least, and uncertainty [Cardenosa et al. 91]. These peculiarities are solved by the incorporation of heuristics whose validation is very awkward, without the systematic application of test cases.
This is one of the main reasons for the KBS development to be directed in many cases by a life cycle for prototypes so as to build initially a consistent knowledge core that can be modelled conceptually and formally [Boehm 88]. This approach of the life cycle aims to define a clear and consistent set of system requirements, question of great difficulty in a previous stage. As the definition of some requirements that assure the reliability of the whole system is essential, the V&V processes on this node are critical to assure that the system development is based on the Quality Assessment standard methods [O'Keefe et al. 87].
In the same way, as in the development of any software system, it is necessary to carry out a Quality Plan in which the V&V activities are included [Cardenosa et al. 96]. In many cases, when the thing developed is the core to obtain the reliable requirements or prototype, this Plan is not drawn up, that is, no systematic tests are applied to the system. This is often for saving resources and for estimating that the requirement specification methods imply a sufficient validation. In this paper, we aim to describe the set of V&V process activities applicable to a KBS core, that is, the essential components of a V&V Plan for this type of system, obviated most of the time but essential to assure the reliability of the system requirements, result of the prototyping phase. This Plan, even if more reduced and more directed towards the conceptual consistency of the knowledge and the user tests, is essential for the purpose commented before.
2. The V&V Process
2.1 Preliminary definitions and "State of the Art"
It is not the point of this paragraph to define concepts that, in some way, the reader already understands. The essential concept in this paper is the one of Verification and Validation that have been subject to the more diverse definitions with significant differences if they refer to conventional software systems or to KBS. By "conventional systems", we mean those in which the knowledge they contain is sufficiently complete and precise to assure the absence of errors. These systems are normally those that reflect knowledge related to systems invented by the man, that is, they obey pre-existing and known principles or their grounds are the so-called "first principles". By "no conventional systems", we mean those in which there is one or several characteristics described in before as uncertainty, imprecision, incompleteness or inconsistency, and that end by requiring specific methods of conceptual and formal representation, usually heuristic methods. A prototype of these systems is the KBS whose specificity has been under study for more than ten years. In the early 80s, the inherent necessities of these activities were raised in diverse forums on the Classic Software Engineering point of view. Then Wilburn [Wilburn 85] who followed, in some way, the more classic guidelines drew up a simple but clear definition of the V&V activities. In the late 80s, a North American project, the EVA project [Chang et al. 90], and another of the European Commission, the VALID project [VALID 89], [Cardenosa 95], dedicated resources to clarify the terminological and methodological questions of these V&V processes. We will not describe them in details but we can say that they did not really impose their taxonomy and definitions, and as a result, some time later, new projects on this subject were raised by the European Commission (all of them within the ESPRIT Information Technologies programme) like the VITAL [Rouge et al. 93] and the VIVA [Craw et al. 95], [VIVA 95]. The later was terminated recently, it is perhaps the clearest and most complete as for the term definitions as well as the methodological questions, and it is compatible with the European Space Agency Standards. The Verification process is associated to a series of activities to carry out during the different phases of the system development so as to check the absence of errors. Therefore, we can say in a very general way that, given that the whole system obeys user requirements and that, as a result, system requirements are defined, by Verification, we mean the set of activities aiming at checking that the system (products and sub products) is adapted to the system requirements (question that gathers the physical system correction as well as the lack of errors).
Likewise, by Validation, we mean the set of activities in charge of checking that the system is adapted to the environment and user requirements, being essential then the tests on the functional requirements among others [IEEE 1012], [IEEE 1008] and [O'Keefe et al. 87].
A specific term of this paper is that of the "core of a KBS". The idea of core is not strict but rather intuitive. This idea is also that of "first prototype". We can find some references as [O'Keefe et al. 90] [Gomaa 83]and [Bischofberger et al. 92], but the intuitive idea is easy to understand. A core, particularly in a KBS, is the minimum part of a Knowledge Base that can operate consistently and is able to solve a number of predetermined test cases. The objective is to determine if the study of the knowledge, its conceptual and formal modelling and even its basic implementation are adequate for the rest of the system. Other authors called it "experimental prototype" and it is more than present in the life cycle of this type of systems that integrate the idea of evolutional prototype with the Validation and Verification activities [Bohem 88], [Gomaa 83] and [O'Keefe et al. 90]. As this core is used to study it and check or evaluate its future effectiveness, on rare occasions a Validation Plan is drawn up, and then its examination is "ad-hoc" to the situation and often not very rigorous. An example of lack of rigour is when a prototype is developed having in view the requirement of a domain composed by a determined number of cases. Then the only V&V process consists of checking that the program solves correctly and exclusively these cases. The problem of this approach with respect to the V&V process is that generally the program solving these cases has been developed without a methodical analysis process, knowledge modelling nor other tasks proper to the development of a prototype. There is a system, it runs with these cases (and not others) and it works, but some things are lacking. This paper will propose a scheme of how to draw up a V&V Plan for this core so as to assure that the requirements obtained from the Prototype or Core are correct.
The idea of Quality Plan includes among other things the drawing-up of a V&V Plan that, on the other hand, is explicitly quoted on the standards [IEEE 1012] and [IEEE 730]. These plans are almost always applied on processes and products, not on prototypes. A prototype should not be considered as a product and not even as a sub-product with respect to Quality Assurance. By core, we mean an "embryo system " but that is not a reason for requiring less to this system. It is important to take into account that the core is the element on which the system can evolve and if the system is not complete, it cannot be tested on real environments. It is maybe why its inspection has to be especially careful...
