Software testing is a complex and critical process for achieving product quality. Its importance has been increasing and well recognized, and there is a growing concern in improving the accomplishment of this process. In this context, Knowledge Management (KM) emerged as an important supporting approach to improve the software testing process. However, managing relevant testing knowledge requires effective means to represent and to associate semantics to a large volume of testing information. To address this concern, we have developed a Reference Ontology on Software Testing (ROoST). ROoST establishes a common conceptualization about the software testing domain, which can serve several KM-related purposes, such as defining a common vocabulary for knowledge workers with respect to the testing domain, structuring testing knowledge repositories, annotating testing knowledge items, and for making search for relevant information easier. In this paper, we present ROoST, and we discuss how it was developed using two ontology pattern languages. Moreover, we discuss how we evaluated ROoST following four complementary approaches: assessment by humans, data-driven evaluation, ontology testing, and application-based evaluation.
Ammann, P. & Offutt, J. (2008). Introduction to Software Testing. Cambridge, UK: Cambridge University Press. doi:10.1017/CBO9780511809163.
2.
Anandaraj, A., Kalaivani, P. & Rameshkumar, V. (2011). Development of ontology-based intelligent system for software testing. International Journal of Communication, Computation and Innovation, 2, 157–161.
3.
Andrade, J., Ares, J., Martínez, M., Pazos, J., Rodríguez, S., Romera, J. & Suárez, S. (2013). An architectural model for software testing lesson learned systems. Information and Software Technology, 55, 18–34. doi:10.1016/j.infsof.2012.03.003.
4.
Arnicans, G., Romans, D. & Straujums, U. (2013). Semi-automatic generation of a software testing lightweight ontology from a glossary based on the ONTO6 methodology. In Frontiers in Artificial Intelligence and Applications (pp. 263–276).
5.
Arantes, L.O.R.A. & Falbo, L.O. (2010). An infrastructure for managing semantic document. In Proceedings 14th International Enterprise Distributed Object Computing Conference Workshops EDOCW 2010 (pp. 235–244). Los Alamitos: IEEE Computer Society. doi:10.1109/EDOCW.2010.17.
6.
Bai, X., Lee, S., Tsai, W. & Chen, Y. (2008). Ontology-based test modeling and partition testing of web services. In International Conf. on Web Services465–472.
7.
Barbosa, E.F., Nakagawa, E.Y. & Maldonado, J.C. (2006). Towards the establishment of an ontology of software testing. In International Conference on Software Engineering and Knowledge Engineering (SEKE), San Francisco, CA (Vol. 1, pp. 522–525).
8.
Barbosa, E.F., Nakagawa, E.Y., Riekstin, A.C. & Maldonado, J.C. (2008). Ontology-based development of testing related tools. In International Conference on Software Engineering & Knowledge Engineering (SEKE), San Francisco, CA.
9.
Barcelos, P.P.F., Santos, V.A., Silva, F.B., Monteiro, M.E. & Garcia, A.S. (2013). An automated transformation from OntoUML to OWL and SWRL. In The Ontology Research Seminar in Brazil (ONTOBRAS), Belo Horizonte, MG (pp. 130–141).
10.
Benjamins, V.R., Fensel, D. & Pérez, A.G. (1998). Knowledge management through ontologies. In The 2nd International Conference on Practical Aspects of Knowledge Management (PAKM), Switzerland.
11.
Black, R. & Mitchell, J.L. (2011). Advanced Software Testing: Guide to the ISTQB Advanced Certification as an Advanced Technical Test Analyst (3th ed.). USA: Rocky Nook.
12.
Bourque, P. & Fairley, R.E. (2014). Guide to the Software Engineering Body of Knowledge, SWEBOK, Version 3.0. Los Alamitos, CA, USA: IEEE Computer Society Press.
13.
Braga, G., Romano, B.L., Campos, H.F., Vieira, R., Cunha, A.M. & Dias, L.A.V. (2009). Integrating amazonic heterogeneous hydrometeorological databases. In Information Technology: New Generations. Sixth International Conference on, Las Vegas, NV (pp. 119–124).
14.
Brank, J., Grobelnik, M. & Mladenic, D. (2005). A survey of ontology evaluation techniques. In Proc. of 8th Int. Multi-Conf. Information Society (pp. 166–169).
15.
Cai, L., Tong, W., Liu, Z. & Zhang, J. (2009). Test case reuse based on ontology. In Pacific Rim International Symposium on Dependable Computing (pp. 103–108).
16.
d’Aquin, M. & Gangemi, A. (2011). Is there beauty in ontologies?Journal Applied Ontology, 6(3), 165–175.
17.
Everett, G.D. & Raymond, M.J. (2007). Software Testing Across the Entire Software Development Life Cycle (3th ed.). Canada: John Wiley & Sons.
18.
Falbo, R.A., Barcellos, M.P., Nardi, J.C. & Guizzardi, G. (2013). Organizing ontology design patterns as ontology pattern languages. In 10th Extended Semantic Web Conference, Montpellier, France.
19.
Falbo, R.A. (2014). SABiO: Systematic approach for building ontologies. In 1st Joint Workshop Onto.Com/ODISE on Ontologies in Conceptual Modeling and Information Systems Engineering, Rio de Janeiro.
20.
Falbo, R.A., Ruy, F.B., Guizzardi, G., Barcellos, M.P. & Almeida, J.P.A. (2014). Towards an enterprise ontology pattern language. In Symposium on Applied Computing, Gyeongju, Korea (pp. 323–330).
21.
Falbo, R.A., Menezes, C.S. & Rocha, A.R. (1998). A systematic approach for building ontologies. In 6th Ibero-American Conference on Artificial Intelligence, Lisbon, Portugal. Lecture Notes in Computer Science (pp. 349–360).
22.
Guizzardi, G. (2005). Ontological Foundations for Structural Conceptual Models. The Netherlands: Universal Press. ISBN 90-75176-81-3.
23.
Guizzardi, G. (2007). On ontology, ontologies, conceptualizations, modeling languages and (meta) models. In O.Vasilecas, J.Edler and A.Caplinskas (Eds.), Frontiers in Artificial Intelligence and Applications, Databases and Information Systems IV. Amsterdam: IOS Press.
24.
Guizzardi, G. & Halpin, T. (2008). Ontological foundations for structural conceptual models. Applied Ontology, 3(1–2), 1–12.
25.
Guizzardi, G., Falbo, R.A. & Guizzardi, R.S.S. (2008). Grounding software domain ontologies in the unified foundational ontology (UFO): The case of the ODE software process ontology. In XI Iberoamerican Workshop on Requirements Engineering and Software Environments, Recife, Brazil (pp. 244–251).
26.
Guizzardi, G., Wagner, G., Falbo, R.A., Guizzardi, R.S.S. & Almeida, J.P.A. (2013). Towards ontological foundations for the conceptual modeling of events. In 32th International Conference on Conceptual Modeling, Hong-Kong, China (pp. 327–341).
27.
Guo, S., Zhang, J., Tong, W. & Liu, Z. (2011). An application of ontology to test case reuse. In International Conference on Mechatronic Science, Electric Engineering and Computer, Jilin, China (pp. 19–22).
28.
Huo, Q., Zhu, H. & Greenwood, S. (2003). A multi-agent software environment for testing web-based applications. In 27th International Computer Software and Applications Conference (COMPSAC), Dallas, TX, USA (pp. 210–215).
29.
Hong, Z. (2006). A framework for service-oriented testing of web services. In Computer Software and Applications Conference. COMPSAC’06. 30th Annual International, Chicago, IL (pp. 145–150).
30.
IEEE (1990). The Institute of Electric and Electronic Engineers (IEEE): standard glossary of software engineering terminology. IEEE Standard 610.12-1990, New York, NY, USA.
31.
IEEE (1998). The Institute of Electric and Electronic Engineers (IEEE): Standard for software test documentation. IEEE Standard 829-1998, New York, NY, USA.
32.
IEEE (2004). IEEE standard for software verification and validation, Std 1012, New York, NY, USA.
33.
IEEE (2013). The international standard for software testing. Software and systems engineering – Software Testing. ISO/IEC/IEEE 29119 Software Testing (Part 1, 2, 3 and 4).
34.
ISO/IEC/IEEE (2008). Standard for Systems and Software Engineering – Software life cycle processes. IEEE Std 12207-2008. c1-138. doi:10.1109/IEEESTD.2008.4475826.
35.
Kim, H.M. (2000). Developing ontologies to enable knowledge management: Integrating business process and data driven approaches. In AAAI Workshop on Bringing Knowledge to Business Processes.
36.
Kitchenham, B. & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering, School of Computer Science and Mathematics Keele University and Department of Computer Science University of Durham, UK, v. 2.3.
37.
Li, X. & Zhang, W. (2012). Ontology-based testing platform for reusing. In Internet Computing for Science and Engineering (ICICSE) (pp. 86–89).
38.
Myers, G.J. (2004). The Art of Software Testing (2th ed.). Canada: John Wiley and Sons.
39.
Mathur, A.P. (2012). Foundations of Software Testing (5th ed.). India: Dorling Kindersley (India), Pearson Education in South Asia.
40.
Nakagawa, E.Y., Barbosa, E.F. & Maldonado, J.C. (2009). Exploring ontologies to support the establishment of reference architectures: An example on software testing. In Software Architecture. WICSA/ECSA, Joint Working IEEE/IFIP Conference on, Cambridge (pp. 249–252).
41.
Nasser, V.H., Du, W. & MacIsaac, D. (2009). Knowledge-based software test generation. In International Conference on Software Engineering and Knowledge Engineering (SEKE) (pp. 312–317).
42.
Perry, W.E. (2006). Effective Methods for Software Testing (3th ed.). Canada: Wiley Publishing, Inc.
43.
Pressman, R.S. (2006). Software Engineering: A Practitioner’s Approach (6th ed.). McGraw-Hill Series in Computer Science. New York: McGraw-Hill.
44.
Ryu, H., Ryu, D. & Baik, J. (2011). A strategic test process improvement approach using an ontological description for MND-TMM. In International Conference on Computer and Information Science (pp. 561–566).
45.
Ruy, F.B., Falbo, R.A., Barcellos, M.P., Guizzardi, G. & Glaice, K.S. (2015). An ISO-based software process ontology pattern language and its application for harmonizing standards. SIGAPP Appl. Comput. Rev., 15, 27–40. doi:10.1145/2815169.2815172.
46.
Ruy, F.B., Falbo, R.A., Barcellos, M.P., Costa, S.D. & Guizzardi, G. (2016). SEON: A software engineering ontology network. In Proceedings of the 20th International Conference on Knowledge Engineering and Knowledge Management (EKAW), Bologna, Italy (pp. 527–542).
47.
Sapna, P.G. & Mohanty, H. (2011). An ontology based approach for test scenario management. In ICISTM (Vol. 141, pp. 91–100).
48.
Engineering Institute SEI (2010). CMMI for Development. Improving processes for developing better products and services. Version 1.3. Technical Report ESC-TR-2010-33.
49.
Souza, E.F., Falbo, R.A. & Vijaykumar, N.L. (2013). Ontology in software testing: A systematic literature review. In The Ontology Research Seminar in Brazil (ONTOBRAS), Belo Horizonte, MG. CEUR Workshop (Vol. 1041, pp. 71–82).
50.
Souza, E.F., Falbo, R.A. & Vijaykumar, N.L. (2015a). Knowledge management initiatives in software testing: A mapping study. Information and Software Technology, 57, 378–391. doi:10.1016/j.infsof.2014.05.016.
51.
Souza, E.F., Falbo, R.A. & Vijaykumar, N.L. (2015b). Using the findings of a mapping study to conduct a research project: A case in knowledge management in software testing. In 41st Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Funchal, Madeira (pp. 208–215).
52.
Staab, S., Studer, R., Schurr, H.P. & Sure, Y. (2001). Knowledge processes and ontologies. IEEE Intelligent Systems, 16(1), 26–34. doi:10.1109/5254.912382.
53.
Suárez-Figueroa, M.C., Góez-Pérez, A., Motta, E. & Gangemi, A. (2012). Ontology Engineering in a Networked World. Springer Science & Business Media.
54.
TMMi Fundation (2012). Test Maturity Model integration (TMMi). Release 1.0, Ireland.
55.
Vrandecic, D. & Gangemi, A. (2006). Unit tests for ontologies. In On the Move to Meaningful Internet Systems 2006: OTM Workshops (Vol. 4278, pp. 1012–1020). doi:10.1007/11915072_2.
56.
Uren, V., Cimiano, P., Iria, J., Handschuh, S., Vargas-Vera, M., Motta, E. & Ciravegna, F. (2006). Semantic annotation for knowledge management: Requirements and a survey of the state of the art. Journal of Web Semantics: Science, Services and Agents on the World Wide Web, 4, 14–28. doi:10.1016/j.websem.2005.10.002.
57.
Yufeng, Z. & Hong, Z. (2008). Ontology for service oriented testing of web services. In Symposium on Service-Oriented System Engineering, Jhongli (pp. 129–134).
58.
Yu, L., Su, S. & Zhao, J. (2008). Performing unit testing based on testing as a service (TaaS) approach. In International Conference on Service Science (pp. 127–131).
59.
Yu, L., Zhang, L., Xiang, H., Su, Y., Zhao, W. & Zhu, J. (2009). A framework of testing as a service. In International Conf. on Management and Service Science (pp. 1–4).
60.
Zhu, H. & Huo, Q. (2005). Developing a software testing ontology in UML for a software growth environment of web-based applications. In Software Evolution with UML and XML (pp. 263–295). IDEA Group.
61.
Zhu, H. & Zhang, Y. (2012). Collaborative testing of web services. IEEE Transactions on Service Computing, 5, 116–130. doi:10.1109/TSC.2010.54.
