Automated Semantic Analysis of IT Project Risks
By Franck-Olivier Kwan, Véronique Nabelsi, Stéphane Gagnon & Wassim El-Kass
What are Automated Semantic Analysis (ASA), Ontologies, and Knowledge Graphs (KG)?
Automated Semantic Analysis (ASA) refers to the use of an “ontology” to annotate textual and other unstructured data. An ontology can be thought of as a way of categorizing objects and the relationships among them. When applied to managing IT project risks, this approach can help to automate or semi-automate the many unstructured relationships contained in project management documents. A key benefit for project managers is that the approach enables forecasting of project risks and proactive management of these risks as a project unfolds.
Think of this as going beyond the traditional “keywords indexing” approach, and instead creating a wide network of multiple-flexible relations between concepts in your project management framework. Akin to Object Oriented Modeling (which is in fact related to ontologies through model transformation standards), but without the complex constraints, and without concern for evolving information schema.
Ontologies are often mentioned when dealing with an increasingly popular NoSQL database platform, Knowledge Graphs (KG) Databases. Some well-known trademarks, such as Neo4j and OrientDB among many others, provide primarily “property graphs” that simplify the constraints on inter-concept relationships. A KG is therefore an ontology or a network of entities, normally defined as concepts or classes and their domain-specific instances, with relationships that define a domain-specific logic with a variety of constraints.
Breakthrough in Using ASA to Analyze IT Project Risks
The use of ontologies to describe IT project risks is still a major scientific challenge. IT projects are complex and representing their risks within a temporal context requires extensive practical experience. Risks can only be understood through in-depth knowledge of the variety of issues throughout the lifecycle of IT projects, from architecture to development, from implementation to end-user experience.
This challenge has successfully been overcome, thanks to a recent completed research project that may be of interest to several government agencies seeking innovative solutions for IT project risk analytics.
First, a new IT project risk ontology was proposed that differentiates clearly between risk occurrences and risk mitigation. It is modelled after PMI’s PMBOK risk management processes and represents a whole breath of academic and practical knowledge found in the literature. As such, this new IT project risk ontology serves as a first step to help extend the knowledge base in the future.
Second, we have demonstrated the feasibility of using an ASA platform entitled Adaptive Rules-Driven Architecture for Knowledge Extraction (ARDAKE) , relying on its ontology-driven annotations to serve as a risk management tool in IT projects. We were able to automatically annotate all artifacts of a project, indicating events related to various risks. Annotated information can be found in several types of project documents, such as project status reports, lessons learned document, business case, and meeting agendas. This diversity in the annotation of project documents seems to provide a perspective that is both direct (i.e., project reports, meeting agendas) and overall risk (cost benefit analysis, lessons learned).
Third, we then used a semantic analysis tool to establish potential links between various events and risks, and thus identify the precursors to trace their occurrence. The annotated dates of the project documents make it possible to create a chronology of events and information to support the dated actions of the mitigation strategies of the risk register. Between the annotated dates of the first events and those of the mitigation strategies, it is possible to estimate a lag ranging from a few days to several months. A risk emergence pattern may be found when the dates of the first annotated sentences were earlier than the dates of the mitigation strategy sentence.
Seeking Government Agencies for Further Pilot Studies
Our findings show the potential of ASA tools to help IT project risk analytics. To advance this research, we need partners to help develop a more systematic research process. This research breakthrough used the detailed PM data from an actual, completed IT project. However, we believe we could radically improve IT project risk forecasting by relying on more detailed and diverse datasets.
To broaden the scope of IT and Business (digital) risks we can forecast and monitor, we can reuse the Business Technology Management Body of Knowledge (BTM BOK). Started recently in v.0.1, BTM BOK will become an integrated reference of 60+ open-source professional standards.Once the BTM BOK ontology is created from integrated standards, it will be possible for any ontology-driven tool to rely on its specifications for IT roles, skills, tasks, processes, artefacts, and outcomes to control risks within complex IT projects.
Government agencies are invited to contact Stéphane Gagnon to develop more pilot projects. We can work with ongoing and completed project datasets, and especially compare IT project risk monitoring from both traditional (manual) methods and more automated analytics features in requirements-to-test tools (e.g., JIRA).
Further Readings about Ontologies
As defined by Studer et al.: “An ontology is a formal, explicit speciﬁcation of a shared conceptualization”. Various standards exist to represent ontologies, but in this research, the object of study is only the W3C’s Web Ontology Language (OWL). Ontologies can contain a thesaurus, glossary, and taxonomy characteristics. It is possible to create multiple ontologies to reflect specific areas of knowledge. For example, research has created and linked three ontologies (software, bug, and version) in a study of information systems to represent various aspects that generate source code. In addition, ontology or a body of knowledge represents elements of a domain of knowledge that can be used in the labeling process to extract various subjects.
Ontologies are valuable tools to help analyze IT project risks. In 2009, a risk ontology was developed for IT projects post-implementation of an integrated management software package. Practitioners can use this detailed ontology in identifying risks in this type of software.  More recently, software that uses a risk ontology has allowed practitioners to know the risks and risk management practices according to project phases. These authors also add that ontology engineers must be able to extract information from various knowledge bases to update the concepts of the ontology.
- Kwan, F.-O. (2021). Development and implementation of an ontology for the automated semantic analysis of IT project risks. D.B.A. STI, Université du Québec en Outaouais, Gatineau.
- El-Kass, W. (2018). Integrating semantic web and unstructured information processing environments: a visual rule-based approach.Ph.D. STI, Université du Québec en Outaouais, Gatineau. https://gagnontech.org/ardake
- Gagnon, S. (2020). Business Technology Management as Transdisciplinary IS-IT Competency Framework. ICIS 2020 Proceedings. Presented at the International Conference on Information Systems (ICIS), Hyderabad, India: AIS. https://aisel.aisnet.org/icis2020/digital_learning_env/digital_learning_env/8/
- Studer, R., Benjamins, V. R., & Fensel, D. (1998). Knowledge engineering: Principles and methods. Data & knowledge engineering, 25(1-2), 161-197.
- Tappolet, J., Kiefer, C., & Bernstein, A. (2010). Semantic web enabled software analysis. Web Semantics: Science, Services and Agents on the World Wide Web, 8(2), 225-240.
- Hindle, A., Ernst, N. A., Godfrey, M. W., & Mylopoulos, J. (2011). Automated topic naming to support cross-project analysis of software maintenance activities. MSR ‘11: Proceedings of the 8th Working Conference on Mining Software Repositories, May 21, 2011. ACM Proceedings Series: 163-172
- Peng, G. C., & Nunes, M. B. (2009). Surfacing ERP exploitation risks through a risk ontology. Industrial Management & Data Systems, 109 (7), 926-942.
- Abioye, T. E., Arogundade, O. T., Misra, S., Akinwale, A. T., & Adeniran, O. J. (2020). Toward ontology‐based risk management framework for software projects: An empirical study. Journal of Software: Evolution and Process, 32(12), e2269.
About The Authors
Franck-Olivier Kwan is Manager of the Case Management Competency Centre at Public Services and Procurement Canada (PSPC). He obtained his DBA in Project Management in 2021 at Université du Québec en Outaouais (UQO). His thesis used the Adaptive Rules-Driven Architecture for Knowledge Extraction (ARDAKE) platform to demonstrate how an IT Risk Ontology can be used to detect early warning signs of emerging risks.
Véronique Nabelsi is Full Professor in Healthcare IT and Chair of the DBA in Project Management at Université du Québec en Outaouais (UQO). She is also Scientific Director of the Psychosocial Medicine Research Center and the Primary Healthcare Research Group at the Centre Intégré de santé et de services sociaux de l’Outaouais (CISSSO). Her research focuses on the management of healthcare establishments, including the integration of medical information systems and processes, in an emerging strategic framework of activity-based funding.
Stéphane Gagnon is Associate Professor in Business Technology Management (BTM) at the Université du Québec en Outaouais (UQO) and founding member of the Government Analytics Research Institute (GARI). His research deals with digital transformation and big data analytics, applied to the healthcare, energy, and financial services industries, as well as public administration.
Wassim El-Kass is Assistant Director at the Advanced Analytics Solutions Centre at Immigration, Refugees and Citizenship Canada (IRCC). He obtained his Ph.D. in Information Science and Technology (IST) in 2018 at Université du Québec en Outaouais (UQO). He developed an ontology-driven text annotation platform entitled Adaptive Rules-Driven Architecture for Knowledge Extraction (ARDAKE).