Challenges in Creating Business Value from Health Information Systems (HIS): A Hybrid Fuzzy Approach

Document Type : Research Paper


1 Assistant Prof., Masaryk University, Brno, Czech Republic.

2 MSc., Mehralborz Institute of Higher Education, Tehran, Iran.


For the last three decades, research on Information Technology Business Value (ITBV) has highlighted how organizations can create greater value from IT investments in different industries, including the healthcare sector. There has been some research investigating the business value of Health Information Systems (HIS). However, the current body of knowledge regarding the challenges in front of proper business value realization of these systems is limited. This study investigates the challenges of HIS business value creation using a combination of the Fuzzy Analytic Hierarchy Process (FAHP) and fuzzy Decision-making Trial and Evaluation Laboratory (DEMATEL). First, this study reviews the literature to identify the main challenges in the field. The outcome of this step revealed three main categories and 22 challenges, including technological (ten challenges), organizational (eight challenges), and environmental (four challenges). Then, this study first uses FAHP to evaluate the weighting for each criterion and then adopts the fuzzy DEMATEL method to establish contextual relationships among those criteria. We find out that technological challenges are the most crucial dimension. Moreover, we observed that “infrastructure costs” has the highest priority. Under the technological dimension, “maintenance costs” and “systems compatibility” challenges are the most critical since they are in the cause area and directly influence the HIS outcomes. Under the organizational dimension, “Change in strategic objectives” is the most important challenge. Moreover, “inter-departmental coordination”, “training costs”, “proficiency”, and “users’ knowledge” are affected by each other as well as influenced by the net causes. The outcome of this study shows that to handle HIS business value creation challenges, healthcare executives should start with infrastructure costs, maintenance costs, and systems compatibility challenges.



For the last three decades, research on Information Technology Business Value (ITBV) has highlighted how organizations can create greater value from IT investments (Grover & Kohli, 2012). The primary focus has been on the operational and economic value of IT, with key importance attached to productivity and profitability (Salge, Kohli, & Barrett, 2015). Research in the field of Health Information Systems (HIS) or Health Information Technology (HIT) has shown that IT not only provides operational and economical business value (Amarasingham, Plantinga, Diener-West, Gaskin, & Powe, 2009), but also improve the quality and safety of care and patient safety (Fichman, Kohli, Krishnan, & Kane, 2011; Khajouei, Gohari, & Mirzaee, 2018). In this line, some researchers assert how to apply current IT tools and applications to effectively improve healthcare service quality and promote electronic case history that has currently become an essential subject in HIS (e.g., Pai & Huang, 2011). HIS are computer systems that collect, store, and transmit clinical and administrative information of patients and organizations (Ahmadian, Dorosti, Khajouei, & Gohari, 2017).

 Despite the business value of HIS, they are at risk due to increasing demand, spiraling costs, inconsistent and inadequate quality of care, and poorly coordinated care processes (Lluch, 2011). In response, governments are developing various strategies, one of which consists of substantial investments in HIS. However, industries in developing countries face several difficulties in proper IT adoption, use and value creation,(Amid, Moalagh, & Ravasan, 2012) and the healthcare sector is not an exception. Therefore, it is of great importance to investigate the main challenges in the business value creation of HIS in the context of a developing country (Abbasi, Khajouei, & Mirzaee, 2020). In line with this motivation, this study intends to identify and address HIS business value creation challenges in the Iranian context. Specifically, this research aims to answer two main research questions as follows:

RQ1. What are the main challenges in creating business value from HIS?

RQ2. What are the cause and effect relationships among HIS business value creation challenges?

 To answer these research questions, first, we investigate the key challenges of HIS business value creation. Then, we classify the identified challenges into technological, organizational, and environmental categories. The identified challenges and their categories were prioritized by calculating the relative weight of each challenge and their categories. Our findings could help the healthcare sector adjust their management strategies to get higher business value from their HIS investment and overcome challenges. Besides, the knowledge about the identified challenges will help develop a generic model that could help the healthcare system execute the ongoing activities successfully.

Regarding prioritizing related HIS challenges, various techniques have been used. The Multi-Criteria Decision Making (MCDM) techniques often tide to ranking and determining the importance of different challenges. However, what has gained less attention is the degree of influence of these challenges on each other. Hence, it seems to be essential for employing a new approach to resolve the problem in the context of prioritizing challenges. This study proposes a hybrid model based on fuzzy set theory to deal with the ambiguity of human judgment, the Fuzzy Analytical Hierarchy Process (FAHP) to determine the rank of each challenge, and the Fuzzy Decision-making Trial and Evaluation Laboratory (DEMATEL) technique to evaluate the cause and effect relationships among HIS challenges.

The rest of this paper is organized as follows. In section 2, the prior literature related to the HIS and respected challenges is reviewed. Section 3 presents the research method. In section 4, an empirical study is illustrated. Finally, section 5 provides conclusions, limitations, and future research.


Literature review

This section provides a review of prior HIS research and business value creation challenges from HIS projects.


Related works to HIS

Reviewing the literature shows that previous research widely discussed the HIS and related fields from different perspectives. For instance, Ludwick and Doucette (2009), using a systematic literature review of peer-reviewed and grey literature, focused on electronic medical record (EMR) adoption and lessons learned from HIS adoption experience in seven countries. This study intended to discover factors affecting HIS outcomes. They suggested that HIS user interface quality, HIS feature/functionality, project management, procurement, and users’ previous experience affect HIS outcomes. HIS projects had concerns about factors such as privacy, patient safety, provider/patient relations, staff anxiety, time factors, quality of care, finances, efficiency, and liability. The most interesting outcome of this study was the fact that HIS do not improve or erode efficiency, quality of care, or patient safety. HIS were not the source of adverse events, per se, but rather, the systems' processes facilitated the errors.

Pai and Huang (2011) presented a conceptual model to investigate the intention to use HIS. They proposed a conceptual model for the intention of HIS use. They adopted the Information System Success Model (ISSM) of DeLone and Mclean (2003) and integrated it with the three dimensions of perceived usefulness, perceived ease of use, and intention to use of Venkatesh and Davis' (2000) updated Technology Acceptance Model (TAM). Their outcome represents that the proposed factors positively influence users' intention to use a healthcare system. Information, service, and system quality influence the users’ intention through the mediating constructs, perceived usefulness, and perceived ease-of-use. Lluch (2011) carried out a systematic literature review to identify and categorize barriers to HIS use. The author categorized barriers under five main headings – (a) structure of healthcare organizations; (b) tasks; (c) people policies; (d) incentives, and (e) information and decision processes. She argued that there is a need to identify each organizational aspect's impact and address them adequately in HIS projects.

 Najaftorkaman et al. (2015), in their research, proposed a reflective pause on the HIS adoption literature to improve our understanding of factors contributing to the user adoption of EMR. Using a systematic literature review, they provided a comprehensive taxonomy of the factors influencing the user adoption of EMR and classified these factors into eight categories: individual, psychological, behavioral, environmental, organizational, financial, legal, and technical. Nguyen et al. (2014) reviewed the literature about electronic health record (Zhang et al., 2012) projects worldwide and highlighted both benefits and issues associated with them. The framework presented in this study is based on the DeLone and McLean’s (2003) dimensions, including information quality, system quality, service quality, the intention of use and usage, user satisfaction, and net benefits, together with contingent dimensions, including systems development, implementation attributes, and organizational aspects. The results confirmed the potential of this technology to aid patient care and clinical documentation, for example, in improved documentation quality, increased administration efficiency, as well as better quality, safety, and coordination of care. Common negative impacts include changes to workflow and work disruption.

 Alsalman et al. (2020) examined the effects of HIS projects in the Eastern province of Saudi Arabia’ hospitals considering the following categories in the structuring of electronic health records: electronic clinical documentation, results viewing, decision support, and barcoding. They found that HIS business value is promising, especially for basic functions related to patient information such as electronic clinical documentation and viewing results. The outcomes of this research indicated that hospitals are implementing HIS in stages. The information obtained in this research is useful for healthcare institutions to develop mechanisms that contribute to the business value realization of HIS.

 Khajouie et al. (2018) focused on HIS's usability and developed a comparative approach to finding out the difference between methods of heuristic evaluation in identifying usability problems of HIS. The results demonstrated that the checklist method had significantly better performance in terms of the number of identified usability problems; however, the performance of the usual method for identifying problems of higher severity was significantly better. Ratwani, Reider, and Singh (2019), in a similar way, explored HIS usability challenges and argued that overcoming usability challenges that have affected HIS will require shared responsibility and greater collaboration among vendors, researchers, policymakers, health care organizations, clinicians, and patients. While policymakers need to initiate many of these actions, success is dependent on true engagement from all groups, particularly vendors who should now consider greater transparency of their products. Teixeira et al. (2019) show how a service design approach can support a national EHR project's successful development and implementation. The service design approach, including the visual models and tools used across the different design stages, was instrumental in envisioning new EHR concepts and designing the system to enhance healthcare users’ experience. A qualitative study performed after implementation showed that the EHR was considered useful and easy to use, and these results are backed by widespread usage of the system.

Dauwed et al. (2019) investigated the challenges of Health information exchange (HIE). HIE is the electronic transmission of healthcare information among large healthcare providers and interconnected hospitals. Their findings reveal HIE challenges, including unified patients’ data, the teamwork of care, security and privacy, address storage, patient consent, compatibility, hospital workflow. Three main recommendations: the use of IT innovations, integration of electronic health records system and increase system capacity are suggested.

Sebetci (2018) assessed HIS to investigate the related factors that might enhance end-user satisfaction through technology compatibility. In his proposed model, information quality, system quality, support resources, and technologies' compatibility were supported as variables that affect user satisfaction. Therefore, adopting such changes and advancements to information systems would improve total user satisfaction. More recently, Meri et al. (2019) proposed a hypothesized research model to determine the utilization of cloud HIS in the Iraqi public healthcare sector. This research developed a model by defining the critical success factors influencing physicians’ confirmation and behavioral control toward utilizing cloud HIS. The results showed that the effects of system compatibility, system complexity, security, and privacy on physicians’ confirmation and behavioral control were statistically significant. Both confirmation and behavioral control had a positive influence on physicians’ utilization of the technology in the Iraqi hospitals.


HIS business value creation challenges

 HIS can help in various operations that are needed in every healthcare organization, such as data acquisition and presentation, record keeping and access, communication and integration of information, surveillance, information storage and retrieval, data analysis, decision support, and education. HIS can have several repercussions, including different workflows, more work for clinicians, changes in communication patterns and practices, and negative emotions. As such, these projects face a variety of social, ethical, and legal challenges (Teixeira et al., 2019). Inline, several attempts have been made to explore the main barriers to adopt, implement, and use HIS. For instance, Thakkar and Davis (2006) identified EHR systems' status in US hospitals regarding the core functionalities and investigated the perceived benefits, risks, and barriers to adoption. They presented barriers to adopting EHR system including software costs, hardware costs, participation from physicians, interoperability among different electronic systems and the true EHR system, inability to find the software that meets the requirements of the true EHR system, organizational culture, participation from nursing staff, standards, return on investment (ROI), and personnel cost.

 Kaye et al. (2010), who developed a field survey about the barriers and critical success factors in health information technology, found a comprehensive typology of barriers based on previous studies, including financial and business barriers, structural barriers, cultural barriers, and technical and professional barriers. Ajami and Bagheri-Tadi (2013) discussed the main barriers in EHR adoption perceived by physicians. They performed a non-systematic literature review to find the significant barriers of EHR adoption, therefore, found 25 main factors gathering from more than 100 papers published in the related area as follows: time, cost, absence of computer skills, workflow disruption, concern about security and privacy, communication among users, interfaces with the doctor-patient relationship, lack of incentives, complexity, physical space, concern about the ability to select an effectively install EHR system, technical support, interoperability, access to computers and computer literacy, vendor trust, expert support, concern about data entry, training and after-sail experiences with their vendor, reliability, inadequate data exchange, concern about patient acceptance, formal training, speed, inter-institutional integration, and wireless connectivity. In another research, Ahmadian et al. (2014) prioritized barriers to the HIS projects using a cross-sectional analytic-descriptive study. They suggested that the lack of powerful information networks, errors in data entry, technical problems related to system design, lack of organizational training, lack of users’ knowledge about the system, and negative attitudes of providers and patients toward systems are the most critical barriers of HIS projects. They classified the identified barriers into five categories: human, system characteristics, human environment, organizational environment, and hardware factors. Later, Ahmadian et al. (2017) examined the challenges of using HIS by nurses and compared academic and non-academic hospitals. The most important challenges in the academic hospitals were human-environment factors, particularly the "negative attitude of society toward using HIS". In the non-academic hospitals, the most common and important challenges were related to human factors, and among them, "no incentive to use the system" was the main factor.

In another research, Gesulga et al. (2017) developed a set of obstacles to EHR projects using a structured literature review. The identified factors organized as people (16 elements), hardware (3 elements), software (7 elements), data (9 elements), network (3 elements), and procedure (19 elements). Their review suggests that people resources (user resistance and lack of skills) and procedure resource (concern for return on investment and lack of administrative and policy support) are the primary barriers to overcome.

 According to the literature alongside interview experts, 22 main challenges were identified and classified into three categories as technological, organizational, and environmental. Tornatzky, Fleischer, and Chakrabarti (1990) introduced the technology, organizational, environmental (TOE) model to model factors that affect technology-based innovations. TOE has been selected here for the following reasons. First, TOE has a holistic and multifaceted view that considers technological, organizational, and environmental factors simultaneously, rather than focusing merely on one aspect and various factors that affect IT initiatives can be classified in technological, organizational, or environmental contexts (Côrte-Real, Ruivo, Oliveira, & Popovič, 2019; Hanafizadeh & Ravasan, 2018). Thus, it is feasible to apply the TOE model to categorize HIS business value creation challenges. In the following, all challenges and dimensions are presented.


Table 1. Major challenges to HIS business value creation



Technological (D1)

Infrastructure costs (F1)

Lack of proficient and stable devices (F2)

Maintenance costs (F3)

Data integration (F4)

Lack of certainty to the migration process (F5)

Customization (F6)

Security and privacy (F7)

Maintenance services (F8)

Lack of communication with other systems (F9)

Compatibility with other systems (F10)

Organizational (D2)

Change in strategic objectives (F11)

Change in healthcare management (F12)

Near real-time availability (F13)

Inter-departmental coordination (F14)

Realized value (F15)

Training costs (F16)

Proficiency (F17)

Users’ knowledge (F18)

Environmental (D3)

The high cost of internet subscription (F19)

Lack of access to international software (F20)

Problems with preparing software licenses (F21)

Poor hardware support (F22)

Sources: (Ahmadian et al., 2014; Ajami & Bagheri-Tadi, 2013; Cresswell & Sheikh, 2013; Fichman et al., 2011; Gesulga et al., 2017; Khajouei et al., 2018; Khalifa, 2014; Meri et al., 2019; Qureshi et al., 2013; Sebetci, 2018; Teixeira et al., 2019)


Research method

This section introduces an overview of fuzzy sets, fuzzy AHP, fuzzy DEMATEL concepts, and prior applications in similar IT domains.


Fuzzy Sets

 To deal with human thought's vagueness, Zadeh (1965) introduced the fuzzy set theory based on rationality in the context of uncertainty due to the vagueness or imprecision. With different daily decision-making problems of diverse intensity, the results can be misleading if the fuzziness of human decision making is not taken into account (Ravasan, Hanafizadeh, Olfat, & Taghavifard, 2017). Fuzzy sets theory providing a more widely frame than classic sets theory has contributed to the capability to reflect the real world. In the fuzzy set theory, the degree of membership to each of the objectives can be assigned a number between 0 and 1 is unlike the CRISP set theory. Hence, the fuzzy set theory can express and handle vague or imprecise judgments mathematically (Rouhani, Ashrafi, & Afshari, 2013).


Fuzzy Analytical Hierarchy Process

AHP is a decision-making tool that, by decomposition of a complex problem into a multi-level hierarchical structure, includes the objectives, criteria, sub-criteria, and alternatives, describes the overall performance of the decision. Saaty (1990) presented the AHP method to prioritize the relative importance of options and features in MCDM issues in 1977. AHP method is repeatedly used in the research literature. One of the advantages of the AHP model is the ease of use. Besides, the AHP method can effectively handle both qualitative and quantitative data simultaneously in the decision-making process (Cebeci, 2009). Though the purpose of AHP is to capture the expert’s knowledge, the conventional AHP cannot reflect the human thinking style (Kahraman, Cebeci, & Ulukan, 2003), as it is unrealistic to assign crisp values for a subjective judgment, especially when information is vague or imprecise (Chang & Wang, 2009). AHP method is often criticized due to its use of an unbalanced scale of judgments and the inability to resolve the inherent ambiguity and uncertainty in the pairwise comparison process (Deng, 1999).

There are various MCDM approaches. However, the uncertainty and vagueness in the expert’s opinion is the major issue in MCDM projects. This imprecision of human judgment could be adequately managed using the FAHP approach. It helps to systematically solve the problem by incorporating the fuzzy set theory concepts and the graphical hierarchical classification of the given problem (Khan, Shameem, Kumar, Hussain, & Yan, 2019). We have used the FAHP approach for the challenges and their categories prioritization. FAHP is a synthetic extension of the classical AHP method when the fuzziness of the decision-makers is considered. The use of FAHP requires scientific approaches for deriving the weights from FAHP pairwise comparison matrices. AHP finds weights, which can be considered as values of “points”; however, FAHP can extend these “points” to “lines” or even “surfaces”, which can estimate the MCDM more precisely and make comparisons for MCDM (Ly, Lai, Hsu, & Shih, 2018). The advantages of AHP are manifold. The AHP method can be successfully applied to qualitative and quantitative data (Rajak & Shaw, 2019). It can transform a complex and multi-criteria problem into a hierarchical structure. The methodology requires minimal mathematical calculations (Javanbarg, Scawthorn, Kiyono, & Shahbodaghkhan, 2012). AHP is a methodology that can consider the consistency of the decision-makers (Durán & Aguilo, 2008).

Numerous studies apply the FAHP method in different contexts to solve problems. For instance, Chou, Sun, and Yen (2012) use FAHP to evaluate the criteria for human resources for science and technology. This research used a crisp judgment matrix to evaluate subjective expert judgments made by perception. Khan et al. (2019), focusing on global software development, developed a prioritization-based framework and taxonomy to understand the success factors of software process improvement using the FAHP approach. Wang, Xu, and Solangi (2020) have used strengths, weaknesses, opportunities, and threats (SWOT) analysis have been employed to assess the internal and external factors which affect renewable energy technologies. They have also employed the FAHP method to grasp this problem from a multi-perspective approach. Boral et al. (2020) proposed an integrated approach for fuzzy failure modes and effects analysis using FAHP to calculate the fuzzy relative importance between the risk factors. Sirisawa and Kiatcharoenpol (2018), who researched the basis of the green supply chain, developed a FAHP-based framework to increase our understanding of the priorities of both barriers and solutions for developing policies and strategies to overcome these barriers.

This paper uses FAHP for determining the weights of the main criteria. FAHP Computational process is based on Hsieh et al. (2004) approach as follows:

Step 1: Build a pairwise comparison matrix. Through expert questionnaires, each expert is asked to assign linguistic terms (as shown in Table 2) to the pairwise comparisons among all the elements/criteria in the hierarchy system's dimensions.


Table 2. The correspondence of Linguistic terms and values

Linguistic variable

Linguistic value

Very High Influence (VH)

(0.75, 1, 1)

High Influence (H)

(0.5, 0.75, 1)

No Influence

(0, 0, 0.25)

Low Influence (L)

(0.25, 0.5, 0.75)

Very Low Influence (VL)

(0, 0.25, 0.5)





Step 2: To use the geometric mean technique to define the fuzzy geometric mean and fuzzy weights of each criterion as follows:


According to the above formula, is the fuzzy comparison value of criterion i to criterion n. Also,  is a comparison of the geometric mean value of criterion i to each criterion and  is the fuzzy weight of ith criterion.



Graph theory has significantly grown in recent years, primarily due to graphs' usefulness as models for computation and optimization. In graph theory, we can visually discover things inside the complex problem because the graph displays mathematical results clearly and unambiguously. The Decision Making Trial and Evaluation Laboratory (DEMATEL) can separate involved factors into cause and effect groups (Wu, 2012). According to Lee, et al. (2011), the main purpose of DEMATEL is to find direct and indirect causal relationships and strength of influence between all variables of a complicated system based on matrix calculation.

The DEMATEL technique originated from the Geneva Research Center of the Battelle Memorial Institute in 1973. DEMATEL technique is one of the MCDM methods. However, it differs from other methods by using graph theory and determine the impact of factors on each other. In this construction, each element can impact all elements that are on the same level, higher level or lower level, and mutually accepted by all of them. This method can discover the cause and effect relationship and provide a plausible structural model (Tseng, 2009). So, it would be the most useful tool for decision-makers to understand the cause and effect of factors within an issue and make their decisions based on those elements' real nature (Zhou, Huang, & Zhang, 2011).

DEMATEL is a commonly used method to analyze and visualize the structure of complex systems that involve causal relationships. It requires a group of experts to deliver subjective estimates of the relationships among the factors, usually in a pairwise form (Asan, Kadaifci, Bozdag, Soyer, & Serdarasan, 2018; Rouhani, Ashrafi, & Afshari, 2014). This method is employed to describe the interrelations and further to determine the weight of each category and factor. Due to its ability to study the complex structure of cause-effect relationships, the method has been popularly used in various practices (Büyüközkan, Güleryüz, & Karpak, 2017; X. Zhang & Su, 2019). Similar to all types of structural modeling approaches, DEMATEL also has a high capability to examine the causes and effects of relationships between the factors of a complex system. It can also approve interdependence between the factors and develop a map, which reflects the relationships among them to solve complicated decision-making problems. DEMATEL has enough capability to transfer the interdependency relations into the two cause and effect groups. Likewise, it can figure out the critical factors using the help of the influential relation map in a complex structural system (Chen, Lu, Ming, Zhang, & Zhou, 2020).

This method's strength lies in its ability to examine indirect relationships (i.e., influence chains and reaction loops) resulting from analyzing the perceived direct ones. This yields valuable information about the problem's structure and reveals factors playing a critical role, which would be otherwise neglected (Hiete, Merz, Comes, & Schultmann, 2012). To sum up, the advantages of the DEMATEL technique are demonstrated threefold, including: (i) that it successfully examines the mutual effects either as direct or indirect influences, between different factors, which subsequently brings a better understanding of cause and effect in a complicated decision-making problem; (ii) DEMATEL can easily provide a general picture of interrelationships between the identified factors using an influential relation map and helping decision-makers to better understand the influence of factors on one another; (iii) it can be used for both ranking factors (alternative/criteria) as well as recognizing the critical factors (Yazdi, Khan, Abbassi, & Rusli, 2020).

 Numerous studies apply the fuzzy DEMATEL method to solve different issues. For example, Mahmoudi et al. (2019) tried to reduce the complexity of the heart failure self-care process and optimize it as a critical success factor (CSFs), used fuzzy DEMATEL method to identify CSFs in an uncertain environment. Zhang and Su (2019) developed a combined fuzzy DEMATEL and TOPSIS approach for estimating participants in knowledge-intensive crowdsourcing. Mohammadfam et al. (2019) aimed to identify interactions among the most crucial individual, situational, and organizational variables affecting situation awareness in industrial workplaces based on the fuzzy DEMATEL method. This study was conducted based on the data collected from experts’ judgments. Singh and Sarkar (2020) proposed a framework based on the fuzzy Delphi and DEMATEL method to achieve the goal of sustainable product development in the Indian automotive industry. Acuña-Carvajal et al. (2019) designed an integrated framework to plan, structure, and validate a business strategy using fuzzy DEMATEL and the balanced scorecard. The objective of this study is to propose the development of a method that supports the overall process of planning, structuring, and validating a business unit strategy while overcoming the limitations of previous works, considering the subjectivity of decision-making in the construction of strategy maps. To achieve this goal, the fuzzy DEMATEL method has been adopted here to support the diagnosis and strategy design as described in the following:

Step 1: Construct the initial direct-relation matrix. Identifying the decision goal and forming a committee of experts to evaluate the direct effect between each pair of elements. Generate the initial direct-relation matrix A = [ ] by converting linguistic assessment into the crisp value. The initial direct-relationship matrix is an n×m matrix and  represents a direct impact of factor i on factor j and Ij when i = j the diagonal axis  = 0.

Step 2: Normalize the initial direct-relations matrix. The normalized direct-relation matrix D= [ ] is calculated ij through Eq. (5). All matrix D elements are between D, and all the principal diagonal elements are equal to 0.


Step 3: The total-relation matrix T can be acquired using Eq.6, in which I is an n×m identity matrix. Each element in this matrix represents the indirect effect of a factor on the others. Therefore, matrix T can reflect the total relation between each pair of system factors.


Step 4: Compute the sum of rows and the sum of matrix T columns through the Eqs. (7-8).


Step 5: A causal diagram can be obtained by mapping the dataset of (D + R, D - R), where the horizontal axis (D + R) is made by adding D to R, and the vertical axis (D - R) is made by subtracting D from R.

Empirical Study and Discussion

This study intends to use a combination of the FAHP and fuzzy DEMATEL method to evaluate the main challenges to HIS business value creation. Specifically, this study first uses FAHP to evaluate each criterion's weighting and then use the FDEMATEL method to establish contextual relationships and illustrate cause and effect diagrams among those criteria. In this section, an empirical study is presented to illustrate the application of FAHP and fuzzy DEMATEL methods.


Step 1. Selecting the committee of experts

 Considering the related HIS challenges, experts were invited to evaluate the criteria. In this study, twelve experts from both academic and healthcare practice environments with over ten years of HIS projects experience were involved. The research period is September 2020.


Step 2. Developing the dimensions and criteria

 This research includes three dimensions and 22 criteria, as mentioned in Table 1. The three dimensions are technological (D1), organizational (D2), and environmental (D3), and the 22 criteria are Infrastructure costs (F1), Lack of proficient and stable devices (F2), Maintenance costs (F3), Data integration (F4), Lack of certainty to the migration process (F5), Customization (F6), Security and privacy (F7), Maintenance services (F8), Lack of communication with other systems (F9), Compatibility with other systems (F10), Change in strategic objectives (F11), Change in healthcare management (F12), Near real-time availability (F13), Inter-departmental coordination (F14), Realized value (F15), Training costs (F16), Proficiency (F17), Users’ knowledge (F18), The High cost of internet subscription (F19), Lack of access to international software (F20), Problems with preparing software licenses (F21), and Poor hardware support (F22).


Step 3. The weights of evaluation dimensions

We adopt the FAHP method to evaluate the weights of different dimensions for the HIS challenges. Following the construction of the FAHP model, it is essential that experts fill the judgment matrix. The outcome of this step is shown in Table 3.


Table 3. The important weights of all criteria




D1. Technological

(0.287, 0.458, 0.696)

F1. Infrastructure costs (0.101, 0.177, 0.297)


F2. Lack of proficient and stable devices (0.098, 0.172, 0.297)


F3. Maintenance costs (0.091, 0.159, 0.277)


F4. Data integration (0.063, 0.118, 0.211)


F5. Lack of certainty to the migration process (0.072, 0.128, 0.231)


F6. Customization (0.047, 0.090, 0.173)


F7. Security and privacy (0.051, 0.082, 0.146)


F8. Maintenance services (0.042, 0.073, 0.137)


F9. Lack of communication with other systems (0.045, 0.073, 0.133)


F10. Compatibility with other systems (0.037, 0.061, 0.117)


D2. Organizational

(0.205, 0.322, 0.506)

F11. Change in strategic objectives (0.099, 0.180, 0.303)


F12. Change in healthcare management (0.101, 0.180, 0.313)


F13. Near real-time availability (0.073, 0.132, 0.230)


F14. Inter-departmental coordination (0.077, 0.132, 0.224)


F15. Realized value (0.061, 0.113, 0.200)


F16. Training costs (0.052, 0.091, 0.182)


F17. Proficiency (0.054, 0.088, 0.149)


F18. Users’ knowledge (0.051, 0.085, 0.158)


D3. Environmental

(0.156, 0.220, 0.342)

F19. High cost of internet subscription (0.219, 0.355, 0.548)


F20. Lack of access to international software (0.197, 0.309, 0.479)


F21. Problems with preparing software licenses (0.123, 0.191, 0.301)


F22. Poor hardware support (0.101, 0.145, 0.233)



According to the table, “F1. Infrastructure costs” is the highest important challenge. This means that healthcare systems need to prepare funding and financial resources to handle the required material and their implementation and maintenance costs. The inability to find appropriate financial resources might hinder and cause a time lag in adopting and using HIS. This issue has also been discussed in Ahmadian et al. (2014), and Ajami and Bagheri-Tadi (2013). The second place is dedicated to the “F3. Maintenance cost”. To be clear, it might frighten healthcare systems to use HIS because they also need to hire some new proficient personnel in the IT field to maintain both software and hardware related facilities. As a solution, the healthcare system should design the annual budgets of the hospitals to capacitate the high operation and maintenance costs of HIS, which should be a part of the regular expenses of operations and not a burden on the hospital resources, by being unscheduled or unplanned (Khalifa, 2014). Besides, we have to make sure that computers and networks are working fine in terms of hardware and have fewer maintenance problems to guarantee that the software will consequently work better. One of the exciting findings of this study is that “problems with preparing software licenses” and “poor hardware support” are ranked as the least important challenges. One explanation for this finding is that several domestic software providers tried to develop software packages to meet consumers’ needs during the years. On this basis, we believe that different issues, such as political sanctions or lack of support from international service providers (in the specific context of Iran (Amid et al., 2012; Hanafizadeh & Ravasan, 2018), might have a lower influence on decision-makers to use and implement HIS. Among the dimensions, the technological issue takes the first place. After that, organizational and environmental issues stand. This shows that respondents believe that technological-related criteria might be the most promising challenges. Therefore, if the healthcare system can handle these challenges, adopting and implementing HIS would be more feasible, and the healthcare sector can get the intended business value from HIS.


Step 4. Capturing the complex relationships among evaluation dimensions and criteria

 We employed the fuzzy DEMATEL method for capturing the complex relationships among these dimensions and criteria. We use this kind of expression to compare criteria by five basic linguistic terms, as “very high influence”, “moderate influence”, “low influence”, “very low influence”, and “no influence,” concerning the fuzzy scale in Table 2.

We obtained the causal and effect diagram (Fig. 1) by mapping a dataset of (R+C, R-C) which horizontal axis (R+C) represents the importance of criteria and (R-C) classifies the identified challenges into cause group as shown in Table 4.


Table 4. The value of (R+C) and (R-C) for technological challenges

Technological challenges





Infrastructure costs (F1)





Lack of proficient and stable devices (F2)





Maintenance costs (F3)





Data integration (F4)





Lack of certainty to the migration process (F5)





Customization (F6)





Security and privacy (F7)





Maintenance services (F8)





Lack of communication with other systems (F9)





Compatibility with other systems (F10)








 It is observable from the causal diagram that these challenges were visually divided into the cause group, including “F1. Infrastructure costs”, “F3. Maintenance costs”, “F5. Lack of certainty to the migration process”, “F8. Maintenance services “, and “F10. Compatibility with other systems”, while the effect group was composed of challenges such as “F2. Lack of proficient and stable devices”, “F4. Data integration”, “F6. Customization”, “F7. Security and privacy”, and “F9. Lack of communication with other systems”. Besides, from Fig. 1, it is clear that both challenges, “F3. Maintenance costs” and “F10. Compatibility with other systems”, are equally might be the most critical criteria.

 By the same token, the causal relationships among the organizational challenges are depicted in Table 5 and Fig. 2.


Table 5. The value of (R+C) and (R-C) for organizational challenges

Organizational challenges





Change in strategic objectives (F11)





Change in healthcare management (F12)





Near real-time availability (F13)





Inter-departmental coordination (F14)





Realized value (F15)





Training costs (F16)





Proficiency (F17)





Users’ knowledge (F18)








 Table 5 depicts that “F11. Change in strategic objectives”, “F12. Change in healthcare management”, “F13. Near real-time availability”, and “F15. Realized value” are net causes. In contrast, challenges including “F14. Inter-departmental coordination”, “F16. Training costs”, “F17. Proficiency”, and “F18. Users’ knowledge” are net receivers by observing (D – R) values. Besides, from Fig. 2, it is clear that “F11. Change in strategic objectives” might be the most critical dimension. Moreover, “F14. Inter-departmental coordination”, “F16. Training costs”, “F17. Proficiency”, and “F18. Users’ knowledge” are affected by each other as well as affected by net causes.

 The causal relationships among three second-tier criteria of environmental dimension are shown in Table 6 and Fig. 3. This shows that “F19. High cost of internet subscription”, and “F22. Poor hardware support” are net causes. In contrast, challenges including “F20. Lack of access to international software” and “F21. Problems with preparing software licenses” are net receivers by observing (D – R) values.


Table 6. The value of (R+C) and (R-C) for environmental challenges

Environmental challenges





The high cost of internet subscription (F19)





Lack of access to international software (F20)





Problems with preparing software licenses (F21)





Poor hardware support (F22)








Finally, Table 7 summarizes the causal relationships among the three dimensions. The digraph of these three dimensions is depicted in Fig. 4. Table 7 depicts that all three dimensions are net causes according to the observation of (D - R) values. Besides, from Fig. 4, it is clear that the organizational dimension is the most critical one. This means that the improvement should be started with organizational-related criteria, particularly highlight the role of “F11. Change in strategic objectives”, “F12. Change in healthcare management”, and “F13. Near real-time availability”.


Table 7. The value of (R+C) and (R-C) for TOE dimensions






D1. Technological





D2. Organizational





D3. Environmental








Conclusions, limitations, and future research

This paper attempts to elaborate on HIS business value creation challenges. To answer the first research question first, we identified 22 challenges in creating business value from HIS through a literature review. Then, we proposed a ranking scheme and cause and effect relationships between those challenges to answer the second research question. It was shown that assessing these challenges in HIS projects is difficult with parameters expressed in linguistic values. Such values are somewhat vague and are subject to expert judgments that involve uncertainties. Therefore, a combination of the FAHP and fuzzy DEMATEL techniques was employed to deal with this problem appropriately. The fuzzy approach is an applicable technique in providing decision-makers with estimated values under uncertainty in the preference judgments. Using an AHP approach in weighting TOE dimensions and challenges made it possible to compare challenges and dimensions. Besides, the DEMATEL technique is employed to investigate the relationship between criteria in terms of cause and effect, so made causal diagrams for each dimension and related challenges. We found out that technological challenges are the most crucial dimension. Moreover, we observed that “infrastructure costs” has the highest priority. Under the technological dimension, “maintenance costs” and “systems compatibility” challenges are the most critical since they are in the cause area and directly influence the HIS outcomes. Under the organizational dimension, “Change in strategic objectives” is the most important challenge. Moreover, “inter-departmental coordination”, “training costs”, “proficiency”, and “users’ knowledge” are affected by each other as well as influenced by the net causes. Using the proposed combination methodology would help understand the most critical challenges in an uncertain environment and help the decision-makers grasp the cause and effect relationships. Therefore, they can invest more in causing challenges to handle those challenges appropriately.

This study suffers from some limitations. In future studies, a methodology based on a combination of the fuzzy set theory concept and the proposed model can be further developed. The currently proposed method is based entirely on human judgment, whereas in some circumstances, we may have access to objective data or even more conventional data-driven methods, which might not be appropriate for our proposed model. Although the provided example demonstrated the usefulness of our approach, we believe that the rooms remain for future validation and improvement. So, further research is necessary to fine-tune the proposed model and assess its validity in other cases. Besides, applying other MCDM methods in a fuzzy environment such as TOPSIS or Vikor to view the problem from a different point of view and comparing the results of these methods is also recommended for future research. Also, it is proposed that other mathematical models or meta-heuristics can be combined with the existing method.



Ahad ZareRavasan acknowledges the financial support of the Grant Agency of the Czech Republic, GAČR (grant no. GA20-12081S).

Abbasi, F., Khajouei, R., & Mirzaee, M. (2020). The efficiency and effectiveness of surgery information systems in Iran. BMC medical informatics and decision making, 20(1), 1-8.
Acuña-Carvajal, F., Pinto-Tarazona, L., López-Ospina, H., Barros-Castro, R., Quezada, L., & Palacio, K. (2019). An integrated method to plan, structure and validate a business strategy using fuzzy DEMATEL and the balanced scorecard. Expert systems with applications, 122, 351-368.
Ahmadian, L., Dorosti, N., Khajouei, R., & Gohari, S. H. (2017). Challenges of using Hospital Information Systems by nurses: comparing academic and non-academic hospitals. Electronic physician, 9(6), 4625.
Ahmadian, L., Khajouei, R., Nejad, S. S., Ebrahimzadeh, M., & Nikkar, S. E. (2014). Prioritizing barriers to the successful implementation of hospital information systems. Journal of medical systems, 38(12), 151.
Ajami, S., & Bagheri-Tadi, T. (2013). Barriers for adopting electronic health records (EHRs) by physicians. Acta Informatica Medica, 21(2), 129.
Alsalman, D., Alumran, A., Alrayes, S., Althumairi, A., Alrawiai, S., Alakrawi, Z., . . . Alanzi, T. (2020). Implementation status of health information systems in hospitals in the Eastern province of Saudi Arabia. Informatics in Medicine Unlocked, 100499.
Amarasingham, R., Plantinga, L., Diener-West, M., Gaskin, D. J., & Powe, N. R. (2009). Clinical information technologies and inpatient outcomes: a multiple hospital study. Archives of internal medicine, 169(2), 108-114.
Amid, A., Moalagh, M., & Ravasan, A. Z. (2012). Identification and classification of ERP critical failure factors in Iranian industries. Information systems, 37(3), 227-237.
Asan, U., Kadaifci, C., Bozdag, E., Soyer, A., & Serdarasan, S. (2018). A new approach to DEMATEL based on interval-valued hesitant fuzzy sets. Applied Soft Computing, 66, 34-49.
Boral, S., Howard, I., Chaturvedi, S. K., McKee, K., & Naikan, V. (2020). An integrated approach for fuzzy failure modes and effects analysis using fuzzy AHP and fuzzy MAIRCA. Engineering Failure Analysis, 108, 104195.
Büyüközkan, G., Güleryüz, S., & Karpak, B. (2017). A new combined IF-DEMATEL and IF-ANP approach for CRM partner evaluation. International Journal of Production Economics, 191, 194-206.
Cebeci, U. (2009). Fuzzy AHP-based decision support system for selecting ERP systems in the textile industry by using a balanced scorecard. Expert systems with applications, 36(5), 8900-8909.
Chang, T.-H., & Wang, T.-C. (2009). Using the fuzzy multi-criteria decision-making approach for measuring the possibility of successful knowledge management. Information sciences, 179(4), 355-370.
Chen, Z., Lu, M., Ming, X., Zhang, X., & Zhou, T. (2020). Explore and evaluate innovative value propositions for the smart product-service system: A novel graphics-based rough-fuzzy DEMATEL method. Journal of Cleaner Production, 243, 118672.
Chou, Y.-C., Sun, C.-C., & Yen, H.-Y. (2012). Evaluating the criteria for human resource for science and technology (HRST) based on an integrated fuzzy AHP and fuzzy DEMATEL approach. Applied Soft Computing, 12(1), 64-71.
Côrte-Real, N., Ruivo, P., Oliveira, T., & Popovič, A. (2019). Unlocking the drivers of big data analytics value in firms. Journal of Business Research, 97, 160-173.
Cresswell, K., & Sheikh, A. (2013). Organizational issues in the implementation and adoption of health information technology innovations: an interpretative review. International journal of medical informatics, 82(5), e73-e86.
Dauwed, M., Yahaya, J., Mansor, Z., Hamdan, A. R., & Meri, A. (2019). Health Information Exchange: Current Trends, Challenges and Recommendations. Paper presented at the 2019 International Conference on Electrical Engineering and Informatics (ICEEI).
Delone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: a ten-year update. Journal of management information systems, 19(4), 9-30.
Deng, H. (1999). Multicriteria analysis with a fuzzy pairwise comparison. International journal of approximate reasoning, 21(3), 215-231.
Du, Y.-W., Wang, S.-S., & Wang, Y.-M. (2019). Group fuzzy comprehensive evaluation method under ignorance. Expert systems with applications, 126, 92-111.
Durán, O., & Aguilo, J. (2008). Computer-aided machine-tool selection based on a Fuzzy-AHP approach. Expert systems with applications, 34(3), 1787-1794.
Fichman, R., Kohli, R., Krishnan, R., & Kane, G. (2011). The Role of Information Systems in Healthcare: Current Research and Future Trends, the lead article by the senior editors in. Information Systems Research, 22(3), 419-428.
Gesulga, J. M., Berjame, A., Moquiala, K. S., & Galido, A. (2017). Barriers to electronic health record system implementation and information systems resources: a structured review. Procedia Computer Science, 124, 544-551.
Grover, V., & Kohli, R. (2012). Cocreating IT value: New capabilities and metrics for multi-firm environments. Mis Quarterly, 225-232.
Hanafizadeh, P., & Ravasan, A. Z. (2018). An empirical analysis of outsourcing decision: the case of e-banking services. Journal of enterprise information management.
Hiete, M., Merz, M., Comes, T., & Schultmann, F. (2012). Trapezoidal fuzzy DEMATEL method to analyze and correct for relations between variables in a composite indicator for disaster resilience. OR spectrum, 34(4), 971-995.
Hsieh, T.-Y., Lu, S.-T., & Tzeng, G.-H. (2004). Fuzzy MCDM approach for planning and design tenders selection in public office buildings. International journal of project management, 22(7), 573-584.
Javanbarg, M. B., Scawthorn, C., Kiyono, J., & Shahbodaghkhan, B. (2012). Fuzzy AHP-based multi-criteria decision making systems using particle swarm optimization. Expert systems with applications, 39(1), 960-966.
Kabir, G., Sumi, R. S., Sadiq, R., & Tesfamariam, S. (2018). Performance evaluation of employees using a Bayesian belief network model. International Journal of Management Science and Engineering Management, 13(2), 91-99.
Kahraman, C., Cebeci, U., & Ulukan, Z. (2003). Multi‐criteria supplier selection using fuzzy AHP. Logistics information management.
Kaye, R., Kokia, E., Shalev, V., Idar, D., & Chinitz, D. (2010). Barriers and success factors in health information technology: A practitioner's perspective. Journal of Management & Marketing in Healthcare, 3(2), 163-175.
Khajouei, R., Gohari, S. H., & Mirzaee, M. (2018). Comparison of two heuristic evaluation methods for evaluating the usability of health information systems. Journal of biomedical informatics, 80, 37-42.
Khalifa, M. (2014). Technical and human challenges of implementing hospital information systems in Saudi Arabia. Journal of Health Informatics in Developing Countries, 8(1).
Khan, A. A., Shameem, M., Kumar, R. R., Hussain, S., & Yan, X. (2019). Fuzzy AHP based prioritization and taxonomy of software process improvement success factors in global software development. Applied Soft Computing, 83, 105648.
Lee, Y.-C., Li, M.-L., Yen, T.-M., & Huang, T.-H. (2011). Analysis of Fuzzy Decision Making Trial and Evaluation Laboratory on the technology acceptance model. Expert systems with applications, 38(12), 14407-14416.
Lluch, M. (2011). Healthcare professionals’ organisational barriers to health information technologies—A literature review. International journal of medical informatics, 80(12), 849-862.
Ludwick, D. A., & Doucette, J. (2009). Adopting electronic medical records in primary care: lessons learned from health information systems implementation experience in seven countries. International journal of medical informatics, 78(1), 22-31.
Ly, P. T. M., Lai, W.-H., Hsu, C.-W., & Shih, F.-Y. (2018). Fuzzy AHP analysis of Internet of Things (IoT) in enterprises. Technological Forecasting and Social Change, 136, 1-13.
Mahmoudi, S., Jalali, A., Ahmadi, M., Abasi, P., & Salari, N. (2019). Identifying critical success factors in Heart Failure Self-Care using fuzzy DEMATEL method. Applied Soft Computing, 84, 105729.
Meri, A., Hasan, M. K., Danaee, M., Jaber, M., Safei, N., Dauwed, M., . . . Al-bsheish, M. (2019). Modelling the utilization of cloud health information systems in the Iraqi public healthcare sector. Telematics and Informatics, 36, 132-146.
Mohammadfam, I., Aliabadi, M. M., Soltanian, A. R., Tabibzadeh, M., & Mahdinia, M. (2019). Investigating interactions among vital variables affecting situation awareness based on the Fuzzy DEMATEL method. International Journal of Industrial Ergonomics, 74, 102842.
Najaftorkaman, M., Ghapanchi, A. H., Talaei‐Khoei, A., & Ray, P. (2015). A taxonomy of antecedents to user adoption of health information systems: A synthesis of thirty years of research. Journal of the Association for Information Science and Technology, 66(3), 576-598.
Nguyen, L., Bellucci, E., & Nguyen, L. T. (2014). Electronic health records implementation: an evaluation of information system impact and contingency factors. International journal of medical informatics, 83(11), 779-796.
Pai, F.-Y., & Huang, K.-I. (2011). Applying the technology acceptance model to the introduction of healthcare information systems. Technological Forecasting and Social Change, 78(4), 650-660.
Qureshi, Q. A., Shah, B., Kundi, G. M., Nawaz, A., Miankhel, A. K., Chishti, K. A., & Qureshi, N. A. (2013). Infrastructural barriers to e-health implementation in developing countries. European Journal of Sustainable Development, 2(1), 163-163.
Rajak, M., & Shaw, K. (2019). Evaluation and selection of mobile health (mHealth) applications using AHP and fuzzy TOPSIS. Technology in Society, 59, 101186.
Ratwani, R. M., Reider, J., & Singh, H. (2019). A decade of health information technology usability challenges and the path forward. Jama, 321(8), 743-744.
Ravasan, A. Z., Hanafizadeh, P., Olfat, L., & Taghavifard, M. T. (2017). A fuzzy TOPSIS method for selecting an e-banking outsourcing strategy. International Journal of Enterprise Information Systems (IJEIS), 13(2), 34-49.
Rouhani, S., Ashrafi, A., & Afshari, S. (2013). Segmenting critical success factors for ERP implementation using an integrated fuzzy AHP and fuzzy DEMATEL approach. World Applied Sciences Journal, 22(8), 1066-1079.
Rouhani, S., Ashrafi, A., & Afshari, S. (2014). Fuzzy DEMATEL Model for Evaluation Criteria of Business Intelligence. Paper presented at the ICEIS (1).
Saaty, T. L. (1990). How to make a decision: the analytic hierarchy process. European journal of operational research, 48(1), 9-26.
Salge, T. O., Kohli, R., & Barrett, M. (2015). Investing in information systems: On the behavioral and institutional search mechanisms underpinning hospitals’ IS investment decisions. Mis Quarterly, 39(1), 61-90.
Sebetci, Ö. (2018). Enhancing end-user satisfaction through technology compatibility: An assessment of the health information system. Health Policy and Technology, 7(3), 265-274.
Singh, P. K., & Sarkar, P. (2020). A framework based on fuzzy Delphi and DEMATEL for sustainable product development: A case of Indian automotive industry. Journal of Cleaner Production, 246, 118991.
Sirisawat, P., & Kiatcharoenpol, T. (2018). Fuzzy AHP-TOPSIS approaches to prioritizing solutions for reverse logistics barriers. Computers & Industrial Engineering, 117, 303-318.
Teixeira, J. G., de Pinho, N. F., & Patrício, L. (2019). Bringing service design to the development of health information systems: The case of the Portuguese national electronic health record. International journal of medical informatics, 132, 103942.
Thakkar, M., & Davis, D. C. (2006). Risks, barriers, and benefits of EHR systems: a comparative study based on the size of the hospital. Perspectives in Health Information Management/AHIMA, American Health Information Management Association, 3.
Tornatzky, L. G., Fleischer, M., & Chakrabarti, A. K. (1990). Processes of technological innovation: Lexington books.
Tseng, M.-L. (2009). Using the extension of DEMATEL to integrate hotel service quality perceptions into a cause-effect model in uncertainty. Expert systems with applications, 36(5), 9015-9023.
Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186-204.
Wang, Y., Xu, L., & Solangi, Y. A. (2020). Strategic renewable energy resources selection for Pakistan: Based on SWOT-Fuzzy AHP approach. Sustainable Cities and Society, 52, 101861.
Wu, W.-W. (2012). Segmenting critical factors for successful knowledge management implementation using the fuzzy DEMATEL method. Applied Soft Computing, 12(1), 527-535.
Yazdi, M., Khan, F., Abbassi, R., & Rusli, R. (2020). Improved DEMATEL methodology for effective safety management decision-making. Safety Science, 127, 104705.
Zadeh, L. A. (1965). Fuzzy sets. Information and control, 8(3), 338-353.
Zhang, L., Stoffel, A., Behrisch, M., Mittelstadt, S., Schreck, T., Pompl, R., . . . Keim, D. (2012). Visual analytics for the big data era—A comparative review of state-of-the-art commercial systems. Paper presented at the 2012 IEEE Conference on Visual Analytics Science and Technology (VAST).
Zhang, X., & Su, J. (2019). A combined fuzzy DEMATEL and TOPSIS approach for estimating participants in knowledge-intensive crowdsourcing. Computers & Industrial Engineering, 137, 106085.
Zhou, Q., Huang, W., & Zhang, Y. (2011). Identifying critical success factors in emergency management using a fuzzy DEMATEL method. Safety Science, 49(2), 243-252.