Evaluation of the quality of website about health using marketing scoring

Introduction

A website is one of the most effective tools for informing the general public about various issues in today's world. Questions about healthy behavior, consumption, care for one's physical and emotional state, information about healthy products should be posted on specialized resources. There are also other types of resources – commercial ones that advertise a particular brand or official sites of business entities, such as clinics, health centers, pharmacy corporations, etc. The specialization of the resource increases trust in it. It stimulates the growth of the user's time on the website. The relevance of the web resource components forms its better position in search queries and increases its popularity through more links from third-party sites.

The issue of health care and the prevention of various diseases is significant. This topic has become especially popular since 2010 and after the outbreak of the global COVID-19 pandemic in 2020. Analysis of search queries in the online service Google Trends (Fig. 1) for the last five years shows that the topic of "healthy" and "healthcare" remains steadily popular without significant peaks. The dynamics of search queries "Pandemic" and "vaccination" is characterized by a rapid increase in the most critical periods (the efforts of governments in many countries to accelerate the vaccination process; March 2020, when the pandemic was the most discussed issue in the world). People are characterized by behaviour when their desire to find information about various aspects of health directly depends on events in society: "Today I want to take care of my health and the health of my loved ones", but when threats disappear, health problems recede, they return to the attractive by profession or personal preferences search engine behaviour on the Internet.

behavior – how users seek health information and what resources they trust. The segmentation of users in the field of health information search shows that the majority are women (72% women vs. 28% men). If before 2019 the leading majority of active health information seekers were people aged from 50 to 64 (over 70% of all those who were interested in health), then after 2019, more than half of users who are interested in sites and news on the health subject, are persons aged 18-29 years. It is due to the growing penetration of the Internet into the lives of modern people, a significant number of referral links on social networks, and, of course, the ever-increasing popularity of "healthy" disease prevention and responsible attitude to themselves and the environment, which provoked, in particular, and events related to the COVID-19 pandemic. Direct search does not yet predominate when going to health sites. Instead, organic is predominant when users visit health sites on a search engine query. More than 60% of active Internet users say that the network has improved how they take care of their health. The Internet remains a popular source for finding medical information due to its accessibility around the clock, the anonymity of the information seeker and a large amount of easily accessible information. Interestingly, almost 30% of users say that they check the accuracy of the information on several health sites at once. People under the age of 40 rely more on health information available online.

Fig.1. The popularity of search queries in the Google Trends service from 2016 to 2021

In our opinion, the tasks of resources that disseminate information in the field of health are:

1) achieving consumer satisfaction with the information received on the resource (its relevance to the search query, reliability, comparability with other available resources);

2) attracting the user's attention due to the quality of the content that accompanies the information posted on the resource (even advertising on the health portal must be accurate);

3) gaining consumer support (forming a user's desire to revisit the resource or recommend it to other users). It could be a recommendation to friends or acquaintances. Another variant is when a person decides to "share" information with a link to the resource on social networks or places a link to the particular web resource on an own website or in a scientific publication (which is typical given the specifics of information posted on health portals, the structure of whose users include, in particular, scientists from various fields of research).

To understand the potential of a website in the field of health, the directions of its further promotion, one should identify its strengths, weaknesses and unique opportunities for development in the online community through the use of appropriate marketing tools. The research aims to determine the quality of several Ukrainian health websites and to propose measures to improve their attractiveness to potential readers and advertisers.

Literature Review

Tahir, M. et al (2020)  focus on researching the quality and availability of online health information. They justify the choice of sites for analysis by prioritizing their issuance in the search engine. The researchers use Google, Bing and Yahoo search engines for gathering this information. They check the quality of the information provided by 25 selected sites using the DISCERN tool. It looks like a questionnaire for assessing the quality of written health information. To increase the reliability of the analysis, they divide all the links into two groups –  professional medical sites and sites for all other entities. In addition, they use the Flesch-Kincaid tool to assess the availability of information on the site for its perception for different categories of the population, mainly distributed by age groups. The DISCERN methodology is used to determine the quality of information on health websites by many other scientists from different countries (Allam et al., 2017, Charnkock et al., 1999, Portillo et al., 2021,  Kaicker et al., 2010). It is noteworthy that these works analyzed many thematic sites, some of which are commercial with the presentation of information about health products. The authors explore websites with the information presented in different languages, interactive and static, sites of international health organizations, etc

Researchers also assess the quality of the information provided by mobile applications, whose ethics and customer support are critical. Nouri et al. (2018) identify criteria for evaluating mobile health applications, including design, quality of content and information, usability, functionality, ethical issues, user security, privacy settings and individual perceived parameters of users of these applications

Petroye et al (2020), Artyukhov et al. (2021), Vakulenko et al. (2021), Oteh et al. (2021), Tielietov & Letunovska (2014) pay attention is paid to a responsible attitude to the disclosure of public information in the media. Scientific works differ in the object of study and reveal different criteria for achieving transparency of information sources and the responsibility of various economic entities or entire countries and regions

Robillard et al. (2018) emphasize that health information posted online is complicated to regulate in any way, and its quality and reliability can vary greatly depending on the source. There is no single approved quantitative tool for assessing the quality of health information to date. However, such a tool should be easy to use, fast and concise. The authors of this article offer a QUEST test for assessing information, which includes seven different questions. They analyze data on the prevalence of Alzheimer's disease in society.

Tao el al. (2018) developed a conceptual framework for assessing the quality of information on the health website according to the relevant criteria and quality drivers, emphasizing the use of this information by young people. Both quantitative and qualitative approaches are used in this study. They conducted semi-structured group interviews and individual assessments. The evaluation is based on visits to various websites and answers to questions on the Likert scale on the importance of health information quality parameters and answers to open-ended questions about possible drivers for improving the quality of information posted on thematic sites. As parameters for quality assessment, the authors propose five components: completeness of information, comprehensibility of data, the relevance of information, depth of knowledge and its accuracy. As a result, the information's completeness and clarity were identified as the main parameters. Among the drivers of information quality, respondents included site content, design, link quality, site functionality, ease of access, resource policy and performance.

Calvano et al. (2021) emphasize that healthcare organizations must provide relevant web resources with quality information. According to their methodology, health websites were evaluated in four categories: accessibility (as the ability of even people with low computer literacy to access website information; marketing component (as the ability of a website to appear in a list of user searches and its rank in this list), content quality (absence of errors in the text, frequency of updating information, the relevance of material and its readability; technological component (page load speed, quality of program code and website infrastructure).

As the COVID-19 pandemic continues to spread worldwide, information on preventing the disease must be of high quality and readable to different categories of readers. Kelly et al. (2021) analyze the quality of data based on English-language search queries about COVID-19 using a combination of evaluation tools such as JAMA benchmarking analysis, DISCERN criterion and HONcode certification. The readability of each website was assessed according to three rating systems: Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL) and Gunning-Fog Index (GFI).

Table 1. Comparison of tools for measuring the quality of online health information (formed according to data from (Robillard et al., 2018, Roberts, 2010)

Table 1 compares the methods to assess the quality of online health information.

From the list of tools in Table 1, those that focus directly on the qualitative component of health assessment of websites and are suitable for sites without restrictions on their focus and content are selected (QUEST test, DISCERN, HONcode and Sandvik's General Quality). It is advisable to compare the components of these methods of assessing the quality of websites in the field of health (Table 2). HONcode method contains the largest number of components. But it does not include such essential components in today's conditions of digital development as site navigation, emotional characteristics of content, complementary of information, opportunities to share it primarily through social networks. The current level of requirements for web resource development creates the need to refine the existing methods and, without overriding undoubted advantages, compose a new comprehensive methodology for assessing the quality of thematic health sites.

 Methodology

Scoring models and scoring cards are actively used in world practice, but this tool is relatively new in marketing. Scoring functions are the types of scoring values ​​available for the selected model. The essence of scoring is that each parameter of the health website is quantified in points. To develop an effective scoring model, one must first choose factors that have the most significant impact on visitors of health sites. The authors have previously analyzed various sources of information on the marketing attractiveness of websites and behavioural aspects of users in the health sector (Yoshida et al., 2020, Wirtz et al., 2020, Zhang et al., 2019, Gupta, 2021).

Table 2. Components of evaluating the quality of health websites using the most common methods

 Results

The authors chose sites with Ukrainian domain names that provide health information. The relevance of the information posted on the site was taken into account. The information in the footer of the site marked © - 2022, which indicates that the site's developers timely correct the text on the pages of the site, was taken into account. The sites are:

- zdorovia.com.ua (news site with sections on health, beauty, cosmetology, medicine with articles on human health, treatment and healthy eating);

- ukrhealth.net (a site with articles and news about health, with expert opinions from specialized specialists);

- medicina.ua (portal-catalog with a list of medical institutions and articles about health);

- health-ua.com (a specialized site in the field of health care, designed for professionals with medical education);

- jazdorov.com.ua (health information portal with thematic articles).

The list of sites also includes, for comparison, the government's recently launched internet project znaimo.gov.ua, which is a site about healthy eating which focuses on such a target audience as students of Ukrainian schools and their parents. This portal results from a memorandum signed between UNICEF, the Embassy of the Swiss Confederation in Ukraine and the Ministry of Education and Science of Ukraine. This web resource is part of the country's information and communication strategy of school nutrition reform.

The portal contains information about a balanced diet. There are categories of articles for parents, educators, founders of educational institutions, business representatives who want, for example, to join the organization of food in educational institutions, etc. The list of sites was also supplemented by a pilot internet project on mental health mh4u.in.ua. This resource includes articles, training courses and webinars on mental health.

Table 3 presents the ranking positions of sites according to the version of the resource Similarweb (for those that are displayed by the resource).

Table 3. Comparison of selected health sites by Similarweb rankings

The metrics in Table 3 with rankings by category show the actual positions of sites by traffic among users in a particular region or category. Based on these data, one could conclude about the leading positions of the site zdorovia.com.ua among competitors both in the global ranking and sites within Ukraine. However, in ranking sites in the category "Health" that interests the researchers, the undisputed leader is another web resource –  health-ua.com. It almost catches up with the site zdorovia.com.ua in the global ranking. However, such rankings are not very informative for understanding the future development potential of the web resource, changes in its popularity among users and qualified health professionals. The authors of this research propose evaluating health sites according to the methodology, which contains three components: technical, content and marketing. It is convenient to use a structured questionnaire with parameters with the scale (Table 4). Indicators 1-12 in Table 4 have a scale from 1 to 3, which is explained by three possible choices for a particular site. Indicators 13-20 have a scale from 1 to 2 and are dichotomous. The number of site pages in the Google index is determined by the search query "site: … ". The date of creation of the site can be determined using various resources. The authors used the online resource WHOIS. The continuity of the web resource is an essential indicator of its reputation and loyalty among users. The uptime of the resource can be determined by various services, for example, Uptime Robot. As for the availability of certain sections of the site, in the absence of the Contacts section, its visitors lose confidence in it. The privacy policy has become a mandatory attribute of sites that care about their reputation in the online space. The FAQ section (with frequently asked questions) establishes communication with potential and regular site visitors. Sites can be designed using various technologies, the most common of which are HTML and the use of SMS engines. It is noteworthy that the first of these technologies sites are mostly more dynamic, with a unique design, but, however, if such a site needs to change in the Menu section at least one word that is on all pages of the site, you have to edit all pages at once. Due to the multifaceted nature of the development of health websites and the importance of constantly updating information on such portals, it is more appropriate and resource-efficient to use CMS engine technology. If for entertainment sites, quiz sites, promotional sites, the use of individual HTML technology in development is a significant competitive advantage, then for sites in the field of health, the technology used to design is not crucial and will not be taken into account when building a site quality scoring map.

Table 4. Scoring map to determine the quality of the health site

The parameters in Table 4 are grouped by questions. The weight of three groups of variables is determined, Table 5. The resulting indicators are as follows: - from 17.2 to 15 – health site of high quality - from 14.99 to 10 – a health site of medium quality; - below 10 –  health site of low quality. Table 6 summarizes the types of health sites based on combinations of defined values of their quality assessment components.

Table 5. Structuring the score map questions to assess the quality of the health site

Fig. 2. The “perfect” combination of health components

Table 6. Typology of health sites depending on combinations of quality assessment components

Table 7. The score of health sites based on the parameters of the proposed scoring map

* 1 –  zdorovia.com.ua; 2 – ukrhealth.net;  3 – medicina.ua;  4 – health-ua.com;  5 – jazdorov.com.ua;  6 – znaimo.gov.ua; 7 – mh4u.in.ua

The combination of the site's components reflects the "ideal" state of the web resource when the technical, content and marketing components receive high marks during the evaluation. It indicates the balance of the functioning of the site. It shows that developers take into account and, if necessary, improve the various components of the success of the web resource, Fig. 2. Table 7 shows the results of the evaluation of health sites by 20 parameters.

According to the analysis results in table 7, the best positions are estimated at the site health-ua.com with a score of 13.8. This site has high rating scores for all three components. It should be noted that none of the analyzed sites, according to the results of calculations, weren't included in the group of high-quality sites. All sites were included in the group of medium quality, except for the site medicina.ua, which was included in the group of low-quality sites.

Experts evaluated the indicators of the content component. Two experts are involved in assessing the content component of the analyzed sites. So it is necessary to check the degree of consistency of their opinions when providing positive and negative assessments of the state of a particular indicator. The best values ​​of the parameter were taken from each of the experts if they did not match. The auhors analyzed the consistency of opinions by the weighted Kappa method, Table 8. Researchers use reliability analysis by the Kappa Fleis method to investigate the coincidence of several expert assessments on an alternative basis. It makes it possible to determine the scale's reliability through the consistency of inter-expert assessment. The value of the criterion could reach a value of 1.0 with the complete concordance of results, and its minimum possible value is zero with the complete mismatch. There are different approaches to interpreting the values ​​of the Kapp test, which are proposed in the works of various scientists (Landis & Koch, 1977, Altman, 1991, Vanbelle, 2018, Letunovska et al., 2020).

The authors considered the values ​​presented in (Landis & Koch, 1977) to assess the consistency of experts' opinions on the quality of health sites in terms of their content component. According to the scale of the Kappa coefficient in (Landis & Koch, 1977), one could talk about the following measures of agreement of experts' opinions: from -1 to 0 – there is no agreement of opinions; from 0 to 0.2 – consistency is very weak; from 0.2 to 0.4 –  good consistency of opinions; from 0.4 to 0.6 – moderate consistency of opinions; from 0.6 to 0.8 – a significant degree of agreement; from 0.8 to 1.0 – full agreement.

Table 8. Assessment of the consistency of experts' opinions on the evaluation of the content component of the analyzed health sites by the weighted Kappa method

The expert assessments of analyzed sites for the majority (six out of seven) showed that the degree of agreement of experts' opinions is quite high. The value of the Kappa indicator is at the level of 0.333 for zdorovia.com.ua site, which indicates a good consistency of opinions, but indicates that experts have given quite different estimates of this site, which is one of the leaders of web sites in the category "Health" among other competing sites.

The proposed method makes it possible to classify sites according to estimates by individual components. This classification, as a result, makes it possible to demonstrate the position of each site for the three components and identify those aspects that need to be refined. Visualization of site positions for the three selected components is presented in Figure 3.

* 1 –  zdorovia.com.ua; 2 – ukrhealth.net; 3 – medicina.ua; 4 – health-ua.com; 5 – jazdorov.com.ua; 6 – znaimo.gov.ua; 7 – mh4u.in.ua

Fig. 3. Component health comparison of sites visualization

The most balanced on the components' development level are the sites marked with a diamond – ukrhealth.net and health-ua.com. All analysis components are at a sufficiently high level of development, and they are comparable, i.e., not significantly different. Such sites need further support and constant monitoring of the pages for the relevance of the content. The site medicina.ua, highlighted in Figure 3 by oval, has the weakest position in the marketing component, and to compete and don't lose positions in search results and grow a loyal audience, it needs work on its design, SEO, review links, etc. Website mh4u.in.ua has the weakest position in a technical component. Despite the excellent marketing attractiveness, the site will not win a leading position in the presence of a significant number of technical errors, downtime, disregard for its productivity for the search engine. Sites not highlighted in Figure 3 occupy intermediate positions and need to be audited on an ongoing basis to review content and follow the mood of the target audience to timely redirect content to that of interest to readers and advertisers.

Conclusion

Modern methods of assessing the quality of health sites and analysis of their features have shown a need to develop an innovative approach to evaluate the competitiveness of web resources with health information. The authors proposed a methodology to assess sites, based on three components of resource functioning, which determine its attractiveness in the eyes of the public, advertisers and, most importantly, the target audience, people interested in health (health professionals, people who monitor their health, people undergoing treatment recover from illness, etc.). This technique involves a combination of scoring according to clearly defined criteria of compliance of each score and expert evaluation with a preliminary analysis of the consistency of experts' opinions by the weighted Kappa method. The assessment made it possible to build a chart that visualizes the positions of the sites for each of the groups of components, followed by the definition of the strategy of working with classified health care sites. The prospect of further research is to find an information gap for health portals in Ukraine, as well as to determine the direction of content and focus on specific important issues for the new Internet resource highlighting problems current of healthy development of various territorial units and businesses in the field of treatment and prevention of public health problems.

This work was supported by the Ministry of Education and Science of Ukraine (research topic № 0122U000781 "The impact of COVID-19 on the transformation of the medical and social security system: economic, financial and budgetary, institutional and political determinants")

 Acknowledgment 

A special thank you to all the teachers and students who took the time to complete the questionnaire.

 Conflict of interest

The authors declare no potential conflict of interest regarding the publication of this work. In addition, the ethical issues including plagiarism, informed consent, misconduct, data fabrication and, or falsification, double publication and, or submission, and redundancy have been completely witnessed by the authors.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article