In the cyberspace, the health webmaster could be assimilated to an editor in chief of a journal, in charge of the content and the design. He/she must follow quality criteria to build any resource: at Rouen University Hospital (RUH), we have chosen the Net Scoring as the tool to design a Web site of quality. The Net Scoring contains a list of 49 criteria which fall into eight categories: credibility, content, links, design, interactivity, quantitative aspects, ethics, and accessibility. The webmaster is the key element of the editorial board process. He/she must regularly monitor his/her Web site to get information whether the site is used and by whom: the method most commonly used is the log analysis. At RUH, an average of 4,000 machines is visiting our Web site each working day. Conclusion: webmaster is a new job opportunity for medical informaticians and medical librarians who are both information science professionals.

At the dawn of the third millenium, the Internet has become a major source of information for the health professional and the Netizen. The resources (sites and documents) available on the Internet about health in general and medicine in particular grows exponentially. Today, CISMeF (Catalog and Index of French-speaking Health Resources on the Internet) indexed over 8,200 resources [1]. Its URL (Universal Resource Locator) is http://www.chu-rouen.fr/cismef. The CISMeF publisher is the Rouen University Hospital (RUH). Extrapolating from the number of HTML pages written in French in 1998 [2], we estimate that French-speaking health resources are only represent 2% of world resources. Therefore, the number of world health should be over 400,000.

All these sites must be managed by a webmaster. The existence of a webmaster is one key quality criterion of a health site. By analogy with a paper journal, a webmaster could be assimilated to the editor in chief. He/she is in charge of the content and the design. We will detail in the paper all the missions of a health webmaster, which are quite heterogeneous.

To help French-speaking webmasters to create a Web site of quality, we have created under "Centrale Santé" umbrella the Net Scoring [3]. "Centrale Santé" is a non-profit organization grouping engineers (from the Ecole Centrale de Paris), physicians, librarians and lawyers. The multidisciplinary of this group should ensure the maximum of objectivity.

The Net Scoring is a list of 49 criteria to assess the quality of health information on the Internet. The French National Council of Physicians was a key member to elaborate the Net Scoring. The Net Scoring first version was finalized in August 1997 (URL http://www.apuis.com/sante/fqualic0.htm). It was then largely inspired by a US study [4]. The updated version is available at the following URL: http://www.chu-rouen.fr/dsii/publi/netscoring.html.

These 49 criteria fall into eight categories: credibility, content, hyperlinks, design, interactivity, quantitative aspects, ethics and accessibility. Every criterion has a weight: essential criterion rated from 0 to 10, important criterion rated from 0 to 5 and minor criterion rated from 0 to 2. The total of these weighted criteria gives the overall score of a site (maximum = 297). All the criteria were chosen by consensus. This set of criteria is an indirect indicator of the relevance and the utility of a Web site [5].

Concerning quantitative aspects, the webmaster need to precisely analyze his/her Web site logs to determine statistics of usage. The RUH Web site is currently using an Apache server on a Linux machine. The webtrends log analyzer version 5 is used to generate statistics of use from the log analyze after excluding requests performed by the RUH 2,000 computers.

The webmaster have to respect several quality criteria concerning the content of his/her Web site. At RUH, there is a couple of webmasters: one medical informatician and one medical librarian. We do respect most of the Net Scoring criteria but not all of them (i.e. we do not distinguish between internal and external hyperlinks). Using auto-evaluation, the score of the RUH is 247/297 (0.83).

Some of these Net Scoring criteria are essential to fulfill: (a) to respect deontological rules and ethics code of conduct, i.e. excluding in the Web site any personnel data from the Netizen or ensuring the maximal security of these data (encryption, authentication); (b) for every document of the site, to establish clearly the source: name and credentials of the authors, name of the institution, including date of creation, date of last update and date of last revision; (c) to follow an editorial board process where the webmaster must play a key role; to list name and credentials of the editorial board. At RUH, in 1996, an editorial board, including the two webmasters, has been created to control quality and coherence of the Web site; (d) to optimize the Web site navigability to help the Netizen and/or the health professional to find quickly and easily the relevant information, using internal search engine, what's new page, help page and map site; (e) to answer systematically to every Email received. At RUH, we receive daily a dozen of Email, most of them from Netizens: i.e. we answer that our site is not performing teleconsultation; and (f) promote the Web site, optimizing its accessibility among the main search engines and catalogues and the main academic and government Web sites.

3.1. Use patterns of a Web site

One of the key role of the webmaster is to monitor as precisely as possible the Web site to get information whether the site is used and by whom: the methods most commonly used are the log analysis and questionnaire. At RUH, during two months (October and November 1999), a questionnaire about the quality of the Web site based on the Net Scoring was proposed to every user on a voluntary basis. We will not detail its results because only 55 questionnaires were filled (less than 0.05% of the users) generating too much biases.

The Net Scoring defined two reproducible criteria to measure quantitatively a Web site: (a) number of machines (IP addresses) visiting the site and (b) number of visualized documents or number of requests for a dynamic site. The number of machines is a criterion to estimate the number of Netizens visiting the Web site. These statistics underestimate the real figures due to the practice of file caching.

At RUH, the Web server software, which provides documents to users on request, does not know the identities of individual users, such as E-mail. We refuse to use cookies to obtain personal data from Netizens; the only identifying data available are the Internet IP addresses of the machines from which the users connect to the site.

Analysis of a representative period, the month of January 2000, showed that every working day over 4,000 machines visited our site (excluding ours), loading over 20,000 HTML pages and performing over 1,000 requests on our internal search engine (18.44% of our visitors use this tool). During the entire month, users from different 72,323 computers made 631,980 requests for HTML documents (1,195,899 hits) from this Web site originating from 124 different countries (34.38% from France, 26.04% from USA, 10.49% from Canada, 3.72% from Belgium, 2.67% from Switzerland and only 0.23% from Africa). The geographical location of the connecting machine could not be determined in the remaining 17.98% of document retrievals (Internet IP addresses without domain name). The average duration of a visit is 9'24''. The average number of different machines was 5,500 in 1996, 11,046 in 1997, 18,826 in 1998 and 36,587 in 1999. The use patterns of RUH Web site is available at the following URL: http://www.chu-rouen.fr/dsii/html/stat.html.

We also use the following indicators to measure the current impact and potential ongoing future usage of our site: (a) Web impact factor [6], number of sites, which have at least one hyperlink to our site [http://www.chu-rouen.fr/dsii/html/pointeur.html]; (b) number of press releases; (c) number of scientific publications about the Web site; and (d) polls.

Currently, our Web impact factor is over 800, including the most prestigious Health resource catalogues (CliniWeb-Us, DDRT-Se, HON-Ch, MedWebplus-Us, and OMNI-Uk). Altavista indicates more than 2,800 pages after exclusion of our internal links with the following request: +link:chu-rouen.fr -url:chu-rouen.fr.

Over 150 press articles released information about our Web site [http://www.chu-rouen.fr/dsii/html/presse.html]. The scientific publications about the RUH Web site (N=33) are available at the following URL: http://www.chu-rouen.fr/general/pubweb.html.

In November 1997, in March 1998 and in November 1998,, three surveys from the CESIM (Centre d'Etudes sur le Support de l'Information Medicale) showed that the RUH's Web site was the most frequently used by general practitioners (GP) with respectively 10%, 17.7 % and 11.5% of their votes. In the last poll, only 11% of the GPs have an Internet connection, we can estimate that the RUH's Web site is currently used by 1.2% of the French GPs. In March 1998, CISMeF has obtained the label "Experimentation of public interest" by an inter-ministry committee (procedure "Information Highways").

The webmaster must analyze the main documents and requests of his/her Web site. He/she may go further, to include a cost-efficiency analysis of the Web site. Some pages are very time consuming to maintain manually (i.e. the WIF page vs. the Alta Vista request). The success of some pages may stay without an answer: i.e. the number of access of the RUH medical library home page (N=4685 in January 2000) is still a mystery for us. At RUH, there is a huge difference between the main requests performed on our internal search engine and the main requests performed on external search engines. Only 2 of 20 were common of these two lists: malaria and pubmed. The webmaster should also check the accessibility of his/her Web site, including its presence in the main external search engines. At RUH, approximately 80% of the requests are performed on three search engines: AtlaVista (N=15,720; 31.59%), Yahoo (N=14,041; 28.22%) and Excite (N=10,689; 21.48%).

The webmaster must look after some technical elements, such as number of files or network traffic. In January 2000, the RUH Web site contained 13,605 hyper-links (including 3,687 internal hyper-links), 3790 files (1965 HTML, 1112 GIF, and 250 JPG), and 218 megabytes of disk space. That is why it is of utmost importance to regularly check their validity. A software checks every hyper-link of the Web site each month. Then the webmaster team manually updates the broken links. The network traffic generated by the different Internet protocols (http, smtp, and nntp) represent between 0.3 and 1.5% (from 100 to 300 Mb/j) of the overall network traffic at RUH.

The role of the webmaster is the success of a Web site has not yet measured, even in our academic world [7]. The following search in Medline using Pubmed [URL: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?CMD=Display&DB=PubMed] is surprising: 2532 references (2-23-00) were found with the MeSH keyword "Internet", although this keyword appeared in the MeSH thesaurus lately in 1999. Webmaster is not (yet) a MeSH keyword. Therefore, we performed the following request in all the fields of the Medline database "webmast*" and found only 5 relevant references.

Webmaster is a new job opportunity for medical informaticians and medical librarians. We consider the information science professionals are the best choice to become health webmasters in charge of the content of a site as an editor in chief. According to us, they are better fit than computer scientists or designers. Nonetheless the two latest have an important role to play in the webmaster team.

The webmaster has to train people from his/her institution that wants to create Web site. Some points are crucial (1) structuring information, using metadata to optimize in the future the search of information. At RUH, we follow the Dublin Core initiative [8] with other institutions; Australian Department of Health and Aged Care [http://www.health.gov.au/], Australian Institute of Health and Welfare [http://www.aihw.gov.au/], Better Health Channel [http://www.betterhealth.vic.gov.au/], National Health and Medical Research Council [http://www.nhmrc.health.gov.au/], WHQW (Women's Health Queensland Wide) [http://www.womhealth.org.au/], and WHO (World Health Organization) [http://www.oms.ch/]; (2) improving the use of quality control. The main role of the health webmaster in the editorial process is not yet established reading the literature. The webmaster work is too often limited to update information without any control of content and design. We consider that the health webmaster must play a key role in the editorial process.

At RUH, the webmaster "couple" (one medical informatician and one medical librarian) could explain lot of the success of the Web site because there is a synergy between these two information scientists [9] although their views about information is different because of their training and experience. The RUH webmaster team also includes one deputy medical informatician in charge of the technical support, including Java script programming and one deputy medical librarian in charge of the daily management of the Web site (statistics, press releases, WIF).

According to us, it is obvious that the health webmaster must be (and become if not yet) a multidisciplinary team leader. His role may sometimes be appreciated as complex and paradoxical because his/her skills must be numerous and various: (a) in charge of the content and the design, following quality criteria to build the Web site; (b) technical ability to parameter statistical tools; (c) able to take strategic decisions (i.e. technical choices, such as the place of XML, Flash technology, use of a database for all or part of the Web site). The role of the webmaster requires multiple competencies which can be resolved more easily by a team more than one person.

Conclusion: Webmastering is a new job opportunity for medical informaticians and medical librarians. We think that webmastering training will soon appear in the academic courses of information science professionals.

CISMeF was supported in part by the grant n° 1998 06 016 from the Agence Universitaire de la Francophonie in the program "French-speaking Virtual University".