The attributes important for inclusion in the documentation of Tools, including performance validation

This document focuses on characterising the models used in VPH research, and discuses the requirements for models to be of maximal benefit in advancing the field. It looks at the importance of modelling for integrative research, the characteristics of the models used in bioengineering modelling, the development of minimum information standards associated with model descriptions, and the model encoding standards being developed for the VPH/Physiome project.

As computational models inevitably become more complex, it is increasingly difficult for anyone other than the author(s) of the publication describing the model to decipher, code and run the model in order to reproduce the results claimed in a publication. It can be very difficult to then use this model as one component of a more complex model. Furthermore, for any of these models to be useful in clinical practice the models must have been validated, be demonstrably reproducible and have a clearly defined dependence on the input data and the assumptions made in their formulation.

We therefore discuss what information is required alongside a model to facilitate its use and reuse within the context of the VPH. However, since this is also a topic of great interest to a wider community, we defer generally to guidelines already in existence, specifically the MIRIAM guidelines for annotation of models, and the MIASE guidelines for the description of reference simulations of models. The key findings are also suitably summarised in the Statement in Support of Joint Action on “Model Reproducibility for Physiome Research”:

The undersigned agree that concrete, coordinated action is vital to ensure accessibility, reproducibility, and reliability of the computational framework for understanding human and other eukaryotic physiology (the Physiome). Specifically, we regard the following points as Priority Items for the implementation of aims set out in the World Integrative Research Initiative agreement of 2007.

1. All publications, including modelling work, shall contain the necessary information required to reproduce published results.
2. Whenever possible, the relevant information should be offered via electronic supplement to journal publications and/or uploading to established international depositories.
3. The development of syntactic standards (such as suitable markup languages) and semantic standards (such as ontologies) shall be supported and their adoption/application will be encouraged where possible.

This document examines the challenge of Data Characterisation, in the context of data sharing. Raw data is of very limited utility to researchers, as ethical and legal issues, provenance, interoperability and sustainability all introduce constraints on data usage; these key concepts are central to the requirement for well-defined data characterisation, and form an essential set of additional criteria that must be made available by data providers if the VPH is to reap the maximum benefit from the successful sharing of data.

The rich variation in VPH data makes the process of characterisation highly complex. The approach taken in this document is not to create a comprehensive characterisation, but to identify the key issues and examine how these have been resolved in various ‘biomedical data-sharing plans’ that have evolved in related arenas. Each such standard is then examined from the VPH standpoint and its utility within the VPH is discussed. Many such plans work well in areas of limited scope, and can serve as a basis for the definition of some VPH characteristics; unfortunately no such plan is adequately comprehensive for the very broad domain spanned by the Virtual Physiological Human community. It is hoped that this document will give sufficient insight for researchers to begin publishing data in a meaningful way, and that conversion to any stricter characterisation that may be introduced in future will be straightforward.

This guideline concerns knowledge representation in relation to the VPH and, in particular, as an overall context for ontological annotation. These guidelines were developed in collaboration with the RICORDO VPH STREP.

This guideline will focus on Interoperability, which has been identified as one of the three principal strands of activity (with engagement and standards) in “The Virtual Physiological Human ToolKit – Evolution and Sustainability”. The focus is put on interoperable content: “High-quality content for the ToolKit needs to be collected. Wherever possible, this is sought from existing developments external to the VPH-NoE, in order both to maximise engagement and to make best use of VPH-NoE resources. The focus is on making existing tools interoperable with other ToolKit content.” This document's intent is to guide researchers and developers in their choices so that to maximise interoperability with other developments.

This Guideline is designed to support delivery of content to the ToolKit, by raising relevant legal, ethics and provenance issues with ToolKit contributors. In so doing it forms part of a wider framework, that helps to give meaning to concepts of "acceptability" and "quality" for material delivered to the ToolKit.

The "take home message" of this guideline document is four-fold:

• Whether you are aware of it or not, as a provider of ToolKit content, you are operating in a territory regulated by law. Personal interaction with the ToolKit will result in you being classified as an author or user.
• Given the legal context, it is prudent to be aware of your legal responsibilities in respect of copyright, data protection, freedom of information and liability.
• The close association of the VPH with industry and the clinic requires that you operate in an ethically considerate manner.
• The interaction of research, clinic and industry under the VPH umbrella implies a degree of interoperability that can only be effective in the context of robust provenance (software tools and data).

This document is divided into three distinct sections (Legal, Ethics, Provenance), but all are underpinned by a common determination to promote delivery of high quality ToolKit content, driven by a desire to encourage its use within the clinical setting.

A common issue for scientists in respect of interoperability is dealing with intellectual property considerations, particularly data and software licensing. The requirements of the host organisation(s) involved need to be taken into consideration, as well as the licences of any software libraries used in a software product. These topics need to be carefully considered when deciding how to release data or software, and under what licence. They are also worth considering when beginning any project, since decisions made earlier in development may have a significant impact on any later decision to release.

While much of the focus of this document is on software licensing, licensing of data and content (e.g. the documentation for a software tool) is no less important. However, historically, licensing associated with data sharing has received less attention, and so this field is less developed. The legal issues are also less clear. It is still crucial, however, that publicly released data and databases have suitable licence terms attached, particularly for the avoidance of doubt among users. It is a sad irony that, at a time when we have the technologies to permit global access to and distributed processing of scientific data, legal restrictions are making it harder to connect the dots.

(Note that licensing of VPH models encoded in a markup language is an emerging aspect of this topic. It is still being debated within the community, and so coverage of this issue has been deferred to a subsequent version of this document.)

This document provides some guidance on licensing issues, with links to further information. However, although recommendations are presented, this document was not produced by lawyers, and should not be considered as legal advice. It is merely a statement of our opinions. We make no warranties regarding the general legal information provided, and disclaim liability for damages resulting from its use. Please consult a lawyer to resolve your specific licensing and intellectual property issues.

The VPH-NoE advocates the use of a business-friendly open source licence whenever possible, in order to promote the wide usage and ongoing development of VPH tools. We further recommend selecting a licence from the list discussed in this document if possible. This is to keep the number of licences used by the VPH community manageable, and avoid the legal complexities that arise from building on software with an unusual licence. Multiple licences may be applied to permit alternative usage scenarios (e.g. to provide a commercial licence for a fee).

For data, we also advocate the use of an open license to encourage sharing and reuse, since innovation in science is best served by access to information. However it should be noted that considerations for clinical data in particular may preclude this in some cases.

The VPH initiative is expected to provide a set of tools that allow scientists and clinicians to access a range of resources distributed across multiple administrative domains to perform patient specific modelling and simulations. These tools will be provided by VPH partners and from other scientific projects where these tools are well known and proven to be very successful.

Usability of these tools is paramount for their acceptance across VPH community. Many tools developed by research projects do not include usability evaluation as part of their standard software development process. A shining example of usability problems is encountered in the use of complex grid middleware and complex security mechanisms used to access High Performance Computing resources. Usability evaluation can provide evidence that there is problem and it has proven itself as a key ingredient in any organisation that wishes to attract users.

One key aspect of the VPH Toolkit is to provide potential end-users with guidelines to evaluate and assess the usability of the tools that will be exposed by the toolkit. The aim of this document is to provide criteria carefully tailored to help the VPH community study the usability factors that end-users are looking for in tools used for scientific research and in security mechanisms enabling secure access to resources used by these tools.