Microsoft Word - 3PiperIanTheBrainMonitoring.doc

12 Romanian Neurosurgery (2010) XVII 1: 12 - 33

The Brain Monitoring with Information Technology
(BrainIT) collaborative network:
Past, Present and Future Direction

Ian Piper1, Iain Chambers2, Giuseppe Citerio3, Per Enblad4,
Barbara Gregson2, Tim Howells4, Karl Kiening5, Julia Mattern5,
Pelle Nilsson4, Arminas Ragauskas6, Juan Sahuquillo7, St.M. Iencean8,
R. Donald9, R. Sinnott10, A. Stell10, on behalf of the BrainIT Group

1Clinical Physics, Southern General Hospital, Glasgow UK; 2Regional Medical Physics
Department, Newcastle General Hospital, Newcastle, UK;
3Neurorianimazione, Hospital San Gerardo, Monza, Italy; 4Neurosurgery, Uppsala
University Hospital, Uppsala, Sweden; 5Neurosurgery, Ruprecht-Karls-Universitat
Hospital Heidelberg, Germany; 6Kaunas University Hospital, Kaunas, Lithuania;
7Neurosurgery, Vall d’Hebron Hospital, Barcelona, Spain
8Neurosurgery, Emergency Hospital “Prof Dr N Oblu ”Iasi, Romania; 9C3 Global Ltd,
Dingwall, Scotland; 10National eScience Centre, University of Glasgow, Scotland
On-behalf of the Brain-IT Group (www.brainit.org).

Abstract
The BrainIT group works collaboratively on developing standards for collection and analyses of
data from brain injured patients and to facilitate a more efficient infrastructure for assessing new
health care technology with the primary objective of improving patient care. European
Community funding supported meetings over a year to discuss and define a core dataset to be
collected from patients with traumatic brain injury using IT based methods. In this paper, we
give an overview of the aims and future directions of the group as well as present the results of
an EC funded study with the aim of testing the feasibility of collecting this core dataset across a
number of European sites and discuss the future direction of this research network. Over a three
year period, data collection client and web-server based tools were developed and core data
(grouped into 9 categories) were collected from 200 head-injured patients by local nursing staff
in 22 European neuro-intensive care centres. Data were uploaded through the BrainIT web site
and random samples of received data were selected automatically by computer for validation by
data validation (DV) staff against primary sources held in each local centre. Validated data were
compared with originally transmitted data and percentage error rates calculated by data category.
Feasibility was assessed in terms of the proportion of missing data, accuracy of data collected and
limitations reported by users of the IT methods. Thirteen percent of data files required cleaning.
Thirty “one-off” demographic and clinical data elements had significant amounts of missing data
(> 15%). Validation staff conducted 19,461 comparisons between uploaded database data with
local data sources and error rates were commonly less than or equal to 6%, the exception being
the surgery data class where an unacceptably high error rate of 34% was found. Nearly 10,000
therapies were successfully recorded with start-times but approximately a third had inaccurate or
missing end times which limits the analysis of duration of therapy. Over 40,000 events and
procedures were recorded but events with long durations (such as transfers) were more likely to
have “end-times” missed. The BrainIT core dataset is a rich dataset for hypothesis generation
and post-hoc analyses provided studies avoid known limitations in the dataset. Limitations in the

Ian Piper et al Brain Monitoring (BrainIT) 13

current IT based data collection tools have been identified and have been addressed. In order for
multi-centre data collection projects to be viable the resource intensive validation procedures
will require a more automated process and this may include direct electronic access to hospital
based clinical data sources for both validation purposes and for minimising the duplication of
data entry. This type of infrastructure may foster and facilitate the remote monitoring of patient
management and protocol adherence in future trials of patient management and monitoring.

Keywords: clinical network, traumatic brain injury, Grid, Internet

Background
Severe traumatic brain injury is a

leading cause of death and survivors have
serious and long term morbidity [1].
There are significant social and economic
effects including loss of employment and
an increased burden of care to the victim
their families and society as a whole.

The aetiology of the disease is complex
often implicating multiple organ systems
causing a high variation in the
presentation of injury and, as a result, a
large number of patients are required
when assessing new health care
technology. Recruiting patients from
multiple centres will significantly reduce
the time to assess new therapies and
monitoring devices. However, despite the
existence of guidelines for the
management of severely head injured
patients [2, 3] this group of patients is
subject to considerable variability in care
across centres [4-9]. To improve the
monitoring and management standards in
this population, the inter and intra-centre
variability in the intensive care
management, physiological monitoring
and treatment of these patients needs to
be assessed on a multi-national basis. To
do so requires a standardised, IT based,
higher resolution methodology for
acquiring multi-centre patient

management and physiological
monitoring information.

One consequence of the variability in
the clinical management across centres
that take care of patients with TBI
(traumatic brain injury) is the
confounding influence that this may have
in multi-centre trials of therapy. Despite
promising pre-clinical results of several
potential neuroprotective drugs most have
failed to show efficacy in the head-injured
population. A number of reasons have
been proposed for these failures which
include: poor study design, insufficient
dose of drug penetrating the blood brain
barrier and inter-species differences in
brain injury mechanisms.

Another factor, not as yet
systematically examined, may be the
occurrence of secondary insults which are
missed through use of inappropriate
monitoring methods. Recent estimates
put the proportion of adverse events
missed by using only end-hour recording
compared with minute by minute
computer based monitored to be in the
region of 30% [13]. Even in large scale
randomised trials an accurate sample size
analysis cannot be made without a
knowledge of the incidence of the true
variability of relevant confounding
factors. Inaccurate sample size estimates

14 Romanian Neurosurgery (2010) XVII 1: 12 - 33

will lead to trials that are not correctly
powered.

Improving the standards and
resolution for multi-centre data collection
will also effect assessment of new medical
technology which is of relevance to the
medical device industry. The majority of
companies that develop or support
devices used to monitor brain injured
patients in intensive care are small to
medium size enterprises (SME). Unlike
the pharmaceutical industry SMEs lack
the resources to independently assess
their devices in multi-centre clinical trials
and this severely limits the ability to
provide an evidence base demonstrating
the clinical utility of their products.

In order to address these issues it is
essential to develop an open, collaborative
network of centres interested in the
realisation of higher resolution and
standardised methods for the collection of
neuro-intensive care monitoring and
management data from patients with
traumatic brain injury. Such an
infrastructure will provide a more
efficient means for assessing new and
developing health care technology which
may be new pharmaceutical compounds,
management approaches or monitoring
devices.

To address these issues, the Brain
Monitoring with Information
Technology (BrainIT) group was formed
(www.brainit.org) . The group has 3 main
aims:

1) To develop and disseminate
standards for the collection, analysis and
reporting of intensive care monitoring
and management data collected from
brain injured patients.

2) To develop and use a standardised
database as a tool for hypothesis
generation and the development, testing
and validation of new data analysis
methodologies.

3) To provide an efficient multi-centre
infrastructure for generating evidence on
the utility of new invasive and non-
invasive intensive care monitoring and
management methods.

The BrainIT group have first defined a
core dataset collected using PC based
tools as part of a European Community
(EC) funded study (QLGT-2000-00454).
A series of meetings spread over one year
enabled key stake holders to meet and the
group to define a minimum set of data
that should be collected from all patients
with traumatic brain injury (TBI). The
outcome of the study was to define a core
dataset which would be useful in most
research projects conducted in this
population of patients [10].

This paper reports on the results of a
subsequent three year EC funded study
(QLGC-2002-00160) that enabled the
group to develop IT methods to collect
the core dataset and to assess the
feasibility and accuracy for collection of
this core-dataset from 22 neuro-intensive
care centres across Europe. Feasibility was
assessed in terms of amount of missing
data, accuracy of data collected and
limitations reported by users of the IT
data collection methods. To assess
accuracy, data validation staff (usually
research nurses) were hired on a regional
basis (normally country by country) to
check samples of the collected data against
the local primary clinical record in order
to quantify the accuracy of the IT based

Ian Piper et al Brain Monitoring (BrainIT) 15

data collection methods. This paper
describes an analysis of the comparison of
the data from 200 patients with that
obtained independently by data validation
staff . The error rates classed by data
category are described and the known
limitations of current IT data collection
methods are considered along with some
proposed solutions .

Methods
Core dataset Definition
Through European Community (EC)

funding (QLRI-2000-00454), a series of
meetings over a one year period brought
together neurosurgeons, intensivists,
scientists and representatives from the
medical device and pharmaceutical
industries to define and discuss a “core-
dataset definition” for data that should be
collected from all patients with traumatic
brain injury (TBI), irrespective of the
underlying project aim. A core dataset was
defined and published [10] that consisted
of the following nine data categories:

i) Demographic and “one-off” clinical
data (pre-neurosurgical hospital data,
neurosurgical hospital admission data and
the first and worst CT scan data). This
data is collected only once per patient.

ii) Daily management data (eg: use of
sedatives, analgesics, vasopressors, fluid
input/output balance etc). This data is
collected as an overview of the day to day
intensive care management of the patient
and is collected only once per day.

iii) Laboratory data (eg: blood gas,
haematology, biochemistry data etc). This
is “episodic” data which is data collected
more than once but at unpredictable
times.

iv) Event data (eg: nursing
manoeuvres, physiotherapy, medical
procedures (line insertion), calibrations
etc - also episodic data).

v) Surgical procedures.
vi) Monitoring data summary (eg: type

and placement location of ICP sensor, BP
lines, etc). Typically this data is only
collected once per patient and is an
overview of the monitoring configuration
for a patient.

vii) Neuro Event data (eg: Glasgow
Coma Score, pupil size and reactivity also
episodic data).

viii) Targeted Therapies A set of
therapy categories have been defined with
some associated therapy type detail. For
every therapy given an intended target
must also be given (eg: mannitol for
raised ICP).

ix) Vital monitoring data. This is
bedside monitoring data which is
collected at regular intervals with a
minimum sampling rate of once per
minute.

Network Structure
The BrainIT group network structure

consists of a central coordinating and data
centre (Glasgow) with individual centres
clustered into language based regions
where each language region contains a
sub-coordinating centre. Each sub-
coordinating centre is responsible for
coordinating the training and validation
activities across centres within their
region and to meet this requirement hire
a “data validation” nurse responsible for
providing training on the data collection
tools and web-services to all centres
within their own language region. The
data validation nurses also provide a data

16 Romanian Neurosurgery (2010) XVII 1: 12 - 33

checking and validation service
coordinated from Glasgow.

Data Collection Tools
Clinical data is entered by local bedside

nursing staff either on hand held PDA’s
or on in-house designed JAVA based
software running on a PC. In
collaboration with Kelvin Connect Ltd
[11] the BrainIT core dataset definition
was implemented in a flexible and easy to
use hand-held PDA based device. A
training course was held for the data
validation nursing staff in Glasgow on the
optimal use of the data collection
instruments which also provides data
entry in six European languages. An
anonymisation routine removed patient
identification elements from the collected
data and labelled the patient data file with
a unique BrainIT study code generated
from the BrainIT web-site. A local
database held in each centre linked the
anonymised data to local centre patient
ID information which was needed during
the data checking stage of the study.
Anonymised data was uploaded via the
BrainIT website. A server side data
converter tool converted data from centre
specific format into a standardised
BrainIT data format generating nine data
category files which are imported into the
BrainIT Microsoft SQL2005 database.

Data Validation Process
Figure 1 graphically represents the data

validation process. Centre staff enter data
using client side tools such as the hand
held PDA. Data is uploaded via the
BrainIT web-services and a server-side
converter formats data into the series of
common data format files which are input
into the BrainIT SQL database. A

validation request tool residing on the
database server randomly samples 20% of
the data uploaded for each data category
and generates a validation request file for
each local data validation nurse listing the
timestamps and data items to be checked
against local data sources. Data validators
move between their designated centres
and enter into a data validation tool the
requested data items from source
documentation held in each local centre.
The resulting validation data file is
uploaded to the BrainIT data coordinating
centre via the website and using data
validation checking software tools, the
validated data is checked against the data
items originally sent from which
percentage accuracy data was calculated.

As part of this validation process, and
in addition to the categorical and numeric
clinical data being checked for accuracy,
we also assessed the minute by minute
monitoring data. Random samples of
monitoring data channels uploaded (eg:
ICP, SaO2) were selected and validation
staff asked to manually enter the hourly
recorded values from the nurses chart (or
local gold standard data source) for the
first and last 24 hour periods of bedside
monitoring for a given patient for a given
channel. These “validation” values could
then be compared with a range of
summary measures (eg: mean, median)
from the computer based monitoring data
acquired from the patient.

Assessing Feasibility
To assess feasibility, we sought answers

to specific questions including:
a) What data cleaning was necessary

prior to analysis?

Ian Piper et al Brain Monitoring (BrainIT) 17

b) What proportion of missing data
was found in each data category?

c) How accurate is the data that was
collected?

d) What are the known limitations of
the existing IT methods for collection of
the data?

Results
Data Description
Over a two year period, core dataset

data (grouped by nine categories) were
collected from 200 head-injured patients
by local nursing staff. One patient’s data
was discarded from the cohort as there
was less than 4 hours of monitoring data
which fell outside our inclusion criteria
leaving 199 patients in the feasibility study
dataset. Table 1 summarises the key
demographic and clinical features of the
study cohort. Mean age was 36 years with
the usual predominance of male patients
in such series. Using the TCDB
classification on worst CT, 100 patients
were coded with diffuse injury and 60
with mass lesions. Using the extended
Glasgow Outcome Scale (GOSe) there
were 33 deaths (20%) with 47% good and
53 % poor outcome respectively. Table 2
summarises the quantity of data collected
per patient classed by data category. There
were 109 “one-off” demographic and
clinical data items collected which
included pre-neurosurgical (PHSH) and
neurosurgical hospital (NSH) data. The
majority of the data were “episodic” in
nature in that they were collected more
than once per patient but at un-
predictable times. These data types
included “ICU monitoring” categories
describing, for example, the location and

type of medical monitoring device placed
(eg: right frontal ICP bolt), neurological
status (ICU GCS scores/pupil size and
reactivity) , therapies delivered, surgeries
performed etc. The largest number of
data items collected fell within the “Other
clinical events” category which included
annotations of blood samples, lab results,
and other nursing and medical
procedures. In this category there was on
average greater than 230 recordings per
patient. The next most common category
of data collected were those of
annotations of target driven therapies. In
this category there were on average
greater than 60 targeted therapies
delivered per patient. By far the largest
category of data collected was the
“periodic” minute by minute
physiological monitoring data with over 2
million records in the patient cohort.
Table 3 lists the number of patients with
specific types of monitoring. Figure 2 is a
histogram plot showing the quantity and
spread of time-series data for the main
monitored channels: ICP, BP, CPP,
CVP, SaO2, Core Temperature, ETCO2
and PtiO2. The number of data points
sampled per channel ranges from 89,524
for PtiO2 to 1.98 million samples for
BPm. (BPm = 1,979,284, ICPm =
1,748,423, CPP= 1,719,166 , CVPm=
541,524, SaO2 = 1,656,614 , Tc =
1,372,641, ETCO2 = 502,524, PtiO2 =
89,524.

18 Romanian Neurosurgery (2010) XVII 1: 12 - 33

Data Cleaning
On average three raw data files were

uploaded per patient giving 600 patient
data files uploaded to the central database
using the BrainIT web-services. All data
files were validated prior to inclusion into
the study dataset and in a proportion of
these errors were found with data values
needing to be re-checked and corrected
by local nursing staff. Seven raw data
patient files required resolving
mismatches between physiological data
patient identifiers and other clinical data
files (1.2 % of files uploaded). Ten raw
data files required trimming of
physiological data outside the range of
clinical data (2% of files uploaded).
Nineteen patient files required correction
of one or more admission, surgery or
discharge time stamps (3% of files
uploaded).

Missing Data
One-Off Measurements
There was missing data across certain

data fields. Figure 3 is a graph listing
those “One-Off” demographic and
clinical data fields with greater than 15%
of missing data. Common patterns in the
types of fields yielding the highest missing
data rates could be identified: A) One
third of the fields with significant
amounts of missing data were “one off”
laboratory data values (eg: glucose,
Haematocrit, PaCO2) which should have
been obtained from admission notes from
either the pre-neurosurgical hospital
(PNSH) or the receiving neurosurgical
hospital (NSH). B) One third of the
missing fields were explanatory variables
associated with either the first or worst
CT scan classification. These explanatory

variables included “yes/no” categories as
to whether or not specific pathologies
were seen on CT such as SAH,
pneumoencaphalopathy, hydrocephalus
etc. C) Fifteen percent of the missing data
fields were explanatory variables
associated with the 6 month Glasgow
Outcome Scale data. These included
fields such as “who was the main
respondent” to the questionnaire and
what was deemed to be the “main cause
of disability” (head injury or systemic
injury).

Episodic Measurements
These data types include therapies,

laboratory values and nursing and medical
procedures that were entered more than
once at un-predictable times. For each
episodic data measurement both a “start-
time” and “end-time” should have been
recorded for each measurement by local
nursing staff. Nearly 10,000 therapies
were successfully recorded with start-
times but approximately a third had
missing end times. Table 4 is a
breakdown of the therapies delivered
classed by type listing the proportion with
missing “end-times”. Clearly the quantity
of missing end-times in this part of the
dataset severely limits analyses assessing
duration of therapy. Over 40,000 events
and procedures were also recorded but
events with long durations (such as
transfers outside of the ITU for theatre or
CT scan) were more than twice as likely
to have “end-times” missed. These short-
comings in the acquired episodic data
have implications for the design of future
data collection/validation tools as well as
project training procedures.

Ian Piper et al Brain Monitoring (BrainIT) 19

Data Accuracy
In total, 19,461 comparisons were

made between collected data elements
and source documentation data by data
validation research nurses. The number
of comparisons made per data category
were in proportion to the size of the data
received for that category with the largest
number checked in laboratory data
(5,667) and the least in the surgery data
(567) (Figure 4). Table 5 summarises
error rates by data class. Error rates were
generally less than or equal to 6%, the
exception being the surgery data class
where an unacceptably high error rate of
34% was found.

In the surgery data category, nursing
staff had to choose surgical procedures
from a fixed list of procedure types: i)
ICP placement, ii) Evacuation of Mass
Lesion, iii) Elevation of depressed skull
fracture, iv) Removal of foreign body, v)
Anterior Fossa repair for CSF Leak, vi)
Placement of Extra Ventricular Drain, vii)
Active external decompression (with bone
removal and duroplastia), viii) Other.
This classification system was used in an
attempt to simplify and reduce the burden
of data entry. However, through
discussions with local nursing and data
validation staff it was found that there was
particular confusion over when to record
ICP sensor placement and the presence of
skull fractures as the primary surgical
procedure. Typically, these procedures
occur during the same operative
procedure as for example during
“evacuation of a mass lesion”. Confusion
over coding these two procedures
between the original data entry nurse and

the validation nurse accounted for the
majority of errors in this data category.

The detection rate of acute events was
also examined (eg: nursing management,
physiotherapy, blood samples etc). It was
found that short duration events were
rarely missed but longer duration events
such as transfer to CT or theatre were
more likely to be not recorded. Through
discussions with local nursing and data
validation staff it is believed that the
intense nursing activity just prior to and
following a transfer is more likely to lead
to omissions in recording these events on
research systems.

Finally, we tested the accuracy of the
minute by minute monitoring data that
was collected. Table 6 shows the
monitoring data validation results for the
6 data types with the most recorded
nursing chart values. Data is expressed in
terms of bias (+- 95% CL) between the
nurses chart recorded values against the
computer collected end hour averages. It
can be seen from this data that the
computer collected end hour data is an
accurate reflection of the nurse’s chart
recorded data.

As an example, Figure 5 shows a scatter
plot of computer monitored minute by
minute ICP data averaged over 60
minutes (ICPavg) plotted against nurses
chart end hour values (ICPvalid)
collected by the data validation nurses.
There is a good correlation between the
two sets of data with a linear regression
best fit R2 value of 0.9773. Figure 6 is an
Altman and Bland plot showing the
average bias (-0.15 mmHg) and 95%
confidence limits (0.12, -0.45) for the
computer monitored end hour averaged

20 Romanian Neurosurgery (2010) XVII 1: 12 - 33

data Vs the nurses chart end hourly
recorded values collected by the validation
nurses.

IT Tool Limitations
The PDA data entry tool and the

website-upload tools did not incorporate
sufficient validation mechanisms. Many
fields with the PDA tool allowed export
and upload of empty data fields. Although
most IT technology nowadays can be
configured to explicitly specify required
fields and prevent upload of data with
specific missing data, at the time this
study was designed, such validation
facilities were not available off the shelf.
Also, the PDA tool was designed to allow
acceptance of new items not part of the
drop down selection menu, which could
generate multiple terms for the same data
element. This caused added burden on
the cleaning process to consolidate
multiple text terms for the same data
element. The most challenging limitation
found with the IT technology used in this
study was an inability to automatically
track “continuous” (non-bolus) therapies
which were started to ensure that a
matching “end-time” was entered. This
resulted in approximately 1/3rd of the
therapies annotated to have missing end
times.

Discussion
This project has studied the feasibility

for collection of the BrainIT core dataset
using our first generation IT based tools.
Feasibility has been assessed in terms of
amount of missing data, data accuracy for
data that was collected and also in terms
of identifying limitations in the IT
technology used to collect the data.

Good laboratory practice dictates that
as part of clinical trial design, acquired
data must be checked for accuracy against
the primary data sources. This is often
implemented through either employing a
contract research organisation or
independent research nurse staff to
perform this duty. In large multi-centre
clinical trials, costs to hire research nurse
data validation staff can become
prohibitively expensive and feasible only
if significant industry or research council
funding support is provided. Now with
the adoption of the new medical device
standard ISO-14155, even small medical
device studies are expected to provide
some form of check on the accuracy of
data collected even as part of a non-
regulatory post-marketing study.

To our knowledge, this study
conducted by the BrainIT group is one of
only a few multi-centre projects to
attempt to prospectively assess the data
capture error rate within an academic
investigator led environment [12].

Monitoring Data Validation
We have shown that computer

collected minute by minute vital signs
data, summarised as end hour averages,
correlated well with nursing chart end
hour recordings. This allows for the end
hour averaged computer records to be
used in database analyses that aim to
assess nurses chart recorded detection of
events with computer based sampling.
Although, end-hour average data
correlates well with the nurses hourly
recorded value, this does not indicate that
important features of the data are not
being missed by employing only hourly
recording. For example, Zanier and

Ian Piper et al Brain Monitoring (BrainIT) 21

colleagues [13] conducted a study
showing that although computer-
monitored end-hour data is accurately
reflected by the nurses’ chart value, more
complex summary measures (such as the
detection of an intracranial pressure (ICP)
of more than 20 mmHg) are less accurate.
Their finding that at least one-third of
secondary insults for raised ICP are not
identified from the nursing chart is
similar to that reported by Corrie and
colleagues [14], who also found a similar
detection error rate for other signals such
as blood pressure, particularly the events
of shorter duration. Importantly, Zanier’s
paper has further shown that when data
are categorised in terms of percentage of
time spent with raised ICP, the patients
exhibiting instability in ICP were most
prone to under estimation of ICP insults.
The data sampling rate may be pertinent
here: Zanier’s study sampled at 600
samples per minute, whereas other
studies used 1 sample per minute [14] or
as few as 4 samples per hour [15]. Our
results here confirm those of other
investigations showing that the end hour
averaged computer values can be used as
estimates of nurse’s paper based end hour
recordings and opens up the possibility
for further studies assessing the clinical
influence of missed short term adverse
physiological events without requiring
tedious recording of nursing chart values.

However, the key question remains
unanswered as to whether missed adverse
events using higher resolution sampling
significantly influences clinical outcome.
Work conducted by Chambers and
colleagues may be relevant in this regard
[16]. They found that in studies of

children with TBI, the choice of
summary measure is also important. They
used an index termed the “Pressure Time
Index (PTI)”, which is a composite index
taking into account both the duration of
the adverse event and the degree of
physiological impairment below a given
threshold. They found, using ROC
analysis of the influence of cerebral
perfusion pressure adverse events upon
clinical outcome (calculated using an age -
related PTI index in children), that
models that included the PTI measure of
CPP burden significantly improved the fit
to clinical outcome compared with
models that did not include measures of
physiological instability. These results are
in contrast to those published by
Signorini et al who they found very little
improvement in outcome prediction
when “Insults” are added to the usual
clinical features in prognostic models of
patient outcome [17]. The approach
developed by Chambers and colleagues
needs to be repeated in the adult TBI
population and is one of the planned
analyses on the BrainIT dataset.

Validation Costs
These validation results calculated on a

subset of patients provides an estimate of
the data quality on a larger patient cohort
of 350 patients collected using the same
methodology but collected outside of the
validation study. However, future data
collection projects will generate datasets
under differing data collection conditions
and will require a separate validation stage
if we wish to maintain our confidence in
the level of data collection accuracy. The
approach used by the BrainIT group to
validate data (using 20% sampling of

22 Romanian Neurosurgery (2010) XVII 1: 12 - 33

uploaded data with some automation of
generating data lists for validation) still
requires significant research nurse time to
track down and enter data for validation
purposes. To maintain a full time data
validation nurse across all participating
countries costs in excess of 1 Million
Euro’s per year. Such large overheads for
an academic network is prohibitively
expensive and not sustainable and a more
cost-effective solution for data validation
must be found.

One promising approach now being
assessed by the BrainIT group is
developing collaborative research with
experts in Grid based middleware
technology. Grid technology comes in a
variety of forms and covers more than just
access to networks of high end servers in
order to solve computationally intensive
problems. There is a considerable amount
of expertise and open source middleware
software solutions now available that
provide secure access to distributed
medical datasets so that the right people
see the correct data in the appropriate
context [23]. Such an approach, provided
local IT policy staff are satisfied with the
system security, will enable remote data
validation systems to directly query
hospital based primary data sources for
the purpose of checking the quality of
previously uploaded data. Most research
datasets contain large portions of data
elements that are collected for routine
patient management purposes and the
difficulty of accessing hospital based data
sources often means that research nurses
are employed to re-enter data extracted
from local hospital sources into research
data entry systems. This results in a high

proportion of double data entry which is
an inefficient use of nursing resources.
Using Grid technology to interface
directly with local hospital data sources
will reduce the burden of double data
entry. Clearly some data validation staff
will still be required to support system
queries but increased use of automatic
data validation procedures and access to
hospital based datasets should
significantly reduce the cost burden to
conduct multi-centre clinical trials.
Towards this end, the BrainIT group as
part of an EC funded framework VII
project are now assessing such an
approach in a group of neuro-intensive
care centres equipped with the latest Grid
technology. This project – the AVERT-IT
project [18] has installed Grid services
behind hospital firewalls in six BrainIT
neuro-intensive care units. Grid services
will interface to local hospital systems,
extract data which maps to the BrainIT
core dataset and integrate data from both
hospital sources and local AVERT-IT data
collection tools (for those elements not
collected as part of routine management)
into a local database. Once every 20
minutes, data is stripped of patient
identifiers, encrypted and “pushed” out of
local hospital networks to an external
secure server cluster hosted at the
University of Glasgow National eScience
Centre [19]. Local databases will be
maintained which link local patient
identifiers with an anonymous patient
identifier. Systems running at the BrainIT
coordinating centre in Glasgow allow
remote monitoring of the data acquired
from all six participating BrainIT centres.
Such a remote monitoring service in

Ian Piper et al Brain Monitoring (BrainIT) 23

quasi real-time (updates every 20
minutes) will allow more efficient
collection and validation of hospital based
data collected for research purposes while
the patient (and their notes) are still
within the ITU environment. This
infrastructure will also support
monitoring of patient management for
adverse events (such as treatments given
for arterial hypotension) and will enable
testing and tracking adherence to study
protocols.

Lessons Learned
A number of lessons have been learned

during this feasibility study. First, our
surgery classification definition is
ambiguous. Specifically our definition
document did not make it clear how to
decide which surgery is the “Primary
Reason for Surgery”. For example if a
patient undergoes surgery for removal of
a mass lesion and repair of depressed skull
fracture, some approach must be used to
provide a consistent classification
response. We are proposing a modified
surgery classification to include a “major
surgery choice matrix” where individual
surgery types are weighted and specific
combinations that do occur can be
resolved to a single surgical priority.

Secondly not all staff favoured use of a
PDA type data tool. By the end of the
feasibility study,

approximately half the centres
collecting data preferred to use PC based
systems rather than the hand-held PDA’s.
Increasingly, nursing and medical staff
have good IT and data entry skills and as a
result we have developed new PC based
data collection tools. Also, our data tools
(although state of the art at the time), did

not provide sufficient local validation
features such as preventing export and
upload of empty data fields. Most IT
technology nowadays can be configured
to explicitly specify required fields and
prevent upload of missing or incorrect
data. Our current generation of data tools
now almost entirely allow only specific
choices to be made from drop-down
“combo boxes” where the default choice
is set to a text value of “not set”. This
makes it explicitly clear that a given field
has not been entered. Our data schema
will not allow mandatory fields to be left
“not-set” before a patient is discharged
from the system. For the entry of
treatment information, every treatment
must be assigned a specific target and
again, the data schema will not accept
treatments that have not been assigned a
target. Furthermore, our next generation
data collection tools, as implemented in
the AVERT-IT project, allows annotation
of any treatment or procedure with only
two mouse clicks providing more rapid
and efficient data entry for the bed side
nurse. The web-client software now
includes data validation routines which
will prevent upload of missing data in any
required fields. Patients cannot be fully
discharged from the system until all
required data is entered. Patients with
missing data can be partially discharged
from the system (when they are
discharged from the ITU) but they
remain in a visible list “Patients with
Missing Data”. A single web page displays
all missing fields in red and must be
completed before the patient can be fully
discharged.

24 Romanian Neurosurgery (2010) XVII 1: 12 - 33

Current Status and Future Direction
The aims of the BrainIT group and

their implementation is a staged process.
We have successfully defined a core
dataset standard, developed some
standardised IT tools to collect the core
dataset and tested the feasibility for
collection of the dataset from 22 centres
across Europe. Limitations in our
methods have been found and attempts
have been made to address those issues
prior to starting future studies. Inevitably,
with each new project, problems will arise
and solutions will be found in a cyclical
process. Our second aim “To develop and
use a standardised database as a tool for
hypothesis generation and the
development, testing and validation of
new data analysis methodologies.” has
been achieved and a number of
publications are now arising from access
to this shared resource [20]. We are
currently using the 2nd database release
with a third release planned, and what is
encouraging is that the existence of the
database resource was directly responsible
for generating and testing the hypothesis
about application of Bayesian neural
network methods for prediction of arterial
hypotension adverse events which lead to
a project now funded under the VIIth EC
information and communications
technology framework [18].

One of the papers arising from the
work of the BrainIT group was a report
on its own internal survey of patient
management which indicated that
international management guidelines are
for the most part adhered with [9].
However, there is a risk with surveys that
there may be differences in results

between what users believe to be the
management applied in their centre and
studies which measure it directly. In this
regard, a recent paper published by one of
our collaborators on analyses of the
BrainIT dataset was to assess, subsequent
to the BrainIT survey, whether the use of
hyperventilation therapy for the
management of raised ICP was indeed
conducted according to international
guidelines. Interestingly they found that
despite what was suggested by the earlier
survey results, and in conflict with
current management guidelines, there
was significant over use of early
prophylactic hyperventilation [22]. This
result highlights the importance of
directly monitoring the applied
management, and if it can be achieved in
near real-time, will enable future
management trials to monitor protocol
adherence and better select when patients
data can be recruited to a study.

The third and most challenging aim of
the BrainIT group is to use its improved
infrastructure to generate new evidence
on the utility of monitoring and
management methods for patients with
TBI. The AVERT-IT project, now
underway, will put in place in six BrainIT
centres, Grid middleware systems
enabling direct access to hospital data and
remote monitoring of patient
management. We believe that this type of
remote monitoring facility is a pre-
requisite for the conduct of a future
multi-centre management trial by the
BrainIT group. Discussions of a
management trial design have been
started at the recent BrainIT group
meeting (Vilnius, September 2009) and

Ian Piper et al Brain Monitoring (BrainIT) 25

the current AVERT-IT project will pilot
the feasibility of the remote monitoring
infrastructure required for the conduct of
such a trial.

Conclusions
In this study we have shown that it is

feasible to collect the BrainIT dataset
from multiple centres in an international
setting with our IT based methods and
the accuracy of the data collected is
greater than or equal to 94%, with the
exception of the surgery data definition
which is being revised. Lessons learned
about weaknesses with our data collection
methods have been met with advances in
client/server tools providing improved
validation support. We anticipate that the
second generation of BrainIT data
collection tools (now being used as part of
the current AVERT-IT project) will
improve missing data and validation
accuracy rates. A future BrainIT
management trial will rely on a Grid
based infrastructure capable of remotely
monitoring patient management and
protocol conformance now being piloted
in six BrainIT centres. Academic led
multi-centre data collection projects must
decrease validation costs and to do so will
require more direct electronic access to
hospital based clinical data sources for
both data validation purposes and for
reducing the research nurse time needed
for double data entry of data currently not
accessible from hospital based systems.

References
1. Olesen J, L. M. The burden of brain disease in
Europe. European Journal of Neurology, 2003, 10, 471-
477.

2. Bullock R, Chesnut RM, Clifton C, et al. (1996)
Guidelines for the management of severe head injury. J
Neurotrauma 13:643-734.
3. Maas AI, Dearden M, Teasdale GM, Braakman R,
Cohadon F, Iannotti F, Karimi A, Lapierre F, Murray
G, Ohman J, Persson L, Servadei F, Stocchetti N,
Unterberg A (1997) EBIC-Guidelines for management
of severe head injury in adults. Acta Neurochir 139:286-
294.
4. Ghajar J, Hariri RJ, Narayan RK, Iacono LA, Firlik K,
Patterson RH (1995) Survey of critical care
management of comatose, head-injured patients in the
United States. Crit Care Med 23:560-567.
5. Jeevaratnam DR, Menon DK (1996) Survey of
intensive care of severely head injured patients in the
United Kingdom. BMJ 312:944-947.
6. Matta B, Menon D (1996) Severe head injury in the
United Kingdom and Ireland: A survey of practice and
implications for management. Crit care Med 24:1743-
1748.
7. Murray GD, Teasdale GM, Braakman R, Cohadon F,
Dearden M, Iannotti F, Karimi A, Lapierre F, Maas A,
Ohman J, Persson L, Servadei F, Stocchetti N,
Trojanowski T, Unterberg A (1999) The European
Brain Injury Consortium survey of head injuries. Acta
Neurochir 141:223-236.
8. Stochetti N, Penny KI, Dearden M, Braakman R,
Cohadon F, Iannotti F, Lapierre F, Karimi A, Maas A,
Murray GD, Ohman J, Persson L, Servadei F, Teasdale
GM, Trojanowski T, Unterberg A, on behalf of the
European Brain Injury Consortium (2001) Intensive
care management of head-injured patients in Europe: a
survey from the European Brain Injury Consortium.
Intensive Care Med 27:400-406.
9. Enblad P, Nilsson P, Chambers I, Citerio G, Fiddes
H, Howells T, Kiening K, Ragauskas A, Sahuquillo J,
Yau YH, Contant C and Piper I. Survey of traumatic
brain injury management in European Brain IT centres
year 2001, Intensive Care Med (2004) 30:1058 –1065.
10. Piper I, Citerio C, Chambers I, Enblad P , Nilsson
P, Chambers I, Citerio G, Fiddes H, Howells T,
Kiening K, Ragauskas A, Sahuquillo J, Yau YH, Contant
C. The BrainIT Group: Concept and Core Dataset
Definition. Acta Neurochir 145:615-629 2003.
11. http://www.kelvinconnect.com/
12. Citerio G, Stocchetti N, Cormio M, Beretta L.
Neuro-Link, a computer assisted database for head
injury in intensive care. Acta Neurochir
(Wien)142(7):769-76. 2000
13. Zanier E, Ortolano F, Ghisoni L, Colombo A,
Losappio S, Stochetti N. Intracranial pressure
monitoring in intensive care: clinical advantages of
computerised monitoring over manual recording. Crit.
Care Med. 11:R7. 2007

26 Romanian Neurosurgery (2010) XVII 1: 12 - 33

14. Corrie J, Piper I, Housely A, Tocher J, Anderson S,
Midgley S, Slattery J, Dearden N, Miller J.
Microcomputer based data recording improves
identification of secondary insults in head injured
patients. British Journal of Intensive Care, June 1993.
226-233.
15. Venkatesh B, Garrett P, Fraenkel DJ and Purdie D.
Indices to quantify changes in intracranial and cerebral
perfusion pressure by assessing agreement between
hourly and semi-continuous recordings. Intensive Care
Med 2004; 30: 510-513.
16. Chambers IR, Jones PA, Lo TWM, Forsyth RJ,
Fulton B, Andrews PJD, Mendelow AD and Minns RA.
Critical thresholds of intracranial pressure andcerebral
perfusion pressure related to age in paediatric head
injury. J. Neurol. Neurosurg. Psychiatry 2006;77;234-
240.

17.[17] Signorini DF, Andrews PJD, Jones PA,
Wardlaw JM and Miller JD. Adding insult to injury: the
prognostic value of early secondary insults for survival
after traumatic brain injury. J. Neurol. Neurosurg.
Psychiatry 1999;66;26-31
18.[18] http://www.avert-it.org
19.[19] http://www.nesc.ac.uk
20.[20] http://www.brainit.org/bit2web/faces/Papers.jsp
21.[21]
http://www.brainit.org/bit2web/faces/Meetings.jsp
22.[22] Neuman J, Chambers I, Citerio G, Enblad P,
Gregson B, Howells T, Mattern J, Nilsson P, Piper I,
Ragauskas A, Sahuquillo J, Yau H, Kiening K on Behalf
of the BrainIT Group. The use of hyperventilation
therapy after Brain Injury in Europe: An analysis of the
BrainIT database. Intensive Care Medicine 2008;
S00134-008-1123-7.
23.[23] http://www.nesc.ac.uk/hub/projects/votes/

Acknowledgements
We wish to acknowledge the

contribution of all data contributing
members of the BrainIT group
(www.brainit.org) who supported the
EEC project: QLGC-2002-01160.

Investigators and Participating Centres
Barcelona Spain, Prof J Sahuquillo;

Cambridge UK., Prof JD Pickard;
Edinburgh UK, Prof R Mins, Prof I
Whittle; Glasgow UK, Mr L Dunn;
Gothenburg Sweden, Dr B Rydenhag;
Heidelberg, Germany, Dr K Kiening; Iasi
Romania, Dr St.M. Iencean; Kaunas
Lithuania, Prof D Pavalkis; Leipzig
Germany, Prof J Meixensberger; Leuven
Belgium, Prof J Goffin; Mannheim
Germany, Prof P Vajkoczy; Milano Italy,
Prof N Stocchetti; Monza Italy, Dr G

Citerio; Newcastle upon Tyne UK, Dr IR
Chambers; Novara Italy, Prof F Della
Corte; Southampton UK, Dr J Hell;
Uppsala Sweden, Prof P Enblad; Torino
Italy, Dr L Mascia; Vilnius Lithuania, Prof
E Jarzemaskas; Zurich Switzerland, Prof
R Stocker

Corresponding Author:
Ian Piper
Brain-IT Group Coordinator
Intensive Care Monitoring
Dept. Clinical Physics, 5th Floor
Institute of Neurological Sciences

Southern General Hospital
1345 Govan Road, Glasgow, UK,

G514TF
Tel:+44(0)141-201-2595
Fax: +44(0)141-201-2592
Email: ipiper@clinmed.gla.ac.uk

Ian Piper et al Brain Monitoring (BrainIT) 27

Table 1
Demographic and Clinical Features of Feasibility Study Data set (n = 199)

Sex TCDB
(Worst)

Male 162 Diffuse1 9
Female 37 Diffuse2 51

Age Diffuse3 34
Mean 36.1 Diffuse4 12
Range 4-83 Mass 60

<14yo 7

Missing 33
Trauma

Type
GOSe

RTA 84 1 (Dead) 31
Pedestrian 16 2 3

Fall 55 3 35
Assault 18 4 8
Sport 6 5 30
Work 5 6 17

Missing 14

7 30

8 24
Missing 21

Table 2

Summary of Data Collected

Data Type No. of Fields Avg No. of Rows per Patient
Demographic
(eg: PNSH/NSH)

109 1

ICU_Monitoring
(eg: Types of Device/Location…)

12 15.0

Neurological Status
(eg: GCS/Pupils)

10 42.3

Other_Clinical_Events
(eg: Blood Samples, Suction…)

20 230.9

Surgery

11 1.4

Target_Therapies

59 69.6

Daily_Observations
(eg: Daily Summaries of Management)

11 8.4

Total 232

28 Romanian Neurosurgery (2010) XVII 1: 12 - 33

Table 3
Monitoring Data Distribution

Channel Number of Patients

BP (blood pressure: mmHg;systolic, diastolic, mean) 199

ICP (intracranial pressure: mmHg;mean) 195

CPP (cerebral perfusion pressure: mmHg;mean) 195

HRT (Heart Rate: bpm) 165

SaO2 (arterial Oxygen saturation: %;pulse oximetry) 164

Tc (core temperature: degrees C) 149

CVP (central venous pressure: mmHg; mean) 105

ETCO2 (end tidal CO2: mmHg) 79

NIBP (blood pressure: mmHg;systolic, diastolic,

mean)

Tp (peripheral temperature: degrees C) 17

PtiO2 (brain tissue oxygen partial pressure: mmHg) 11

SjO2 (jugular venous oxygen saturation: %) 10

CO (cardiac output: ml/hour) 7

brTemp (brain temperature: degrees C) 3

PrX (bp-icp reactivity:dimensionless) 1

Ian Piper et al Brain Monitoring (BrainIT) 29

Table 4
Therapy Type Vs Missing “End Times”

Therapy Start Entries End Entries Missing End Entries
Sedation 1108 499 55%
Analgesia 1032 574 44%
Paralysis 741 460 38%
Volume Expansion 1674 1308 12%
Inotropes 614 199 68%
Anti-Hypertensives 63 22 65%
Anti-Pyretics 788 505 36%
Hypothermia 22 10 55%
Steroids 51 6 88%
Cerebral Vasoconstr. 0 0 ---
Osmotics (Mannitol) 807 538 33%
Barbiturates 90 45 50%
Other 2576 2026 21%
Totals 9566 6192 35%

Table 5

Percentage Error Rate by Data Type Class with Description of Common Error Types.

Data Class Error Rate (%) Common Fields with

Errors
Laboratory 2 pCO2, FiO2 value wrong
Demographic 4 Monitoring time on arrival

at neurosurgery, intubation
present on arrival at
neurosurgery wrong

Neuro Observations 5 Pupil Size, GCSv (code 1
Vs Unknown code error)

Monitoring Summary 5 ICP type, ICP Location
wrong

Daily Management Summary 5 Infusion type (bolus Vs
infusion or both), drug
number (1 or > 1)

Targeted Therapy 6 Non-standard target, no
Target specified

Surgeries 34 ICP placement, Skull #,
mass lesion wrong

30 Romanian Neurosurgery (2010) XVII 1: 12 - 33

Table 6
Monitoring Data Validation Results – Bias (+- 95% CL) Between Nurses Chart

Recorded Values Vs Computer Collected End Hour Averages

Data Type

Value ICP
(mmHg)

BP
(mmHg)

CPP
(mmHg)

SaO2
(%)

Tc
(C)

Bias -0.15 0.16 0.46 0.46 -0.29
+95% 0.32 1.57 1.81 1.23 0.09
-95% -0.62 -1.25 -0.88 -0.31 -0.67
n 749 558 457 499 223

Figure 1 Graphical representation of the data validation process. Centre staff enter data using client side

tools such as the hand held PDA. Data is uploaded via the BrainIT web-services and a server-side convertor
converts data into the series of common data format files which are input into the BrainIT SQL database. A

validation request tool residing on the database server randomly samples 20% of the data uploaded for each data
category and generates a validation request file listing the timestamps and data items to be checked by local

data validators. Data validators move between their designated centres and enter into a data validation tool the
requested data items from source documentation held in each local centre. The resulting validation data file is
uploaded to the BrainIT data coordinating centre via the website and using data validation checking software

tools, the validated data is checked against the data items originally sent from which percentage
accuracy data is calculated

Ian Piper et al Brain Monitoring (BrainIT) 31

Figure 2 is a histogram plot showing the quantity and spread of time-series data for the main
monitored channels: ICP, BP, CPP, CVP, SaO2, Core Temperature, ETCO2 and PtiO2. The number of
data points sampled per channel ranges from 89,524 for PtiO2 to 1.98 million samples for BPm. (BPm =

1,979,284, ICPm = 1,748,423, CPP= 1,719,166 , CVPm= 541,524, SaO2 = 1,656,614 , Tc = 1,372,641, ETCO2
= 502,524, PtiO2 = 89,524

32 Romanian Neurosurgery (2010) XVII 1: 12 - 33

1 0

2 0

3 0

4 0

5 0

6 0

7 0

8 0

NS
H

G
OS

e_
m

on
th

s
po

st
in

ju
ry

NS
H

T
im

e_
of

_D
is

ch
ar

NS
H

p
O2

NS
H

F
irs

t_
CT

_ M
ain

Le
sio

n_
vo

lu
m

PN
SH

S
aO

NS
H

F
irs

t_
C T

_ S
A H

NS
H

F
irs

t_
C T

_ P
N E

NS
H

F
irs

t_
CT

_ I
VE

NS
H

F
irs

t_
CT

_ H
YD

NS
H

F
irs

t_
C T

_I
SC

NS
H

D
is

ch
ar

ge
d_

NS
H

p
CO

NS
H

p
H

NS
H

W
or

st
_C

T_
IS

NS
H

W
or

st
_C

T_
SA

NS
H

W
or

st
_C

T_
PN

NS
H

W
or

st
_C

T_
IV

NS
H

W
or

st
_C

T_
H Y

NS
H

H
CT

NS
H

G
LU

NS
H

W
or

st
_C

T_
M

ai
nL

es
io

n_
vo

lu
m

NS
H

G
OS

e_
m

et
ho

NS
H

G
OS

e_
R e

sp
on

de
nt

NS
H

G
OS

e_
Ou

tc
om

e _
Fa

ct
or

NS
H

B
Ps

NS
H

B
Pd

NS
H

M
on

ito
re

d_
ic

PN
SH

B
Ps

PN
SH

B
Pd

NS
H

C
au

s e
o

f D
ea

D a ta F i e ld T y p e

%
M

is
si

n
g

Figure 3 Graph showing “One-Off” demographic and clinical data fields with greater than 15% of missing

data

Figure 4 Pie chart showing the distribution of the 19,461 data validation comparisons which were made

in proportion to the size of the data received with the largest number checked in laboratory data (5,667)
and the least in the surgery data (567)

Ian Piper et al Brain Monitoring (BrainIT) 33

Figure 5 Scatter plot of computer monitored minute by minute ICP data from an example patient

showing the data averaged over 60 minutes (ICPavg) plotted against nurses chart end hour values
(ICPvalid). Linear regression best fit R2 value = 0.9773

Figure 6 Altman and Bland plot from an example patient showing the average bias (-0.15 mmHg) and

95% confidence limits (0.12, -0.45) for the computer monitored end hour averaged data Vs the nurses chart
end hourly recorded values collected by the validation nurses