Site Map   Home   About Us!   NEWS   The Farm   Government   The Law   The Vet   Science   Petitions       

   Downloads   The Message Boards   Links   Search   Contacts


Home Inquiries pdf/FMD_1968-9_northumberland_report_01.pdf pdf/FMD_1968-9_northumberland_report_02.pdf pdf/FMD-2002 northumberland_report.pdf INQUIRIES - LUSMORE STATS pdf/inquiry_cumbria_questions_for_defra.pdf pdf/inquiry_fmd_cumbria_report.pdf pdf/inquiry_cumbria_chairmans_letter_to_margaret_beckett.pdf pdf/iah_report_2001.pdf pdf/inquiry_eu_draft_report_sept2002.pdf pdf/inquiry_eu_mandate2002.pdf Royal Society Inquiry

Inquiries - SUBMISSION TO THE LESSONS LEARNED INQUIRY INTO FOOT AND MOUTH

Valerie Lusmore

The White House, Orchard Rise, Pwllmeyric, Monmouthshire, NP16 6JT

Tel: 01291 623573 Email: val_lusmore@hotmail.com

AUTHOR

I have degrees in Mathematics and Applied Mathematics from the University of Natal and I have worked in the computer industry since 1969. My early work was as a consultant to large international companies, very much at the cutting edge of technology and I was involved in computer modelling on many occasions. In the past 10 years I have evolved into a data specialist.

This role developed as I realised that many of the problems in complex computer systems lie in the data rather than the system. The problems are exacerbated by databases containing very large numbers of records, some of which are not consistent with the original data specifications. This leads to anomalies in calculations and processes that do not work correctly.

During the 2001 FMD outbreak I was a founder member of the National Foot and Mouth Group. When people kept telling me that they could not understand the daily data published by MAFF/DEFRA publicly on their FMD website, I made a daily analysis and published regular summaries for those who had an interest. I kept as complete a record as I could so that other people could use my data as a resource to get accurate and meaningful figures.

INDEX 

Page

1.     - Introduction

2.     - Executive summary

3.     - Mathematical Model

3.1   -  Pirbright scientists and modelling FMD

3.2   - The mathematical modelling teams

3.3   - The computer modelling tools

3.4   - Papers written by the modelling teams

3.5   - Main problem with modelling - the data

3.6   - Modelling backwards

4.    - Information Collection

4.1   - Initial data set-up

4.2   - Data integrity

4.3   - Summarising the Data

4.4   - Example of Inconsistency and Inaccuracy in MAFF statistics

4.5   - Collection History for IP data

4.6   - County Tables for Slaughter Statistics

5.     - Examples of Data Problems

5.1   - Analysis of Statistics

5.2   - Name and Address anomalies

5.3   - Data validation

5.4   - Constituency Tables - why do they exist?

5.5   - Changing the Tables

5.6   - Analysis of the new County List by Local Authority

6.    - Other Methodology

6.1   - Other modelling methods by volunteers

6.2   - Maps 

6.3   - Tracings

6.4  - Picture of Epidemic spread using more traditional methods 

6.5.  - Results of Picture

6.6   - Local Spread?

7.    - IT systems and resources

7.1   - How could the data have been improved?

7.2  - Organisation and communication

7.3  - Contingency planning

8.   - Conclusions

9.   - Appendices

9.1. - Bibliography

9.2. - County and Local Authority Lists - a comparison

9.3  - Example of ‘pictorial’ weekly reports drawn from DEFRA statistics

                    1. Introduction

The policies used during the FMD crisis of 2001 were mainly driven by epidemiologists and bio-mathematicians. These policies were brought in hurriedly at the end of the fifth week into the crisis when there were concerns that the disease was already out of hand. The main policy that the modellers developed was that of the contiguous cull.

Along with many other people with an interest in the subject I tried to understand the factors behind this model - and found it difficult to understand why this particular approach had been used especially as there was much information available on the Internet of other methods of controlling the disease that were used in other parts of the world.

Having trained as a mathematician and scientist originally I was extremely concerned that ‘mathematicians and scientists’ appeared to be making critical decisions as to the policy to be used to control an animal disease epidemic. Their methods affected the life of the whole rural community and their decisions and methodology were reflecting badly on both mathematics and science.

In particular, when I began to look at the available data about the UK outbreak and where it was situated, it very quickly became apparent to me that the information was incomplete, inaccurate, inconsistent and difficult to use. This led me to develop consistent ways of reporting the MAFF/DEFRA information so that it was clear and simple to understand.

2. Executive summary

This paper covers the mathematical models - and what I consider important about them.

This includes brief descriptions of the groups of people involved, their experience of the subject; the computer modelling tools available. There are a few comments on the scientific papers produced by the different groups justifying their work and a section on the main problems with the modelling itself.

I then move on to the major importance of the information needed to drive the models, how it was set up, examples of what information was available; discussion of the management and control needed for such information; and a commentary on the integrity and accuracy of the data itself.

The section on the data problems covers simple examples which demonstrate how I came to believe that the quality of the data was such that it was impossible for the modellers to give an accurate picture of what was happening.

The next section then postulates whether other methods could and should have been used. It describes some traditional techniques I used with the information available to the modellers at the time and the pictures that I created of how the epidemic was spreading. This leads into a section on IT systems and how they were just not up to modern standards. IT should have been used far more effectively to free up resources and to aid with communications.

My conclusions follow - the main one being that there was pressure from political sources to come up with a quick solution. The data available for the mathematical modelling was such that it would have been better to use other more traditional methodology right at the beginning to get control of what was happening to the disease. Modelling should have been used as a simple adjunct to other methods - not to shape the policies.

3. Mathematical Models

3.1 Pirbright scientists and modelling FMD

The scientists at Pirbright, world reference laboratory for FMD, had been working with mathematical models to study various aspects of foot and mouth disease, since the 1970s. The EUFMD research group discussed research papers on modelling at their annual conferences in both 1999 and 2000 as well as in earlier years. The group was led by Alex Donaldson and Paul Kitching who were world class specialists. Both had worked on the EUFMD research group for a number of years and knew most of the other international specialists on this subject.

The EUFMD group had a great deal of knowledge of the pan-Asian O strain of FMD and there had been increasingly serious discussions for at least two years of how to cope with this strain when it arrived in Europe. There had also been discussions of how to cope with the logistics of slaughter and disposal of large numbers of carcasses as it was felt that increasingly this would lead to a public outcry in many countries.

In Alex Donaldson’s 1999 paper he wrote about the earliest model created through collaboration between IAH, Pirbright, and the UK Meteorological Office, a computer-based model was developed during the 1970s for assessing the risk of airborne spread of FMD. It was created by bringing together data on the aerobiology of FMD with data on the physical behaviour of particles in the atmosphere under different climatic conditions. The model was shown to be capable of giving a prediction within a few of hours of the confirmation of an outbreak of FMD of whether there was a risk of spread and, if so, which farms were in jeopardy. The model could predict accurately up to a distance of 10 km from the source. Any farms considered to be at risk could be placed under intensive surveillance so that suspected cases could be quickly identified and eliminated. The model was used successfully under operational conditions during the outbreaks of FMD on Jersey and the Isle of Wight in March 1981.

3.2 The mathematical modelling teams

MAFF invited 3 teams of mathematical modellers to assist with analysis and prediction of the outbreak - a fourth team from Imperial College independently created their own model.

I will refer to them as:

THE VLA team - Professor Wilesmith from the State Veterinary Labs agency was backed up by colleagues from Massey University in New Zealand - they had worked together previously on the BSE problem.

The Cambridge team - Professor Grenfell from Cambridge and his colleagues - they were very experienced at analysing data from an epidemic against various factors to see if fresh insights could be obtained (eg measles, Soay sheep, etc)

The Edinburgh team - Professor Woolhouse and colleagues - had the most expertise in Geographic Information systems and were called upon by the other groups in this area.

The Imperial team - Professor Anderson and his colleagues from Imperial College were well known to the Chief Scientist but were not among the groups originally asked to take part. They had experience of BSE modelling and various (mostly human) epidemic diseases. The team had recently moved (Nov 2000) from Oxford to a new department researching Human Health headed by Professor Anderson at Imperial College. Their only experience of FMD was that Ms Donnelly had co-authored a paper in July 2000 where the data from the FMD epidemics in UK 1967 and Taiwan in 1997 was run through epidemic simulations. The conclusions of that simulation exercise was that it was imperative that herds be slaughtered on the day that disease was confirmed and that resources should be available to implement this policy should an outbreak occur.

3.3 The computer modelling tools

Epiman database - developed at Massey University in the early 1990s and used to track and manage outbreaks. Had been adapted for EU conditions and tested by various European groups of FMD researchers. Purchased by MAFF some years previously to the 2001 UK outbreak but not set up with data. Needs time to be set up

Quote from a Dutch team of researchers in the mid-1990s ‘The EpiMAN(EU) GIS application was user friendly and provided the user with good tools to facilitate certain tasks in the control of a FMD outbreak. The system could be used in the Netherlands, and has potential for other countries as well. However, digitised data has to be available in advance of an outbreak, which is not completely the case yet in the Netherlands. To fully use the possibilities of a DSS such as EpiMAN(EU), a permanent, updated database with farm full information, including farm locations, is necessary.’

This database provided the information for the associated Interspread model which was used for predictions and modelling.

Cambridge model - more complex model - contains more detail, in terms of describing transmission between individual farms, as a random process, allowing for more heterogeneity: differences between farms, in terms of numbers of animals, different species.

Imperial model - adapted by Neil Ferguson and Christl Donnelly from calculations that used the transmission of human sexually transmitted diseases to model spread together with knowledge gleaned from their work on BSE. Simple model - generalised animal species, and constant infectivity assumed.

Rimpuff Model together with GIS system - developed by University of Denmark in 1990s and adapted for use to predict plumes of virus and spread where predicted weather factors are included.

3.4 Papers written by the modelling teams

There were a number of scientific papers written by the teams of modellers whose work was used by the government to determine their policies.

The paper published by the Imperial team in May describes the ‘model’ they used which led government policy to use the contiguous cull of all livestock within 3kms of an Infected Premise (IP).

This model relies heavily on the ‘contact tracing’ carried out by MAFF in the first 3 weeks and discovered from this information that farms within 3 kms of an IP appeared to be at greater risk of infection. There was no differentiation between different species although information about the different infectivity of sheep, cattle and pigs to this particular strain of FMD was readily available. There was another assumption that infectivity on a farm is constant from day 3 after infection to day 11.

Presumably the 3km spread was assumed to be via windborne transmission, although the Pirbright team knew that this strain of virus did not spread that way over more than 200 metres.

In the subsequent paper published by this team in October significant bias in the contact tracing was uncovered and it was suggested that ‘local’ spread may have been via personnel or vehicles; it was also later discovered by analysis of what had happened that there was significant differences in infectivity between different sizes of farm and types of animal.

In a paper by the Edinburgh and Cambridge teams analysing the epidemic data afterwards, it was again discussed that there was a bias in the contact tracing data towards ‘local’ contacts without discovering how the disease was spread. They also suggest that in reality there are epidemic dynamics within a farm which means that infectivity changes over time. These became more significant as delays in culling infected animals (as well as all the contiguous stock) built up. Analysis found significant differences in infectivity between species with cattle being more liable to infection and sheep being relatively little affected.

All these facts were known by the Pirbright team before this epidemic occurred (published in May 2001) - but were not fully taken account of by the modellers.

3.5 Main problem with modelling - the data

The main problem with all the models was not the methodology or the assumptions, but the quality and integrity of the available data. This was not of a standard consistent with modern practice - there are comments in most of the papers by the modelling teams about the data. These range from the relatively mild comments from the Cambridge team about ‘lacunae’ in the data, to Anderson of Imperial’s comments to the Parliamentary agriculture committee that several of the farms were, according to MAFF’s figures, situated in the North Sea.

From my own experience of the data, checked and gathered every single day since I became involved in this epidemic, I know that the data which MAFF/ DEFRA published is extremely inaccurate. Any experienced data analyst would have realised that there was no point in continuing with the modelling unless significant effort was put into validating, verifying and correcting the available data. Until the data validity was improved only extremely simplistic models could be run.

3.6   Modelling backwards

When running simulations using a computer, there is very often a situation where the people requesting the results know what their desired ‘answer’ is, and they run several scenarios making adjustments to the input parameters until the ‘right’ answer appears. This is a perfectly valid methodology under certain situations eg when modelling for financial decision making and the maximum spend budgetted for is already known.

Under these circumstances, an experienced modeller may recognise that the question is no longer ‘what happens to the totals if we use x, y and z as values ?’ BUT ‘which values of x, y and z will give the acceptable answer for the totals?’ and adjust the basic equations (depending on complexity) to solve for the desired values of the parameters x, y and z.

This can be feasible - and saves wasting a lot of time running simulations. The ‘models’ used for studying the FMD spread might reasonably have changed from the questions on ‘where is this spreading to, and how long will it take to come under control?’ to ‘ what do we need to do to get this over by a certain date?’.

After all there were various scenarios, such as ‘kill every susceptible animal immediately’ which would have achieved the desired result within a very short period, always supposing that infinite resources were available.

The problem in this situation is knowing whether the people building the model are sufficiently experienced to recognise this is happening and thus have sufficient understanding of what is really required, as opposed to what they have been asked for.

The people involved in actually doing the modelling may have been relatively inexperienced in such a ‘political’ environment where the questions asked are not straightforward. Too many of the people on the science committee were the senior academics and researchers who spend most of their time organising funding for their academic department rather than actually doing any academic work. They are more used to a political environment.

Whether the ‘right’ questions were asked is unknown but purely from reading the press releases on the subject of modelling I got a distinct impression that ‘backward’ modelling techniques may well have been used. The values which are discovered by these ‘back’ techniques were then run through the simulation models to give the required answers. This would no doubt account for very similar results being returned from all the different models.

Another example which concerns me about the questions asked was that the scenarios used for the modelling of possible vaccination strategies were not realistic and could not have been practically implemented. Clearly the wrong questions were asked.

‘Local’ as a reason for the spread of FMD is used without any explanation or justification as if just being within a certain distance of an IP made a farm more likely to develop the disease.

Again the question of why ‘local’ spread occurred was not asked in a sufficiently rigorous manner to elicit the answers.

Some of the later scientific papers appear to have been written more to justify their earlier decisions and to defend the ‘contiguous cull’. Further simulated results are used to justify what ‘might have happened’ without necessarily using better or more correct information. The papers appear to be more to ensure further funding of research than to genuinely find out what happened.

                   4. Information Collection

                   4.1 Initial data set-up

The information required for the models is in two sections -

  • Firstly the general information of the overall picture of the locale both for farming and geography of the areas
  • Secondly, the particular details of each IP in the FMD outbreak - by location and type of livestock involved

Ideally the general information should already have been available and fed into a Database (presumably that was why the Epiman database was purchased by the MAFF two years previously). This would be useful and appropriate under contingency procedures for ANY outbreak of disease.

Once the foot and mouth disease outbreak was identified in Britain, four New Zealand experts were sent immediately to work with British colleagues to get the system up and running urgently. Professor Morris of Massey University said that loading all the data and getting the program running was done in four days, by cutting corners to get available data into the system as quickly as possible, "warts and all", rather than methodically and calmly as intended.

In this case, the data had to be hurriedly gathered and converted from many different sources - not all of which were compatible. There was very little time for checking the accuracy or consistency of the available data on the 144000 holdings in the UK. Data was collected from:

June 2000 agricultural census

Local MAFF office databases (Vetnet)

Database created for Swine fever outbreaks

From these sources a database of all farms was created to be used in conjunction with the data collected from the Infected Premises. Basic data from this database was then immediately available to be passed to the Epiman database of information about the IPs and used in conjunction with the Interspread model.

4.2 Data integrity

Unfortunately this data was flawed from the start:

The June census figures are not consistently collected and the numbers of animals on holdings change considerably during the year, especially in February where the number of sheep on each holding need to be maximised for the sheep premium.

Overall geographical mapping data loaded from current sources did not necessarily match the Vetnet data collected from a number of sources over a period of years. This data was often seriously out of data with regard to crucial factors such as map coordinates, addresses, postcodes, local authorities and counties.

Holding data had not been removed for farms that had stopped farming many years before. Holding numbers were not unique. I was told that there were no computer systems available in local MAFF offices at the start and inexperienced personnel were drafted in to copy all this data in from the Vetnet system, and that inadequate procedures were followed for checking the data for accuracy.

All of these items were crucial for accurate modelling. The data for the infected premises was not sufficiently accurate or consistent to model clearly, nor was it particularly accurate for the MAFF officials who needed to go round the local area to put on the D notices and movement restrictions. This was certainly observed at almost every stage throughout the epidemic and must have been even worse at the beginning.

Data on the outbreak was gathered by MAFF but, as they were not ready to deal with the outbreak, the accuracy and consistency of that data was also poor. This became very apparent as soon as data was published on the MAFF website daily for public viewing.

This data was widely published in the newspapers and on the BBC.

Almost as soon as the data was published the inconsistencies began to show.

4.3 Summarising the Data

One of the main decisions to be made when setting up data of this type, is the levels to which the data may be aggregated for various purposes. There may be several different data fields that are stored merely so that data can be aggregated - for example, counties, type of holding, which office they report to, etc etc. The decision must be taken when setting up the data as to exactly which lists are to be used, otherwise all data previously loaded has to be changed whenever decisions are made to use a different categorisation.

In the case of the county lists on which the data was based this was never sorted out and even today some of the data is aggregated to one set of ‘counties’ (local authorities) and other information is aggregated to the ‘counties’ as they were in 1979. This may seem in itself a small point - but it ensures that it is impossible to completely reconcile the information in the two sets of tables!

This has been true throughout the running of the epidemic - the ‘local authority’ table used to tell the general public where the Infected Premises were located, has been amended constantly since last year, and has only just (Jan 18th 2002) been corrected.

Because the data has been changed so often, the total number of IPs (Infected premises) assigned to a particular county on the Totals page is often different from the number of Names and addresses on the actual Table; the total number of all these sub-totals is often different from the actual number of IPs at any point in time.

When producing large amounts of data in a computer it is common practice to produce ‘control totals’ which ensure that all the data is actually entered into the system. At any point, adding up details of the data gives a check as to whether all the data is included. This principle has been totally ignored throughout the course of this epidemic - the total of the number of cases on the county summary tables often did not add up to the number of cases in the database.

4.4 Example of Inconsistency and Inaccuracy in MAFF statistics

A telling example is the slaughter statistics - three different sets of numbers are published on the DEFRA website every day which give the following details:

List 1: the number to date of animals slaughtered rounded to thousands - categorised by cattle, sheep, pigs, goats, deer and ‘other’ - this is over all premises whatever the status - (infected, direct contact, contiguous and ‘slaughtered on suspicion’)

List 2: a complete table by counties of the actual animals slaughtered with one row each for cattle, sheep, pigs, goats, deer and ‘other’ - and one column each for the 4 different kinds of premises

                List 3: a summary over all counties of the data on List 2

Logically these 3 lists should match each other but they do not. To demonstrate this here is an example from a random day taken recently.

Examples below: (Source - taken from DEFRA as at 14/1/2002 published on 15/1/2002)

List 1: from DEFRA as at 17:00 on 14/1/2002 published on 15/1/2002

4,050,000 animals recorded as slaughtered (594,000 cattle, 3,310,000 sheep, 142,000 pigs, 2,000 goats, 1,000 deer, 1,000 other animals slaughtered

List 2: base slaughter data by county TOTALLED over all counties - from DEFRA as at 17:00 on 14/1/2002 published on 15/1/2002

Animals

IPs

DC

Non-contiguous

SOS

Total

Cattle Total

304934

193267

82151

14356

594708

Sheep Total

953989

978262

1270834

110902

3313987

Pigs Total

20204

48944

70714

2543

142405

Goats Total

934

664

544

293

2435

Deer Total

25

578

411

3

1017

Other Total

283

306

0

3

592

Grand Total

1280369

1222021

1424654

128100

4055144

 

 

 

 

 

 

 

 

List 3 - from DEFRA summary of data over all counties as at 17:00 on 14/1/2002 published on 15/1/2002

Total animals slaughtered

Infected
premises

DC Contiguous
premises

DC Non contiguous
premises

Slaughter on suspicion

Grand Total

Cattle

304934

193267

82151

14356

594708

Sheep

953989

979505

1270834

110902

3315230

Pigs

20204

51594

70714

2543

145055

Goats

934

666

544

293

2437

Deer

25

578

411

3

1017

Other

283

306

0

3

592

Grand Total

1280369

1225916

1424654

128100

4059039

Comparison of Data on Lists 1, 2 and 3 above

Looking at the above 3 Lists the assumption must be that the data on List 2 is the most accurate (as the others are only summaries and List 2 contains ALL the basic data by county)

List 3 has anomalies for sheep, pigs and goats in the DC contiguous premises column (italicised numbers) and the same anomalies in the Grand Total column.

List 1 has anomalies in both the details for sheep and cattle. This data should be:

(amended List 1)

4,055,000 animals recorded as slaughtered (595,000 cattle, 3,314,000 sheep, 142,000 pigs, 2,000 goats, 1,000 deer, 1,000 other animals slaughtered)

Key: highlighted and italicised data shows differences

Note: On 20th February 2002, List 3 has finally been corrected to match the totalled information over all counties on List 2 - however the rounded data on List 1 is still different from the details on the other list by some 5000.

4.5 Collection History for IP data

There are 2 different tables on the MAFF/ DEFRA website that list all the Infected Premises - one in numerical order and one list of similar information by county. This latter table has always been rather strange and caused much confusion. Let me explain what happened chronologically:

March 2001

During the first month of the outbreak the only list that became available was the list of IPs by county.

The information on this table is (by column)

Case number - Date confirmed - Name and address of Owner and premises

April 2001

Sometime during April the List of All Infected Premises became available and contained an extra column - the number and type of animals (apparently this information provided from the valuation details which would account for why so many of them appeared to have nicely rounded numbers such as 2000 sheep).

As I had collected all the county data in my own spreadsheet, I extracted the list of the numbers of animals so I would have the most complete data available. It soon became apparent to me, when checking the data from one list to the other, that the names and addresses from the two lists were slightly and subtly different - as if typed by different people.

Typing data twice would seem to most people to be just a waste of time and resources - BUT to any Information Technology expert it is a disaster. Data should only be saved in one place so that it can remain consistent - and then when changes need to be made they only need to be made in that one place. At the beginning of my study of the data I thought this must just be due to a shortage of staff in a pressurised department - but as time went on, it became just another factor demonstrating that MAFF really had no proper controls and plan for how to organise and validate the information needed.

This was clearly true in the main computer database they used for ALL the data about the farms. IACS holding numbers are not unique - anecdotal evidence from farmers indicated there were often several farms with the same (supposedly unique) holding number - probably because numbers were generated by local offices and not centrally - a throwback to old manual pre-computerisation methods.

May 2001

It had become apparent that there were other inconsistencies with the way data was treated. We were always told that once an IP was ‘confirmed’ it remained on file even if it was later proved to have not had FMD.

At least two cases were ‘downgraded’ from IPs to ‘slaughter on suspicion’ and the numbers re-used for later cases - as you see from the data below extracted from the 2 different sets of tables:

Table 4.5.1 Extracts of IP data (Source MAFF web site)

 

County

IP no

Date

Premises

Cattle

Sheep

Dumfries & Galloway

229*

14/03/01

Mr D Stoddart High Law Lockerbie Dumfries and Galloway

296 cattle

145 sheep

Devon

590*

25/03/01

Mr B Drury Winstode Farm Crediton

Devon EX17 5HQ

 

Northumberland

590

02/04/01

Mr W Aynsley Whiteside Law Hallington Newcastle Northumberland

600 cattle

71 sheep

Cumbria

229

11/04/01

M/S R W & J J Steel Lesson Hall Wigton Cumbria CA7 0EA

89 cattle

224 sheep

The two original cases 229 and 590 were revoked and the numbers reused later (note dates!), so sorting the list into numerical order does not give it in date order.

However in all other cases, when the blood tests came back as negative, the cases were left on the list of infected premises and remained there. Contiguous farms or farms on a ‘D’ notice were still culled out or kept on movement restrictions - taking up much needed veterinary and surveillance resources.

                   4.6 County Tables for Slaughter Statistics

The list of counties was changed several times during the outbreak - mostly trying to get the data more consistent. However, near the end of May, tables of slaughter statistics became available.

This is a very large table with 7 rows and 5 columns for each county:

One row each for: cattle, sheep, pigs, goats, deer and ‘other’ animals killed and a Total row

And one column per premise type: IPs, CPs, DCs, SOS and a Total column (see below)

             Definition of type of premises

Infected Premises (IPs): premises where foot and mouth disease has been confirmed.

Dangerous Contacts (DCs): premises where animals have been subject to direct contact with infected animals or have in any way been exposed to infection. The figures in the ‘other DCs’ column exclude data separately recorded for contiguous premises (CPs).

Contiguous Premises (CPs): a category of dangerous contacts where animals may have been exposed to infection on neighbouring infected premises.

Slaughtered on Suspicion (SOS): premises where a veterinary inspection detects some symptoms of disease, but these are insufficient to confirm that foot and mouth disease is present. Animals are culled and samples taken to confirm the presence/absence of disease. The tables include only those animals where samples have proven negative or remain unconfirmed; SOS cases giving positive results, or that are subsequently confirmed on clinical grounds, are classified as IPs, and slaughtered animals are recorded in the IP column.

(Source: DEFRA website)

The table below gives an example of a section of the DEFRA slaughter statistics.

Table 4.6.1.Example of slaughter statistics for County of Avon:

 

County

Data

IP

Contiguous

Non Contiguous

SOS

Total

Avon

Cattle

116

1291

0

6

1413

Avon

Sheep

40

829

0

534

1403

Avon

Pigs

5491

9

0

2058

7558

Avon

Goats

8

3

0

0

11

Avon

Deer

0

0

0

0

0

Avon

Other

0

0

0

0

0

Avon

Total

5655

2132

0

2598

10385

The county list for these statistics is very different from that used to describe the IPs, and caused much confusion when trying to match the data. It was finally described by the DEFRA website (during early January 2002) by the phrase ‘These statistics are classified into county boundaries as previously determined in 1979’.

It is difficult to cross-reference this list of counties to the original list classifying the IPs.

Much of the data provided by DEFRA in answer to various Parliamentary questions is in this format. It is not transparent and also makes it difficult to check whether the inconsistencies are purely due to the ‘county’ boundary problems or whether the data is just inconsistent.

For examples see Appendix 9.2.

5. Examples of Data Problems

                5.1 Analysis of Statistics - Original County List

The list of counties was totally uncontrolled and inconsistent - people doing their best but with no direction and management. Quite often the total of the number of IPs by county (on the front page of the website where you can access data by county) did not add up to the total number of cases to date.

It was apparent (from the OIE reports provided to the EU by the chief vet every fortnight) that the totals by county are an international convenience for management and reporting - especially as the management of many government people who need to be involved.

However Counties have in many cases been replaced by new Local Authorities. They are responsible for regulating and enforcing various matters to control FMD such as:

Implementing local contingency plans; serving Forms A, D and E on behalf of MAFF;

enforcing the licensing regimes; issuing licences for (a) movement of animals for slaughter for human consumption, (b) Collection Centres and (c) for farm to farm and Autumn movements of livestock; collection of stray animals; controlling movements of animal by-products; restriction (and enforcement) of countryside access and also providing advice and communicating local control measures to the wider general public audience.

However, instead of providing a full list of local authorities for the data entry clerks to enter the data correctly and consistently, it was left to the people entering data to work it out for themselves. This they tried to do from the addresses - however much of the data on the database created by MAFF was out of date and, especially since the new Unitary Authorities arrived in the 1990s, these are no longer part of most people’s postal address.

The counties and local authorities changed several years before, post codes (if available) had also been extensively changed in the last few years- names and addresses had been extracted from internal MAFF databases and did not tie up at all with current accurate information.

This was obvious to me from the beginning - from the data published, it was difficult to locate the farms involved on the online mapping systems - very quickly I realised that, as with so many enterprises of this nature, the data was not very good and was being badly managed.

5.2 Name and Address anomalies

During May I found one of the most curious anomalies - still there on both tables: Case 1557 has two different names and addresses (many miles apart)

Current numeric list of all IPs

1557 May 5 JN & GM Hadwin, High Aulthurstside Farm, Woodland
Broughton in Furness, Cumbria, LA20 6AE Cattle 32

Current list for Cumbria (January 2002)

FMD 2001/1557 Stoddart, Hillside, Wigton, Cumbria 06/05/2001 10:39 CUMBRIA

                 How can this be?

              5.3 Data validation

To show what I mean about lack of validation of data entered:

An example that demonstrates the differences and lack of checking is the following set of information extracted from the two different parts of the database a few weeks apart:

 

Source

COUNTY

IP no

Date

Premises

Cattle

Sheep

Numeric list

Herefordshire

535

23/03/01

M/S SN & SJ Gibbens Llwynbrain Farm Llanigon Hay on Wye Hereford HR3 5QF

41 cows

220 sheep

County list

Devon

536*

24/03/01

M/S AF & SJ Gibbens,            North Down Farm, Lewdown, Okehampton Devon EX20 4EB

Numeric list

Devon

536

24/03/01

M/S AF & SG Loud & Sons North Down Farm, Lewdown Okehampton,Devon EX20 4EB

480 cows

 

Clearly the person putting the data in for case number 536 on the county list started doing it correctly, ‘M/S AF & ‘ and was then interrupted. This person then continued typing ‘SJ Gibbens’ as in the previous address, then continued again, presumably after another break, with the remainder of the address for the Loud family without ever checking that the correct name and address had been entered.

I picked up these differences because one name and address was on the County list, and the other was on the ‘all cases’ list. The case labelled 536* above later disappeared as the correction was made unlike many of the other mistakes which remain to this day.

5.4 Constituency Tables - why do they exist?

Another table created in May 2002 (and still updated regularly) is Infected Premises by Parliamentary Constituency - this table has columns for

                Constituency name

Constituency MPs name

Party to which the MP belongs

Number of IPs in that Constituency

This table was 23 cases short when first published and has been ever since, however often it was updated. My explanation for this is that presumably the person who was given the task of creating this table (relatively easy on the Internet as long as you have the exact postcodes for all the IPs) extracted the data at a particular point. Once the table had been created, it was a few days later and there were 23 more cases of Foot and Mouth - however whoever was in charge did not check that the total number of cases added up to that on the IPs. After that, whoever updated this table on the daily list merely updated for the new cases that day. So that table was still 23 cases short!

This table was finally updated on 8th January 2002 and has at last had the totals corrected.

It worries me as to who this table was for - there are less than 700 MPs and this specialised group of people certainly didn’t need some of the limited resources available in DEFRA to create this table for them. It concerns me that this table continued to be regularly updated and corrected right up until the beginning of 2002.

5.5 Changing the Tables

In January 2002 the website data for the Infected Premises by County was transformed once more into a list which classified each IP according to the Local Authority which controls it

The OLD County tables looked like this:

 

FMO 2001/nn

Date confirmed

Owner/premises

1470

25 Apr

Mr W J Francis
Goitre Coed Farm
Abercynon
Mountain Ash
Mid Glamorgan
CF45 4EN

1426

21 Apr

Mr EH Leyshon
Pentwyn Farms
Crymlyn Road
Skewen
Neath
West Glamorgan
SA10 6NL

1108

8 Apr

Mr M A J Jarrold
Parc Farm
Nelson
Caerphilly
Mid Glamorgan
CF46 6DR

This data is now on 3 different data tables and looks like this:

(1 table for each unitary authority)

 

INFECTED PREMISES

(IP) FMD NUMBER

OWNER NAME & IP

ADDRESS

CONFIRMATION

DATE

COUNTY NAME

DESCRIPTION

FMD 2001/1108

Jarrold

Park Farm

Nelson

Caerphilly

Mid Glam

 

07/04/01 20:10

CAERFFILI -

CAERPHILLY

 

Unitary Authority

FMD 2001/1426

Leyshon

Pentwyn Farms

Crymlyn Road

Skewen

West Glasmorgan

 

21/04/01 17:53

CASTELL-NEDD PORT

TALBOT - NEATH

PORT TALBOT

 

Unitary Authority

FMD 2001/1470

Francis

Goitre Coed Farm

Abercynon

Mountain Ash

Mid Glamorgan

 

25/04/01 13:10

RHONDDA, CYNON,

TAF - RHONDDA,

CYNON, TAFF

 

Unitary Authority

You will notice that the new tables have additional data, the name of the Local authority and a column that describes what type of authority it is.

However, the names and addresses have been retyped yet again and, in doing so, have lost both the owners initials and the Post Code. This makes the farms difficult to locate for anyone interested in studying the outbreak.

The minor spelling mistakes and the changes in dates demonstrate yet again that there is insufficient checking and validation of data.

5.6 Analysis of the new County List by Local Authority

I spent some time trying to match the county tables from the December County Tables to those in the new lists of Local Authorities created in January 2002. The numbers appeared to match fairly reasonably until I looked up specific data and found that the new lists contained as many anomalies in the first rough analysis of the comparison as the old lists.

When I tried to find whether the new list matched up for counties I knew well, I found that the information as recoded is more accurate in some places and less accurate in others - there are complications with Hereford, Monmouthshire, Powys, Shropshire. Some of the old IPs have been corrected but new ones have now been moved into the ‘wrong’ counties.

Take case 1 (Cheale Meats abattoir) and case 2 (A Cheales farm) which is right next door - literally a few hundred meters, as is the farm of case 3 - Mr Gemmill. These all used to be in Essex - now case 1 and 3 are in Essex and case 2 is in Thurrock

Shropshire used to have 12 cases in the old list - it now has 15 cases of which 3 cases are in the old Powys list (and are still in Montgomeryshire which is in Wales and hence definitely not in Shropshire), 10 cases are from the old Shropshire list and 2 cases are from the old Staffordshire list. Two cases that used to be on the Shropshire list have moved over to Staffordshire!

When I tried to sort out the relationship between the old and new Powys lists I found links to cases previously in Monmouthshire, Herefordshire, and Shropshire- it was just too complex to sort it all out in a hurry.

So the new lists do not seem any more accurate than the old ones - and with even less data available, it is more difficult to find the locations of individual IPs.

                    6. Other Methodology

6.1 Other modelling methods by volunteers

On the Internet there were several other people with different experience of modelling who have been involved in trying to predict how the foot and mouth epidemic behaved. Some of these used mathematical modelling techniques such as logistic equations (used in population biology) and simple parameterised models. Others used traditional mapping techniques using standard computerised maps which are easily available on the Internet.

Most of these techniques were of a less complex nature than those created by the teams of bio-mathematicians employed by MAFF and because of the inherent simplicity of the techniques it was often easier to ‘see’ how changing the parameters produced different results.

Hence these independent researchers were able to validate their ‘models’ against the predictions made at various stages and check the assumptions against the actual results. Many of these people published extensively during the outbreak - because of their concerns at the way the epidemic was being tackled and the reliance on complex mathematical models to determine policy.

Internet discussion groups all over the world became involved in trying to find a better way forward. Researchers volunteered and shared knowledge; volunteers worked very hard

doing whatever they could to assist. Many of these discussion groups became conduits of knowledge - there was the opportunity to get information from all the world experts, many of whom gave their time and expertise freely.

                 6.2 Maps

Several volunteers mapped all the outbreaks using computerised tools. Others tried to predict the spread via these maps. Many theories abounded as to what caused the spread.

                 6.3 Tracings

The tracing information created by MAFF/DEFRA did not become available to the general public until the outbreak was finished. However, when I tried to use it to understand the information in the paper by the Cambridge group, I soon found the data in the scientific paper did not tie up to the tracing information published to the Parliamentary agriculture committee by Mr Scudamore. Again the anomalies - this time in the number of abattoirs and markets shown. And of course, by that point I had become aware that a considerable proportion of the Infected Premises had tested negative and had probably not had FMD at all. So I abandoned that particular line of investigation until I have more accurate information.

                6.4 Picture of Epidemic spread using more traditional methods

When trying to comment on the mathematical modelling techniques available and used in the early days of the epidemic, I tried to put myself in the position of having only the data that was available at the time the models were created in March, and the ordinary tools available at the time.

Naturally the modellers had the advantage over me in that they had more complete data on each infected premise and of any neighbouring premises as well as models.

However I had the complete list of cases as and when they were reported with all the public details available and I also had a simple road map. I would have preferred to use the standard ordinance survey maps as (especially the older ones) they show nearly all farms, but then I would have needed the sort of ‘war-room’ size table that the Army and logistics people presumably used.

Anyway, I just took the Infected Premises and numbered small round white labels with the case numbers and coloured the labels differently for weeks 1,2,3 and 4 so that the pattern emerged on the map. I then located the IPs as accurately as I could (more difficult with the early ones because Post Codes were not at that stage included on the MAFF website)

This gave me the similar sort of information that was available to the logistics teams AND the modellers. The picture that emerged gave the following figures over the first 4 weeks, translating the little dots on my map:

The following Table demonstrates the picture I could see on my maps.

Table 6.4.1 Weekly cases of FMD over first 4 weeks analysed as IPs, Counties and Areas

 

Week number

1

2

3

4

 

Ending Sun

25th Feb

4th Mar

11th Mar

18th Mar

First 4 weeks

Weekly

 

 

 

 

 

IPs

7

61

96

161

325

Counties

3

16

7

4

30

Areas

3

26

20

16

65

 

 

 

 

 

Total

Cumulative

 

 

 

 

 

IPs

7

68

164

325

325

Counties

3

19

26

30

30

Areas

3

29

49

65

65

Key: IPs = confirmed cases ie infected premises

Counties are mostly as defined in the postal address (some anomalies)

Areas are defined by groups of dots showing several cases round a local geographical area

Week 1 was characterised by just a few cases BUT already it was clear that these were in 3 very geographically dispersed areas - Essex, Devon and Northumberland

At the end of Week 2 there were 68 confirmed IPs in 19 counties. 29 different areas of the country affected. This week was the worst for spread.

Week 2 and week 3 show the results of spread from markets and increasing recognition of the disease by farmers and vets.

At the beginning of week 4, although the number of cases continued to rise, the results of the stopping of markets and movements began to work. The cases in the 4th week were mainly due to spread from other IPs or ‘not known’ according to Mr Scudamore’s tracing diagrams.

Adding in the data from the Tracing document (for first 325 cases) and grouping them by what MAFF believed to be the method of spread we get the following table:

Table 6.4.2 Weekly cases analysed by Tracing document for method of spread of FMD

 

Tracing From

Week 1

Week 2

Week 3

Week 4

Total 1st 4 weeks

Index case

1

 

 

 

1

Markets

 

27

21

6

54