Subject: Re: Airline Software-safety database (RISKS-14.08)

From: Pete Mellor <[email-masked]>
Date: Sun, 22 Nov 92 17:21:22 GMT

Dave "Van Damme" Ratner <[email-masked]> writes in RISKS-14.08: 

> I am posting this for Robert Ratner, Ratner Associates Inc, which does
> international consulting in air-traffic control and aviation safety issues.  
> He is looking for a public-accessible data base on software-related incidents 
> in this area.  Email correspondence can be sent to me at [email-masked].
> Thanks.            Dave "Van Damme" Ratner    [email-masked]

In my experience, all major manufacturers of software keep databases of 
incidents reported by users of their software and the faults ("bugs") which 
give rise to those incidents. I know for a fact that IBM, ICL, DEC, Unisys 
(or whatever it is now), and Sun all do this. 

Such a database is essential to their efforts to improve the quality of their 
software by identifying and fixing bugs, and to reduce their maintenance 
workload by informing customers about known problems so that repeated reports 
are suppressed. 

The interesting phrase is "public-accessible". If you are a customer of a large 
manufacturer of system or application software, you will almost certainly have 
access to the *relevant* parts of the database (those which concern the 
products you have bought). This will be provided either on-line, or as printed 
or micro-fiche extracts, updated on a regular basis. 

The other interesting phrase is "in this area" (i.e., of air-traffic control 
and aviation safety). 

The users of safety critical on-board avionics software are the companies that 
buy the aircraft. They are provided with regular information about all sorts 
of design glitches in the aircraft they have bought, including those in the 
software. Such information is provided in the form of "OEBs" (Operating 
Engineering Bulletins), which are distributed to the flight crews. 

Information about software faults in safety-critical avionics systems *must*, 
therefore, be kept on a database somewhere. These databases are public in the 
sense that any pilot on that type of aircraft would have access, but Joe 
Public (as far as I know) does not. 

Incidents in flight must (or should) be reported via offical channels by the 
crews. These reports drive the manufacturers' quality improvement programmes. 
After the fault which caused an incident has been diagnosed, it may result in 
an OEB or similar, and in a modification. 

Databases of such incident reports are not generally widely accessible. 
Published reports sometimes appear, however. In addition, there are channels 
for anonymous reporting of incidents. In the UK, "CHIRP" is such a forum. In 
the US, I believe the FAA used to run such a scheme, but it was compromised 
when the guarantee of anonymity was removed. 

For further information I suggest you contact ALPA. 

Given the increasing use of safety-critical software, a central database for 
each major application area would be highly desirable, to say the least. 
Obviously, sensitive issues of commercial confidentiality are involved. In 
particular, it may be difficult to obtain corresponding figures for the 
operating time so as to be able to estimate reliability, and it may be 
difficult to correlate incidents with faults, and so determine which incidents 
are due to software. 

I stand to be corrected if anyone *does* know of an official channel for 
public access to flight incident and system fault reports. 

Regarding ATC incidents, again I am certain that these are recorded, but access 
is not likely to be easy. 

Peter Mellor, Centre for Software Reliability, City University, Northampton 
Sq., London EC1V 0HB, Tel: +44(0)71-477-8422, JANET: [email-masked] 
-----------------------------------------------------------------------------

---

From: [email-masked] (Michael T. Palmer)
Date: 01 Dec 92 13:54:52 PST

Pete Mellor <[email-masked]> writes:

[etc]

>an OEB or similar, and in a modification. 

[etc]

>For further information I suggest you contact ALPA. 

[etc]

>public access to flight incident and system fault reports. 

Okay, here goes.  In the USA, NASA and the FAA have teamed up to deploy
the Aviation Safety Reporting System (ASRS), which is managed by folks at
NASA Ames Research Center, Moffet Field, CA.  I believe the original
poster misunderstood the intent and operation of the ASRS, so I will try
to fill in some details.  Anyone from the ASRS Office at Ames is free
to jump in and correct any mistakes I make.

The ASRS collects incident and accident reports for all aviation-related
activities, including flight, air traffic control, and maintenance.  The
way it works is that whenever an individual is involved in an incident,
he or she is encouraged to submit an ASRS report describing what happended,
why they think it happended, and what should be done to correct the problem
so it doesn't happen again.  The fact that a report is submitted (receipts
are kept for proof) is accepted by the FAA as a sheild from legal
retribution except in cases of gross misconduct or criminal intent.  There
is no anonymity, per se, in the filing of the report.

When the reports are collected and entered into the ASRS database, they
have keywords identified to allow easier searches on related topics.  At
this time, they are also "de-identified."  This may be what the original
poster misunderstood as a guarantee of anonymity.  During de-identification,
all references that would lead a later reader of the report to be able to
identify the exact person, place, and aircraft (by N-number) involved are
removed and replaced by generic terms.  This protects filers from, say,
unscrupulous company or government people that seek to harrass them later
for possibly unrelated reasons.

Now, for the good part.  This database, which is HUGE, is publicly
"accessible."  However, the access is controlled, and database search
requests must be submitted to and approved by the ASRS office.  This
helps to prevent frivolous or duplicative use, which could rack up *very*
large costs *quickly* due to the sheer size of the database.  The actual
searches are performed by Battelle, Inc. under contract to NASA Ames.
The Ames (NASA) person to call for more information about the ASRS is
Vince Mellone at (415) 969-3969 or (415) 604-6467.  The database search
requests are actually sent to:

     Battelle ASRS Office
     625 Ellis Street, Suite 305
     Mountain View, CA  94043

If you have never used the ASRS before, I suggest you give Vince a call
first to find out what information you need to provide in your search
request so the Battelle people can help you find what you're actually
looking for.

Note that the ASRS database is NOT "on-line" such that anyone could just
dial in and peruse it at their leisure.  So, this may not suffice for
what the original poster had in mind.  However, I would venture to guess
that any database with enough information in it to provide a reasonable
basis for design would end up being so large that access MUST be controlled
somehow.  I think the ASRS is a good compromise.

-- 
Michael T. Palmer, M/S 152, NASA Langley Research Center, Hampton, VA 23681
Voice: 804-864-2044,   FAX: 804-864-7793,   Email: [email-masked]
PGP 2.0 Public Key now available -- Consider it an envelope for your e-mail