45TH ANNUAL CONFERENCE, Kaohsiung, Taiwan, 27-31 March 2006
WP No. 93
Review the Use of Safety Cases
Presented by Chairman Committee B
1.1. During previous conferences, a number of working papers made reference to the use of safety cases as a guarantee of the safety of a system or sub-system. The increasing use of this condition suggested that there may be a number of interpretations concerning safety cases and that there is a need for a greater understanding about the content, validity and the use of safety cases.
1.2. This paper considers the basic components of a safety case and its operational context in relation to system safety. This would provide the basis of a consistent understanding within the Federation about the use of safety cases.
2.1. The Abstract paper presented to the AFM regional meeting held on 28-30 Nov 05 in Dubai included the WP C 5.18 – Safety Management from the Hong Kong Annual Conference. This WP was wide ranging and made some important points for MAs to consider. With regard to this paper, there are 3 issues regarding safety cases that need emphasising:
i. “….to understand safety everybody working in aviation should have a basic knowledge about elements such as TLS, Risk Management, Functional Hazard Analysis and Mitigation measures….. by reading or making available material from various sources talking about a target level of safety, a functional hazard analysis, a safety case, a risk assessment, a mitigation procedure”.
ii. a safety case (an analysis presenting an overall justification for the declaration that a particular system satisfies its safety requirements (Eurocontrol)) will establish that a new procedure is meeting the TLS as required. Normally this is achieved with a Functional Hazard analysis. Operational and system experts sit together and elaborate a list of possible Hazards which could occur introducing a new piece of equipment, a new procedure and a new human factors/resources related issue. This analysis is documented and all possible hazards are identified. Then the Hazards are quantified and mitigation means are proposed. The second part of the Safety Case will concentrate on what has to be done to have the mitigation means in place the day of the introduction of any new elements in the current aviation system.
iii. The WP contained a schematic diagram that indicated a process for safety assessment that becomes the basis for the Safety Case.
2.2. Point (i) is excellent advice and this WP intends to expand on this theme however points (ii) and (iii) are only part of the rationale for a safety case. In answer to the question – why do we need a safety case we need to add more detail to support the Eurocontrol definition extracted from the earlier WP which is:
“A safety case is a document which contains claims, arguments, and evidence that make a case for the safety of a system, operation or service” (CAP 760)
For every change to existing systems*, or introduction of a new system, a safety assessment should be carried out and documented. Depending upon the safety significance of these processes will determine the amount of justification that is required to satisfy both the operator and the regulator that the system is safe.
Note: Systems, sometimes referred to as projects in this context, can be considered to include the following constituents:
- any technical system or part of a system
- any procedure (e.g. operational procedure used by the aerodrome operator or air traffic service provider or, alternatively, a maintenance procedure for related equipment)
- any operation
For example, the national authority may mandate a change to SSR code allocation however a safety assessment should be completed before introduction so that its effects on the ATS operation can be determined. In this case, it is probable that the safety significance will be either beneficial or neutral however the documented safety assessment will provide the evidence. The evidence may be contained in a single page justification whereas, on the other hand, a complex procedure such as a RNAV arrival/departure TMA route system may be proposed and this could involve a number of stakeholders (ANSPs, airport authority, airlines, pilot and controller organizations etc). In this case, a fully documented safety case (see para 2.5) will be required to record the claims, arguments and evidence that the procedures are safe.
2.3 When preparing for system changes or new introductions (project), it should be recognized that there are a number of phases during the life of the project including the possible decommissioning. Safety needs to be planned throughout all these phases which will include safety assessments at each stage. Performing safety assessment early in the project can identify hazards that impact on the design of the system. It is better that these hazards and their impact are identified early in a project, rather than incurring expense trying to change a design after hazards have been discovered.
2.4. Planning for safety is as important a part of a project as planning for operational use. Consideration should be given to developing a Safety Plan for a project detailing:
i. The scope of the project or system that is being considered
ii. The safety activities planned to be carried out in the different project phases
iii. When or at what stage in the project the safety activities will be carried out
iv. The staff responsible for contributing to the safety activities
v. The authority of staff e.g. having the authority to approve safety documentation or having the authority to accept unresolved risks on behalf of the organisation etc.
2.5. The project safety lifecycle is likely to include some or all of the following phases:
i. Preliminary (feasibility and concept) phase
ii. Design and development phase
iii. Transition to Service phase
iv. Operational (ongoing) service phase
The safety case documentation follows these phases and normally falls into 4 parts (excluding decommissioning) however the safety case is a living document which remains active during the whole life cycle of the project. Subsequent changes to the project may require a further safety case to be produced so that the project remains fully documented.
2.6. The original Eurocontrol definition of a safety case refers to:
“…overall justification for the declaration that a particular system satisfies its safety requirements.”
Therefore we need to consider safety requirements. What is a safety requirement? It is a risk mitigation measure that is necessary for the system/project to meet the safety objective. There are 2 basic types of safety requirement: derived safety requirements and statutory safety requirements. A derived safety requirement is one, which has been generated by safety assessment process whilst the statutory safety requirement is one that has been specified by a Standard and/or a national regulatory authority. Safety requirements may take various forms such as organisational, operational, procedural, performance and interoperability requirements.
Using the RNAV example, the hazard identification and risk assessment of the safety assessment process may have produced a derived safety requirement which stated that:
“ the probability of aircraft deviating from track by more than 0.1nm shall be no less than remote (10-6 per hour)”
whilst the statutory safety requirement (decided by the national authority) may state that:
“RNAV procedures may only take place when radar monitoring is available”.
The preliminary phase of the safety case (Part 1) will produce the top-level safety requirements (derived and statutory) which have to be satisfied by the subsequent parts of the safety case using the appropriate arguments and evidence.
As the project develops through its various phases, further safety assessments are required to test the original safety requirements, to refine them with more detail or even to reject them because they have become redundant.
2.7 As an example of what the safety case contents should look like, the following has been taken from the Eurocontrol Safety Case development manual:
Aim: what the Safety Case is trying to show, and directly related to the Claim that the service or system is safe;
Purpose: why is the Safety Case being written and for whom;
Scope: what is, and is not, covered;
Strategic Safety Objectives: Strategic Safety Objectives, which include Safety Criteria to define what is safe;
System Description: a description of the system change;
Justification: for project Safety Cases, the justification for introducing the change (and therefore potentially for incurring some risk);
Argument: a reasoned and well-structured Safety Argument showing how the Aim is satisfied;
Evidence: supporting Safety Evidence to substantiate the Safety Argument;
Caveats: all Assumptions, outstanding safety Issues, and any Limitations or restrictions on the operation of the system;
Conclusions: a simple statement to the effect that the Aim has been satisfied, subject to the stated Caveats.
It is not intended to cover the detail of the safety case contents however it is useful to consider Arguments and Evidence. First, the definition of Argument is:
“a statement that points to evidence justifying why a claim is valid. Depending on the amount of evidence available supporting a claim being made, more than one argument per claim or sub-claim may be required to encompass all of the evidence”
Evidence is used to demonstrate that a Safety Requirement has been met through claim-argument-evidence statements or GSN diagrams (Goal Structured Notation – is an intuitive form of claim/argument representation that is growing in popularity with Safety Case authors across industry). There can be many sources of evidence, however it is important to think about two key types of evidence:
Direct Evidence – This is evidence clearly linked to the Safety Requirement itself e.g. a test result showing that a safety critical parameter is within tolerance, or the results and analysis of a RNAV simulation.
Backing Evidence – This is evidence that supports the Direct Evidence, thus giving Direct Evidence more credibility. In the above examples, it could be that a qualified test engineer using certified test equipment carried out the testing or that fully operational TMA controllers participated in the RNAV simulation.
The authority of a safety case proving that a system is safe depends on the credibility of the evidence produced to support the claims and arguments. If it does not stand up to scrutiny then the safety case has failed and the system should not be put into operation until there is sufficient and reliable evidence. For MAs and individual controllers, this is the most difficult area to make comment because the evidence is often highly technical particularly when the integrity of software is being tested.
To overcome this issue, there are a number of possibilities including, inter alia:
i. dedicated safety training for operational staff, including safety diplomas/degrees,
ii. use of safety specialists as consultants, and/or
iii. liaison with engineering, pilot and other aviation organisations
3.1. The safety case is an essential element of the safety management process for ATM. All changes to a system or the introduction of a new system should not be implemented until a satisfactory safety case has been produced with justified claims backed with strong arguments and reliable evidence to prove that the system is safe to operate. The level of justification is dependent on the complexity of the system and the number of involved stakeholders.
3.2. The existence of a safety case for system changes is not a guarantee of safety unless the documented evidence is appropriate, reliable, and sufficiently detailed. MAs and individual controllers are likely to require either further training or external assistance to determine the validity of the evidence contained in a safety case.
It is recommended that;
4.1 This paper is accepted as information material.
CAP 760 – Guidance on the Conduct of Hazard Identification, Risk Assessment and the Production of Safety Cases For Aerodrome Operators and Air Traffic Service Providers (UK CAA Jan 06).
Draft – Safety Case Development Manual (Eurocontrol).
Last Update: September 29, 2020