top - download
⟦1b35b541a⟧ Wang Wps File
Length: 22726 (0x58c6)
Types: Wang Wps File
Notes: CPS/210/SYS/001 ISSUE 2.1
Names: »0403A «
Derivation
└─⟦ad67f9f62⟧ Bits:30006075 8" Wang WCS floppy, CR 0032A
└─ ⟦this⟧ »0403A «
WangText
T̲A̲B̲L̲E̲ ̲O̲F̲ ̲C̲O̲N̲T̲E̲N̲T̲S̲…01…S̲e̲c̲t̲i̲o̲n̲ ̲4̲
…02…4 QUALITY ASSURANCE PROVISIONS ................. 345
4.1 TEST AND VERIFICATION METHODS ............
345
4.2 VERIFICATIONS ............................
345
4.2.1 Hardware Verification ................
346
4.2.1.1 Prototype Verification ...........
346
4.2.1.2 Preproduction Verification .......
347
4.2.1.3 Production Verification ..........
347
4.2.1.4 DSMT (Prototype) System
Verification .....................
347
4.2.1.5 Factory Acceptance Verification ..
348
4.2.1.6 COMSEC In Plant Verification .....
348
4.2.1.7 Availability Verification ........
348
4.2.1.7.1 MTBF Verification ...........
349
4.2.1.7.1.1 MTBFs of 3500 Hours or
Less .....................
349
4.2.1.7.1.2 MTBFs Greater than 3500
Hours ....................
351
4.2.1.7.2 MTTR Verification ...........
352
4.2.1.7.2.1 MTTRs of 45 Minutes or
Less .....................
352
4.2.1.7.2.2 MTTRs Greater than 45
Minutes ..................
353
4.2.1.7.3 R&M Unit or Module Test Plan .
354
4.2.2 Software Verification ................
354
4.2.2.1 Software Development Verification
354
4.2.2.2 In Plant Software Verification ...
355
4.2.2.3 Software Functional Verification .
356
4.2.2.4 Software Operational Verification
356
4.2.3 System Verification ..................
357
4.2.3.1 COMSEC On-Site Verification ......
357
4.2.3.2 Site Provisional Acceptance (SPA)
357
4̲ ̲ ̲Q̲U̲A̲L̲I̲T̲Y̲ ̲A̲S̲S̲U̲R̲A̲N̲C̲E̲ ̲P̲R̲O̲V̲I̲S̲I̲O̲N̲S̲
This section specifies the quality assurance provisions
applicable to verification of equipment and software.
4.1…02…T̲E̲S̲T̲ ̲A̲N̲D̲ ̲V̲E̲R̲I̲F̲I̲C̲A̲T̲I̲O̲N̲ ̲M̲E̲T̲H̲O̲D̲S̲
The unit test requirements will be met by one of the
following methods:
…02…a. E̲x̲a̲m̲i̲n̲a̲t̲i̲o̲n̲.̲ This method is a non-functional verification,
such as visual inspection of the physical characteristics
of the item or of the documentation associated with
the item.
…02…b. A̲n̲a̲l̲y̲s̲i̲s̲.̲ This method is a non-functional verification,
such as deduction or translation of data, review of
analytical data, or performance of a detailed analysis.
…02…c. T̲e̲s̲t̲ ̲D̲e̲m̲o̲n̲s̲t̲r̲a̲t̲i̲o̲n̲.̲ This method is a functional verification,
such as actual operation wherein the element of verification
is instrumented, measured, or displayed directly (test)
or where the element of verification is logically obvious,
as the result of some other verification, but not itself
displayed (demonstration).
4.2…02…V̲E̲R̲I̲F̲I̲C̲A̲T̲I̲O̲N̲
a) The verification efforts are divided into hardware,
software and system verification.
b) The verification will be divided into two classes
which are
…02… I - Internal verification
…02… II - External verification
c) Only class II verifications shall be approved by
SHAPE.
d) Class I verifications shall be approved by CR's
internal QA only, but may be inspected by SHAPE's
QAR.
e) The verification efforts shall be divided into
the verification type defined below.
f) H̲A̲R̲D̲W̲A̲R̲E̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
…02… C̲l̲a̲s̲s̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲ ̲T̲y̲p̲e̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲
…02…I…02…Prototype Verification
…02……02…I Preproduction Verification
…02…I…02…Production Verification
…02…II…02…DSMT (Prototype) System Verification
…02…II…02…Factory Acceptance Verification
…02…II…02…COMSEC In-Plant Verification
…02…II…02…Availability Verification
g)…02…S̲O̲F̲T̲W̲A̲R̲E̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
…02…C̲l̲a̲s̲s̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲ ̲T̲y̲p̲e̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲
…02…I…02…Software Development Verification
…02…II…02…In Plant Software Verification
…02…II…02…Software Functional Verification
…02…II…02…Software Operational Verification
h) S̲Y̲S̲T̲E̲M̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
C̲L̲A̲S̲S̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲ ̲t̲y̲p̲e̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲
̲ ̲ ̲
II Comsec On-site Verification
II Site Provisional Acceptance (SPA)
4.2.1…02…H̲a̲r̲d̲w̲a̲r̲e̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
4.2.1.1…02…P̲r̲o̲t̲o̲t̲y̲p̲e̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
The Prototype Verification is an internal verification
(class I). The verification methods used are examination
and test. The purpose of the prototype verification
is
to verify that the prototype meets the basic requirements
of its product specification. This is only applicable
to new design of units, modules and subsystems or modifications
to existing design.
4.2.1.2…02…P̲r̲e̲p̲r̲o̲d̲u̲c̲t̲i̲o̲n̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
The preproduction verification (First Article Qualification
Test) is an internal verification (class I). The verification
methods used are examination and test. This verification
shall verify that the preproduction item meets all
the requirements of its product specification. This
verification is only applicable to new design of units,
modules and subsystems or modifications to existing
design.
4.2.1.3…02…P̲r̲o̲d̲u̲c̲t̲i̲o̲n̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
The production verification which is internal (class
I) is the verification to be performed on each unit,
module and subsystem following manufacture to ensure
that all items are in accordance with product specification
prior to system integration.
4.2.1.4…02…D̲S̲M̲T̲ ̲(̲P̲r̲o̲t̲o̲t̲y̲p̲e̲)̲ ̲S̲y̲s̲t̲e̲m̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
a) When the hardware of DSMT System has been integrated
and the system software completed, a DSMT (Prototype)
Acceptance Verification of the integrated system
will be performed.
b) This is an external verification (class II). Upon
SHAPE approval of the results of the tests, the
manufactured systems are baselined.
4.2.1.5…02…F̲a̲c̲t̲o̲r̲y̲ ̲A̲c̲c̲e̲p̲t̲a̲n̲c̲e̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
a) When a CAMPS has been integrated at the factory,
a test will be carried out to verify that all hardware
fulfils the functional specifications. The test
is a subset of the DSMT (Prototype) System Verification.
The subset of test procedures will be identical
to the performance test procedures used for the
DSMT (Prototype) tests, except for the exclusion
of the environmental tests.
b) This test is according to contract document para.
17.4 This is an external verification (class II).
4.2.1.6…02…C̲O̲M̲S̲E̲C̲ ̲I̲n̲-̲P̲l̲a̲n̲t̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
a) The CAMPS equipment will be tested by Purchaser
at the factory. The purchaser will liaise directly
with the national COMSEC authorities where appropriate.
b) If the screened cage solution is chosen by CR the
computer equipment will be operated within the
screened cage, and test measurements will be made
outside of the cage to ensure that the attenuation
provided by the cage is adequate to meet the security
requirements.
c) If EMI-Racks are chosen for the central equipment
the CAMPS system will be tested without the use
of a screened cage.
d) All equipment which process classified information
in a clear electrical form shall meet the NATO
COMSEC tempest requirements as laid down in AMSG
720A.
4.2.1.7 A̲v̲a̲i̲l̲a̲b̲i̲l̲i̲t̲y̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
a) The availability and reliability performance characteristics
specified in section 3.4.4 shall be demonstrated
by performing availability measurements/calculations,
in which the values of the module/unit MTBFs and
MTTRs are verified by:
1) Testing/analysis in accordance with sections
4.2.1.7.1, 4.2.1.7.2 and 4.2.1.7.3 or,
2) Provision of acceptable evidence of conformity
to requirements, i.e. evidence supported by
prior observation of system or module performance
over a length of time sufficient to have the
same degree of confidence as the test procedure
that the required availability can be archived.
3) Availability/reliability/maintainability observations
which the contractor wishes to submit in lieu
of testing shall be provided as part of the
R&M Plan for agreement by the purchaser.
b) The set of units and modules, taken together, shall
make up the CAMPS equipment of the R&M model(s)
used for verification of system availability. A
definition of the terms module and unit is given
in section 3.4.4.7 l and m.
c) The R&M Characteristics shall be considered acceptable
if it is verified that the reliability performance
requirements specified in section 3.4.4. are met.
Demonstration that the specified availability requirements
have been met shall be accomplished prior to acceptance
of the first site.
4.2.1.7.1 M̲T̲B̲F̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
4.2.1.7.1.1 M̲T̲B̲F̲s̲ ̲o̲f̲ ̲3̲5̲0̲0̲ ̲H̲o̲u̲r̲s̲ ̲o̲r̲ ̲L̲e̲s̲s̲
a) Tests shall be performed to measure the MTBF of
all units and modules having a predicted MTBF value
of 3500 hours or less. The contractor need not
test all the individual units of a module being
tested unless the unit has a MTBF value less than
3500 hours and is part of another module which
is not being tested.
b) R&M testing shall be conducted in accordance with
an approved unit or module test plan. The overall
system test programme shall be planned so that
reliability data is obtained from as many functional
tests as possible. In this manner the performance
of reliability testing, per se, will be reduced
to the minimum necessary for the demonstration
of system compliance with the R&M availability
requirements.
c) The R&M Test Programme shall make provisions for
the accumulation of sufficient testing time on
each type of module. The accumulated testing time
shall be equivalent to at least three multiples
of either:
1) the predicted MTBF value of the module with
an MTBF of 3500 hours or less, or
2) the largest predicted unit MTBF value for units
having a predicted MTBF value less than 3500
hours and forming part of a module with a MTBF
value greater than 3500 hours.
d) The R&M test programme shall provide for demonstrating
that unit and module MTBF values are equal to or
greater than the respective predicted values, to
a lower confidence limit value in accordance with
the following schedule:
1) 70% lower confidence limit value at the time
of Factory Acceptance Verification (site 1)
2) 80% lower confidence limit value at the time
of SPA (Site 1)
e) The lower confidence limit values of MTBF shall
be determined from the test data using the following
equation:
2T…0f…M…0e…
MTBF…0f…LCL…0e… …0f…=…0e… …0e… ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲…0f…
2
,f
where
T…0f…M…0e… = the total cumulative module
or
unit operating time obtained
from the test programme
2
,…0f…f…0e… = chi-square value with
probability and f degrees
freedom
= (̲1̲0̲0̲-̲L̲C̲L̲)̲
100
LCL = Specified Lower Confidence
Level, expressed as a percentage
f = 2x(N…0f…p…0e… + 1)
N…0f…p…0e… = the actual number of va-
lid module or
unit failures observed
during the period T…0f…M…0e….
4.2.1.7.1.2 M̲T̲B̲F̲s̲ ̲G̲r̲e̲a̲t̲e̲r̲ ̲T̲h̲a̲n̲ ̲3̲5̲0̲0̲ ̲H̲o̲u̲r̲s̲
a) The contractor shall justify the use of MTBF values
greater than 3500 hours for individual units and
for modules, without redundant units, by means
of analytical calculations. Such calculations shall
use values taken from actual test data on "like"
or similar equipment or from a component analysis,
utilizing values taken from recognized sources,
e.g. reliability and maintainability handbooks
maintained by National Government and by Industry.
b) The preliminary and final R&M Programme Plans shall
identify those units and modules which have predicted
MTBF values greater than 3500 hours, and which
will have the MTBF values justified rather than
tested. The list shall include the predicted value
of MTBF for such units and modules. The list may
be changed as the programme progresses provided
notification is given at least 60 days in advance
of starting or cancelling any planned tests. The
change process contemplated for this list is the
same as that provided in the contract for all items
(data) under Configuration Control except:
1) The contractor's proposals will not include
any consideration of the cost impact for contractor's
proposed change.
2) Such proposals, once mutually agreed for all
project impacts other than cost, will be contractually
implemented at no change in contract price.
3) These change proposals shall be treated as
"critical documents" within the meaning of
Contract Special Provision No. 11, Approval
and Submittal of Documentation.
The analytical calculations for all units, and
modules in this category shall be submitted to
the purchaser for approval at least 60 days prior
to the start of any acceptance testing.
4.2.1.7.2 M̲T̲T̲R̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
4.2.1.7.2.1 M̲T̲T̲R̲s̲ ̲o̲f̲ ̲4̲5̲ ̲M̲i̲n̲u̲t̲e̲s̲ ̲o̲r̲ ̲L̲e̲s̲s̲
a) Tests shall be performed to measure the MTTR of
all types of units and modules with a predicted
MTTR value of 45 minutes or less.
The contractor need not test individual units of
a module that is being tested unless the unit is
also a part of another module that is not being
tested.
b) The tests shall be carried out in accordance with
an approved R&M Unit or Module Test Plan. The overall
test programme shall be planned so that data related
to maintainability is obtained from as many functional
tests as possible. In this manner the performance
of maintainability testing, per se, will be reduced
to the minimum necessary for the demonstration
of system compliance with the R&M availability
requirements.
c) The MTTR measurements shall include corrective
and preventive maintenance actions, the time to
localise the failure to a particular module, time
to replace the failed module, verify the repair
action and restoring the system to operational
configuration exclusive of any software load requirements.
The MTTR does not include the repair time of the
failed module. The R&M test programme shall provide
for demonstrating that unit and module MTTR values
are less than or equal to the respective predicted
MTTR values to an upper confidence limit value
in accordance with the following schedule:
1) 70% upper confidence limit value at the time
of Factory Acceptance Verification (Site 1).
2) 80% upper confidence limit value at the time
of SPA (Site 1).
d) The upper confidence limit values of MTTR shall
be determined from the test data by the following
equation:
̲
MTTR…0f…UCL…0e… = X + (t …0f…,f…0e…) . ̲S̲ ̲
…0e…N…0f…
where
̲
X = the observed statistical mean of
the measured maintenance action
periods:
̲
X
X = ̲ ̲i̲
N
Where X…0f…i…0e… are the individual observations and N is the
number of test measurements.
t …0f…,f = Students t-value with probability and
f degrees of freedom,
= (̲1̲0̲0̲-̲U̲C̲L̲)̲
100
UCL = Specified Upper Confidence Limit expressed
as
a percentage.
f = (N-1),
S = Observed standard deviation of the measured
maintenance action periods:
̲
S…0e…2…0f… = (X…0f…i…0e… - X)…0e…2…0f…/(N - 1)
4.2.1.7.2.2 M̲T̲T̲R̲s̲ ̲G̲r̲e̲a̲t̲e̲r̲ ̲T̲h̲a̲n̲ ̲4̲5̲ ̲M̲i̲n̲u̲t̲e̲s̲
a) The preliminary and final R&M Programme Plans shall
identify those units and modules that have MTTR
values greater than 45 minutes, together with the
predicted MTTR values. Reasons for assigning a
MTTR value in excess of 45 minutes to a unit or
module shall be given and shall explain the maintenance
procedures involved in restoring such a unit or
module to service when a malfunction occurs.
Justifying analyses to prove that the overall availability
requirements can be met with units having MTTR
values in excess of 45 minutes shall be provided.
b) The list of units and modules having MTTR values
greater than 45 minutes may be changed as the programme
progresses provided notification is given at least
60 days in advance of starting any testing or cancelling
any planned tests. The change process contemplated
for this list is the same as that provided in the
contract for all items (data) under Configuration
Control except:
1) The contractor's proposals will not include
any consideration of the cost impact for contractor's
proposed change.
2) Such proposals, once mutually agreed for all
project impacts other than cost, will be contractually
implemented at no change in contract price.
3) These change proposals shall be treated as
"critical documents" within the meaning of
Contract Special Provision No. 11, Approval
and Submittal of Documentation.
4.2.1.7.3 R̲&̲M̲ ̲U̲n̲i̲t̲ ̲o̲r̲ ̲M̲o̲d̲u̲l̲e̲ ̲T̲e̲s̲t̲ ̲P̲l̲a̲n̲s̲
The MTBF and MTTR for each unit and module shall be
part of a R&M Unit and module test plan which in detail
will specify how the MTBF and/or MTTR is to be verified
either by test, analysis or presentation of evidence
of conformity.
4.2.2 S̲o̲f̲t̲w̲a̲r̲e̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
4.2.2.1 S̲o̲f̲t̲w̲a̲r̲e̲ ̲D̲e̲v̲e̲l̲o̲p̲m̲e̲n̲t̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
The software development verification is an internal
verification (class I) performed during the development
of the software.
All software and parts thereof shall be thoroughly
tested at each step of development and integration.
4.2.2.2 I̲n̲ ̲P̲l̲a̲n̲t̲ ̲S̲o̲f̲t̲w̲a̲r̲e̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
In plant software verification (class II) shall be
performed as outlined in the software test plan.
The software test will be performed in the factory
with a simulated environment specified in section 3.5.11.5.2.
The test criteria requires an accumulated test time
of 240 hours with the following allowable errors:
a) C̲a̲t̲e̲g̲o̲r̲y̲ ̲1̲
a total of 1 error allowed of the following types:
- Loss of a message or other transaction.
- Corruption of a message or other transaction.
- A message or transaction being missent.
- Misapplication of security rules.
- Failure of accounting procedures.
b)\C̲a̲t̲e̲g̲o̲r̲y̲ ̲2̲
A total of 2 errors allowed of the following types:
1) loss of service to more than 25% of all
channels and user connecting points.
2) loss of service to more than 50% of all
operating positions or system reporting
facilities.
3) Loss of ability to recover the system.
c) C̲a̲t̲e̲g̲o̲r̲y̲ ̲3̲
a total of 3 errors allowed of the following types:
1) errors not included in 1 and 2 above.
d) The software Test Plan shall specify the tests
which will provide the reliability production to
calculate the probability that the MDSD of:
1) category 1 SW errors is 240 hours
2) category 2 SW errors is 120 hours
3) category 3 SW errors is 80 hours
4.2.2.3…02…S̲o̲f̲t̲w̲a̲r̲e̲ ̲F̲u̲n̲c̲t̲i̲o̲n̲a̲l̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
a) At the first site (SHAPE) CR shall conduct a formal
functional test (class II) to verify basic system
functions and interfaces. The test will be supervised
by CR test engineers and SHAPE operational staff
will assist in operating the system.
b) NICS-TARE, ACE CCIS, and SCARS II are planned to
be available, however, in the event one or more
of the interfaces are not available, the contractor
shall proceed with all of those components of the
system that are independent of the missing interfaces.
c) In this event SHAPE will give Site Provisional
Acceptance, not withstanding the inability to test
all interfaces, providing all other aspects of
the test are satisfactorily completed.
4.2.2.4…02…S̲o̲f̲t̲w̲a̲r̲e̲ ̲O̲p̲e̲r̲a̲t̲i̲o̲n̲a̲l̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
a) Immediately after successful completion of the
Software Functional Verification at the SHAPE site,
an Operational Test (class II) shall take place
in order to verify the reliability of the software
system.
b) The operational test will take place under CR control
but SHAPE personnel will operate the system in
a live operational mode in accordance with agreed
test plans and procedures.
c) The acceptance test criteria shall be that the
CAMPS system shall be able to execute the basic
software functions according to the approved System
Requirements Specification for a period of 60 calendar
days in a live operational mode without any software
failure.
d) Any software failure, which concept will be defined
in the acceptance test plan, shall require a complete
retest cycle. The duration of the test or retest
may be shortened at SHAPE's discretion.…06…1
…02… …02… …02… …02…
4.2.3…02…S̲y̲s̲t̲e̲m̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
4.2.3.1…02…C̲O̲M̲S̲E̲C̲ ̲O̲N̲-̲S̲i̲t̲e̲ ̲V̲e̲r̲i̲f̲i̲c̲a̲t̲i̲o̲n̲
a) When the CAMPS computer equipment and peripherals
have been installed on-site, a COMSEC verification
will be performed by the national COMSEC authorities.
b) Purchaser will supervise the verification and Christian
Rovsing A/S test engineers will assist in operating
the system. The COMSEC on-site verification is
a formal verification (class II). The verification
shall take place in conjunction with the SPA.
4.2.3.2…02…S̲i̲t̲e̲ ̲P̲r̲o̲v̲i̲s̲i̲o̲n̲a̲l̲ ̲A̲c̲c̲e̲p̲t̲a̲n̲c̲e̲ ̲(̲S̲P̲A̲)̲
a) Site provisional acceptance is the act whereby
SHAPE will acknowledge by protocol that CR has
fully demonstrated that a site is complete and
ready for initial operation and will take place
when the following requirements have been met:
1) Completion of an agreed on-site acceptance
test.
2) Verification of the site inventory.
3) If applicable, availability of a mutually agreed
discrepancy list showing the agreed date for
clearance of each listed discrepancy.
