top - download
⟦53768c122⟧ Wang Wps File
Length: 16265 (0x3f89)
Types: Wang Wps File
Notes: MOM NR. 38
Names: »4213A «
Derivation
└─⟦e9ca1be93⟧ Bits:30006183 8" Wang WCS floppy, CR 0382A
└─ ⟦this⟧ »4213A «
WangText
…02…
…02…KNN/831028…02……02…#
MINUTES
OF MEETING
NO. 38
…02……02…CAMPS
CR/SHAPE
831028 38 CR
A/S
DRAFT CAMPS in Plant Software
Test Conduct
Maj. York K. Nybroe-Nielsen
Mr. A. Gardiner
1. General
2. Schedule
3. Test Structure and Test, Team Organization
4. Physical Environment
5. Test Procedures
6. Test Conduct
7. Documentation of Test Results
8. Error Definition
9. Error Documentation, Categorization,
Resolution
10. Termination of Test
1. G̲E̲N̲E̲R̲A̲L̲
1.1 This documentation establish the conduct
of the CAMPS 240 hour test to be conducted
during November 1983- December 1984.
This document will be an informal working
paper for a basis of subsequent discussion
between CR and SHAPE:
1.2 The general conduct of the 240 hour test
to include administrative support will
be discussed. Test cases and test condition
fare been previously defined and are
conditionally changed/modified below.
2. S̲C̲H̲E̲D̲U̲L̲E̲
2.1 The testing schedule will be as follows:
D̲a̲y̲ E̲v̲e̲n̲t̲ R̲e̲s̲p̲o̲n̲s̲i̲b̲i̲l̲i̲t̲y̲
T-7 Notification
of
start
of
test CR
T-4 General
of
SHAPE
Test
Deviation
and
support SHAPE
T-1 General
of
SHAPE
Test
Team SHAPE
T Familization/Review
of
CR/SHAPE
Conduct
of
Test
2.2 T̲E̲S̲T̲ ̲T̲E̲A̲M̲ ̲S̲C̲H̲E̲D̲U̲L̲E̲
Daily Activity
The daily activity is as follows:
Test Test Test Admin.
H̲o̲u̲r̲s̲ W̲i̲t̲n̲e̲s̲s̲e̲s̲
̲ S̲u̲p̲e̲r̲v̲i̲s̲o̲r̲
̲
̲
̲
̲
̲
̲
̲
̲
̲
̲
̲
̲
̲
8.30- 9-30 1 Test Meeting
CR Test
9.30-12.00 2.5 Test Test Test
12.00-13.00 ------------lunch------------------
13.00-16.30 3.5 Test Test Collect/
Collate
16.30-17.30 ---------SHAPE
meeting-------------
This results in 7 test hours
per day, which will mean that
120 hours of functional test
will last about 17 days if
there are no breaks. Within
the above, a break of up to
10 minutes per hour per terminal
should be foreseen, further
reducing the test time accumulated.
The breaks will be jointly
decided by the SHAPE and CR
test directors and be recorded
by the test administration.
3 T̲E̲S̲T̲ ̲S̲T̲R̲U̲C̲T̲U̲R̲E̲ ̲A̲N̲D̲ ̲T̲E̲S̲T̲ ̲T̲E̲R̲M̲
̲O̲R̲G̲A̲N̲I̲Z̲A̲T̲I̲O̲N̲
The test is organised into
two phases.
Firstly, there is a functional
test phase where the various
features of CAMPS are demonstrated
on a number of terminals.
This involves SHAPE test witnesses
at each terminal.
Secondly, there is a stress
or load test where the CAMPS
is subjected to a heavy traffic
load from a traffic load generator
(the Test Drive System)
3.1 F̲U̲N̲C̲T̲I̲O̲N̲A̲L̲ ̲T̲E̲S̲T̲ (120 hours test time).
The test procedures of "In-plant
Software Test Specifications
and Procedures", CPS/TRP/007
are to be executed and witnessed
at the following number of
terminals (vosual display
units):
T̲e̲r̲m̲i̲n̲a̲l̲ N̲u̲m̲b̲e̲r̲ ̲o̲f̲
T̲e̲s̲t̲ S̲H̲A̲P̲E̲
F̲u̲n̲c̲t̲i̲o̲n̲ T̲e̲r̲m̲i̲n̲a̲l̲s̲
C̲a̲s̲e̲s̲ W̲i̲t̲n̲e̲s̲s̲e̲s̲
1) User - 3
30 SHNMO/CISD
2) Release - 1
5 SHNMO/CISD
3) Supervisory - 2
43 SSG
4) MSO - 1
54 SSG
5) MDCO - 1
43 SECCOS'
6) Operator - 1
19 SSG/SHNMO
The test case activity at each terminal
will be repeated many times over in
order to achieve continous representative
CAMPS activity for 120 hours of test
executive time.
During this time, the external channels
of CAMPS will be driven by the Test
Drive System to provide incoming traffic.
Deliberate and known faults in message
preparation and incoming message formats
will ensure that the MSO and MDCO
functions are invoked.
The test cases for the external channels
are as follows:
C̲h̲a̲n̲n̲e̲l̲ N̲u̲m̲b̲e̲r̲ ̲o̲f̲ T̲e̲s̲t̲
S̲H̲A̲P̲E̲ ̲S̲t̲a̲f̲f̲
C̲i̲r̲c̲u̲i̲t̲s̲ C̲a̲s̲e̲s̲
I̲n̲v̲o̲l̲v̲e̲d̲
CCIS 36
CISD/SSG
SCARS II 39
CISD/SSG
NICS TARE 40
CISD/SSG
TRC/PT-TO-PT 40
CISD/SSG
3.2 S̲T̲R̲E̲S̲S̲ ̲T̲E̲S̲T̲S̲
During this test, a defined set of
messages is automatically and continously
sent by the TDS to the CAMPS. For
the first 80 hours the transmission
shall be error free, and for the 40
subsequent hours, transmission errors
will be injected into the system.
The system connectivity will be larger
than for the first part of the test
with a full complement of the user,
supervisory and message assistance
positions. The witnesses will be allocated
to the various functions as for the
functional test.
3.3 S̲H̲A̲P̲E̲ ̲T̲E̲S̲T̲ ̲T̲E̲A̲M̲ ̲O̲R̲G̲A̲N̲I̲Z̲A̲T̲I̲O̲N̲
The team is organized as shown in
Fig. 1. It consists of three parts:
the test supervisor, the test witness
and the test administration and support.
3.3.1 T̲e̲s̲t̲ ̲S̲u̲p̲e̲r̲v̲i̲s̲o̲r̲
The team has a leader and a person
enpowered to speak on site for SHAPE.
This individual will discuss test
results with the CR Test Director
and present and defend the SHAPE position
as required. He may be required to
call for quidance from SHSOC, the
SHNMO Legal Advisor or the CAMPS Project
Manager. In addition, this individual
must chair all SHAPE team meetings
to gather test results and to discuss
problem reports. He is the single
point of contact for both CR and SHAPE
personnel. This individual should
probably be the Program Manager since
all above responsibilities clearly
fall within his scope of responsibility
and due to the importance of this
particular test. However, the CAMPS
P.M. with the agreement of Head of
SHNMO may choose to delegate this
duty to another on-site SHNMO person
as required.
If at any point during the testing
the SHAPE team leader feels that testing
or quality assurances procedures are
being violated, he will first request
the CR Test Director to cease testing
and correct the deficiency. If the
matter is not resolved he will state
an official complaint. If the matter
is still not resolved he will immediately
inform the SHNMO Management personnel
(Chief SHSOI or Chief SHNMO) and request
approval to withdraw the test evaluators.
3.3.2 T̲e̲s̲t̲ ̲A̲d̲m̲i̲n̲i̲s̲t̲r̲a̲t̲i̲o̲n̲ ̲S̲u̲p̲p̲o̲r̲t̲
This individual will collect all test
reports and problems report and create
a daily summary file of problems for
presentation and discussion at the
regular end of day meeting. The support
will also record:
- SW
and
HW
configurations
and
configuration
changes.
- Test
case
and
clock
values
for
errors
occurrences.
- The
240
hour
error
category
count.
3.3.3 Test Witnesses
Figure 1
3.4 C̲R̲ ̲T̲E̲S̲T̲ ̲T̲E̲A̲M̲ ̲O̲R̲G̲A̲N̲I̲Z̲A̲T̲I̲O̲N̲
3.4.1 CR Test Team Organization is shown
at Figure 2.
Figure 2.
4 P̲H̲Y̲S̲I̲C̲A̲L̲ ̲E̲N̲V̲I̲R̲O̲N̲M̲E̲N̲T̲
4.1 T̲E̲S̲T̲ ̲R̲O̲O̲M̲
The VDU's and printers shall be grouped
to minimize personnel movement.
They shall be clearly marked as to function
and relationship to other VDU/printers.
The terminal/user profile shall also be
clearly shown.
4.2 T̲E̲S̲T̲ ̲S̲U̲P̲E̲R̲V̲I̲S̲O̲R̲ ̲R̲O̲O̲M̲
Preferably this shall be in close proximity
to the test room. A separate meeting room
shall also be nominated.
4.3 S̲U̲P̲P̲O̲R̲T̲
Adequate rest-room and refreshment facilities
shall be provided
4.4 R̲E̲F̲E̲R̲E̲N̲C̲E̲ ̲D̲O̲C̲
A complete set of reference documentation
shall be available in the test room
5 T̲E̲S̲T̲ ̲P̲R̲O̲C̲E̲D̲U̲R̲E̲
Hand marked copies of revised test procedures
CPS/TPR/007 are to be carried to SHAPE.
Via telex SHAPE will inform CR if they
can proceed to update the procedures.
As an objective SHAPE shall complete their
review within one week.
Any further updates to the procedures
will be telexed to SHAPE immediately.
The updated version of the test procedures
should be available in SHAPE prior to
start of test.
6 T̲E̲S̲T̲ ̲C̲O̲N̲D̲U̲C̲T̲
Thos covers behaviour of test clock and
text activity when various events occur.
6.1 T̲E̲S̲T̲ ̲C̲L̲O̲C̲K̲S̲
There shall be three independent test
clocks, one for each of the error categories.
6.1.2 C̲l̲o̲c̲k̲ ̲B̲e̲h̲a̲v̲i̲o̲u̲r̲ ̲f̲o̲r̲ ̲a̲n̲ ̲E̲r̲r̲o̲r̲ ̲O̲c̲c̲u̲r̲r̲a̲n̲c̲e̲
The test clock for each category shall
be reset to the accummulated test time
of the maximum allowable error for that
category whenever the allowable number
of errors in that category has been exceeded.
6.1.3 C̲l̲o̲c̲k̲ ̲B̲e̲h̲a̲v̲i̲o̲u̲r̲ ̲D̲u̲r̲i̲n̲g̲ ̲D̲e̲c̲i̲s̲i̲o̲n̲ ̲o̲n̲
̲C̲a̲t̲e̲g̲o̲r̲i̲z̲a̲t̲i̲o̲n̲
If the categorization of the error cannot
be determined immediately then only one
more full day of testing will be allowed
before all clocks are stopped.
If the categorization results in ca. I,
II, or III then the X time of the actual
error shall be used in deciding the test
clock reset for that category.
Until the decision is made all test clocks
shall remain stopped.
6.2 E̲R̲R̲O̲R̲ ̲C̲O̲U̲N̲T̲I̲N̲G̲
If it can be proved that the error is
a repetition of an error previously occurring
in the same test case then this error
shall not count towards the total for
this category.
6.3 E̲R̲R̲O̲R̲ ̲C̲O̲R̲R̲E̲C̲T̲I̲O̲N̲
6.3.1 P̲r̲i̲o̲r̲ ̲t̲o̲ ̲M̲a̲x̲ ̲A̲l̲l̲o̲w̲a̲b̲l̲e̲ ̲E̲r̲r̲o̲r̲s̲ ̲i̲n̲ ̲E̲a̲c̲h̲
̲C̲a̲t̲e̲g̲o̲r̲y̲
CR is free to choose to correct the error.
In general CR will attempt not to hold
up the 240 hour test by correcting the
error using another system.
With the agreement of the SHAPE Test Director
the 240 hour test may be stopped for the
purpose of error correction using the
test machine.
6.3.2 A̲f̲t̲e̲r̲ ̲M̲a̲x̲ ̲A̲l̲l̲o̲w̲a̲b̲l̲e̲ ̲E̲r̲r̲o̲r̲s̲ ̲h̲a̲v̲e̲ ̲b̲e̲e̲n̲
̲A̲c̲c̲u̲m̲m̲u̲l̲a̲t̲e̲d̲
CR shall correct all errors occurring
in all categories.
6.4 T̲E̲S̲T̲ ̲P̲R̲O̲C̲E̲D̲U̲R̲E̲S̲
Since the test procedures will be cycled
through at least 4 times during the test
and since all three test clocks are independent,
there is no need to restart the test at
a particular test procedure.
However, by agreement between the test
directors, the test sequence may be restarted
at any point.
7. D̲O̲C̲U̲M̲E̲N̲T̲A̲T̲I̲O̲N̲ ̲O̲F̲ ̲T̲E̲S̲T̲ ̲R̲E̲S̲U̲L̲T̲S̲
7.1 Functional Test
7.1.1 External Channels:
All messages passing over external channels
will be printed out. Each message will
have an origin identifier in the text
pattern of the message.
7.1.2 U̲s̲e̲r̲ ̲V̲D̲U̲ ̲T̲r̲a̲n̲s̲a̲c̲t̲i̲o̲n̲s̲
All user VDU transactions will be logged,
a log trace performed and the I/C and
OG Message Status display printed at appropriate
intervals. All comments will be printed
out and will have a reference to the transaction
commeented upon.
7.1.3 M̲S̲O̲/̲M̲D̲C̲O̲/̲S̲u̲p̲e̲r̲v̲i̲s̲o̲r̲ ̲V̲D̲U̲ ̲T̲r̲a̲n̲s̲a̲c̲t̲i̲o̲n̲
All of these transactions will be logged.
After each cycle of text activity a log
trace will also be done for each of these
VDU's.
7.2 L̲O̲A̲D̲ ̲T̲E̲S̲T̲
7.3 T̲E̲S̲T̲ ̲E̲N̲G̲I̲N̲E̲E̲R̲S̲ ̲A̲C̲T̲I̲V̲I̲T̲Y̲ ̲R̲E̲C̲O̲R̲D̲
The test engineers will record the progress
through the test cases on a summary sheet.
The information will be recorded on a
form similar to Enclosure 1A (Test Report)
8 E̲R̲R̲O̲R̲ ̲D̲E̲F̲I̲N̲I̲T̲I̲O̲N̲S̲
8.1 S̲O̲F̲T̲W̲A̲R̲E̲ ̲E̲R̲R̲O̲R̲S̲
8.1.1 C̲o̲s̲m̲e̲t̲i̲c̲ ̲E̲r̲r̲o̲r̲s̲
Category 1, 2, and 3 errors are defined
in the acceptance test plan. In addition,
a new category of errors is defined -
cosmetic errors.
Included in cosmetic errors are:
1) P̲R̲I̲N̲T̲ ̲F̲O̲R̲M̲A̲T̲ ̲E̲R̲R̲O̲R̲S̲
o The contents of the report or
print-out is correct but the format
is wrong.
o Misspellings.
o Page Numbering.
2) V̲D̲U̲ ̲F̲O̲R̲M̲A̲T̲ ̲E̲R̲R̲O̲R̲S̲
o Fields are misplaced in relation
ot ICD's
o Misspellings
3) E̲R̲R̲O̲R̲ ̲C̲O̲D̲E̲S̲ ̲A̲N̲D̲ ̲A̲S̲S̲O̲C̲I̲A̲T̲E̲D̲ ̲T̲E̲X̲T̲
o Updated version of error codes/text
in ICD's
o Appropriateness of error text.
E̲X̲C̲L̲U̲D̲E̲D̲
o Erroneous error codes
4) E̲R̲R̲O̲R̲S̲ ̲I̲N̲V̲O̲K̲I̲N̲G̲ ̲C̲O̲R̲R̲E̲C̲T̲ ̲R̲E̲C̲O̲V̲E̲R̲Y̲ ̲M̲E̲C̲H̲A̲N̲I̲S̲M̲
o LTU/LTUX line blocked due to COPSY
detected errors and supervisor/operator
functions set back into service
o Switchover and recovery.
E̲X̲C̲L̲U̲D̲E̲D̲
o Frequent and regular occurrences
of such errors interfering with
the execution of the planned test.
5) ERRORS TRACEABLE TO AMBICUITY IN BASELINE
S̲P̲E̲C̲I̲F̲I̲C̲A̲T̲I̲O̲N̲S̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲
̲ ̲ ̲ ̲ ̲ ̲ ̲ ̲
o Message deletion
6) E̲R̲R̲O̲R̲S̲ ̲D̲U̲E̲ ̲T̲O̲ ̲A̲C̲C̲I̲D̲E̲N̲T̲A̲L̲ ̲M̲O̲S̲O̲P̲E̲R̲A̲T̲I̲O̲N̲
̲O̲F̲ ̲T̲H̲E̲ ̲S̲Y̲S̲T̲E̲M̲
o
7) E̲R̲R̲O̲R̲S̲ ̲D̲U̲E̲ ̲T̲O̲ ̲M̲I̲S̲O̲P̲E̲R̲A̲T̲I̲O̲N̲ ̲O̲F̲ ̲E̲N̲V̲I̲R̲O̲N̲M̲E̲N̲T̲
̲S̲I̲M̲U̲L̲A̲T̲O̲R̲S̲
o OCR simulation
o Test drive system failures
8.2 H̲A̲R̲D̲W̲A̲R̲E̲ ̲E̲R̲R̲O̲R̲S̲
The burden of proof that an error is caused
by HW operation shall be on CR.
Should such errors be due to hardware
functions not tested at FAT, but designated
to be deferred until 240 hour test this
shall contitute an error in cat. I, II,
III as appropriate.
(Above Para to be checked).
9 E̲R̲R̲O̲R̲ ̲D̲O̲C̲U̲M̲E̲N̲T̲A̲T̲I̲O̲N̲,̲ ̲C̲A̲T̲E̲G̲O̲R̲I̲S̲A̲T̲I̲O̲N̲ ̲A̲N̲D̲
̲R̲E̲S̲O̲L̲U̲T̲I̲O̲N̲
9.1 E̲R̲R̲O̲R̲ ̲D̲O̲C̲U̲M̲E̲N̲T̲A̲T̲I̲O̲N̲/̲C̲A̲T̲E̲G̲O̲R̲I̲S̲A̲T̲I̲O̲N̲
When an error occurs a Problem Report
Form similar to Enclosure 1B is completed.
The number of the problem report is indicated
in the text summary sheet along with test
clock time.
The test supervisor should be notified
immediately if the error is judged to
be Cat.I, II or III, so that a fast resolution
of error category may be made.
When the error seems to fall into a cosmetic
error category then the test engineer
and the SHAPE test witness can record
the error for final resolution of category
at the end of day meeting.
9.2 S̲H̲A̲P̲E̲ ̲T̲E̲A̲M̲ ̲E̲N̲D̲ ̲O̲F̲ ̲D̲A̲Y̲ ̲M̲E̲E̲T̲I̲N̲G̲
At the end of each day, or at a time specified
by the SHAPE test leader, the evaluation
team will gather to review the daily test
results and problem reports with the objective
of providing a SHAPE position on the systems
performance. Examples of the outcome of
the discussions may be:
1) The team can approve all or part of
the tests accomplished.
2) The team can accept a test result
that varies from CPS/TPR/007 because
it may be advantageous to SHAPE to
do so.
3) The team can officially decide that
a test result does not conform with
the CAMPS requirements and provide
a formal statement of that decision
to the CR Test Director for resolution
and determination of error category.
If disagreements occur and cannot
be resolved, the Shape team must confer
with SHNMO Management personnel prior
to deciding on a course of action.
The final course of action chosen
could be as extreme as a resusal of
SHAPE to continue with the test or
as minor as a decision to table the
issue and continue testing. Naturally,
a full range of options must be considered
here in light of particulars associated
with the problem.
4) The team may decide that where a test
result does not conform to requirements,
that the requirement may be modified
where it is clearly in SHAPE's interest
to do so.
9.3 T̲E̲S̲T̲ ̲L̲E̲A̲D̲E̲R̲S̲ ̲M̲E̲E̲T̲I̲N̲G̲
After the end of day meeting prior to
the start of the testing each day or ad
HOC for CAT Errors, SHAPE and CR will
meet to state the SHAPE position and reconcile
differencies where possible. If agreement
is not possible then discussion with and
between higher level CR management and
SHAPE management will ensue.
10 T̲E̲S̲T̲ ̲T̲E̲R̲M̲I̲N̲A̲T̲I̲O̲N̲
The test shall end successfully when the
test clocks for all three categories have
each reached 240 hours.
The test shall be interrupted for at least
7 days for the following reasons:
o Hardware instability resulting in
unavailability of the system for more
than one day in a week.
o CR wishes to proceed with error corrections
for more than 3 days.
o Mutually agreed suspension of tests.
o The current test attempt has lasted
longer than 90 days.
