|
DataMuseum.dkPresents historical artifacts from the history of: Rational R1000/400 Tapes |
This is an automatic "excavation" of a thematic subset of
See our Wiki for more about Rational R1000/400 Tapes Excavated with: AutoArchaeologist - Free & Open Source Software. |
top - downloadIndex: ┃ T ┃
Length: 19066 (0x4a7a)
Types: TextFile
Names: »TESTMATE_RELEASE2_2_0_LPT«
└─⟦db1c56801⟧ Bits:30000748 8mm tape, Rational 1000, TESTMATE 2_2_0
└─ ⟦866d14df1⟧ »DATA«
└─⟦09991f156⟧
└─⟦this⟧
Rational TestMate
Release Information
Rev2_2_0 Release
\f
Copyright 1992 by Rational
Part Number: 508-003286-004
October 1992 (Software Release Rev2_2_0)
Rational and R1000 are registered trademarks and Compilation
Integrator, Rational Environment, Rational Subsystems, and
Universal Compiler Integration are trademarks of Rational.
Rational
3320 Scott Boulevard
Santa Clara, California 95054-3197
\f
Rev2_2_0 Release
1. Overview
This note documents the Rev2_2_0 release of Rational TestMate.
TestMate is a comprehensive set of tools that automates the
process of developing, performing, and evaluating software tests.
TestMate includes a test management system and a non-intrusive
coverage analyzer, both tightly integrated with the Rational
Environment.
This release has been verified to work under Environment releases
D_12_5_0 and D_12_6_5.
1.1. Components of This Release
This release includes the following:
* !TOOLS.TEST.TESTMATE.REV2_2_SPEC
* !TOOLS.TEST.TESTMATE.CODE2_2_0
* !TOOLS.TEST.TESTMATE_TOOLS.REV2_2_SPEC
* !TOOLS.TEST.TESTMATE_TOOLS.CODE2_2_0
* !MACHINE.INITIALIZATION.RATIONAL.TESTMATE_START
* !MODEL.TESTMATE
2. New Features
2.1. Procedure Setup_Rci_Tms
TestMate now supplies a procedure, Setup_Rci_Tms, that is used to
create the RCI TMS package. This package is used by RCI tests to
signal target test results back to the R1000. Any RCI tests that
call package TMS must have this package coded in the RCI world.
Setup_Rci_Tms works by parsing the text file "TMS", which is
included with each code view, into an RCI world. It accepts a
single parameter that identifies the world in which the TMS
package is to be created:
procedure Setup_Rci_Tms (In_Library : String := "$");
2.2. Formatted Closure Report
A sample routine to examine the .Closure subobject of a specified
test run file and generate a formatted report can be found in the
"Tools" directory of the TestMate code view. This routine, named
"Print_Closure", is supplied in source form.
RATIONAL October 1992 1\f
Rational TestMate Release Information
The Print_Closure routine can be used as an example of how a
user-written routine would access the information in the .Closure
file, and as an example of how the programmatic interfaces to
TestMate data can be used.
3. Limitations
3.1. General
3.1.1. Copying, Moving, or Renaming Test Data
The Test_Run files have full path names to all associated files -
such as logs, scripts, etc. - to enable resolution at a later
time independent of the current Test_Context settings. If any of
these associated files are moved, the Test_Run references must be
updated. These references can be easily updated by executing the
Update_Test_Run_References procedure (found in the Tools
subdirectory of the TestMate subsystem). Pass to
Update_Test_Run_References the name of the test run file and the
name of a new test context file that indicates where the relative
files are now located.
3.1.2. Temporary Files
When the user executes Edit_Run_Group and does not specify a
group name, a temporary file is created in !Machine.Temporary.
These files are not currently deleted by TestMate, although they
will be removed when the R1000 is next rebooted. Due to the fact
that these temporary files are quite small, this is not expected
to pose any real problem.
3.2. Editors
3.2.1. Test Case used Multiple Times in Test Set
A Test_Set cannot include multiple Test_Cases with the same
simple name. The current product does not detect this condition.
3.2.2. Applying Multiple Filters with a Single Command
The Testmate.Filter command must be used exactly as specified in
the User's Guide; the command will only use the first item if
multiple filters are specified. Thus, for "status => pass,fail",
only "pass" will be applied.
2 October 1992 RATIONAL\f
Rev2_2_0 Release
3.2.3. Editing Test Sets Containing Deleted Test Cases
The test-set editor does not remove from a test set those test
cases whose test-case files have been deleted (using
Library.Delete, for example). It is up to the user to manually
delete these cases from the set.
3.3. Scripts
3.3.1. Consistency Checking At Script Generation
TestMate does not check test sets and their constituent test
cases for consistency at script generation. Consistency is
checked at script execution, however. If the script's test cases
are "compatible" with the test cases that were used to generate
the script, the test script will execute correctly. Refer to the
"Ensuring Test Set Consistency" section in Chapter 2 of the
TestMate User's Manual for more information.
3.4. Coverage Analyzer
3.4.1. Coverage of Library Elaboration Code
The coverage analyzer does not gather coverage data for library
package elaboration unless the subject program uses pragma Main.
3.4.2. Code View and Coverage Analyzer
The Coverage Analyzer uses debug information for each unit in
order to operate in a non-intrusive manner. Code Views do not
contain debugging information, so the Coverage_Analyzer is unable
to gather coverage data for units inside of a code view.
4. Known Problems
4.1. General
4.1.1. Version Information and Code Views
Version information is not saved for units that are located
inside code views.
RATIONAL October 1992 3\f
Rational TestMate Release Information
4.2. Coverage Analyzer
4.2.1. Decision Coverage
Under certain, not yet well-defined circumstances, specifying the
collection of decision coverage data can cause the test to hang
or give incorrect results. When the particular criteria have
been fully characterized, a note will be sent to all customers
indicating under which circumstances decision coverage can safely
be used. Until then, usage of the decision coverage mode is not
recommended.
4.2.2. Null Case Statement Arms
Null case arms do not have code generated for them; therefore,
the Coverage Analyzer cannot detect segment coverage in those
arms. Null arms are not counted as missed segments. This can
result in 100% decision coverage being indicated when, in fact,
one or more null case arms were not exercised.
This is expected to be fixed in a later release.
4.2.3. Very Large Units
A sufficiently large unit (in excess of 32,000 executable R1000
instructions) will cause problems for the Coverage Analyzer due
to an inability to properly handle multiple code segment debug
table information. The Coverage Analyzer will issue a log
message and not process that unit. This should be a very rare
event as Ada units requiring multiple code segments must be very
large.
4.2.4. Multiple Subject Programs
If a test case specifies multiple subject programs, and the
Coverage_Analyzer_Params field in that test case is set to ""
(the default), the coverage analyzer will not collect coverage
data for all of the specified subject programs.
This problem can be worked around by explicitly specifying the
subject program names in the Coverage_Analyzer_Params field of
the test case. Note that the subject programs can be specified
using relative pathnames.
4 October 1992 RATIONAL\f
Rev2_2_0 Release
4.2.5. Specific Routine Within a Unit
If a test case specifies as its subject program a routine in a
particular unit ("sppkg.foo_proc", for instance), and the
Coverage_Analyzer_Params field in that test case is set to ""
(the default), the coverage analyzer will not correctly collect
coverage data for the specified routine.
This problem can be worked around by specifying the package name
in the Coverage_Analyzer_Params field of the test case.
4.3. Editors
4.3.1. Coverage Editor and Missed Segments
When the user does a [Definition] to view the missed segments for
an Ada unit, the Editor brings up that Ada unit and marks the
segments. If there are no missed segments, the Editor provides a
warning message to that effect and brings up the Ada editor.
Under certain complex circumstances involving the viewing of
coverage data for multiple test runs, the Ada editor fails to
come up properly. This only occurs for 100% covered units, and
the warning message is still properly posted.
4.3.2. Test Run Group Editor
The following sequence of events can result in an incorrect
editor image:
* Invoke the test run editor for a group of test runs
* Expand the image
* Place the cursor after the text on one of the expanded rows
(such as the test run log row) and press [Definition]. As
expected, an error message will be generated indicating that
[Definition] is not supported for this column.
* Expand the image once again.
The two title rows as well as the test run rows will all be
erased, leaving only the expanded rows.
To regain the proper image, abandon the current image and
re-execute the command used to invoke the test run editor.
RATIONAL October 1992 5\f
Rational TestMate Release Information
5. Errors
This section documents areas in TestMate where potentially
unclear or confusing error messages may arise.
5.1. General
5.1.1. Format of Persistent Objects
If the format for the textually-manipulated persistent objects
(Test_Context, Script_Construction_Control, and
Script_Execution_Control) is not followed explicitly, data layout
errors can occur. In particular, something like:
xx =>
....long name.....
will fail. The correct form is:
xx => ....long name...
An alternative form is:
xx => (
.......long name ....)
Data layout errors do not supply a detailed explanation of the
failure mode. The user will have to look at the file for
details. Note that the most common mistake is a misspelling of
one of the field identifiers.
5.2. Scripts
5.2.1. Connected and Separate Job Run Modes
If the user is running a test case with run mode set to
Separate_Job or Connected_Job, a job command is created which
must be parsed by the Environment and then executed. That
command can fail to execute for a number of reasons; TestMate
will post the entire command to the log. It is up to the user to
figure out why the command failed. This is usually done by
running the user's part of the job command string from the error
log. If that succeeds, check for missing links to the TestMate
units mentioned in the job command: the links may not be
available from the specified execution context.
Another common failure is due to the number of parameters being
incorrect either in type or number.
6 October 1992 RATIONAL\f
Rev2_2_0 Release
5.2.2. Test Drivers and Package TMS
If a test driver directly references package TMS using default
values (i.e., makes reference to the current test case or test
run), then running that driver outside of the script will cause
an exception instead of a more graceful error message.
6. Documentation
This product has been documented in the TestMate User's Guide,
product #4000-00720, dated November 1992.
6.1. Links to TestMate Packages From Worlds
The TestMate User's Guide shows how to add links to the TestMate
packages and how to update your activity to specify the correct
views. The discussion for Worlds, however, lists these
operations in the wrong order (see the bottom of page 4). Your
activity should be updated before you establish links to a
TestMate world. Otherwise, the Links.Add command will use what
it finds in your default activity (and it will fail if it doesn't
find entries for TestMate and TestMate tools).
6.2. Specifying Testmate_Tools
Pages 4 and 5 of the TestMate User's Guide specify that the
following entries must appear in your activity:
Subsystem Spec View Load View
Context
============== =============== =======================
============
TESTMATE Rev2_n_SPEC CODE2_0_m
!TOOLS.TEST
TESTMATE_TOOLS Rev0_0_n_SPEC CODE0_0_m
!TOOLS.TEST
For this release, this should read:
Subsystem Spec View Load View
Context
============== =============== =======================
============
TESTMATE Rev2_2_SPEC CODE2_2_0
!TOOLS.TEST
TESTMATE_TOOLS Rev2_2_SPEC CODE2_2_0
!TOOLS.TEST
RATIONAL October 1992 7\f
Rational TestMate Release Information
The "Subsystems" section on page 5 also discusses the importing
of the Testmate and Testmate_Tools specs; the correct specs to
import are:
!TOOLS.TEST.TESTMATE.REV2_2_SPEC
!TOOLS.TEST.TESTMATE_TOOLS.REV2_2_SPEC
7. Training
TestMate training is available; the current draft is dated
November, 1992. TestMate training is identified by product
#7000-00720.
8 October 1992 RATIONAL\f
Rev2_2_0 Release
Contents
1. Overview 1
1.1. Components of This Release 1
2. New Features 1
2.1. Procedure Setup_Rci_Tms 1
2.2. Formatted Closure Report 1
3. Limitations 2
3.1. General 2
3.1.1. Copying, Moving, or Renaming Test Data 2
3.1.2. Temporary Files 2
3.2. Editors 2
3.2.1. Test Case used Multiple Times in Test Set 2
3.2.2. Applying Multiple Filters with a Single Command 2
3.2.3. Editing Test Sets Containing Deleted Test Cases 3
3.3. Scripts 3
3.3.1. Consistency Checking At Script Generation 3
3.4. Coverage Analyzer 3
3.4.1. Coverage of Library Elaboration Code 3
3.4.2. Code View and Coverage Analyzer 3
4. Known Problems 3
4.1. General 3
4.1.1. Version Information and Code Views 3
4.2. Coverage Analyzer 4
4.2.1. Decision Coverage 4
4.2.2. Null Case Statement Arms 4
4.2.3. Very Large Units 4
4.2.4. Multiple Subject Programs 4
4.2.5. Specific Routine Within a Unit 5
4.3. Editors 5
4.3.1. Coverage Editor and Missed Segments 5
4.3.2. Test Run Group Editor 5
5. Errors 6
5.1. General 6
5.1.1. Format of Persistent Objects 6
5.2. Scripts 6
5.2.1. Connected and Separate Job Run Modes 6
5.2.2. Test Drivers and Package TMS 7
6. Documentation 7
6.1. Links to TestMate Packages From Worlds 7
6.2. Specifying Testmate_Tools 7
7. Training 8
RATIONAL October 1992 iii\f
Rational TestMate Release Information
iv October 1992 RATIONAL\f