forked from boostorg/website
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathtest.html
170 lines (137 loc) · 6.57 KB
/
test.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<title>Boost Test Policies and Protocols</title>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii" />
<link rel="icon" href="/favicon.ico" type="image/ico" />
<link rel="stylesheet" type="text/css" href=
"../style-v2/section-development.css" />
<!--[if IE 7]> <style type="text/css"> body { behavior: url(/style-v2/csshover3.htc); } </style> <![endif]-->
</head><!--
Note: Editing website content is documented at:
https://www.boost.org/development/website_updating.html
-->
<body>
<div id="heading">
<!--#include virtual="/common/heading.html" -->
</div>
<div id="body">
<div id="body-inner">
<div id="content">
<div class="section" id="intro">
<div class="section-0">
<div class="section-title">
<h1>Boost Test Policies and Protocols</h1>
</div>
<div class="section-body">
<p>The Boost libraries are intended to be both reliable and
portable. Every experienced programmer knows that means each
library must be tested against a suitable number of test cases,
on a wide range of platforms, and then tested again (regression
tested) every time a change is made and before every
release.</p>
<p>"Quality assurance based on a wide range of targeted tests"
as one of the key answers to C.A.R Hoare's question "How did
software get so reliable without proof."</p>
<h2>Regression test</h2>
<p>Boost uses an automatic <a href=
"/doc/tools/regression/index.html">regression
test suite</a> which generates HTML <a href=
"/development/testing.html#RegressionTesting">compiler status
tables</a>.</p>
<h2>Test Policy</h2>
<h3>Required</h3>
<ul>
<li>Every Boost library should supply one or more suitable
test programs to be exercised by the Boost <a href=
"/doc/tools/regression/index.html">regression
test suite</a>. In addition to the usual compile-link-run
tests expecting successful completion, compile-only or
compile-and-link-only tests may be performed, and success for
the test may be defined as failure of the steps.</li>
<li>Test program execution must report errors by returning a
non-zero value. They may also write to stdout or stderr, but
that output should be relatively brief. Regardless of other
output, a non-zero return value is the only way the
regression test framework will recognize an error has
occurred. Note that test programs to be included in the
status tables must compile, link, and run quickly since the
tests are executed many, many, times.</li>
<li>Libraries with time consuming tests should be divided
into a fast-execution basic test program for the status
tables, and a separate full-coverage test program for
exhaustive test cases. The basic test should concentrate on
compilation issues so that the status tables accurately
reflect the library's likelihood of correct compilation on a
platform.</li>
<li>If for any reason the usual test policies do not apply to
a particular library, an alternate test strategy must be
implemented.</li>
<li>A <a href=
"/doc/tools/regression/index.html#Adding_new_test">
Jamfile</a> to drive the regression tests for the
library.</li>
</ul>
<h3>Optional (but highly recommended)</h3>
<p>The <a href="/doc/libs/release/libs/test/index.html">Boost
Test Library</a> provides many useful components which ease the
construction of test programs.</p>
<ul>
<li>Use the library's <a href=
"/doc/libs/release/libs/test/doc/html/boost_test/testing_tools.html">
Test Tools</a> for the construction of simple test programs
that do not need much structure.</li>
<li>Use the library's <a href=
"/doc/libs/release/libs/test/doc/html/boost_test/tests_organization.html">Unit
Test Framework</a> for the construction of more complex test
programs that need to be structured into individual tests and
test suites.</li>
</ul>
<h2>Suggested Protocol for Fixing Bugs or Adding Features.</h2>
<ul>
<li>First, add regression test cases that detects the bug or
tests the feature. Sometimes adding one case suggests similar
untested cases, and they are added too.</li>
<li>Second, for bugs, run the regression test and verify that
the bug is now detected.</li>
<li>Third, then, and only then, fix the bug or add the
feature.</li>
<li>Finally, rerun the full regression tests - sometimes the
change breaks something else.</li>
</ul>
<h2>History</h2>
<p><a href=
"/doc/tools/regression/index.html#History">See
Regression Test History</a>.</p>
<h2>Acknowledgements</h2>
<p>Written by Beman Dawes. Jens Maurer, Paul Moore, Gary Powell
and Jeremy Siek contributed helpful suggestions.</p>
<hr />
</div>
</div>
</div>
</div>
<div id="sidebar">
<!--#include virtual="/common/sidebar-common.html" -->
<!--#include virtual="/common/sidebar-development.html" -->
</div>
<div class="clear"></div>
</div>
</div>
<div id="footer">
<div id="footer-left">
<div id="revised">
<p>Revised $Date$</p>
</div>
<div id="copyright">
<p>Copyright Beman Dawes 2001.</p>
</div><!--#include virtual="/common/footer-license.html" -->
</div>
<div id="footer-right">
<!--#include virtual="/common/footer-banners.html" -->
</div>
<div class="clear"></div>
</div>
</body>
</html>