Changes between Version 1 and Version 2 of TestingProfiling
- Timestamp:
- 01/12/10 09:57:44 (12 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
TestingProfiling
v1 v2 5 5 === randomness from timing information === 6 6 A profile will contain a randomness in the form of timing information. It does not make sense to test for absolute values since tests must be able to succeed on slow machines as well as fast. It may make sense to test relative timing information, but even this is questionable. 7 The best approach may be to enable the runcore to emit constant dummy timing information where all values are 1. This would also allow testing that times were added correctly and would simplify sanity checking.7 The best approach may be to enable the runcore to emit a canonical form of the profile in which all data that vary between runs (timing information, memory addresses) would be changed to constants. This would also allow testing that times were added correctly and would simplify sanity checking. 8 8 9 9 === data volume === 10 10 The profile for even a short PIR program will be non-trivial in size. It must be easy for the testing code to specify which part of a profile it wants to test. Ideally the testing code would also avoid using PGE so that profiling tests could be run as part of coretest. 11 12 One solution is to use annotations to delimit the part of the code to be tested. The testing code could examine the first chunk of profiling data between two predefined annotations (e.g. .annotate "begin_profiling_test" 1 ... .annotate "end_profiling_test" 1). This would allow testing the profile of an arbitrary contiguous subset of a piece of PIR code. 13 example: 14 {{{ 15 pir_delimited_profiling_output_is(<<'PIR', <<'PROFILE', "..."); 16 .sub main :main 17 say "im not in ur profile" 18 .annotate "begin_profiling_test", 1 19 say "HELO" 20 .local int i 21 i = 3 22 loop: 23 dec i 24 if i > 0 goto loop 25 say "BYE" 26 .annotate "end_profiling_test", 1 27 noop 28 i = 9 29 .end 30 PIR 31 OP:{x{line:4}x}{x{time:1}x}{x{op:say}x} 32 OP:{x{line:6}x}{x{time:1}x}{x{op:set}x} 33 OP:{x{line:8}x}{x{time:1}x}{x{op:dec}x} 34 OP:{x{line:8}x}{x{time:1}x}{x{op:lt}x} 35 OP:{x{line:8}x}{x{time:1}x}{x{op:dec}x} 36 OP:{x{line:8}x}{x{time:1}x}{x{op:lt}x} 37 OP:{x{line:8}x}{x{time:1}x}{x{op:dec}x} 38 OP:{x{line:8}x}{x{time:1}x}{x{op:lt}x} 39 BYE 40 OP:{x{line:-1}x}{x{time:1}x}{x{op:say}x} 41 PROFILE 42 }}} 43 44 45 46 47