Changes between Version 1 and Version 2 of TestingProfiling

Show
Ignore:
Timestamp:
01/12/10 09:57:44 (4 years ago)
Author:
cotto
Comment:

more thoughts on dealing with profile size

Legend:

Unmodified
Added
Removed
Modified
  • TestingProfiling

    v1 v2  
    55=== randomness from timing information === 
    66A profile will contain a randomness in the form of timing information.  It does not make sense to test for absolute values since tests must be able to succeed on slow machines as well as fast.  It may make sense to test relative timing information, but even this is questionable. 
    7 The best approach may be to enable the runcore to emit constant dummy timing information where all values are 1.  This would also allow testing that times were added correctly and would simplify sanity checking. 
     7The best approach may be to enable the runcore to emit a canonical form of the profile in which all data that vary between runs (timing information, memory addresses) would be changed to constants.  This would also allow testing that times were added correctly and would simplify sanity checking. 
    88 
    99=== data volume === 
    1010The profile for even a short PIR program will be non-trivial in size.  It must be easy for the testing code to specify which part of a profile it wants to test.  Ideally the testing code would also avoid using PGE so that profiling tests could be run as part of coretest. 
     11 
     12One solution is to use annotations to delimit the part of the code to be tested.  The testing code could examine the first chunk of profiling data between two predefined annotations (e.g. .annotate "begin_profiling_test" 1 ... .annotate "end_profiling_test" 1).  This would allow testing the profile of an arbitrary contiguous subset of a piece of PIR code. 
     13example: 
     14{{{ 
     15pir_delimited_profiling_output_is(<<'PIR', <<'PROFILE', "..."); 
     16.sub main :main 
     17  say "im not in ur profile" 
     18  .annotate "begin_profiling_test", 1 
     19  say "HELO" 
     20  .local int i 
     21  i = 3 
     22 loop: 
     23  dec i 
     24  if i > 0 goto loop 
     25  say "BYE" 
     26  .annotate "end_profiling_test", 1 
     27  noop 
     28  i = 9 
     29.end 
     30PIR 
     31OP:{x{line:4}x}{x{time:1}x}{x{op:say}x} 
     32OP:{x{line:6}x}{x{time:1}x}{x{op:set}x} 
     33OP:{x{line:8}x}{x{time:1}x}{x{op:dec}x} 
     34OP:{x{line:8}x}{x{time:1}x}{x{op:lt}x} 
     35OP:{x{line:8}x}{x{time:1}x}{x{op:dec}x} 
     36OP:{x{line:8}x}{x{time:1}x}{x{op:lt}x} 
     37OP:{x{line:8}x}{x{time:1}x}{x{op:dec}x} 
     38OP:{x{line:8}x}{x{time:1}x}{x{op:lt}x} 
     39BYE 
     40OP:{x{line:-1}x}{x{time:1}x}{x{op:say}x} 
     41PROFILE 
     42}}} 
     43 
     44 
     45 
     46 
     47