Changes between Initial Version and Version 1 of AutomatedHLLTesting

01/07/11 01:25:15 (11 years ago)

initial free-form braindump


  • AutomatedHLLTesting

    v1 v1  
     1Parrot needs to become a stable platform for HLL development.  A big component of that is finding out quickly when a change breaks an HLL, and automated HLL testing will help.  We also need to fix Parrot's culture of looking at our own test suite as the primary arbiter of what constitutes a safe change.  Hopefully regular automated HLL testing will help ingrain that in our collective mindset. 
     3=== HLL testing needs === 
     5An HLL CI environment needs to: 
     6 * test the latest version of Parrot against known-good versions of a variety of HLLs and libraries 
     7 * intelligently increment the versions of HLLs 
     8   * needs to hold parrot steady so that we can be relatively certain that the upgrade didn't introduce an HLL bug 
     9   * testing old versions of HLLs that are expected to retain backwards-compatibility may be desirable 
     10 * be smart enough to know that an old HLL won't work on a current Parrot 
     11   * deprecations and support periods may come into play here 
     12 * notify #parrot when something breaks an HLL 
     13   * granularity can be configurable, but every couple hours would be sufficient.  Hourly or per-commit would be ideal. 
     14 * keep notifying #parrot every time the HLL stays broken 
     15   * it might even retry on every commit once an HLL is known to fail 
     16 * notify #parrot when an HLL stops being broken 
     17 * live somewhere where it can be reasonably modified by more than one parrot developer 
     18   * any interested and trusted developer should be able to get an admin bit 
     19   * low bus numbers lead to pain