Version 13 (modified by cotto, 4 years ago)

add concurrency

There is a lack of clarity about the path that we plan on taking to implement and use Lorito. This page exists as a roadmap to document what we Parrot developers plan on doing with Lorito and how we expect to get there.

I'm dividing the road to Lorito into several stages. Most of the tasks during a given stage can happen independently, making it easy to parallelize.

Stage 0: Preparation

opsc (C mangling)

  • Implement an nqp-based ops compiler ("opsc") capable of generating C code equivalent to what the perl-based ops2c generates.
  • Check the generated C code for ops into version control to allow bootstrapping and reduce the need for rebuilding ops.

Lorito Design

  • Define Lorito ops and semantics.
    • Primary issues (should be decided before serious prototyping begins):
      • purpose (why does it exist, what role does it fill, what is it, what is it not)
      • how to deal with native types and call C-level functions (ffi)
      • arg format (fixed three-arg format, variable, other)
      • classes of ops, specific op set
      • security model
      • concurrency
      • interaction with PMCs
      • registers (will PMCs have distinct registers or be represented simply as pointers)
    • Secondary issues (can be decided as implementations progress):
      • binary format (Will PBC eventually contain Lorito ops? If not, how will composed ops be represented?)
      • textual format
      • backward and forward compatibility
      • composition of PIR ops from Lorito ops
      • how will bytecode be used (e.g. local caching optimization, distributed directly, only present as an intermediate step during compilation)
      • minimizing bytecode bloat (if PBC will eventually become Lorito-based)
  • Implement some PMCs, ops and library bindings in Lorito to demonstrate that Lorito is sufficiently powerful.
    • Because Lorito is not intended to be written directly, this stage may involve some annoyance on the part of the implementors.
    • An HLL may be suitable if it can be shown without excessive hand-waving how its code could be compiled to Lorito.
  • Identify and deal with blockers to implementing core PMCs and systems in Lorito.
  • Get a rough idea of how PIR -> Lorito translation will work. The same goes for Lorito -> C code and Lorito -> C functions.

Stage 1: Prototype

Lorito Prototype

  • Implement Lorito as PIR dynops, allowing them to be processed by the imcc and other existing tools.
  • Modify PCT to optionally emit Lorito dynops. Once it's usable, start using it.
  • Figure out how to make user-defined ops (i.e. ops defined in Lorito) work.
  • Define PIR -> Lorito translation.
  • Define Lorito -> C code translation (this is a JIT, among other things)
  • Define Lorito -> C function translation (similar to the current function-based runcore).

Lorito Field Testing

  • Make opsc able to process HLLs and spit out Lorito ops. Implement some ops in terms of an HLL, such as NQP, that compiles down to Lorito.
  • Start work on a Lorito setting, if needed and/or wanted.
  • Start implementing core PMCs in something that compiles to Lorito.
  • Make it possible to implement core functionality in Lorito. Encourage early adopters to start doing so to help flush out bugs, design problems and other sharp edges.

Stage 2: Implement and Bootstrap

Pervasive Lorito

  • Modify the current PIR compiler (ideally  PIRATE, possibly imcc or pirc) to emit Lorito.
  • Implement core PIR ops in Lorito.
  • Switch to a Lorito-centric PBC format. (maybe)
    • If we want to change the PBC format, this would be a good time to do it.
  • Make the final switch. All ops and PMCs get compiled down to Lorito during the normal build process (possibly using a bootstrapping step).
  • Start rewriting core systems in Lorito (strings, mmd, packfiles, pcc, whatever's feasible).

Stage 3: Convert and Optimize

Lorito Translations

  • Implement translation from Lorito to LLVM IR, libjit, GNU Lightning, LOLCODE, etc.
  • Lorito to C will be the default fallback.
  • Work in earnest on Lorito-level optimizations.

See also