NCI Tasks

The current Parrot NCI system has a number of weaknesses; the following is a list of the major areas to be addressed:

Improve Signature => Thunk Generation

• Currently, if Parrot has been configured using the default, static frame builder, the NCI signatures useable from Parrot are fixed at compile time.
• This is more restrictive than necessary: extra thunks can be loaded at runtime into interp.iglobals.nci_funcs.
• nativecall.pl already generates thunks for core, could be extended to make runtime libraries to accomplish this
  • If we are already refactoring nativecall.pl, we can re-write in a Parrot language, to reduce the dependancy on Perl (this has been done with nativecall.pir)
  • Advantages of nativecall.pir:
    • All knowledge of signature-specific behaviour is now in sig_table, not in program logic
    • No more 15 positional parameter functions.
    • Runs on Parrot, no need for Perl to use.
    • Available for use by other Parrot tools (eg: tests)

Generalize Function Handling

• Generalize the callback subsystem (see TT #1192)
• Support GetProcAddress-style functions (library functions that return addresses for other library functions)

Improve Type Handling

• Basic types
  • long long (see TT #1182)
  • Types larger than native machine word (long double, 64 bit int on x86, 128 bit int on x86_64, etc.)
  • Exact sized types, including smaller types (16 bit int and float, 32 bit float, etc.)
  • Wide characters
  • Endian conversion
  • Handle signed/unsigned properly
• Structures, unions, and arrays
  • Bit packed fields
  • Unions
  • Less painful and *faster* API for structures
  • Proper handling of packed single- and multi-dimensional arrays
  • Clean SoA and AoS handling (to arbitrary depth)
• Buffers and protocol data
  • Counted strings
  • Padded strings/buffers
  • Zero-copy R/W access to buffer contents

Improve documentation

• Expand function signature information in source:/trunk/docs/pdds/draft/pdd16_native_call.pod to include all possible param types in source:/trunk/tools/build/nativecall.pl (or its replacement)
• Add full documentation for structure/union/buffer/array handling
• Expand example cases considerably; at least all param types, structure/array forms, and callback/GetProcAddress variants should be represented

Discussion

The above tasks were discussed  in #parrot on 2009-11-05, leading to the proposed solutions below.

Proposed solutions

• Create and document pluggable (configure-time) framebuilder system
  • 2 varieties: call-out (current framebuilder), and call-in (callbacks)
  • implementations are not required to handle all signature strings (but document where they lack functionality). notably, the fallback static builder only handles a predefined set of strings (configure-time modifiable)
  • for callbacks without an opaque userdata parameter, call-in framebuilders will DTRT if they can, and explode otherwise
  • Fixes:
    • callback problems
    • external ffi/jit library feature-envy
• replace current signature strings with new, more general strings and/r structured definitions
  • more general strings:
    • similar to pcc signatures ("args->retv"), each position's type is defined by an upper-case character followed by non-upper-case-modifiers
    • types: I => integer, F => float, S => string, P => pointer/buffer, ... others?
    • modifiers: & => pass by ref (PMC required), various => size modifier (size defaults to parrot register type)
    • examples: I => INTVAL, Is => short, I[32] => int32, Ic& => pass by reference char (from inside an Integer PMC)
    • transition strategy: initially require new-style signature strings to be prefixed by a character not in either scheme (eg: '+')
  • structured definitions:
    • similar to struct handling API
    • transition strategy: different types (PMC vs String) for signatures => different operations => no conflict
  • fixes (potentially):
    • long long
    • exact sized types
    • wide chars
    • endianness
    • signedness
  • what needs to be done:
    • make src/pmc/nci.pmc understand the new signatures
    • make tools/dev/nci_thunk_gen.pir understand the new signatures
    • deprecation notice (if necessary)
• allow more direct PIR access to NCI PMC (particularily creation)
  • example:

  $P0 = get_c_pointer_somehow()
  $P1 = new 'NCI'
  $P1['FooBarSig'] = $P0

• fixes:
  • GetProcAddress woes

• replace/augment (un)?managed struct with a struct type factory (JIT or a set of types that efficiently handle common cases)
  • could unify type signatures with NCI strings (probably more of an NCI change than a struct handling change)
  • fixes:
    • structure/union/array problems

Useful Information

Some useful information has been compiled at CFFI