Ticket #1020: plandoc.txt

----
The status as it is on 28 okt 2009. See branch tt_1020 for more details. 
(broken at the moment will be fixed tomorrow I hope,small svn problem)

This will give some extra detail on the plan and will give a view on the current status.
Any comments are welcome. 

The functionality in object.pmc will move to default.pmc. With this action everything will
become an object. (default.pmc will get renamed back to object.pmc). 
The C pmc can keep using there C structures as they did before. But with a few restrictions.
All field must be one off the following types.
(AT_POINTER)     void* (to some C structure/data)
(AT_PMC_POINTER) pmc* (pointer to a pmc)
(AT_STRING)      STRING* (pointer to a String)
(AT_INTEGER)     INTVAL (integer)
(AT_FLOAT)       FLOATVAL (float value)
The usage of attributes as an array is no longer allowed. (ATTR INTVAL vals[4] = no long allowed)
The following thing has been done to make the set/get attributes function also work for the attributes accessed via the
C structure. The attributes array (DPOINTER **) array has been mapped over this C structure (union). 
The "get_attr_str","get_attr_keyed","set_attr_str","set_attr_keyed" use this array to access these attributes.
This array has element size off 32 bits. Using a 32 bits I386 machine as base. Pointers and Integers are 32 bits. 
But Float (=double) are 64 bit. So they take 2 entries of this attributes array. To keep track of this sizes the 
following attributes in the class.pmc are used.
-attrib_index2type 
Tells which type of value can be found at each index. If it is index halfway of double value then it is a invalid value.
That is a small gap.
-attrib_markwalk
Array with the indexes to the items which has to marked.
-attrib_indexes 
Array with the indexes to all items.

For exmaple when using the following attributes.
attr Context * ctx;
attr PMC * some_pmc;
attr INTVAL myval;
attr FLOAT myfloat;
attr STRING * mystring;
Then these attributes will look like:
attrib_index2type = [AT_POINTER,AT_PMC_POINTER,AT_INTEGER,AT_FLOAT,invalid,AT_STRING]
attrib_mark_walk = [1,5]
attrib_indexes = [0,1,2,3,5]

The sizeof(void*), INTVAL and FLOAT are determined by a config script and placed in the 
core_pmcs.c->attrib_type2index_size function. Note that I placed some restriction on the this, see config/gen/core_pmcs.pm.
See code oo.c "get_attrib_type_from_desc", "cache_class_attribs","build_attrib_index" for extra detail.

All core pmcs get a full describing pmcclass object as a 
pir created class does have, instead of the vtable struct information. See the code in lib/Parrot/Pmc2c.

A extra load pass had been added. These are the attributes of the class.pmc which the load pass
at which they are loaded.
 
0    ATTR INTVAL id;             /* The type number of the PMC. [deprecated: See RT #48024] */
0    ATTR STRING *name;          /* The name of the class. */
0    ATTR STRING *fullname;      /* The name of the class. */
2    ATTR PMC *_namespace;       /* The namespace it's linked to, if any. */
-    ATTR INTVAL instantiated;   /* Any instantiations since last modification? */
1    ATTR PMC *parents;          /* Immediate parent classes. */
2    ATTR PMC *all_parents;      /* Cached list of ourself and all parents, in MRO order. */
2    ATTR PMC *roles;            /* An array of roles. */
2    ATTR PMC *methods;          /* Hash of method names to methods in this class. */
-    ATTR PMC *vtable_overrides; /* Hash of Parrot v-table methods we override. */
2    ATTR PMC *attrib_metadata;  /* Hash of attributes in this class to hashes of metadata. */
2    ATTR PMC *attrib_index;     /* Lookup table for attributes in this and parents. */
2    ATTR PMC *attrib_cache;     /* Cache of visible attrib names to indexes. */
-    ATTR PMC *resolve_method;   /* List of method names the class provides to resolve
                                 * conflicts with methods from roles. */
-    ATTR PMC *parent_overrides;
0    ATTR INTVAL attrib_count;   /* Number of attributes in attrib object store of the
                                   object of this type
                                   Field could be saved by combining it with instantiated */

0    ATTR VTABLE *vtable;        /* The vtable function of this class */
0    ATTR VTABLE *ro_vtable;

The roles field uses a temporary function which create a "empty" role.pmc object.
In the future someone could implement something, which makes it possible to define a role as 
in the same way a pmc is now.

The namespace is created when the class is loaded.

The attrib_metadata is created, the "type" field is now used to describe the type of the
attributes. Which can be one of the following items "String","Void*","Integer","Float","PMC".

Since all pmc will become an object, there is will be a field pointing to the pmcclass.
The pmcclass object will contain the full describtion of the pmc/object. 
The pointer to the vtable struct will get removed since all information is stored in the
pmcclass object. The interp->vtables[] array get replaced with the interp->classes[] array.
This last step has already been completed.

The vtable fields and initianlization of "base_type","attribute_defs","whoami","provides_str",
"isa_hash","_namespace",*"mro",*"attr_size",*"pmc_class" will become reduntant.
* are already removed

So after completion of this change the vtable struct will only hold pointers to the vtable functions.
The pmc will have no more pointer to the vtable struct, all needed information is stored in the pmcclass object.

Because a pmcclass object, which describes a class has got a more central role and the information is used
for the removal of an object, are they stored in a seperate gc small object arena. Class object will
run the mark function and mark other object, but will for now will not get automatically removed. 
This means when a class is defined it will stay in the memory until shutdown. 
These can ofcourse be fixed in the future, but this "problem" has also been
for a while in the jvm, so I think it will not be a big problem. Once the gc system get finalized, the pmcclass object
which describes the class itself will get removed as the last one. See code src/gc/*.c. 

The "get_attr_str","get_attr_keyed","set_attr_str","set_attr_keyed" are the function which are
used by pir code to access the attributes. These function do lookup via a hash, as it was before
this change. 
When a INTVAL,FLOAT or STRING is stored it will get unbox and stored.
When rertrieving them they will get boxed again.
Attributes of type void* can not be accessed, they are "private".
Note that the code in the core pmc's just keep using the c structure.
The difference is there will be no more proxy object so setting an attribute in an extended core pmc object
will actually set the attribute values. This will make the following test succeed.

.sub int_subclass_test
    .local pmc myint
    $P5 = box 10 
    myint = subclass 'Integer', [ 'MyInt2' ]    
    $P1 = new myint
    setattribute $P1,'iv',$P5
    $I1 = set $P1
    is ($I1, 10, 'setattribute integer value')
    getattribute $P3,$P1,'iv'
    $I1 = set $P3
    is ($I1, 10, 'getattribute integer value')
    $S0 = $P1
    is ($S0, "10", 'integer to string')
.end

The following work has already been doen.

-Replace interp->vtables[] by interp->classes[]
-Added needed extra fields in class.pmc
-Make bootstrap/load code in lib/Parrot/pmc2c/ for loading the full class information of the core pmcs
-Addapt the class registration code, (some changes still be done)
-Creation of the new attribute indexing system  
-Store pmcclass object into seperate gc pool.
-Because of the restriction of the c structure some attributes of some pmc had to wrapped in a struct
or pointer. This work has been completed. 
-All pmc has become an object, this resulted in the removal of the auto_attr fields. All changes
needed for that has been completed. See for example the context.pmc. The pointer to the context has
become an attribute instead of being directly stored in the pmc->data field.
-Move the functionality of the object.pmc to default.pmc partly done
--The mark,clone,thaw,freeze,visit and finishthaw has still to be done.

Work to do and thing not yet build

This is the work that has still to be done, to complete this change. 

-mark,clone,freeze,thaw,visit,finishthaw functions
Pir code can extends a core pmcs class and extra attributes to it. These extra attributes also needs to get handled.
This applies to the following function.
Marking -> mark
Cloning -> clone
Freeze/thaw -> freeze,thaw,visit,finishthaw
These function are currently implemented in the pmc files, but they should the "'object.pmc'" also to handle
the optionally added extra attributes. But since all attributes of type of PMC,STRING,INVTAL and FLOAT can be handled
by this "'object.pmc'" basic code, they do not need to be done by the specific pmcs code itself, except there is
something special to them. Attributes of type Void* has to be handled by the pmcs class itself.
The pmcÂŽs has to changed in such a way the optionally added extra attributes get also handled.

-freeze/thaw system
The freeze/thaw system, make use of the imformation stored in the pmcclass objects. These need special handling in
the freeze/thaw system. First needs the classes get thawed before the object, because the information in the classpmc
object is needed for the thawing. 
The current completed changed code stores the class information in the object freeze/thaw data, 
but that cost extra space, which is not needed.

-get/set attr overrides
"get_attr_keyed","set_attr_keyed" are free and not implemented by any core pmc
"get_attr_str","set_attr_str" implemented in callsignature, exception,task,eventhandler

-kill pmcproxy.pmc
--make sure the init and init_pmc vtable function still get called in rigth order
--step by step removal of pmcproxy functions
--switch over to class.pmc initialize function
--kill pmcproxy.pmc file 

-kill old object.pmc file
-rename default.pmc to object.pmc

-clean up
--step by step of vtable struct fields, which are no longer used
--remove vtable field from pmc struct 
--remove old no longer used code
--fixup broken tests
---threads.t and interpreter cloning may need some extra time
--fixup coding standaard

-fixup the documentation


-

Discussion on Performance 
Take in mind that the above described int_subclass_test test is not working at the moment and fixing that is going to cost
some performance.

+no more pmc proxy object 
 = gaining speed (no more looking up needed)
 = use less memory in case of extending core pmc (lots less memory)
-vtable function acces pmc->class->vtable->function instead of pmc->vtable->function
-some non allowed attribute types are wrapped up
-auto_attr is gone, see for example context.pmc. If there are no attributes then no mem is allocated.
-call to vtable get/set attribute functions must do extra index 2 type action
-call to vtable get/set attribute functions do unbox and boxing, but only in case of accessing
a core pmc attribute, so this one does not count :)
+allowing later on patch for vtable override function, which does the following thing:
When there is a vtable override the vtable function pointer will get replaced by pointer
pointing to a lookup function which search in the vtable overrides code. Instead of having
to do a lookup search all the time. Gaining lots of speed.
+no more special code inplace for handling core pmcs
-the class.pmc needs some more initialization. 
+- the mark,clone,freeze and thaw function will change
- mark function must only walk the pmc* and String* attributes
- clone function must take care of the different attribute types

+the already nice parrot code will get better smoothed out, gaining dev time (important one for me :)) 


I hope have mentioned every thing and did not forget stuff.

--------