Timing of GC runs

[nwellnhof]

Currently, a GC run is triggered after 1MB of headers newly allocated (GC_SIZE_THRESHOLD). Similarly for buffers. This causes memory allocation to get slower the more memory a process consumes. Attached is patch that sets this threshold dynamically to half of the memory currently used, so the amortized cost of memory allocations stays roughly constant.

On my machine, a test script creating a hash with a million strings now completes in 1.0 seconds instead of 2.7 before. Inserting two million strings takes 2.0 seconds instead of 8.7.

[chromatic]

revision of first patch

[chromatic]

The original patch slowed down Rakudo startup, so I added back in the skipping of constant pools and gained back all that speed and more. How does this fare with your benchmark?

[nwellnhof]

Yes, it was probably a bit rash to remove the GC_SKIP logic. I can't test it now, but I'm pretty sure that it doesn't have any effect on my benchmark.

[chromatic]

On Thursday 12 August 2010 at 03:17, Parrot  wrote:

>  Yes, it was probably a bit rash to remove the GC_SKIP logic. I can't test
>  it now, but I'm pretty sure that it doesn't have any effect on my
>  benchmark.

If we can get some testing on this on various platforms before Saturday 
(especially with Rakudo and Lua), I'm comfortable committing it for 2.7.

-- c

[chromatic]

Per Moritz's suggestion, I've created a branch for testing and further tuning; it's gc_threshold_tuning.

[nwellnhof]

Here is another fix that doesn't subtract freed buffers from the used memory count. The memory of freed buffers isn't available until they have been compacted, so the GC shouldn't be delayed because of them.

Also, my initial suggestion of setting the dynamic threshold to 50% of used memory might be a bit aggressive. Maybe we should try 25% (i.e. right shift by 2)?

[nwellnhof]

Here's another fix that I think is important. This part was removed by gc-timing-two.patch and maybe it interferes with the skipping of constant pools again, but it fixes the memory leak reported in TT#1465.

[nwellnhof]

Fixed in r48585