Ticket #1737 (closed patch: fixed)

Opened 4 years ago

Last modified 4 years ago

Timing of GC runs

Reported by: nwellnhof Owned by:
Priority: normal Milestone:
Component: GC Version: 2.6.0
Severity: medium Keywords:
Cc: Language:
Patch status: Platform:

Description

Currently, a GC run is triggered after 1MB of headers newly allocated (GC_SIZE_THRESHOLD). Similarly for buffers. This causes memory allocation to get slower the more memory a process consumes. Attached is patch that sets this threshold dynamically to half of the memory currently used, so the amortized cost of memory allocations stays roughly constant.

On my machine, a test script creating a hash with a million strings now completes in 1.0 seconds instead of 2.7 before. Inserting two million strings takes 2.0 seconds instead of 8.7.

Attachments

gc-timing.diff Download (11.2 KB) - added by nwellnhof 4 years ago.
gc-timing-two.patch Download (9.3 KB) - added by chromatic 4 years ago.
revision of first patch
gc-timing-fix.diff Download (2.8 KB) - added by nwellnhof 4 years ago.
gc-timing-fix2.diff Download (0.7 KB) - added by nwellnhof 4 years ago.

Change History

Changed 4 years ago by nwellnhof

Changed 4 years ago by chromatic

revision of first patch

Changed 4 years ago by chromatic

  • component changed from core to GC

The original patch slowed down Rakudo startup, so I added back in the skipping of constant pools and gained back all that speed and more. How does this fare with your benchmark?

Changed 4 years ago by nwellnhof

Yes, it was probably a bit rash to remove the GC_SKIP logic. I can't test it now, but I'm pretty sure that it doesn't have any effect on my benchmark.

Changed 4 years ago by chromatic

On Thursday 12 August 2010 at 03:17, Parrot  wrote:

>  Yes, it was probably a bit rash to remove the GC_SKIP logic. I can't test
>  it now, but I'm pretty sure that it doesn't have any effect on my
>  benchmark.

If we can get some testing on this on various platforms before Saturday 
(especially with Rakudo and Lua), I'm comfortable committing it for 2.7.

-- c

Changed 4 years ago by chromatic

Per Moritz's suggestion, I've created a branch for testing and further tuning; it's gc_threshold_tuning.

Changed 4 years ago by nwellnhof

Here is another fix that doesn't subtract freed buffers from the used memory count. The memory of freed buffers isn't available until they have been compacted, so the GC shouldn't be delayed because of them.

Also, my initial suggestion of setting the dynamic threshold to 50% of used memory might be a bit aggressive. Maybe we should try 25% (i.e. right shift by 2)?

Changed 4 years ago by nwellnhof

Changed 4 years ago by nwellnhof

Here's another fix that I think is important. This part was removed by gc-timing-two.patch and maybe it interferes with the skipping of constant pools again, but it fixes the memory leak reported in TT#1465.

Changed 4 years ago by nwellnhof

Changed 4 years ago by nwellnhof

  • status changed from new to closed
  • resolution set to fixed

Fixed in r48585

Note: See TracTickets for help on using tickets.