Ticket #2090 (new bug)

Opened 3 years ago

Last modified 3 years ago

Parrot underestimates memory on OS X

Reported by: benabik Owned by:
Priority: normal Milestone:
Component: core Version: 3.2.0
Severity: low Keywords:
Cc: GeJ, jkeenan Language:
Patch status: Platform: darwin

Description

On OS X, Parrot uses the HW_PHYSMEM sysctl to determine the amount of memory on the system. Unfortunately, the value returned from it is a signed 32b integer, so we will never know if a system has more than 2GiB of RAM. The sysctl man page suggests using HW_MEMSIZE to get a full 64-bit value.

This doesn't break anything, but as-is, Rakudo spends about 60% of the time building Rakudo's core.pir in gc_gmc_mark_and_sweep, which is triggered when 10% of total memory is allocated. On my system, it would trigger literally half as often if parrot knew much RAM my laptop really had.

Of course, this is complicated by the fact that we previously changed from MEMSIZE to PHYSMEM already in SVN r49633 (git 5977674) due to build errors. Does anyone know what version of Darwin MEMSIZE was introduced in? Perhaps we should check if the HW_MEMSIZE macro exists and only use PHYSMEM if it doesn't?

Change History

  Changed 3 years ago by jkeenan

Some relevant links:

TT #1829 -- particularly last 3 or 4 posts

 a stackoverflow discussion

 a monetdb discussion

in reply to: ↑ description   Changed 3 years ago by jkeenan

Replying to benabik:

Does anyone know what version of Darwin MEMSIZE was introduced in?

I haven't been able to find this out.

Perhaps we should check if the HW_MEMSIZE macro exists and only use PHYSMEM if it doesn't?

Can you prepare a patch?

Thank you very much.

kid51

follow-up: ↓ 4   Changed 3 years ago by jkeenan

There is a possibility that we could revert to HW_MEMSIZE.

On Darwin/PPC, I made this change:

$ git diff
diff --git a/src/platform/darwin/sysmem.c b/src/platform/darwin/sysmem.c
index cb2f0f4..6b05ca3 100644
--- a/src/platform/darwin/sysmem.c
+++ b/src/platform/darwin/sysmem.c
@@ -45,7 +45,8 @@ Parrot_sysmem_amount(PARROT_INTERP)
     char         *err_msg;
     unsigned long length = sizeof (memsize) ;
 
-    int selection[2] = { CTL_HW, HW_PHYSMEM } ;
+/*    int selection[2] = { CTL_HW, HW_PHYSMEM } ; */
+    int selection[2] = { CTL_HW, HW_MEMSIZE } ;
 
     err = sysctl(selection, 2, &memsize, &length, NULL, 0) ;
 

I was able to build and test, failing only the same test in t/src/extend_vtable.t that has been failing regularly (cf. TT #2084).

I didn't time these tests, nor did I attempt to build Rakudo. We'd need results on Darwin/i386 on more recent versions of OS X before reverting.

I can't keep track of which GC system we're using as default these days, but perhaps changes we've made elsewhere in Parrot since last October now permit us to use HW_MEMSIZE.

Thank you very much.

kid51

in reply to: ↑ 3   Changed 3 years ago by jkeenan

Replying to jkeenan:

There is a possibility that we could revert to HW_MEMSIZE.

That possibility is now dubious. Testing with HW_MEMSIZE today, make failed with this error:

./miniparrot -Iruntime/parrot/include config_lib.pir > 
  runtime/parrot/include/config.fpmc
src/string/api.c:653: failed assertion 'encoding'
make: *** [runtime/parrot/include/config.fpmc] Error 134
/usr/local/bin/perl tools/build/parrot_config_c.pl > \
    src/parrot_config.c
'runtime/parrot/include/config.fpmc' is truncated. 
  Remove it and rerun make
make: *** [src/parrot_config.c] Error 255

Reverting to HW_PHYSMEM avoids this build error.

kid51

Note: See TracTickets for help on using tickets.