Jit code buffers need to be allocated via mmap()

[Infinoid]

Currently on linux x86, jit code buffers are allocated from the heap. On certain platforms and under certain conditions, the heap is not executable. This results in a segfault immediately upon attempting to execute some JIT code. We are currently able to reproduce this on feather3, when executing tene's build from within a screen session (don't ask).

Parrot does call mprotect() in an attempt to set the right permissions on the buffer. However, the mprotect manpage on my linux box says that POSIX mprotect behavior is undefined when called on a non-mmaped buffer. So, it looks like we need to switch mem_alloc_executable()/mem_free_executable()/mem_realloc_executable() over to using mmap.

[Infinoid]

I've attached patches to do this. Three patches implement the functionality in stages (API change, ManagedStruct change, and then finally the JIT change), the fourth patch is for debugging/testing it.

It doesn't quite work for me. It runs fine under gdb, and the debugging statements output the right values, but then it segfaults when calling the free-function pointer (apparently trying to call an address other than the one it just printed to stderr).

If we can get that part working, I believe it is the correct fix for the issue on feather3.

[Infinoid]

Small followup: the old "don't run it in a screen session" trick is no longer making things work, so polyglotbot is currently configuring parrot with "--jitcapable=0". This seems to be an effective workaround.

[santtu]

As a side comment and further rationale on using mmap: selinux in fedora particularly tends to disallow changing heap or stack to executable (using mprotect), the only more-or-less guaranteed working way is to use mmap (with MAP_SHARED -- even MAP_PRIVATE will fail if allow_execmod=0).

[santtu]

Updated Infinoid's 01 patch to work in HEAD (r37916)

[santtu]

Another approach to calling mem_free_executable with correct values (see Inifinoid's 02 patch): use custom and only-for-JIT PMC (JitCode) instead of generic ManagedStruct

[santtu]

I added two patches, first one is just update on earlier 01* patch. The second one takes alternative approach to 02*.

The 02 patch is needed, because if mmap/munmap is used, the munmap *must* be called with correct size value. Thus parrot needs to keep the actual JIT memory block allocation size somewhere around. ManagedStruct already has size parameter (int struct value), but it for JIT code it needed to be passed to mem_free_executable. Thus the 02 patch added custom free routine.

I chose instead to not modify existing ManagedStruct, which probably has other uses too and is overloaded with many accessors (via extending UnmanagedStruct). The JIT code basically just needs to be GCable and accessible from parrot C core -- not really PIR code at all so any accessors etc. are unused. Thus the JitCode PMC is very simple -- it just holds memory pointer and its size, and calls mem_free_executable when GCed.

[Infinoid]

Updated patch from Santtu++, thanks!

[Infinoid]

Implement custom free and clone callback functions (and opaque private pointer arguments).

[Infinoid]

This latest version of the patch set is what fell out of this morning's design discussion on #parrot. It preserves existing behavior as much as possible, and allows feature reuse as much as possible.

It could use some testing on selinux/x86 and on win32.

[Infinoid]

Minor update to Makefile interdependencies (this fixes builds on non-jit architectures)

[Infinoid]

Summary patch (contains 01, 02, 03) for easier testing.

[Infinoid]

I've applied my newest versions of these patches as r37991, r37992, and r37993, with additional fixes in r37994. Thanks very much to Santtu++ for patches and guidance.

This should get basic jit working. Santtu has some ideas for how to get jit working in even more restricted selinux configurations, but I believe we should make a new ticket for that.

(If the checkins I mentioned above broke anything, please reopen this ticket.)

[Infinoid]

Minor bump, TT #581 appears to be the same issue, again on Fedora. The requester complains that selinux in restricted mode doesn't work, but permissive mode does work.