Ticket #1927 (closed bug: fixed)

Opened 11 years ago

Last modified 11 years ago

Parrot doesn't build with spaces in directory name

Reported by: whiteknight Owned by: jkeenan
Priority: critical Milestone: 3.0
Component: build Version: master
Severity: high Keywords: win32
Cc: Language:
Patch status: applied Platform: all

Description

I just tried building a fresh checkout of Parrot on Win32 in a directory with spaces in the name "C:/Documents and Settings/awhitworth/...". In the makefile and in config_lib.pir, this was mangled to "C:/Documents\ and\ Settings/awhitworth", which failed. In the config PMC, this was unescaped to "C:/Documents\\ and\\ Settings/awhitworth", which looks even worse.

Even after hand-editing the makefile and the config_lib.pir text, the build still does not complete because of issues finding include files.

When I copied my build directory to a path without spaces ("C:/Parrot"), it builds without issue.

Attachments

spaces_in_paths.diff Download (18.5 KB) - added by jkeenan 11 years ago.
diff between spaces_in_path branch and master

Change History

  Changed 11 years ago by jkeenan

whiteknight:

Is this the same issue as TT #930?

kid51

  Changed 11 years ago by nwellnhof

See also TT #196.

  Changed 11 years ago by nwellnhof

  • owner set to nwellnhof
  • platform changed from win32 to all
  • status changed from new to assigned

Linux build in a directory with spaces is also broken.

  Changed 11 years ago by nwellnhof

Should be fixed in cd40e3589390. Tested on Debian and Windows 7 with Activestate/MSVC.

  Changed 11 years ago by jkeenan

I successfully built and ran make moretests on Linux/i386 in a directory called My Parrot.

But then I tried to build on Darwin/PPC in a directory called My Freakin Parrot. The build failed here: 

/usr/local/bin/perl -MExtUtils::Command -e mkpath blib/lib
/usr/bin/g++ -dynamiclib -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib -L/Users/jimk/gitwork/My Freakin Parrot/blib/lib -L/sw/lib  -o blib/lib/libparrot.2.11.0.dylib -install_name /Users/jimk/gitwork/My Freakin Parrot/blib/lib/libparrot.dylib \
        src/string/api.o src/ops/core_ops.o src/core_pmcs.o src/datatypes.o src/debug.o src/dynext.o src/embed.o src/embed/api.o src/embed/strings.o src/embed/pmc.o src/string/encoding.o src/exceptions.o src/exit.o src/extend.o src/extend_vtable.o src/gc/alloc_memory.o src/gc/api.o src/gc/gc_ms.o src/gc/gc_inf.o src/gc/gc_ms2.o src/gc/mark_sweep.o src/gc/system.o src/gc/fixed_allocator.o src/gc/variable_size_pool.o src/gc/string_gc.o src/global_setup.o src/hash.o src/hll.o src/call/pcc.o src/call/args.o src/interp/inter_cb.o src/interp/inter_create.o src/interp/inter_misc.o src/call/ops.o src/call/context.o src/call/context_accessors.o src/key.o src/library.o src/list.o src/pointer_array.o src/misc.o src/multidispatch.o src/namespace.o src/nci/api.o src/nci/signatures.o src/nci/core_thunks.o src/nci/extra_thunks.o src/oo.o src/platform.o src/packfile/object_serialization.o src/pmc.o src/runcore/main.o src/runcore/cores.o src/runcore/profiling.o src/scheduler.o src/spf_render.o src/spf_vtable.o src/sub.o src/thread.o src/runcore/trace.o src/utils.o src/vtables.o src/warnings.o src/packfile/api.o src/packfile/output.o src/packfile/pf_items.o src/longopt.o src/gc/alloc_resources.o  src/io/core.o src/io/api.o src/io/utf8.o src/io/buffer.o src/io/unix.o src/io/win32.o src/io/portable.o src/io/filehandle.o src/io/socket_api.o src/io/socket_unix.o src/io/socket_win32.o src/pmc/default.o src/pmc/null.o src/pmc/env.o src/pmc/key.o src/pmc/unmanagedstruct.o src/pmc/managedstruct.o src/pmc/exception.o src/pmc/parrotlibrary.o src/pmc/parrotinterpreter.o src/pmc/parrotthread.o src/pmc/lexpad.o src/pmc/timer.o src/pmc/pointer.o src/pmc/sub.o src/pmc/continuation.o src/pmc/coroutine.o src/pmc/eval.o src/pmc/nci.o src/pmc/float.o src/pmc/integer.o src/pmc/bigint.o src/pmc/bignum.o src/pmc/complex.o src/pmc/string.o src/pmc/boolean.o src/pmc/fixedintegerarray.o src/pmc/iterator.o src/pmc/fixedstringarray.o src/pmc/hash.o src/pmc/orderedhash.o src/pmc/oplib.o src/pmc/opcode.o src/pmc/addrregistry.o src/pmc/arrayiterator.o src/pmc/bytebuffer.o src/pmc/callcontext.o src/pmc/capture.o src/pmc/class.o src/pmc/eventhandler.o src/pmc/exceptionhandler.o src/pmc/exporter.o src/pmc/filehandle.o src/pmc/fixedbooleanarray.o src/pmc/fixedfloatarray.o src/pmc/fixedpmcarray.o src/pmc/handle.o src/pmc/hashiterator.o src/pmc/hashiteratorkey.o src/pmc/imageiofreeze.o src/pmc/imageiosize.o src/pmc/imageiostrings.o src/pmc/imageiothaw.o src/pmc/lexinfo.o src/pmc/mappedbytearray.o src/pmc/multisub.o src/pmc/namespace.o src/pmc/nativepccmethod.o src/pmc/object.o src/pmc/orderedhashiterator.o src/pmc/packfile.o src/pmc/packfileannotation.o src/pmc/packfileannotations.o src/pmc/packfileconstanttable.o src/pmc/packfiledebug.o src/pmc/packfiledirectory.o src/pmc/packfilerawsegment.o src/pmc/packfilesegment.o src/pmc/pmcproxy.o src/pmc/resizablebooleanarray.o src/pmc/resizablefloatarray.o src/pmc/resizableintegerarray.o src/pmc/resizablepmcarray.o src/pmc/resizablestringarray.o src/pmc/role.o src/pmc/scalar.o src/pmc/scheduler.o src/pmc/schedulermessage.o src/pmc/sockaddr.o src/pmc/socket.o src/pmc/stringbuilder.o src/pmc/stringhandle.o src/pmc/stringiterator.o src/pmc/task.o src/pmc/threadinterpreter.o src/pmc/undef.o src/string/encoding/shared.o src/string/encoding/tables.o src/string/encoding/null.o src/string/encoding/ascii.o src/string/encoding/latin1.o src/string/encoding/binary.o src/string/encoding/utf8.o src/string/encoding/utf16.o src/string/encoding/ucs2.o src/string/encoding/ucs4.o compilers/imcc/imcparser.o compilers/imcc/imclexer.o compilers/imcc/imc.o compilers/imcc/main.o compilers/imcc/symreg.o compilers/imcc/instructions.o compilers/imcc/cfg.o compilers/imcc/reg_alloc.o compilers/imcc/sets.o compilers/imcc/debug.o compilers/imcc/optimizer.o compilers/imcc/pbc.o compilers/imcc/parser_util.o compilers/imcc/pcc.o -lm -lgmp -lreadline -lintl 
powerpc-apple-darwin8-g++-4.0.1: Freakin: No such file or directory
powerpc-apple-darwin8-g++-4.0.1: Parrot/blib/lib: No such file or directory
powerpc-apple-darwin8-g++-4.0.1: Freakin: No such file or directory
powerpc-apple-darwin8-g++-4.0.1: Parrot/blib/lib/libparrot.dylib: No such file or directory
make: *** [blib/lib/libparrot.2.11.0.dylib] Error 1

Is it possible that the revision works with one whitespace in a path -- but not two?

Thank you very much.

kid51

follow-up: ↓ 7   Changed 11 years ago by jkeenan

Some more data from lib/Parrot/Config/Generated.pm:

On linux/i386, trying to build in a directory called My Freakin Parrot:: 

grep -n Freakin lib/Parrot/Config/Generated.pm 
79:  'build_dir' => '/home/jimk/gitwork/My Freakin Parrot',
109:  'coveragedir' => '/home/jimk/gitwork/My Freakin Parrot',
294:  'libparrot_linkflags' => '-L"/home/jimk/gitwork/My Freakin Parrot/blib/lib" -lparrot',
355:  'rpath_blib' => '-Wl,-rpath="/home/jimk/gitwork/My Freakin Parrot"/blib/lib',

Note the weird value for rpath_blib. Notwithstanding this, I built and tested successfully.

On Darwin/PPC, trying to build in a directory called My Parrot: 

$ grep -n My lib/Parrot/Config/Generated.pm
77:  'build_dir' => '/Users/jimk/gitwork/My Parrot',
107:  'coveragedir' => '/Users/jimk/gitwork/My Parrot',
286:  'ldflags' => '-L/usr/local/lib -L/opt/local/lib -L/Users/jimk/gitwork/My Parrot/blib/lib -L/sw/lib',
292:  'libparrot_linkflags' => '-L"/Users/jimk/gitwork/My Parrot/blib/lib" -lparrot',
295:  'libparrot_soname' => '-install_name /Users/jimk/gitwork/My Parrot/blib/lib/libparrot.dylib',
353:  'rpath_blib' => '-L"/Users/jimk/gitwork/My Parrot"/blib/lib',

Here I failed with: 

powerpc-apple-darwin8-g++-4.0.1: Parrot/blib/lib: No such file or directory
powerpc-apple-darwin8-g++-4.0.1: Parrot/blib/lib/libparrot.dylib: No such file or directory
make: *** [blib/lib/libparrot.2.11.0.dylib] Error 1

So I think this problem has to do with the way different OSes add to ldflags, libparrot_linkflags, libparrot_soname and rpath_blib.

kid51

in reply to: ↑ 6   Changed 11 years ago by jkeenan

Replying to jkeenan:

So I think this problem has to do with the way different OSes add to ldflags, libparrot_linkflags, libparrot_soname and rpath_blib.

I have created the spaces_in_paths branch to work on this problem. I am trying to insert double-quotes in appropriate locations as per Nick's work in config/inter/libparrot.pm.

kid51

  Changed 11 years ago by jkeenan

On Darwin and Linux, I have had success so far with this patch in the spaces_in_paths branch: 

$ git diff master...spaces_in_paths | cat
diff --git a/config/gen/makefiles/root.in b/config/gen/makefiles/root.in
index 3ab392e..6f841ce 100644
--- a/config/gen/makefiles/root.in
+++ b/config/gen/makefiles/root.in
@@ -859,7 +859,7 @@ $(PARROT) : frontend/parrot/main$(O) $(GEN_HEADERS) $(LIBPARROT) \
     $(MINIPARROT)
        $(LINK) @ld_out@$@ \
        frontend/parrot/main$(O) src/parrot_config$(O) src/longopt$(O) \
-       @rpath_blib@ $(ALL_PARROT_LIBS) $(LINKFLAGS) $(LINK_DYNAMIC)
+       $(RPATH_BLIB) $(ALL_PARROT_LIBS) $(LINKFLAGS) $(LINK_DYNAMIC)
 #IF(win32):    if exist $@.manifest mt.exe -nologo -manifest $@.manifest -outputresource:$@;1
 
 $(PBC_TO_EXE) : $(DEV_TOOLS_DIR)/pbc_to_exe.pir runtime/parrot/library/config.pir $(PARROT) $(DYNEXT_DIR)/os$(LOAD_EXT) $(DYNEXT_DIR)/file$(LOAD_EXT)
@@ -885,7 +885,7 @@ $(PARROT_CONFIG) : $(DEV_TOOLS_DIR)/parrot-config.pir $(PARROT) $(PBC_TO_EXE)
 $(MINIPARROT) : frontend/parrot/main$(O) include/parrot/api.h include/parrot/longopt.h $(LIBPARROT) \
     src/null_config$(O) src/longopt$(O)
        $(LINK) @ld_out@$@ frontend/parrot/main$(O) src/null_config$(O) src/longopt$(O) \
-       @rpath_blib@ $(ALL_PARROT_LIBS) $(LINKFLAGS)
+       $(RPATH_BLIB) $(ALL_PARROT_LIBS) $(LINKFLAGS)
 #IF(win32):    if exist $@.manifest mt.exe -nologo -manifest $@.manifest -outputresource:$@;1
 
 $(INSTALLABLEPARROT) : frontend/parrot/main$(O) $(GEN_HEADERS) $(LIBPARROT) \
@@ -1054,7 +1054,7 @@ $(PDB) : frontend/parrot_debugger/main$(O) src/parrot_config$(O) $(LIBPARROT)
        $(LINK) @ld_out@$@ \
     frontend/parrot_debugger/main$(O) \
     src/parrot_config$(O) \
-    @rpath_blib@ $(ALL_PARROT_LIBS) $(LINKFLAGS)
+    $(RPATH_BLIB) $(ALL_PARROT_LIBS) $(LINKFLAGS)
 #IF(win32):    if exist $@.manifest mt.exe -nologo -manifest $@.manifest -outputresource:$@;1
 
 $(INSTALLABLEPDB) : frontend/parrot_debugger/main$(O) $(LIBPARROT) src/parrot_config$(O)
@@ -1078,7 +1078,7 @@ $(DIS) : src/pbc_disassemble$(O) $(LIBPARROT)
        $(LINK) @ld_out@$@ \
     src/pbc_disassemble$(O) \
     src/longopt$(O) \
-    @rpath_blib@ $(ALL_PARROT_LIBS) $(LINKFLAGS)
+    $(RPATH_BLIB) $(ALL_PARROT_LIBS) $(LINKFLAGS)
 #IF(win32):    if exist $@.manifest mt.exe -nologo -manifest $@.manifest -outputresource:$@;1
 
 $(INSTALLABLEDIS) : src/pbc_disassemble$(O) \
@@ -1099,7 +1099,7 @@ $(PDUMP) : frontend/pbc_dump/main$(O) frontend/pbc_dump/packdump$(O) $(LIBPARROT
        $(LINK) @ld_out@$@ \
     frontend/pbc_dump/main$(O) \
     src/longopt$(O) \
-    frontend/pbc_dump/packdump$(O) @rpath_blib@ $(ALL_PARROT_LIBS) $(LINKFLAGS)
+    frontend/pbc_dump/packdump$(O) $(RPATH_BLIB) $(ALL_PARROT_LIBS) $(LINKFLAGS)
 #IF(win32):    if exist $@.manifest mt.exe -nologo -manifest $@.manifest -outputresource:$@;1
 
 frontend/pbc_dump/main$(O) : \
@@ -1132,7 +1132,7 @@ $(PBC_MERGE) : frontend/pbc_merge/main$(O) $(LIBPARROT) src/parrot_config$(O)
     frontend/pbc_merge/main$(O) \
     src/parrot_config$(O) \
     src/longopt$(O) \
-    @rpath_blib@ $(ALL_PARROT_LIBS) $(LINK_DYNAMIC) $(LINKFLAGS)
+    $(RPATH_BLIB) $(ALL_PARROT_LIBS) $(LINK_DYNAMIC) $(LINKFLAGS)
 #IF(win32):    if exist $@.manifest mt.exe -nologo -manifest $@.manifest -outputresource:$@;1
 
 $(INSTALLABLEPBC_MERGE) : frontend/pbc_merge/main$(O) $(LIBPARROT) $(INSTALLABLECONFIG)
@@ -2871,7 +2871,7 @@ win32-inno-installer : world installable
 ###############################################################################
 
 exec : src/parrot_config$(O) $(LIBPARROT)
-       $(LINK) @ld_out@$(EXEC)$(EXE) $(EXEC)$(O) src/parrot_config$(O) @rpath_blib@ $(ALL_PARROT_LIBS) $(LINKFLAGS)
+       $(LINK) @ld_out@$(EXEC)$(EXE) $(EXEC)$(O) src/parrot_config$(O) $(RPATH_BLIB) $(ALL_PARROT_LIBS) $(LINKFLAGS)
 
 ###### OS depend targets ##########
 
diff --git a/config/init/hints/darwin.pm b/config/init/hints/darwin.pm
index 2f6184f..82820fd 100644
--- a/config/init/hints/darwin.pm
+++ b/config/init/hints/darwin.pm
@@ -43,7 +43,7 @@ sub runstep {
     _set_deployment_environment();
 
     my $lib_dir = $conf->data->get('build_dir') . "/blib/lib";
-    $flagsref->{ldflags} .= " -L$lib_dir";
+    $flagsref->{ldflags} .= ' -L"' . $lib_dir . '"';
 
     if ($ENV{'MACOSX_DEPLOYMENT_TARGET'} eq '10.6') {
         $flagsref->{ccflags} .= ' -pipe -fno-common ';
@@ -91,9 +91,9 @@ sub runstep {
         libparrot_shared_alias => "libparrot$share_ext",
         rpath                  => "-L",
         libparrot_soname       => "-install_name "
-            . $lib_dir
+            . '"'. $lib_dir
             . '/libparrot'
-            . $conf->data->get('share_ext')
+            . $conf->data->get('share_ext') . '"'
     );
 }
 
diff --git a/config/inter/libparrot.pm b/config/inter/libparrot.pm
index ab2f541..03af147 100644
--- a/config/inter/libparrot.pm
+++ b/config/inter/libparrot.pm
@@ -83,10 +83,10 @@ sub runstep {
     $conf->data->set( rpath_blib => ( ! $disable_rpath
                                      && $parrot_is_shared
                                      && $conf->data->get('rpath') )
-        ? $conf->data->get('rpath')
-            . '"' . $conf->data->get('build_dir') . '"'
+        ? '"' . $conf->data->get('rpath')
+            . $conf->data->get('build_dir')
             . '/'
-            . $conf->data->get('blib_dir')
+            . $conf->data->get('blib_dir') . '"'
         : ''
     );

I suspect that we should make similar changes in all the config/init/hints/<os>.pm files.

kid51

  Changed 11 years ago by nwellnhof

  • owner changed from nwellnhof to jkeenan
  • status changed from assigned to new
  • summary changed from Parrot doesn't build on Win32 with spaces in directory name to Parrot doesn't build with spaces in directory name

kid51: Your fixes work for me. Changing summary and reassigning to you.

  Changed 11 years ago by jkeenan

  • status changed from new to assigned

  Changed 11 years ago by jkeenan

Running make fulltest revealed instances where building in a directory with spaces in the path caused test failures, mostly in make buildtools_tests and make install_tests. I am working on fixing them in branch.

If anyone wants to help out by running make fulltest the spaces_in_paths branch on Win32, that would be appreciated.

Thank you very much.

kid51

Changed 11 years ago by jkeenan

diff between spaces_in_path branch and master

follow-up: ↓ 13   Changed 11 years ago by jkeenan

I got very close to solving this problem -- only to run into the kind of problem we used to have frequently.

On Linux/i386, I merged the spaces_in_path branch into master. (See attached diff; I have not pushed the merge to github.) In a directory that had spaces in its path, I then configured and installed like this: 

$> pwd
/home/user/gitwork/My\ Freakin\ Parrot
$> perl Configure.pl --prefix=/tmp/pseudoinstall
$> make install

(By this point, I was passing make fulltest.)

I then went to the prefix directory and called: 

./bin/parrot --version

... and got the expected output. I then went back to my build directory, called make realclean, went back to the prefix directory, and again called ./bin/parrot --version. Once again, I got the expected output.

I then moved to Darwin/PPC, pulled the latest changes from the branch and (without attempting a merge into master), replicated the above procedure closely. I was able to build Parrot in the prefix directory and get the expected version output. 

[pseudoinstall] 507 $ ./bin/parrot --version
This is Parrot version 2.11.0-devel built for ppc-darwin.
Copyright (C) 2001-2011, Parrot Foundation.

This code is distributed under the terms of the Artistic License 2.0.
For more details, see the full text of the license in the LICENSE file
included in the Parrot source tree.

However, when I then called make realclean and then called ./bin/parrot --version from the install directory, I got this output: 

[pseudoinstall] 508 $ ./bin/parrot --version
dyld: Library not loaded: /Users/jimk/gitwork/My Freakin Parrot/blib/lib/libparrot.dylib
  Referenced from: /Users/jimk/work/pseudoinstall/./bin/parrot
  Reason: image not found
Trace/BPT trap

The install directory is still somehow dependent on the build directory.

Can anyone help? The spaces_in_paths branch is up-to-date on github, so I would appreciate someone taking a look. config/init/hints/darwin.pm is something which will need inspection, but the problem may not be there. In any event, if I observe this problem on Darwin there's a chance that it will crop up on FreeBSD as well. So I would really like to nail this.

Thank you very much.

kid51

in reply to: ↑ 12 ; follow-up: ↓ 14   Changed 11 years ago by jkeenan

Replying to jkeenan:

The install directory is still somehow dependent on the build directory.

It turns out that this problem (a) has nothing to do with the problem of spaces in paths and (b) has been lurking in Darwin all along. Working in master, I configured in a directory that had no spaces in its path. I then installed. When I called ./bin/parrot --version I got the expected output. I then called make realclean in the build directory and re-called ./bin/parrot --version. I got this output: 

[pseudoinstall] 529 $ ./bin/parrot --version
dyld: Library not loaded: /Users/jimk/gitwork/parrot/blib/lib/libparrot.dylib
  Referenced from: /Users/jimk/work/pseudoinstall/./bin/parrot
  Reason: image not found
Trace/BPT trap

Since the problem I'm having on Darwin has to do with Darwin and not this ticket, I will merge the spaces_in_paths branch into master and open a new ticket for the Darwin problem.

Thank you very much.

kid51

in reply to: ↑ 13 ; follow-up: ↓ 15   Changed 11 years ago by jkeenan

  • patch set to applied

Replying to jkeenan:

Since the problem I'm having on Darwin has to do with Darwin and not this ticket, I will merge the spaces_in_paths branch into master and open a new ticket for the Darwin problem.

This was merged. It turns out that we have encountered the Darwin problem previously: see TT #1890.

I will keep this ticket open for 2-3 days for reports of any further problems that are specific to "spaces in paths".

Thank you very much.

kid51

in reply to: ↑ 14   Changed 11 years ago by jkeenan

  • status changed from assigned to closed
  • resolution set to fixed

Replying to jkeenan:

I will keep this ticket open for 2-3 days for reports of any further problems that are specific to "spaces in paths".

No complaints received; closing.

Note: See TracTickets for help on using tickets.