SVN misclassifying .t files as binary, preventing 'svn blame'

[jkeenan]

Tonight I tried to get a line-by-line SVN revision history on a '.t' file, only to be told that SVN had to skip that file because it was a binary:

$ svn blame t/compilers/imcc/syn/macro.t 
Skipping binary file: 't/compilers/imcc/syn/macro.t'

I hypothesized that this was a problem with all files in the directory, but there are several that do 'svn blame' without incident.

pmichaud hypothesized that it was the SVN property 'svn:mime-type' which was set to 'text/plain' and that there may be binary characters within the file. I couldn't see any (but I'm not an expert with 'od'), but we set that property on lots of files.

I investigated further. This problem is widespread, at least among files with names ending in .t.

[li11-226:parrot] 654 $ fnsa ./t '*.t' | xargs svn blame | grep Skipping 
Skipping binary file: 't/pmc/signal.t'
Skipping binary file: 't/pmc/boolean.t'
Skipping binary file: 't/pmc/timer.t'
Skipping binary file: 't/pmc/globals.t'
Skipping binary file: 't/pmc/fixedstringarray.t'
Skipping binary file: 't/pmc/eval.t'
Skipping binary file: 't/pmc/sub.t'
Skipping binary file: 't/pmc/iterator.t'
Skipping binary file: 't/pmc/integer.t'
Skipping binary file: 't/pmc/fixedpmcarray.t'
Skipping binary file: 't/pmc/complex.t'
Skipping binary file: 't/pmc/fixedintegerarray.t'
Skipping binary file: 't/pmc/array.t'
Skipping binary file: 't/pmc/resizablepmcarray.t'
Skipping binary file: 't/pmc/hash.t'
Skipping binary file: 't/pmc/threads.t'
Skipping binary file: 't/pmc/namespace.t'
Skipping binary file: 't/pmc/resizableintegerarray.t'
Skipping binary file: 't/pmc/sarray.t'
Skipping binary file: 't/pmc/resizablefloatarray.t'
Skipping binary file: 't/pmc/coroutine.t'
Skipping binary file: 't/pmc/pmc.t'
Skipping binary file: 't/pmc/prop.t'
Skipping binary file: 't/pmc/freeze.t'
Skipping binary file: 't/pmc/intlist.t'
Skipping binary file: 't/pmc/orderedhash.t'
Skipping binary file: 't/pmc/key.t'
Skipping binary file: 't/pmc/multiarray.t'
Skipping binary file: 't/pmc/fixedbooleanarray.t'
Skipping binary file: 't/pmc/undef.t'
Skipping binary file: 't/pmc/float.t'
Skipping binary file: 't/pmc/string.t'
Skipping binary file: 't/pmc/io.t'
Skipping binary file: 't/pmc/sys.t'
Skipping binary file: 't/pmc/resizablebooleanarray.t'
Skipping binary file: 't/pmc/object-meths.t'
Skipping binary file: 't/pmc/fixedfloatarray.t'
Skipping binary file: 't/pmc/env.t'
Skipping binary file: 't/pmc/objects.t'
Skipping binary file: 't/pmc/exception.t'
Skipping binary file: 't/pmc/managedstruct.t'
Skipping binary file: 't/pmc/multisub.t'
Skipping binary file: 't/pmc/resizablestringarray.t'
Skipping binary file: 't/pmc/tqueue.t'
Skipping binary file: 't/pmc/nci.t'
Skipping binary file: 't/pmc/delegate.t'
Skipping binary file: 't/pmc/bigint.t'
Skipping binary file: 't/pmc/ref.t'
Skipping binary file: 't/stress/gc.t'
Skipping binary file: 't/src/basic.t'
Skipping binary file: 't/src/extend.t'
Skipping binary file: 't/src/exit.t'
Skipping binary file: 't/benchmark/benchmarks.t'
Skipping binary file: 't/dynpmc/gdbmhash.t'
Skipping binary file: 't/dynpmc/foo.t'
Skipping binary file: 't/dynpmc/rational.t'
Skipping binary file: 't/compilers/imcc/imcpasm/cfg.t'
Skipping binary file: 't/compilers/imcc/imcpasm/opt0.t'
Skipping binary file: 't/compilers/imcc/imcpasm/opt1.t'
Skipping binary file: 't/compilers/imcc/imcpasm/opt2.t'
Skipping binary file: 't/compilers/imcc/imcpasm/optc.t'
Skipping binary file: 't/compilers/imcc/imcpasm/pcc.t'
Skipping binary file: 't/compilers/imcc/imcpasm/sub.t'
Skipping binary file: 't/compilers/imcc/syn/op.t'
Skipping binary file: 't/compilers/imcc/syn/file.t'
Skipping binary file: 't/compilers/imcc/syn/pod.t'
Skipping binary file: 't/compilers/imcc/syn/bsr.t'
Skipping binary file: 't/compilers/imcc/syn/const.t'
Skipping binary file: 't/compilers/imcc/syn/eval.t'
Skipping binary file: 't/compilers/imcc/syn/tail.t'
Skipping binary file: 't/compilers/imcc/syn/objects.t'
Skipping binary file: 't/compilers/imcc/syn/clash.t'
Skipping binary file: 't/compilers/imcc/syn/keyed.t'
Skipping binary file: 't/compilers/imcc/syn/macro.t'
Skipping binary file: 't/compilers/imcc/syn/labels.t'
Skipping binary file: 't/compilers/imcc/syn/pcc.t'
Skipping binary file: 't/compilers/imcc/syn/scope.t'
Skipping binary file: 't/compilers/imcc/reg/spill.t'
Skipping binary file: 't/compilers/pge/pge.t'
Skipping binary file: 't/perl/Parrot_IO.t'
Skipping binary file: 't/perl/Parrot_Docs.t'
Skipping binary file: 't/perl/Parrot_Distribution.t'
Skipping binary file: 't/library/streams.t'
Skipping binary file: 't/library/md5.t'
Skipping binary file: 't/library/sort.t'
Skipping binary file: 't/library/pcre.t'
Skipping binary file: 't/library/dumper.t'
Skipping binary file: 't/library/parrotlib.t'
Skipping binary file: 't/library/yaml_parser_syck.t'
Skipping binary file: 't/distro/manifest.t'
Skipping binary file: 't/native_pbc/integer.t'
Skipping binary file: 't/native_pbc/string.t'
Skipping binary file: 't/native_pbc/number.t'
Skipping binary file: 't/codingstd/c_cppcomments.t'
Skipping binary file: 't/op/00ff-dos.t'
Skipping binary file: 't/op/spawnw.t'
Skipping binary file: 't/op/globals.t'
Skipping binary file: 't/op/jit.t'
Skipping binary file: 't/op/trans.t'
Skipping binary file: 't/op/gc.t'
Skipping binary file: 't/op/string_cs.t'
Skipping binary file: 't/op/integer.t'
Skipping binary file: 't/op/comp.t'
Skipping binary file: 't/op/interp.t'
Skipping binary file: 't/op/jitn.t'
Skipping binary file: 't/op/lexicals.t'
Skipping binary file: 't/op/string.t'
Skipping binary file: 't/op/calling.t'
Skipping binary file: 't/op/basic.t'
Skipping binary file: 't/op/64bit.t'
Skipping binary file: 't/op/ifunless.t'
Skipping binary file: 't/op/number.t'
Skipping binary file: 't/op/hacks.t'
Skipping binary file: 't/op/stringu.t'
Skipping binary file: 't/op/literal.t'
Skipping binary file: 't/op/time.t'
Skipping binary file: 't/op/debuginfo.t'
Skipping binary file: 't/op/bitwise.t'
Skipping binary file: 't/op/00ff-unix.t'
Skipping binary file: 't/op/arithmetics.t'

Does anyone have any idea what is wrong?

Thank you very much. kid51

[coke]

It's not every .t file; t/compilers/imcc/syn/errors.t in the same directory as macro.t blames fine.

My guess is that when these files were initially added into the repository, they did not have the svn:mime-type set properly. looks like this was added for macro.t in r7803.

svn blame -r7804:HEAD macro.t

works.

so, this is failing for the same reason that we have to add svn:mime-type in the first place.

I wonder if we can change the default sense of what a .t file is on our existing repository; that might fix this issue, and allow us to remove a distro sanity check.

[jkeenan]

 How Subversion classifies a file as binary

[jkeenan]

Seems like this is a long-standing problem with Subversion.  Back in 2004, someone commented on a list:

Seems like the correct behavior would be to determine the MIME type of the file from the MIME type of the revision you're "blaming", not from the pessimistic union of all contributing revisions.

[jkeenan]

Further research leads to a solution:

svn blame --force t/compilers/imcc/syn/macro.t

DWIMs as of Subversion 1.40, 10 September 2006,  as documented here.

Closing ticket.