mk_language_shell.pl and create_language.pl cause confusion

[cotto]

It's confusing to have two scripts that appear to do the same thing without a clear indication why they both exist. We need to differentiate the documentation of these scripts to the point where a potential HLL dev can look the inline POD of both of them and easily decide which one is appropriate for his or her needs.

[jkeenan]

tools/dev/create_language.pl was added to the repository by pmichaud last December:

r38147 | pmichaud | 2009-04-16 01:21:25 -0400 (Thu, 16 Apr 2009) | 13 lines
Add tools/dev/create_language.pl, a substantially modified version 
of tools/dev/mk_language_shell.pl .

... and further maintained by pmichaud:

r42889 | pmichaud | 2009-12-04 12:26:19 -0500 (Fri, 04 Dec 2009) | 3 lines

[tools]  Update create_language.pl to build for nqp-based toolchain.
Add create_language.pl to files in installed Parrot.

tools/dev/mk_language_shell.pl now appears to be generated from tools/dev/mk_language_shell.in. gerd and fperrad appear to be the principal contributors to this program.

Gerd, François, Patrick: Can you comment on any of the issues cotto has raised?

Thank you very much.

kid51

[gerd]

Okay here is my comment without knowing much about it.

mk_language_shell.pl is the one that the PCT-Tutorial in examples/languages/squaak/doc refer to.

create_language.pl seams to be newer.

> perl /usr/lib64/parrot/2.7.0/tools/dev/create_language.pl Blabla Blabla.dir
...
Your new language has been created in the Blabla.dir directory.
To do an initial build and test of the language:

    cd Blabla.dir
    perl Configure.pl [--gen-parrot]
    make
    make test

If the language is created with mk_language_shell.pl, the same is done with this commands:

> cat README 
Language 'Blabla' was created with /usr/lib64/parrot/2.7.0/tools/dev/mk_language_shell.pl, r47087.

    $ parrot setup.pir
    $ parrot setup.pir test

It seams to be a question of design.

That is what I am thinking:

If "create_language.pl" is newer and better then "mk_language_shell.pl" can removed und the PCT Tutorial should be updated to have a nice starting point for potential HLL developer. Perhaps the PCT-Tutorial can refer to both.

[coke]

My vote is to pick ONE. I don't care which one, but having 2 is just confusing to incoming HLL devs.

[gerd]

The basic language files (Action.pm, Compiler.pm ...) generated from both scripts have the same content. So it should be possible to merge "mk_language_shell.pl" and "create_language.pl" and give the result an option to choose the design that should be used for the generating of a language. I will start putting both scripts together. I will call the result of the merge "create_language.pl", because I like this name more.

create_language.pl [--generate-with=make|setup]

I will give the option "generate-with" as default the value "setup". Better names for the option as "generate-with" are welcome.

[jkeenan]

Replying to gerd:

So it should be possible to merge "mk_language_shell.pl" and "create_language.pl" and give the result an option to choose the design that should be used for the generating of a language. I will start putting both scripts together.

Gerd, Thanks for taking this on. I'm just hoping that we can hear from Patrick as to his thoughts around create_language.pl.

kid51

[pmichaud]

On Mon, Sep 06, 2010 at 12:33:12PM -0000, Parrot wrote:
> Comment(by jkeenan):
> 
>  Replying to [comment:4 gerd]:
>  > So it should be possible to merge "mk_language_shell.pl" and
>  > "create_language.pl" and give the result an option to choose the design
>  > that should be used for the generating of a language. I will start putting
>  > both scripts together.
> 
>  Gerd, Thanks for taking this on.  I'm just hoping that we can hear from
>  Patrick as to his thoughts around ''create_language.pl''.

Sorry it's taken me so long to respond -- I've been trying (and largely 
failing) to formulate a positive and clear response.  Overall I'm -0.5 on
the proposal to merge the two scripts into a single tool.

As gerd++ points out earlier in the thread, behind the two tools lie 
strong differences of opinion regarding the mechanics of the build and
install system.  In fact, the primary reason I abandoned mk_language_shell.pl 
and started create_language.pl is because I was extremely uncomfortable 
with the many modifications being made to mk_language_shell.pl, and it was
far easier to start again from scratch with a new tool than to try to
work around the changes being made to the existing one.

At this point I'm still strongly in favor of a build system similar to
what create_language.pl, NQP, and Rakudo currently use, and I don't want 
to feel "encumbered" by setup.pir.  I don't find the proposal to provide
a "make vs. setup" option switch to create_language.pl to be all that
compelling; to a large degree I think it just converts the newbie question
of "Which script do I choose, and why?" to "Which --generate-with= option 
do I choose, and why?".  The fact that --generate-with= is proposed to
default to "setup" makes me even less happy with the proposed merger, as
it basically eliminates the whole reason behind why I started 
create_language.pl in the first place.

The reason my vote is -0.5 instead of a full -1.0 is because I recognize
the current situation is not ideal, and I don't have any strong alternatives
to offer at this time.  I'm also trying to avoid an appearance of
"my way of doing things is the only correct answer", because I do 
acknowledge that there are justifications for the setup.pir approach 
as well and I respect the efforts the setup.pir authors have put into
that system.

So, I'm not wanting to say "absolutely don't go down this path".  I'm
just trying to flag up that I think that trying to develop a single
language creation script that supports setup.pir isn't likely
to work out at all well for the projects I'm working on, I'm not likely
to support the resulting script if it is merged, and that perhaps
we could continue searching for other means of resolving the
existing confusion.

Pm

[gerd]

I don't care about the default value of "--genetate-with=". Setting it to "setup" was only a suggestion. I only think that the "setup.pir" approach should not completely ignored. I think a script with an option is more easy to maintain then two scripts and an option is easier to explain then a reason for two scripts.

The "setup.pir" part could go in a separate subroutine and even in a separate file and this part could be maintained from the setup.pir group.

In consideration to Patick's comment I still suggest the merge:

- by start writing a merge in create_language.pl.in

- generate the file create_language_new.pl from it

- setting --generate-with to "make" as default

- working on it as long everybody agree with it

- then delete create_language.pl and mk_language_shell.pl and let create_language.pl be generated from create_language.pl.in

- I would like that create_language.pl has a correct shebang, is executable and installed in the bin path

Would it be okay if I start to do this? Or may be the composers of the scripts want to do this by themselves? May be someone has a better or complete other idea. Means are welcome.

[dukeleto]

I am +1 to gerd++'s merging of the scripts, which should default to "make". Another possibility is to keep the scripts separate but have them use the same backend code.

At the least, we need some basic tests for these scripts.

As for installing the scripts, I think they should have the word "parrot" in them somewhere, such as "create_parrot_language", if we are going to install them.

[whiteknight]

I vote for the shared backend code. If people insist, we can have two thin front-ends which call the shared backend code. The benefit to this kind of setup is less duplicated logic, easier maintainability, etc. Also, if this is a tool that we're going to be using and relying on in the long term, we need to add unit tests and lots of good documentation and examples (and tests for those examples).

It's probably worth taking some effort to get the design of this tool correct, especially since this is going to be the first introduction to Parrot that many developers are going to have, and if it's an absolute piece of shit a first impression will be created that is going to be hard for us to overcome later. Plus, this tool is going to create a foundation for new projects, and problems in this setup will be difficult for people to correct later.

I suggest the following:

• Merge core logic between the two modules. Add a script with a front-end that takes a switch. Whichever default we use is inconsequential
• Create two new front-end scripts, create_language and mk_language_shell, that are thin wrappers around the core utility to force the switch argument to one value or the other
• Abstract away the generated files and folder structure into some kind of template file format. That way people will be able to use any existing template, or create their own. This may be important on certain platforms where the default setup.pir or makefile might not be correct, or in cases where certain types of project need different templates. One thing that comes to mind here is creating extension/embedding applications or shared bytecode libraries that do not use the same format as a new HLL project, but for which a template would be very valuable.

[dukeleto]

Fixing this ticket will make  http://trac.parrot.org/parrot/ticket/1845 trivially easy, which is a very important ticket to close.

[jkeenan]

Replying to dukeleto:

Fixing this ticket will make TT #1845 trivially easy, which is a very important ticket to close.

That ticket has been closed, but see also TT #2003 and TT #2009.

[jkeenan]

Replying to jkeenan:

duke, Can we get an update on the status of this ticket?

Thank you very much.

kid51