Discuss derived files in the git repo in the doc.

1 files changed, 222 insertions, 5 deletions

diff --git a/doc/gawk.texi b/doc/gawk.texi
index ceea9a92..4bab87e7 100644
--- a/doc/gawk.texi
+++ b/doc/gawk.texi
@@ -712,6 +712,8 @@ particular records in a file and perform operations upon them.
                                    @command{gawk}.
 * New Ports::                      Porting @command{gawk} to a new operating
                                    system.
+* Derived Files::                  Why derived files are kept in the
+                                   @command{git} repository.
 * Future Extensions::              New features that may be implemented one
                                    day.
 * Basic High Level::               The high level view.
@@ -28503,15 +28505,45 @@ the @option{-l} command-line option
 (@pxref{Options}).
 
 @item
-The ability to use GNU-style long-named options that start with @option{--}
+The
+@option{-b},
+@option{-c},
+@option{-C},
+@option{-d},
+@option{-D},
+@option{-e},
+@option{-E},
+@option{-g},
+@option{-h},
+@option{-i},
+@option{-l},
+@option{-L},
+@option{-M},
+@option{-n},
+@option{-N},
+@option{-o},
+@option{-O},
+@option{-p},
+@option{-P},
+@option{-r},
+@option{-S},
+@option{-t},
+and
+@option{-V}
+short options. Also, the
+ability to use GNU-style long-named options that start with @option{--}
 and the
+@option{--assign},
 @option{--bignum},
 @option{--characters-as-bytes},
 @option{--copyright},
 @option{--debug},
 @option{--dump-variables},
-@option{--exec},
+@option{--execle},
+@option{--field-separator},
+@option{--file},
 @option{--gen-pot},
+@option{--help},
 @option{--include},
 @option{--lint},
 @option{--lint-old},
@@ -28525,13 +28557,13 @@ and the
 @option{--sandbox},
 @option{--source},
 @option{--traditional},
+@option{--use-lc-numeric},
 and
-@option{--use-lc-numeric}
-options
+@option{--version}
+long options
 (@pxref{Options}).
 @end itemize
 
-
 @c       new ports
 
 @item
@@ -30311,6 +30343,8 @@ as well as any considerations you should bear in mind.
                                 @command{gawk}.
 * New Ports::                   Porting @command{gawk} to a new operating
                                 system.
+* Derived Files::               Why derived files are kept in the
+                                @command{git} repository.
 @end menu
 
 @node Accessing The Source
@@ -30629,6 +30663,189 @@ operating systems' code that is already there.
 In the code that you supply and maintain, feel free to use a
 coding style and brace layout that suits your taste.
 
+@node Derived Files
+@appendixsubsec Why Generated Files Are Kept In @command{git}
+
+@c From emails written March 22, 2012, to the gawk developers list.
+
+If you look at the @command{gawk} source in the @command{git}
+repository, you will notice that it includes files that are automatically
+generated by GNU infrastructure tools, such as @file{Makefile.in} from
+@command{automake} and even @file{configure} from @command{autoconf}.
+
+This is different from many Free Software projects that do not store
+the derived files, because that keeps the repository less cluttered,
+and it is easier to see the substantive changes when comparing versions
+and trying to understand what changed between commits.
+
+However, there are two reasons why the @command{gawk} maintainer
+likes to have everything in the repository.
+
+First, because it is then easy to reproduce any given version completely,
+without relying upon the availability of (older, likely obsolete, and
+maybe even impossible to find) other tools.
+
+As an extreme example, if you ever even think about trying to compile,
+oh, say, the V7 @command{awk}, you will discover that not only do you
+have to bootstrap the V7 @command{yacc} to do so, but you also need the
+V7 @command{lex}.  And the latter is pretty much impossible to bring up
+on a modern GNU/Linux system.@footnote{We tried. It was painful.}
+
+(Or, let's say @command{gawk} 1.2 required @command{bison} whatever-it-was
+in 1989 and that there was no @file{awkgram.c} file in the repository.  Is
+there a guarantee that we could find that @command{bison} version? Or that
+@emph{it} would build?)
+
+If the repository has all the generated files, then it's easy to just check
+them out and build. (Or @emph{easier}, depending upon how far back we go.
+@code{:-)})
+
+And that brings us to the second (and stronger) reason why all the files
+really need to be in @command{git}.  It boils down to who do you cater
+to---the @command{gawk} developer(s), or the user who just wants to check
+out a version and try it out?
+
+The @command{gawk} maintainer
+wants it to be possible for any interested @command{awk} user in the
+world to just clone the repository, check out the branch of interest and
+build it. Without their having to have the correct version(s) of the
+autotools.@footnote{There is one GNU program that is (in our opinion)
+severely difficult to bootstrap from the @command{git} repository. For
+example, on the author's old (but still working) PowerPC macintosh with
+Mac OS X 10.5, it was necessary to bootstrap a ton of software, starting
+with @command{git} itself, in order to try to work with the latest code.
+It's not pleasant, and especially on older systems, it's a big waste
+of time.
+
+Starting with the latest tarball was no picnic either. The maintainers
+had dropped @file{.gz} and @file{.bz2} files and only distribute
+@file{.tar.xz} files.  It was necessary to bootstrap @command{xz} first!}
+That is the point of the @file{bootstrap.sh} file.  It touches the
+various other files in the right order such that
+
+@example
+# The canonical incantation for building GNU software:
+./bootstrap.sh && ./configure && make
+@end example
+
+@noindent
+will @emph{just work}.
+
+This is extremely important for the @code{master} and
+@code{gawk-@var{X}.@var{Y}-stable} branches.
+
+Further, the @command{gawk} maintainer would argue that it's also
+important for the @command{gawk} developers. When he tried to check out
+the @code{xgawk} branch@footnote{A branch created by one of the other
+developers that did not include the generated files.} to build it, he
+couldn't. (No @file{ltmain.sh} file, and he had no idea how to create it,
+and that was not the only problem.)
+
+He felt @emph{extremely} frustrated.  With respect to that branch,
+the maintainer is no different than Jane User who wants to try to build
+@code{gawk-4.0-stable} or @code{master} from the repository.
+
+Thus, the maintainer thinks that it's not just important, but critical,
+that for any given branch, the above incantation @emph{just works}.
+
+@c So - that's my reasoning and philosophy.
+
+What are some of the consequences and/or actions to take?
+
+@enumerate 1
+@item
+We don't mind that there are differing files in the different branches
+as a result of different versions of the autotools.
+
+@enumerate A
+@item
+It's the maintainer's job to merge them and he will deal with it.
+
+@item
+He is really good at @samp{git diff x y > /tmp/diff1 ; gvim /tmp/diff1} to
+remove the diffs that aren't of interest in order to review code. @code{:-)}
+@end enumerate
+
+@item
+It would certainly help if everyone used the same versions of the GNU tools
+as he does, which in general are the latest released versions of
+@command{automake},
+@command{autoconf},
+@command{bison},
+and
+@command{gettext}.
+
+@ignore
+If it would help if I sent out an "I just upgraded to version x.y
+of tool Z" kind of message to this list, I can do that.  Up until
+now it hasn't been a real issue since I'm the only one who's been
+dorking with the configuration machinery.
+@end ignore
+
+@enumerate A
+@item
+Installing from source is quite easy. It's how the maintainer worked for years
+under Fedora.
+He had @file{/usr/local/bin} at the front of hs @env{PATH} and just did:
+
+@example
+wget http://ftp.gnu.org/gnu/@var{package}/@var{package}-@var{x}.@var{y}.@var{z}.tar.gz
+tar -xpzvf @var{package}-@var{x}.@var{y}.@var{z}.tar.gz
+cd @var{package}-@var{x}.@var{y}.@var{z}
+./configure && make && make check
+make install	# as root
+@end example
+
+@item
+These days the maintainer uses Ubuntu 10.11 which is medium current, but
+he is already doing the above for @command{autoconf} and @command{bison}.
+
+@ignore
+(C. Rant: Recent Linux versions with GNOME 3 really suck. What
+    are all those people thinking?  Fedora 15 was such a bust it drove
+    me to Ubuntu, but Ubuntu 11.04 and 11.10 are totally unusable from
+    a UI perspective. Bleah.)
+@end ignore
+@end enumerate
+
+@ignore
+@item
+If someone still feels really strongly about all this, then perhaps they
+can have two branches, one for their development with just the clean
+changes, and one that is buildable (xgawk and xgawk-buildable, maybe).
+Or, as I suggested in another mail, make commits in pairs, the first with
+the "real" changes and the second with "everything else needed for
+ building".
+@end ignore
+@end enumerate
+
+Most of the above was originally written by the maintainer to other
+@command{gawk} developers.  It raised the objection from one of
+the devlopers ``@dots{} that anybody pulling down the source from
+@command{git} is not an end user.''
+
+However, this is not true. There are ``power @command{awk} users''
+who can build @command{gawk} (using the magic incantation shown previously)
+but who can't program in C.  Thus, the major branches should be
+kept buildable all the time.
+
+It was then suggested that there be a @command{cron} job to create
+nightly tarballs of ``the source.''  Here, the problem is that there
+are source trees, corresponding to the various branches! So,
+nightly tar balls aren't the answer, especially as the repository can go
+for weeks without significant change being introduced.
+
+Fortunately, the @command{git} server can meet this need. For any given
+branch named @var{branchname}, use:
+
+@example
+wget http://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawk-@var{branchname}.tar.gz
+@end example
+
+@noindent
+to retrieve a snapshot of the given branch.
+
+
 @node Future Extensions
 @appendixsec Probable Future Extensions
 @ignore