Module Basics

Latest revision as of 19:38, 26 October 2013

In Lunar parlance, software packages are called modules. The collection of all modules is the moonbase, which is simply a directory (usually /var/lib/lunar/moonbase/) containing sections (i.e. directories) which in turn contain the module directories.

A module is simply a directory containing the scripts necessary to build a software package, and optionally configuration files which may be needed in /etc. Some modules require only a DETAILS file, however this is only the case for a few of the modules in the entire moonbase. In each case, after DETAILS, DEPENDS, and CONFIGURE, where a module can use lunar's default internal function(s), there is no need for a module-specific script.

• DETAILS sets version, source URL(s) and other critical data
• CONFLICTS specifies modules which must (will) be removed by module
• CONFIGURE interactive script where build options can be set
• DEPENDS specifies required and optional packages
• PRE_REMOVE used by lrm; actions which must preceed removal
• PRE_BUILD most often used for patching, unpacking addional source tarballs
• BUILD runs necessary variations on: configure; make; make install
• POST_BUILD install configuration scripts and data.
• POST_INSTALL messages, notes more cleanups, configuration fixes
• POST_REMOVE used by lrm; actions which must follow removal

Note: modules that require changes for 64-bit systems may also have DETAILS.x86_64, etc.

Package Build and Install Scripts

The following scripts are used by lin or indirectly by lunar when building modules.

The DETAILS script

Every module is required to have at least a DETAILS file. A minimal DETAILS may appear as follows: (/var/lib/lunar/moonbase/editors/emacs/DETAILS)

          MODULE=emacs
         VERSION=21.3
          SOURCE=$MODULE-$VERSION.tar.gz
      SOURCE_URL=$GNU_URL/$MODULE
      SOURCE_URL=ftp://ftp.gnu.org/pub/gnu/$MODULE
      SOURCE_VFY=sha1:94d7ae9cb3aef05159cfff148265fc9ce0973980
        WEB_SITE=http://www.gnu.org/software/emacs
         ENTERED=20010922
         UPDATED=20020529
           SHORT="the extensible, self-documenting real-time display editor"

cat << EOF
Emacs is the extensible, customizable, self-documenting real-time
display editor. 
EOF

Other SOURCE_URLS in lunar :

• $GNOME_URL : GNOME mirrors
• $GNU_URL : GNU mirrors
• $KDE_URL : KDE mirrors
• $SFORGE_URL : SourceForge mirrors
• $LRESORT_URL : lunar-linux mirrors
• $XFREE86_URL : XFree86 mirrors
• $XORG_URL : x.org packages mirrors
• $KERNEL_URL : kernel, kernel tools mirrors
• $PATCH_URL : the place for *.patch or *.diff files which are not downloadable from other public hosts.
  • See Module Guidelines about creating patches for use in the BUILD file.
  • Such patches may be sent unpacked to Lunar developers mailing list: lunar-dev (at) lunar-linux (dot) org so that a Lunar developer can check it, pack it as *.bz2 archive then upload it to $PATCH_URL server.
  • Otherwise it may be copied to the module directory to be included as part of the lvu submit

The use of _URL mirrors is very important because:

• downloads are faster because mirror is usually very close to user
• if any mirror server fails there is another one on list so user is not left with broken download
• international Internet connections are less blocked

Other optional fields :

• If the application does not compile on more than one thread, add: PSAFE="no" or PSAFE=no
• If the tarball is not extracting into the default $MODULE-$VERSION (e.g. emacs-21.7), add: SOURCE_DIRECTORY=$BUILD_DIRECTORY/"$MODULE"_"$VERSION"_src
• If you want your name in the module and be listed as maintainer and like to be notified when somebody modifies the module, add : MAINTAINER=youremailadress
• If you have more than one SOURCE_URL, list them as: SOURCE_URL[0]=, SOURCE_URL[1]=, SOURCE_URL[2]=, ...
• If you have more than one SOURCE, list them as: SOURCE ,SOURCE2, SOURCE3, ... (don't forget to tell the BUILD script what to do with it...) and add a SOURCE2_URL as well...
• If you want to force lunar to use the gcc4 compiler for this module, add: LUNAR_COMPILER=GCC_4_0 or GCC_3_4 for gcc3

Update the UPDATED field only when you have added something that will change the compile behaviour, updated the version or you added configure options... don't change it just for outlining the DETAILS or BUILD or... file, so in general don't force ppl to recompile if all you did what tweaking around a bit...

• Check Module_Guidelines for special handling of UPDATED for *-cvs or *-svn modules in zbeta.

The SOURCE_VFY field can be used to check that the downloaded source file has not been corrupted, or changed, since the module was last updated. It should be omitted for modules that download snapshots of sources from svn, cvs, git repositories or similar because the sources could change between downloads. sha1 checksums are preferred, md5sums can be listed as SOURCE_VFY=md5:

If you find a MAINTAINER field please respect this and notify the person listed there that you will or have updated his module.

With comments, default values:

          MODULE=emacs                                       # Module name, yes it's redundant
         VERSION=21.3                                        # Version, changes *often*
          SOURCE=$MODULE-$VERSION.tar.gz                     # Source filename
SOURCE_DIRECTORY=$BUILD_DIRECTORY/$MODULE-$VERSION           # Where source unpacks
                                                             # ($BUILD_DIRECTORY=/usr/src)
   SOURCE_URL[0]=$GNU_URL/$MODULE                            # Download URL
   SOURCE_URL[1]=ftp://ftp.gnu.org/pub/gnu/$MODULE   # Alternate URL(s)
      SOURCE_VFY=sha1:94d7ae9cb3aef05159cfff148265fc9ce0973980 # Sets sha1 hash or pgp/gpg sig url
        WEB_SITE=http://www.gnu.org/software/emacs           # where to learn more
         ENTERED=20010922                                    # First appearance in moonbase
         UPDATED=20020529                                    # Date of latest change.
                                                             # Force update by setting this

# The remaining lines are used for input to the 'lvu what' command
# and are best copied from the source-maintainer's own description.

           SHORT="the extensible, self-documenting real-time display editor"

cat << EOF
Emacs is the extensible, customizable, self-documenting real-time
display editor. 
EOF

The DEPENDS script

The DEPENDS script is essential to configuration management, and is the key to the overall operation of lunar. Dependencies should be exactly specified, preferably not assuming the presence of any other modules, while knowing the sub-dependencies of the modules which are added and not adding those explictly where not needed.

Warning - Getting this right is difficult. Because the state of installed packages may vary widely, it's important to have a good understanding of what might be or not be installed on a target system.

Note - By convention Lunar does not include the X Window System, xfree86 or XOrg, in any dependency. There are two reasons for this choice. First we expect that users must understand that to use a graphical application locally, the X Window System must be installed. Second, due to the sligtly unusual definition of client and server used by X11, it is often in fact possible to build graphical applications and tools for remote display, without the server components locally installed. At some future date we may elect to provide a client-only installation of xfree86.

DEPENDS may include both required and optional dependencies. The depends function statement simply determines one required package. The optional_depends function is a little more complex. It consists of the required package, necessary --options to give to ./configure for yes and no respectively, an explanatory comment telling the user the purpose of the option being presented, and an optional preferred choice. Normally the preferred choice depends on the modules state of the required package, this option can change that behaviour and always default to y or n. A typical DEPENDS file might appear as follows: (/var/lib/lunar/moonbase/devel/subversion/DEPENDS)

depends zlib
depends openssl
optional_depends "db4" "--with-berkeley-db"  ""   "for creating local repositories"
#                  ^            ^            ^                  ^
#                  |            |            |                  |
#     optional package       if "Y"       if "N"       explanatory comment
#                        { ./configure strings }

Many old modules use '&&' in DEPENDS to be consistent with BUILD files, but the '&&' delimiter is not required and new DEPENDS files should not use it.

• Important - You may not use if module_installed... or other general bash programming in the DEPENDS file to handle conditional dependencies. This is to keep the dependency tracking code in the lunar tools as simple and fast as possible.

Aliases

Aliases are a mean to select a generic module. When you need a functionnality that can be provided by two or more softwares, you can select one of them to provide a correct dependency.

Example /var/lib/lunar/moonbase/aliases:

%APACHE:apache apache2 apache-mod_ssl
%FAM:fam gamin
%GECKO_RENDERER: firefox thunderbird mozilla
%GHOSTSCRIPT:espgs ghostscript
%MTA:postfix exim sendmail esmtp
%SLANG:slang slang2
%X:XOrg XOrg-test xfree86 xfree86-beta
%XMLRENDERER: libxml2 expat
%XSCREENSAVER:xscreensaver xscreensaver-gtk1 xscreensaver-kde

For example you can choose %X instead of XOrg in a module that would depends on any X server:

depends %X

The CONFLICTS script

This script is simply used to specify modules which will be removed when a given module is installed. An example would be: (/var/lib/lunar/moonbase/editors/elvis/CONFLICTS)

conflicts vim

The CONFIGURE script

The CONFIGURE script is used to collect interactive input from the user on optional parameters for the software build. use the query function and provide a default answer to each question. The results of the answers are then used to store configuration variables needed in configuration state files. An a simple example might be: (/var/lib/lunar/moonbase/crypto/gnupg/CONFIGURE)

if ! grep -q CONFIGURED $MODULE_CONFIG ; then
  if query "Enable experimental external HKP keyserver interface? " n ; then
    OPTS="$OPTS --enable-external-hkp"
  fi
  echo 'CONFIGURED="y"' >> $MODULE_CONFIG
  echo 'OPTS='\"$OPTS\" >> $MODULE_CONFIG
fi

Another way is using mquery like the lilo module does:

mquery RUN_LILO "Run LILO automatically upon LILO upgrades?" y

mquery ENABLE_FOO "Enable foo?" n "--enable-foo --enable-foo2" "--disable-foo --disable-foo2"

Where "No" would be the default answer for the user. When he chooses to enable-foo, then answer "yes" would be stored in the ENABLE_FOO variable and --enable-foo and --enable-foo2 will get added to the ./configure command in the BUILD script.

The PRE_BUILD script

PRE_BUILD is used where special processing is needed before undertaking the actual build steps. Typical requirements include unpacking multiple sources, creating necessary system or source-tree direcotries and applying source patches. And example would be: (/var/lib/lunar/moonbase/doc-tools/html2db/PRE_BUILD)

mk_source_dir $SOURCE_DIRECTORY  &&
unpack $SOURCE                   &&
cd $MODULE
unpack $SOURCE2
cd tidy
patch_it $SOURCE_CACHE/$SOURCE3 0
cd /usr/src/$MODULE

The BUILD script

BUILD is used where the default_build() function does not work for a given software package. For reference the commands run by default are:

Function default_build() calls default_config which executes:

./configure  --build=$BUILD            \
             --prefix=/usr             \
             --sysconfdir=/etc         \
             --localstatedir=/var      \
             --infodir=/usr/share/info \
             --mandir=/usr/share/man   \
             $OPTS

Next, default_build() calls default_make which executes:

make &&
prepare_install &&
make install

The prepare_install should only be called immediately before the make install phase, so that the package management system only tracks the installed files and not any intermediate build files.

2011-02-20: Be aware that several modules in the moonbase appear to get this wrong, so take care!

If you have modules from git,svn or cvs, the configure script is missing most of the time. You can then use the default_cvs_build() function. The commands run are:

./autogen.sh --prefix=/usr  
   

it will then call default_make()

Where this build configuration does not work, the BUILD script is used to provide the needed steps. About 75% of modules need a BUILD script. Two examples include: (/var/lib/lunar/moonbase/archive/gzip/BUILD)

./configure --build=$BUILD            \
            --prefix=/usr             \
            --bindir=/bin             \
            --infodir=/usr/share/info \
            --mandir=/usr/share/man   &&
make &&
prepare_install &&
make bindir=/bin install

and: (/var/lib/lunar/moonbase/editors/ex/BUILD)

cd $SOURCE_DIRECTORY                    &&
sedit 's/usr.local/usr/' Makefile       &&
sedit 's/= man/= share\/man/' Makefile  &&
sedit 's/ucb/bin/' Makefile             &&
sedit 's/= termlib/= ncurses/' Makefile &&
make                                    &&
prepare_install                         &&
make install

The first example is a build which needs non-standard 'configure' and 'make install' commands. The second is a build which does not use gnu auto-tools' 'configure' script.

In the examples above, the BUILD scripts contain multiple commands separated by '&&' to ensure that execution stops as soon as any command fails. The '&&' are 'and' operations: the command after the '&&' will only be run if the one before the '&&' completed successfully. The use of '&&' is required in multi-command BUILD scripts.

The POST_BUILD script

POST_BUILD runs in place of the default_post_build routine which is used to install minor documentation and transfer/enable initialization scripts and similar system data, mostly into /etc.

POST_BUILD script usage is deprecated. You should install your config files in BUILD (remember not to overwrite previous config files!) or install defaults from POST_INSTALL (again, do not overwrite present files!). The ability to use a POST_BUILD script is purely for certain internal functions.

The POST_INSTALL script

POST_INSTALL has no equivalent functions, and is run to handle post-installation work in a general manner. An example is: (/var/lib/lunar/moonbase/compilers/gcc/POST_INSTALL)

cd /usr/lib/gcc-lib/$BUILD/$VERSION                      &&
ln    -sf /usr/bin/cpp cpp                               &&
cd /lib/                                                 &&
ln    -sf /usr/bin/cpp cpp                               && 
if [ ! -e /usr/bin/cc ] ; then
  ln -s gcc /usr/bin/cc
fi

As in the BUILD scripts, the '&&' represent 'and' operations, but their use in POST_INSTALL scripts is preferred rather than required.

Package Removal Scripts

Module removal is handled by lrm. Because installation is monitored and backup tarballs are created using installwatch, most of package removal is handled automatically using the logs created by installwatch. However we provide for additional actions to be taken through the PRE_REMOVE and POST_REMOVE scripts.

The PRE_REMOVE script

PRE_REMOVE is needed to execute any tasks needed prior to the main task of removing all files installed by the module. An example would be: (/var/lib/lunar/moonbase/mail/docbook-3.1/PRE_REMOVE)

CENTRALIZED=/etc/sgml/catalog
DOCBOOK_INSTALL_DIR=/usr/share/sgml/docbook/$VERSION
install-catalog -r $CENTRALIZED $DOCBOOK_INSTALL_DIR/catalog

The POST_REMOVE Script

POST_REMOVE may be used to remove data not tracked by installwatch and to correctly adjust remaining configuration files and data. Examples would include: (/var/lib/lunar/moonbase/devel/binutils/POST_REMOVE)

install-info  --delete as         --info-dir /usr/info
install-info  --delete bfd        --info-dir /usr/info
install-info  --delete binutils   --info-dir /usr/info
install-info  --delete configure  --info-dir /usr/info
install-info  --delete gasp       --info-dir /usr/info
install-info  --delete gprof      --info-dir /usr/info
install-info  --delete ld         --info-dir /usr/info

or: (/var/lib/lunar/moonbase/compilers/php/POST_REMOVE)

if    module_installed  apache;  then
  cp        /etc/httpd/httpd.conf       /tmp/httpd.conf
  grep  -v  "LoadModule php4_module"    /tmp/httpd.conf  |
  grep  -v  "AddModule mod_php4.c"   >  /etc/httpd/httpd.conf
  rm    -f  /tmp/httpd.conf
elif  module_installed  apache_mod_ssl;  then
  cp        /etc/httpsd/httpd.conf      /tmp/httpd.conf
  grep  -v  "LoadModule php4_module"    /tmp/httpd.conf  |
  grep  -v  "AddModule mod_php4.c"   >  /etc/httpsd/httpd.conf
  rm    -f  /tmp/httpd.conf
fi