software

From Cliquesoft
Jump to: navigation, search

Are you coming from a Microsoft Windows environment where software can be hard to locate and install? Or perhaps you are a bit confused on how to use dpkg (or if you should use aptitude) or some other Linux package management software and want an easiler solution. Welcome to our version of software package managment! This project is appropriately named 'software' and was designed to be easy and logical in its use. Not unlike our other bash-script-based software, this project relies on our clAPI framework for various functionality, so be sure this dependency is installed before using. It's also worth mentioning, when using this software from the command prompt, that the placement of the OPTION's must follow their respective ACTION (or parent script) which can be determined via the --help output. It might also help to read over the basics of clAPI to get a better understanding when running software from the command line.


License

This projects' codebase is licensed under the CPLv2 unless a valid CPLv1 license has been purchased. More information about both of these licenses can be found under the "Our Licenses" link of our homepage.



Dependencies

Currently the only dependencies are clAPI, cURL, and common binaries such as cp, mv, rm, mkdir, etc. Each script in this project contains all the binaries required in its header.



Details

Before getting in the installation and operation of this project, lets discuss some details of the inner workings of this software. This package manager is a series of scripts designed as a framework to provide various functionality with the software currently installed on the device or the software available to be installed. Aside from those scripts, there is a master configuration file as well as a personal configuration file that can be placed in your home directory (typically /home/username). Each package also contains up to six files that are referred to as 'control files' since they control the behavior of the project scripts. Each of those files is located in the root directory of the packaged software and contain a preceeding period (.) in the filename to hide the each file (in Linux). Below we will cover each of those scripts in detail so you have a better understanding of each:

  • .compile This control file stores all the information relevant to (cross-) compile the source code for packaging or installation. It is important to note that this file is optional and should only be included if the software needs to be compiled. If it is absent during the packaging process, the wizard will ask if it should be created and help you generate this file if requested.
  • .info Contains all the general information about the package including the package name (e.g. libc6, bash, etc), optional sales price, optional software key, dependencies, and more. This control file is mandatory and gets built during the packaging wizard if this file is absent during that process.
  • .license Used by the software during installation and other functions, to show the user what the license is of the software package. This is a mandatory control file and should be present before any packaging efforts begin.
  • .manifest This control file contains a wealth of information on the files that make up the package including the manufacturers ownership and permissions settings, privacy levels, and integrity information. This file is mandatory and can automatically be generated in its absence during the package process. Package maintainers please note, any directories that are listed in this file are considered property of the package and will be deleted upon the appropriate uninstallation process. This is useful during the uninstallation process since some software creates files dynamically. In that instance, listing the parent directory will ensure that all those dynamic files are deleted during uninstallation without having them specifically documented in this file. A great real world example of this is the '/etc/software' directory. That directory contains a variety of variable information under it and so by just listing this directory in the software package manifest, you can be certain that all the dynamic content is removed when desired.
  • .post This is another optional control file that is actually an executable script used to handle various post action tasks. Please note the current working directory for each action shown in the following template when writing the syntax for each applicable section. Below is a template that should be used when building this file:
#!/bin/bash
# .post		This is the post-whatever script that runs after the below actions
#		have been performed.
#
# created	YYYY/MM/DD by AUTHOR (EMAIL)
# updated	YYYY/MM/DD by AUTHOR (EMAIL)


case $1 in
   compile)					# code to run AFTER a successful compilation
	# NOTE: the pwd is inside the uncompressed directory
	;;

   install)					# code to run AFTER a successful installation
	# NOTE: the pwd is inside the uncompressed directory
	;;

   purge)					# code to run AFTER a successful purge
	# NOTE: the pwd is TARGET/DIRCONF/packages
	;;

   uninstall)					# code to run AFTER a successful uninstallation
	# NOTE: the pwd is TARGET/DIRCONF/packages
	;;
esac
  • .pre This file is exactly the same as the .post control file, but just executed during before whatever action is about to take place. A template that can be used for this file is shown below:
#!/bin/bash
# .pre		This is the pre-whatever script that runs before the below actions
#		have been performed.
#
# created	YYYY/MM/DD by AUTHOR (EMAIL)
# updated	YYYY/MM/DD by AUTHOR (EMAIL)


case $1 in
   compile)					# code to run BEFORE a successful compilation
	# NOTE: the pwd is inside the uncompressed directory
	;;

   install)					# code to run BEFORE a successful installation
	# NOTE: the pwd is inside the uncompressed directory
	;;

   purge)					# code to run BEFORE a successful purge
	# NOTE: the pwd is TARGET/DIRCONF/packages
	;;

   uninstall)					# code to run BEFORE a successful uninstallation
	# NOTE: the pwd is TARGET/DIRCONF/packages
	;;
esac

Some other nice features of this software are...

  • has a small foot print (~200KB) with large flexibility
  • can install dd (*.image) and ntfsclone (*.clone) images to partitions or devices
  • can install master boot records
  • can install pre-compiled and compilable software
  • can create and restore restore-points
  • can rebuild a complete system after any type of failure or as a template for mass produced goods
  • handles all software interaction instead of using different programs (e.g. dpkg or aptitude)



Installation

Installation could not be more easy! Currently it's a simple 2-step process which involves obtaining the actual software and then installing. To complete the first step, run the commands below. Please note that the last two commands accomplish the same goal and were included incase one was unavailable on your system.

$ mkdir /tmp/install

$ curl -# http://repo.cliquesoft.org/vanilla/all/software/stable -o "/tmp/install/software.soft"
  -OR-
$ wget --progress=bar http://repo.cliquesoft.org/vanilla/all/software/stable -O "/tmp/install/software.soft"

Now the second step can be accomplished using the following commands. After you execute the installation script, answer all the questions and your software will be ready for use!

$ cd /tmp/install
$ ./install



Using the CLI

ACTION's

You can see what actions are available to this project by running 'software --help' at the command prompt. Towards the top of that output, you will see a section entitiled "[PROJECT ACTIONS]" where a list will be presented for your review. Each action name should indicate what its function is, but in the event where it does not make sense, a small description to the right of each item has been included to help you to better understand. Below we will outline each action along with its associated options.


fix

This action is responsible for repairing any software that has been installed using this project. There are several tests that are run per execution of this action that include ownership, permissions, file integrity, and that dependencies are installed. If any of these checks fail, steps to resolve each problem will be presented upon completion. There are currently only two options that can be used with this action which we will cover below. It is important to note that using this action does NOT check for configuration errors, not issues with any data created by any software.

  • --all If this option was passed, then the user wants to run each of the afore mentioned checks against all of the currently installed packages. This can be a timely process depending on the speed of the device and how much software has been installed. If you are only interested in performing these tests on a particular package, then use the '--name' option instead of this option as outlined below:
$ software --verbose fix --all
  -OR-
$ software --verbose --name='bash' fix
  • --prefix This option was included mainly for system or network administrators, so the vast majority of users will not use this option. An example of its use would be where an admin might be working on a storage device that contains all the installed software for another device that currently does not start or work correctly. In that scenario, that storage device can be plugged into working equipment that can perform this task against it. So lets attach some details to this example. Lets say that a hard drive from a non-working device is plugged into a working system and is mounted to '/mnt/source'. An administrator can run this action on the "bad" drive by running the following command:
$ software --verbose fix --all --prefix='/mnt/source'


install

Just as the name suggests, this action installs the desired software, regardless if the package must be downloaded from online repositories or from a pre-downloaded local file. Unlike other package managers, a user can not only install pre-packaged software, but can install from source code or even images! There are quite a few options to this action since it is so comprehensive, which we will cover below:

  • --code Using this parameter indicates that you would like to download, compile, and install the package source code instead of the pre-compiled version. This will be a much more lengthy process of installing software than using the pre-compiled versions and is not recommended for the typical user.
$ software --verbose --name='gcc, make' install --code          # this will download, compile, and install the software to your home directory
  • --docs Unless this option has been specified, the manufacturers help documentation is not added to the device when installing software. The alternative to using this parameter is to rely on any (online) documentation from the manufacturer of the operating system.
$ software --verbose --name=bash install --docs                 # this installs bash and its related manufacturer documentation
  • --delete After performing an install, the software packages that were downloaded and uncompressed will remain on the device unless this option has been passed.
$ software --verbose --name=bash install --delete
  • --device When using the '--image' option, this parameter specifies what device needs to receive the contents of the image.
$ software --verbose install --image=/opt/xinix.image --device=/dev/hda3
  • --dir This option instructs the package manager to install all the packages that are in the directory specified by this value. It is important to know that by using this parameter, there is no dependency checking so the directory needs to contain all the neccessary packages. Also see the '--file' option below.
$ software --verbose install --dir=/opt/template
  • --file You should use this parameter when you have just a single file that needs to be installed. This option does check dependency requirements are met (unless --ignore is used) for the package. Also see the '--dir' option above.
$ software --verbose install --file=/opt/bash.soft
  • --ignore Under rare circumstances, you may need to install software without having its dependency requirements met. In those instances, you can use this parameter to accomplish that goal. This will work even when using the '--code' option.
$ software --verbose install --file=/opt/bash.soft --ignore
  • --image Defines the path and filename of the image to install. When using this option, you must also use the '--device' parameter to indicate were to install the image. Currently the only images that this option can use are dd (identified by a .image filename extension) and ntfsclone (identified by a .clone filename extension). It is important to note that there is no restore point created when using this option. Also you should not use '--name' when imaging devices.
$ software --verbose install --image=/opt/xinix.image --device=/dev/sda1
  • --mbr When imaging a device (using the --image option above), you can also specify a master boot record to install during the process. This file should be a 512 byte file made by 'dd' or something comparible, and unlike the '--image' option, this parameter does not rely on a filename extension.
$ software --verbose install --mbr=/opt/xinix/xinix.mbr --images=/opt/xinix.image --device=/dev/sda1
  • --nrp In some instances you may not want to create a restore point when installing new software. In those cases, pass this option and this step will be skipped.
$ software --verbose --name=bash install --nrp
  • --prefix This option defines where the software should be installed. Currently there are three valid values: device|private|public. A value of 'device' installs software in the root (/) of the filesystem and is considered a change in the "firmware" of the device. 'private' installs the software in the users home directory (~/) and allows specific (versions of) software per user. And lastly, 'public' installs the software in the Public directory (/home/.public) and is considered software available to all users.
$ software --verbose --name=bash install --prefix=/opt/target
  • --rebuild This option not only allows you to rebuild a system from scratch, but it also makes a great way to create a template for an OS installation. Similar to how the '--dir' option works, this parameter installs all the packages located in the directory passed as the value. Unlike '--dir' however, this option does perform dependency checks, so all that is required to be in the directory is that top-most packages. For example, you should not have to list 'libc6' in there since it is a dependency for a variety of other packages. Another big difference is that the files contained in that directory just need to be the .info files of the packages you wish to install. Those .info files do not even need any contents!
$ software --verbose install --rebuild=/opt/rebuild
  • --version Similar to how '--prefix' works, this option specifies the version number of the package you are interested in installing. Also like '--prefix', there are keywords that can be used which are 'beta' and 'stable'. By specifying either of those keywords, the latest version of each type will be installed. Providing any other value will install that specified version if it's valid. It is important to remember that if you specify a version, you can only install one package at a time!
$ software --verbose --name=bash install --version='3.01'       # this specifically installs version 3.01 of bash
$ software --verbose --name=bash install --version=stable       # this installs the latest stable version of bash


list

If you are interested in seeing what software is currently installed on your device, then this is the action you will want to use. Since this is such a simple action, there are not many options for it.

  • --all This is an optional parameter and indicates that you would like to display a list of all the currently installed software. In the absense of this parameter, you must specify a '--name' value which will give you much more detailed information on the package. Examples will be shown below:
$ software --verbose list --all
  -OR-
$ software --verbose --name='bash,libc' list
  • --license This is also an optional parameter, but only when using the '--name' option. By specifying this option, you will be shown the detailed package information along with the license of each.


package

Developers will appreciate this action as it assists in creating the packages that are distributed through the software repositories. The wizard will help create the various packages (e.g. .soft, .code, .docs, etc) and contained control files along with the option of uploading to a specified repo. Despite all the work, there is a surprisingly small number of options:

  • --build Some software requires that the build directory must be different than where the source code resides. In those instances, you can use this option to accomplish that very goal. The value here needs to be an empty directory that will contain all the files associated with the package. Please note, this is not the installation directory, but a temporarily used directory when needed.
$ software --verbose package --staging=/opt/package --source=/opt/source --build=/opt/build
  • --source This option will be used in packaging all the compilable software and specifies the directory were the source code resides. The directory passed must be a full path (e.g. /opt/staging), not one that is relative (e.g. ../staging).
  • --staging Similar to the --source option, this one specifies the full path to where the compiled software will be "installed" for packaging. If the software does not need to be compiled, it must reside in this directory before the packaging task begins. Also, the directory referenced by this option needs to have any pre-existing mandatory control files present beforehand. To find out which ones will be created using the wizard, see the 'Details' section above.

NOTE: An important point to remember is that software pacakge maintainers are able to include patches specific for their distro that will automatically be applied during the pre-compilation stage by creating a case-sensative directory named 'PATCHES' in the root directory of the source code. This way the vanilla source code can be shipped with specific modifications allowed per distro. Each of the patch files located in that directory are incorported in the order that 'ls -1' sorts them. If you would like them applied in a specific order, you must name them so they are in the correct order according to the 'ls' command as previously presented. When generating the patches, they must be in unified context and created from the root of the source directory to incorporate relative paths to the target files.


search

Used as a way to see what software packages are available in the configured repositories, this action returns a list of full or partial matching results. This action can also find what package(s) contains a desired file depending on the passed parameters.

  • --details This option indicates that you would like to see the details of a particular package in the same formatting as the list action. There are a couple of caveats that need to be mentioned. First when using this option, you must use the '--name' parameter as well. Second before you can use this option, you have to run a '--query' first (or how else would you know what package to show the details of).
# first run...
$ software --verbose search --query='secure shell'              # which might return things like dropbear, ssh, etc

# next get the details of a particular package
$ software --verbose --name='dropbear,ssh' search --details     # this will show the package details for both dropbear and ssh
  • --filenames If this option is passed, you indicate that the keywords in '--query' should apply to searching for filenames rather than packages. This can be helpful if you need to install software and don't know what the package name is, but you do know the name of a file used in that package.
$ software --verbose search --filenames --query='scp'           # this will return package names like dropbear, ssh, etc
  • --query By using this option, you will be searching for packages that contain the keyword(s) specified by this (comma separated) value. Alternatively, you can use the '--name' parameter to search for specific packages.
$ software --verbose search --query='command,line,shell'        # this might return packages like bash, ksh, tsh, ...

$ software --verbose --name='bash' search                       # this will return only the information on bash
  • --repo If you would like to search a particular software repository instead of all available when performing a '--query', you can use this option with a value pertaining to the name of a repo: official|professional|public|private|media
$ software --verbose search --query='secure shell' --repo='public'


uninstall

When certain installed software is no longer desired, you can execute this action in order to remove it from your device. There are several options to protect configuration files in the event that the software may be reinstalled at a later date. This action is also responsible for restoring restore-points if requested.

  • --depends As you uninstall software, the dependencies of that software still remain on your device. By using this option, you indicate that the package manager should also uninstall any non-shared dependency packages as well. This will keep only the absolute minimum amount of software installed in order to run the system. By removing this additional unused software, you are getting rid of potential exploits that can compromise your equipment.
$ software --verbose --name=tsh uninstall --depends             # this would uninstall the shell, but not libc6 since others rely on it
  • --ignore This option will mainly be used by administrators or users who find themselves in a bad situation. In the instance that you are working with a broken, non-working system and you have to adjust some of the packages, you can use this option to ignore any dependency issues when uninstalling/installing/updating various packages.
$ software --verbose --name=libc6 uninstall --ignore
  • --level As indicated in the description of the parent action, sometimes a user may want to uninstall just the software and leave the configuration files behind for later use or backup OR they may want to get rid of everything related to the package. Using this option will enable you to specify which of those tasks should transpire. Current valid values are 'uninst' or 'purge' where the former retains software configuration and the latter removes everything.
$ software --verbose --name=dropbear uninstall --level=uninst   # this leaves the configuration files for dropbear on the device
  • --nrp Most of the time when you uninstall software you will want to make a restore point, just in case you would like to restore that package at a later date. One of the instances when you may not want to create a restore point is when you are cycling restore points to find a particular one. Obviously creating a restore point in this instance would not be desired since we already have one. In that case you can pass the '--nrp' option to prevent that task from occurring.
  • --prefix This option works in the same way that the '--prefix' parameter works in the 'fix' action above.


update

In order to keep your system up-to-date, use this action to download and install all the updates for installed or specific packages. This can help prevent malicious activity from occuring on your equipment by patching bugs and other exploits.

  • --repo If you would like to acquire updates from a particular software repository (instead of all that are available in the absence of this parameter), then you can specify this option with a value that indicates which repo you would like to use. Currently the available values for this parameter include: official|professional|public|private|media
  • --state Most of the time users will want to install the latest stable version of software. Using this option, you can overwrite that default behavior, but we would only recommend this to beta testers or other users that need the most bleeding edge version due to the potential to break a working system. The valid values that can be passed are: beta|stable



Chained ACTION's

At some point you may want to run multiple actions in a single call, and thanks to the clAPI framework, we can do just that! Lets run through a couple of examples where this may be applicable. Please remember that the ACTION's will need to come in the order that they need to be executed!

Our first example will be to uninstall a current package and replace it with a restore point. This will involve having to do a little more work (unless you are using a graphical interface) other than just executing a 'software' call. We'll outline the steps below:

$ cd /var/restore                                               # change into the restore point directory
$ ls -1 | grep ^'bash'                                          # list all the filtered files in the directory
         ...
bash_20130503145604.soft
bash_20130608224316.soft
bash_20130609052533.soft
         ...
$ software --verbose --name=bash uninstall --nrp install --file='/var/restore/bash_20130608224316.soft' --nrp --prefix=device



Using the Web Interface

This aspect of the project has not been developed yet, but it is planned. Please visit us again soon to see if this part has been implemented.



Repositories

Currently there are up to six software repositories that can be accessed based on the settings within the configuration file of this project. If any repo should not be accessible, simply set it with a blank value in the config file. Below we will discuss each of the repositories that can be enabled:

ARCHIVE contains all the prior versions of software from each repo OFFICIAL contains only the software that has official support PROFESSIONAL contains only third-party software with access to a minimum of 8am-5pm support for their products PUBLIC contains all other third-party software - buyer be ware! PRIVATE contains the collection of software residing within your personal networks MEDIA contains the repository for available media


[Administrator notes] Restricting repositories on a per user basis can also be accomplished by leaving the global configuration file with the correct values and the desired blank repo values in the personal '~/.etc/software/software.conf' file of each user.

If your Linux distro utilizes upgrades instead of rolling releases, you can simply add the release name as part of the SCHEMA value in the software configuration file. Then all that is required to upgrade to a new branch version, you would simply modify the SCHEMA value and run 'software --verbose update'

If you are interested in setting up your own PRIVATE repository or one(s) as part of your own Linux distro, then you will need a particular directory structure as well as a server-side script to perform the interaction with the client-side package manager. Before getting into the server-side scripts, lets discuss the directory structure. If you're using a vanilla version of our package management software, the there are several root directories that need to be present. The first one will need to be called 'categories' with all subfolders being the actual name of each software category supported in your distro. For example, XiniX contains the following categories by default:

accessibility development education firmware games miscellaneous multimedia network productivity services system utilities

Under each of these directories needs to be the current version of the .info and .manifest control files of each package available in the repository. The next set of root directories correspond to the SCHEMA layout in your software configuration file. By default, in XiniX, this value equals "EDITION/ARCH" indicating the edition of the operating system (currently 'Vanilla' or 'Server') and the device architecture (currently 'all', 'amd64', 'arm', 'i386', 'ia64', 'mips', and 'ppc'). Please note, if you are creating your own repository, you will need to match the SCHEMA value to your directory layout. Under each of these directories needs to be a parent directory for the packages available for that group. So again, as an example, XiniX would have a full path of 'vanilla/i386/bash' which would then contain the various files for that package (e.g. .soft, .code, .docs, etc).

Now we need to discuss the server-side script. This script needs to reside in the root of your repository and have the name 's.php' (feel free to submit in additional languages). Below you will find the minimal script to use to accomplish this task written in php:

<?php
# s.php         the search script that finds related packages in the repo
#               and returns them to the local device.
#
# created       2013/08/15 by Dave Henderson (dhenderson@cliquesoft.org or support@cliquesoft.org)
# updated       2013/08/17 by Dave Henderson (dhenderson@cliquesoft.org or support@cliquesoft.org)
#
# This script is public domain.
#
# This program is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
# FOR A PARTICULAR PURPOSE. See the appropriate Cliquesoft License for details.


# perform some safety measures...
if ($_POST['t'] == '/') { exit(1); }                            # make sure root (/) wasn't passed as the value.
if (strpos($_POST['t'], '../') !== false) { exit(1); }          # make sure the user isn't passing something like /home/dave/../../ to get to root (/).
if (strpos($_POST['t'], ';') !== false) { exit(1); }            # make sure the user isn't passing in a series of commands.
if (strpos($_POST['q'], ';') !== false) { exit(1); }            # same as above
if (strpos($_POST['e'], ';') !== false) { exit(1); }            # same as above
if (strlen($_POST['t'] >= 48)) { exit(1); }                     # make sure we're not having some type of malious activity (which usually requires longer variable length)
if (strlen($_POST['q'] >= 24)) { exit(1); }                     # same as above
if (strlen($_POST['e'] >= 10)) { exit(1); }                     # same as above

if ($_POST['p'] == '/') { exit(1); }                            # make sure root (/) wasn't passed as the value.
if (strpos($_POST['p'], '../') !== false) { exit(1); }          # make sure the user isn't passing something like /home/dave/../../ to get to root (/).
if (strpos($_POST['s'], '../') !== false) { exit(1); }          # same as above
if (strlen($_POST['p'] >= 48)) { exit(1); }                     # make sure we're not having some type of malious activity (which usually requires longer variable length)
if (strlen($_POST['s'] >= 10)) { exit(1); }                     # same as above


header("Content-Type: text/plain");

if ($_POST['q'] != '') {                                        # IF WE ARE SEARCHING THE REPO, THEN...
   chdir(escapeshellcmd($_POST['t']));                          #   change into the TARGET directory containing the files to be parsed
   system('grep -Ril "'.escapeshellcmd($_POST['q']).'" --include=*.'.escapeshellcmd($_POST['e']).' *');         # q = query, e = filename extension

} else if ($_POST['p'] != '') {                                 # IF WE ARE CHECKING FOR UPDATES, THEN...
   $ver=readlink($_POST['p'].'/'.$_POST['s']);                  #   p = package, s=state
   echo basename($ver,'.soft');                                 #   return just the version number portion of the filename - removing path and filename extension
}

?>



Developers

Dave Henderson [dhenderson (at) cliquesoft (dot) org]