PBI Manager/9.2

From PC-BSD Wiki
Revision as of 12:40, 9 August 2012 by Drulavigne (Talk | contribs)

Jump to: navigation, search

(Sorry for the inconvenience)

Contents

PBI Manager is a suite of command line utilities which can be used to install, remove, create and manage PBIs. It is also available for FreeBSD systems. The PBI Manager is released under the [1][1].

Features

The 9.x PBI format introduced the following features:

  • Upgrade deltas: since PBIs are self-contained, installation files tend to be large. In the previous implementation, updating a PBI required the re-downloading of the entire installation archive. For larger applications this could be a time-consuming process, especially over low bandwidth connections. The new PBI specification performs updates using binary diff patches known as PBP (Push Button Patch) files. PBPs are a fraction of the PBI's size; in some cases less than 5% of the original PBI archive. An upgrade automatically checks for for the presence of a PBP file and attempts to use it, only falling back to the original archive should the process fail.
  • Library and file sharing: in the previous implementation of the PBI format it was common that identical files existed between various applications. These duplicates, while necessary to provide the self-contained functionality, wasted both disk space and runtime memory. This waste of space has been greatly reduced through the use of a hash-dir directory where libraries and common files are shared through a system of hard links. When an identical file is found in a PBI, the original will be removed and a hard-link stored in the hash-dir. After a PBI has been removed, any unneeded files left in the hash-dir are cleaned up by the pbid(8) daemon which monitors and maintains the integrity of the shared files.
  • Repository management: allows administrators and PBI builders to create and manage their own repositories of PBIs. This repository system provides a number of tools for PBI distribution, release management, and repository browsing.
  • Digitally signed PBIs: each repository includes an openssl public key file which is installed on the end user's system. Each PBI includes several signatures for the content archive and installation and removal scripts. During the PBI installation process, these signatures are checked to confirm that the archive has not been tampered with during transit. This key file is also used to associate a particular PBI with a parent repository for upgrade purposes, since it is possible that multiple repositories will have the same applications.
  • Root password not required: most applications can be installed and upgraded by regular user (non-root) accounts. This allows enhanced security in office or home situations, where users can now add/remove desktop applications without needing access to the root password. PBIs that impact on the security of the system (e.g. installing a web server) will require root access.
  • Implementation: the previous PBI design was developed in QT/KDE C++, making it inappropriate for use at the command line. The new format is implemented 100% in shell and is comprised of command-line utilities, each with an associated man page. These utilities are discussed in more detail in the rest of this section.

Additional Resources

Release History

0.9.7 (12-07-11) - Kris Moore

  • Use -a flag for fetch to retry after soft failures
  • Don't create binary patch files, if they are larger than the source
  • Add ability to use /etc/pbi-make.conf on system, letting us set universal make options
  • Simplify the startup with less parsing commands
  • Add new -R flag for pbi_add, which only downloads a PBI from a mirror, without installing it
  • Add support for using "ccache" automatically if configured on the host system
  • Fixed a bug using "pbi_create" on a directory manually, make sure file end up in proper PBI application directory
  • Make sure we use the systems share/icons directory to use shared cursors / icons across all PBIs


0.9.6 (11-07-11)

  • Fix some bugs doing cleanup of stagedir / build directories
  • Properly kill fetch process when canceling a download
  • Improve buildworld on FreeBSD to use /usr/src if it already has source
  • Fix some random warnings when building via sudo


0.9.5 (11-03-11)

  • Add sizes of PBIs to the meta index files
  • Fix bug adding default icon to PBIs when none is specified
  • Fix handling of PBI_PROGREVISION
  • Make sure we unmount nullfs mounts completely


0.9.4 (10-19-11)

  • Fix issue using system-fonts in PBIs
  • Fix issue removing files which have been set r--
  • Slow down fetch process to fix issues with fetch protocol errors
  • Fix bug coping linux libraries into final PBI file
  • Allow user settings in external-links to overwrite auto-detected values
  • Fix bug running "stat" on files with spaces in name
  • Validate the build checksum before doing patching
  • Speed up builds with optional --tmpfs flags
  • Add priority system to building modules
  • Fix issues setting proxy support from pbi.conf / pcbsd.conf files
  • Validate repo URL's when creating .rpo file


0.9.3 (07-18-11)

  • Unset variables between auto-builds
  • Fixed some warnings with 'cut' when parsing i18n strings
  • When doing auto-builds, run through alphabetically
  • Increase check frequency when first trying to download index's
  • Speed up the hashdir merging
  • Show repo MD5 with listings
  • If no index files, do not fail on listing, just show empty repo


0.9.2 (05-23-11)

  • Fixed usage errors with "tr"
  • Syntax fixes when trying to install app with no meta-data
  • Fixed some bugs with binary patching
  • Fixed auto-builder using the PBI_BUILDKEY in modules pbi.conf file
  • When building ports, don't export PREFIX since it confuses some linux ports
  • Perform check if user is root, if PBI is flagged root-only
  • Auto-builder now checks if port is compatible with arch type of system


0.9.1 (05/3/11)

  • Initial public release

What is the PBI format?

One of the things holding back mainstream adoption of open source desktops is the package management format. With almost all open source desktops, software is simply treated as part of the operating system. Thus, when performing an update of some seemingly trivial application, you run the risk of potentially needing to upgrade other packages which could break other critical parts of your desktop.

While package management systems have gotten better at resolving dependency issues, trying to fix conflicts to prevent breakage before it occurs, they still don't address the underlying problem: every software package is a part of the system, and pulling on any one thread has the potential of causing a break somewhere farther down the line.

The PBI format tries to correct this underlying flaw. Rather than making every application a part of the base system, PBIs are self-contained, including their own dependent library tree and related data. As a result, when you install a PBI there are no dependency issues to resolve, and applications can be added or removed freely, without fear of causing breakage to the desktop or any other installed software.

Installing PBI Manager

If you are running PC-BSD 9 or higher, then the PBI Manager is already installed, and you can use it via the command-line, or a front-end such as the AppCafe®.

FreeBSD users can install it from the [4][4].


Users interested in installing the development version from our [5][5] can do so with the following commands:

svn co svn://svn.pcbsd.org/pcbsd/current/src-sh/pbi-manager
cd pbi-manager
make install

NOTE: The development version may be unstable / buggy, use at your own risk

Getting PBI Files

A list of approved and available PBIs can be viewed with pbi_info -i or pbi_browser, and then installed with pbi_add -r <pbiname>.

NOTE: PBI files previously created for PC-BSD 7.x/8.x will NOT work with PBI Manager as it was designed for version 9.x.

Underlying File / Directory Structure

The underlying files and directories used by PBI Manager are as follows:

  • /usr/local/etc/pbi.conf: location of the PBI Manager configuration file
  • /usr/pbi/: where PBIs are installed on the system
  • /var/db/pbi/: contains data files related to installed PBIs and repositories
  • /usr/local/sbin/pbi_*: location of the PBI Manager commands
  • /usr/local/share/pbi-manager/module-examples/convertoldmod.sh: script which can be used to convert an existing 7.x or 8.x PBI module to the new 9.x format

Feedback / Reporting Problems

Feedback, problem reports, and general discussion are welcome on the [6][6].

Command Reference

The following commands are installed by PBI Manager. For more details, refer to that command's man page.

pbi_add(1)

Similar to FreeBSD's pkg_add, the pbi_add command is used for adding/installing PBIs on a system, either from a local file or remotely from a repository. This utility supports the options listed in Table 7.2a.

Table 7.2a: pbi_add Options

Switch Description
-e extract only, do not install; will extract the archive to ~/<pbidirname> unless the -o option is used
-f force installation, overwriting an already installed copy of the application
-g show path to icons and images for GUI installations
-i display information about specified PBI
-l display license for specified PBI
-o outdir specify the directory to use when extracting the PBI with -e
-r remote fetch installation file from update server; the system architecture and version will be automatically determined in order to fetch the correct file
-R remote fetch the install file from the update server but do not install
-v enable verbose output
--checkscript display any custom scripts used in the installation/removal of the PBI
--licagree agree to license terms and conditions; to view the license, use -l
--no-checksig skip the openssl signature verification of the PBI data
--no-checksum skip the checksum verification of the archive data
--no-hash disable using the shared hash dir
--repo repoid specify which repository to use
--rArch arch manually specify the PBI architecture type of i386 or amd64
--rVer version specify which version of the PBI to install

For security reasons, it is recommend that users first use the -e and --checkscript options to view archive contents and installation scripts prior to installing a PBI file.

To install a PBI, use: pbi_add -r PBINAME. The following example will install the alpine PBI:

pbi_add -r alpine
Downloading ftp://ftp.pcbsd.org/pub/mirror/PBI/mail/alpine/9/x32/alpine-2.00_3-i386.pbi
/usr/pbi/.alpine-2.00_3-i386.pbi.42391   100% of  11 MB 295 kBps 00m00s
Verifying Checksum...OK
Extracting to: /usr/pbi/alpine-i386
Installed: Alpine-2.00_3

pbi_addrepo(8)

The pbi_addrepo command is used to register a new PBI repository on a system. If the pbid daemon is running, the repository's index and meta files will be automatically fetched and made ready for browsing. The command has one argument: the name of the repository file. Repository files have a .rpo extension and are created with the pbi_makerepo command.

pbi_autobuild(8)

The pbi_autobuild command is used on the PBI build system to build any out-of-date or new packages. The utility is also of interest to PBI testers and system administrators who create and maintain their own internal repository of PBIs. It can traverse the FreeBSD ports and Meta-data trees, building missing PBI files or PBIs in which the target port version has been updated. Table 7.2b summarizes this command's options:

Table 7.2b: pbi_autobuild Options

Switch Description
-c confdir specify the directory containing the PBI configuration modules; any found pbi.conf files will be parsed, and if PBI_MAKEPORT is set, the target port will be used for the build; if PBI_MAKEPORT is unset, the auto-build will attempt to match the module to a FreeBSD port based upon the dirname of pbi.conf
-d portsdir specify an alternative ports directory; defaults to /usr/ports/
-h script specify a helper script to call after building a PBI
-o outdir the directory to place the finished PBI files; also used to determine which apps are in need of a rebuild if the associated FreeBSD port has been updated
--genpatch when building a new PBI, check for archived copies and generate smaller patch updates to the new version (*.pbp files)
--keep num when building new PBIs, keep <num> copies of past versions of working PBI in <outdir>/archived/ folder; these archived copies can be used with the --genpatch command to generate update patch files
--pkgcache enable caching of .txz pkg files which greatly speeds up subsequent builds of a PBI
--prune remove any PBIs which no longer have an associated module in confdir
--tmpfs automatically create and mount a tmp filesystem and use it for WRKDIRPREFIX; can speed up port compiles on systems with available RAM
--sign keyfile digitally sign the PBI file with the specified openssl private key file

pbi_browser(1)

The pbi_browser command provides a CLI front-end to browsing a repository's available PBIs. Options for viewing categories and searching by keyword are available, and once the desired PBI is located, it will show the pbi_add command which can be used to install the application. Table 7.2c summarizes the available options.

Table 7.2c: pbi_browser Options

Switch Description
-c category displays a list of PBIs in the specified category
-s search search for PBIs containing the specified string in the name, description, or keywords
--listcats list the available categories
--viewall list all available PBIs

pbi.conf(5)

pbi.conf is an ASCII text configuration file containing values that are used by the various pbi_* commands. The proxy variables are only needed if the system uses a proxy server to access the Internet. Table 7.2d lists the supported variables.

Table 7.2d: pbi_conf Variables

Switch Description
PBID_REFRESH wakeup time in seconds for pbid to run its checks
PBI_INDEXREFRESH number of hours representing how often pbid refreshes the index and meta files from repos; default is every 24 hours
PBI_PROXYURL proxy server IP address
PBI_PROXYPORT proxy server port number
PBI_PROXYTYPE can be HTTP or SOCKS5
PBI_PROXYUSER username used to authenticate with proxy server
PBI_PROXYPASS password used to authenticate with proxy server

pbi_create(1)

The pbi_create command provides a way for packagers to manually specify a target directory to be compressed into a PBI file. The option -b can also be used to re-package an already installed PBI back to an archive. PBI creators are encouraged to send a tarball of the resulting PBI module to the [7][6] so they can be added to the PC-BSD PBI repository and made available to other PC-BSD users.

Table 7.2e summarizes the available options:

Table 7.2e: pbi_create Options

Switch Description
-a author specify the author for this PBI
-b make a backup of an installed PBI; specify the target PBI name instead of the PBI directory
-c confdir specify the metadata configuration directory; while not required it is highly recommended as metadata is required to create icons and binary entry-points
-d portsdir specify an alternative ports directory; defaults to /usr/ports/
-i icon specify a default icon, relative to pbidir/
-n name specify a name for this PBI
-o outdir place the finished .pbi file into the specified directory; defaults to $HOME
-p port use the given port to get PBI name and version
-r version specify a version for this PBI
-u weburl specify a website URL for the PBI
--no-hash disable using the shared hash directory which uses hard links to share files between applications
--sign keyfile digitally sign the PBI file with the specified openssl private key file

As the superuser, you can create a PBI with the pbi_create command using the following syntax:

pbi_create -a <author> -n <name> -r <version> -w <weburl> <target directory>

Inside the target directory place the application's binaries or scripts along with any required dependencies. To indicate which file(s) represent the runtime command(s), include a file named external-links in the target directory. That file contains an entry for each command, as seen in the following example:

# Files to be symlinked into the default LOCALBASE
# One per-line, relative to %%PBI_APPDIR%% and LOCALBASE
# Defaults to keeping any existing files in LOCALBASE
# Use bin-files/ for binaries that need wrapper functionality

# TARGET                LINK IN LOCALBASE       ACTION
bin/myapp               bin/myapp               binary,nocrash

This entry instructs pbi_create to make the wrapper scripts for the myapp binary, along with placing it in the user's PATH at install time.

It is also possible to include desktop icons and mime entries using the xdg-mime/, xdg-desktop/ and xdg-menu/ directories. The section on how to Create PBIs contains more details about creating these files. These directories should be created as subdirectories of the target directory of your application.

pbi_delete(1)

Similar to FreeBSD's pkg_delete, the pbi_delete command removes an installed PBI from the system. It also schedules cleaning for the shared library directory, which is performed by pbid. Table 7.2f summarizes its options:

Table 7.2f: pbi_delete Options

Switch Description
-v enable verbose output
--clean-hdir perform a full cleaning of the shared hash directory, removing any unused files; should only be required after a system crash or failure in removing a PBI

When removing a PBI, you must give its full name. The full name can be found in the output of pbi_info. The following example searches for the ntop PBI and removes it:

pbi_info | grep ntop
ntop-4.0.1_1-i386
pbi_delete -v ntop-4.0.1_1-i386
Running pre-removal script: /var/db/pbi/installed/ntop-4.0.1_1-i386/pre-remove.sh
Removing: /usr/pbi/ntop-i386
Removing: /var/db/pbi/installed/ntop-4.0.1_1-i386

pbi_deleterepo(8)

The pbi_deleterepo command can be used to remove a registered repository from the system. It takes the repository's ID as the only command argument.

pbi_icon(1)

The pbi_icon command provides a number of options for adding desktop icons, menu entries, and mime data for an installed PBI. Not all PBIs will contain desktop/menu/mime data. Additionally, the window manager must be [8][7]-compliant to understand a PBI's icon and mime settings. Table 7.2g summarizes this command's options:

Table 7.2g: pbi_icon Options

Switch Description
add-desktop installs desktop icon; should be run as regular user
add-mime installs mime information; should be run as root
add-menu installs menu icons; should be run as root
add-pathlnk installs any $PATH links to ~/bin when run as user or to $LOCALBASE when run as root
del-desktop removes desktop icon; should be run as regular user
del-menu removes menu icons; should be run as root
del-mime removes mime information; should be run as root
del-pathlnk removes any $PATH links to ~/bin when run as user or to $LOCALBASE when run as root

pbi_indextool(1)

The pbi_indextool command is useful for repository maintainers. It allows PBI files to be added and removed from the repository's INDEX file rather than editing the file manually. Table 7.2h summarizes the available options:

Table 7.2h: pbi_indextool Options

Switch Description
add adds a PBI to the target INDEX file
rem removes a PBI from the target INDEX file

Run the command with the desired switch to receive detailed usage options.

pbi_info(1)

Similar to FreeBSD's pkg_info command, the pbi_info command is used to determine which PBIs are currently installed. Table 7.2i summarizes the available options:

Table 7.2i: pbi_info Options

Switch Description
-a list all PBIs installed on the system; same as running pbi_info without an argument
-i list all available PBIs from any repo
-v enable verbose output

pbi_listrepo(1)

The pbi_listrepo command manages installed repositories on a system. Table 7.2j summarizes this command's options:

Table 7.2j: pbi_listrepo Options

Switch Description
--down move the targeted repoID down a single number in priority
-mirror URL change the specified repoID's mirror URL
--up move the targeted repoID up a single number in priority

Run the command without any options to list the IDs of the available repositories.

pbi_makepatch(1)

The pbi_makepatch command is automatically used by pbi_autobuild to create small *.pbp (Push Button Patch) files. These files can be downloaded to a user's system in order to update a PBI's version without re-downloading the entire archive. This allows users to download only the incremental changes when a PBI is upgraded. The command can also be run manually by providing two PBI archives to compare and generate a patch file for.

Table 7.2k summarizes the available options:

Table 7.2k: pbi_makepatch Options

Switch Description
-o outdir save the resulting *pbp file to the specified directory
--sign keyfile use the specified openssl key to digitally sign the patch file

pbi_makeport(1)

The pbi_makeport command can be used by packagers to build a target FreeBSD port and convert it into a PBI file. Many options are provided to fine-tune the build process, and meta-data modules can also be specified to further improve the resulting PBI file. The first time this command is run, it will build a fresh chroot sandbox environment which can be used for clean-room building of the target port without affecting the host system. How to create a PBI using this command is discussed in more detail here.

NOTE: the pbi_makeport command has support for using [9][8] to speed up the compile process. If ccache is installed on the host system and the CCACHE_DIR variable is set, the pbi_makeport command will automatically utilize it for the port compile phase. This can be disabled by setting NO_CCACHE=yes in /etc/pbi-make.conf on the host system, or as an optional make flag in a module's pbi.conf file.

Table 7.2l summarizes the available options:

Table 7.2l: pbi_makeport Options

Switch Description
-B build-only; generally used with -k to build a port before running pbi_create manually
-c confdir specify the metadata configuration directory; while not required it is highly recommended as metadata is required to create icons and binary entry-points
-d portsdir specify an alternative ports directory; defaults to /usr/ports
-k keep the build files after building the PBI
-o outdir the directory to place the finished PBI file; defaults to user's $HOME directory
-p prefix manually provide a PREFIX which determines the location where the PBI will be installed on the end-user's system
--delbuild remove any existing build directories before starting the build
--mkdebug will drop to a debugging shell should the port make fail
--no-prune disable auto-pruning of non-REQUIREDBY ports after the compile phase; by default any ports which are used solely for building and which are not required for program execution will be pruned
--tmpfs automatically create and mount a tmp filesystem and use it for WRKDIRPREFIX; can speed up port compiles on systems with available RAM
--sign keyfile digitally sign the PBI file with the specified openssl private key file

pbi_makerepo(1)

The pbi_makerepo command allows repository maintainers to create a single *.rpo file containing various information about the new repository. This .rpo file can then be installed on the target system with pbi_addrepo. Table 6.2.4.15 summarizes the available options.

Table 6.2.4.15: pbi_makerepo Options

Switch Description
--desc description required; description of the repo to be shown in the repo list
--key keyfile required; OpenSSL public key used to verify the digital signature of PBIs installed from this repo
--mirror URL required; URL in http://, https://, or ftp:// format to download PBIs and updates from
--url URL required; URL in http://, https://, or ftp:// format to use when downloading the master INDEX file of available PBIs

pbi_metatool(1)

The pbi_metatool command provides a way for repository maintainers to modify the PBI metadata in their repository in order to add or remove application categories or specified PBIs. This metadata may contain information such as Name, Description, Icon, Keywords, License, and more for the respective PBI file. Table 6.2.4.16 summarizes the available options:

Table 6.2.4.16: pbi_metatool Options

Switch Description
add add a category or application to the target metafile
rem remove a category or application to the target metafile

Run the command with the desired switch to receive detailed usage options.

pbi_patch(1)

The pbi_patch command is used to update an installed PBI to a different version using the small diff Push Button Patch *.pbp file. This allows the user to perform an incremental upgrade of an installed PBI. The available options are summarized in Table 6.2.4.17.

Table 6.2.4.17: pbi_patch Options

Switch Description
-e extract only, do not install; will extract the archive to ~/<pbidirname> unless -o is used.
-g extract image data from header; commonly used for GUI installations
-i display information about this PBI file
-o outdir specify the directory to use when only extracting the PBI with -e
--checkscript display any custom scripts used in the installation/removal of this PBI file; recommended if the PBI file is suspect in any way
--no-checksig skip the openssl signature verification of the PBI data
--no-hash disable using the shared hash directory which uses hard links to share files between applications

pbi_update(1)

The pbi_update command is used to display information about which PBIs have available updates and to perform the updates. Table 6.2.4.18 summarizes the available options.

Table 6.2.4.18: pbi_update Options

Switch Description
-c check only the specified PBI for available updates
--check-all run a full check of all installed PBIs and display list of available updates
-disable-auto disable auto-updating of the target PBI
--enable-auto enable auto-updating of the target PBI
--update-all update all installed PBIs to the latest versions

pbi_update_hashdir(1)

The pbi_update_hashdir command is used by the pbid daemon to merge the contents of a PBI into the hash directory.

pbid(8)

The pbid command runs a small daemon which performs maintenance of installed PBIs, merges files into the shared hashdir, fetches the repository INDEX and meta files, and makes the adding and removing of PBIs much faster. It will automatically be started from the /usr/local/etc/rc.d/pbid startup script if pbid_enable="YES" is in the /etc/rc.conf file.

This utility supports the option summarized in table 6.2.4.20:

Table 6.2.4.20: pbid Options

Switch Description
-v enable verbose output when the daemon starts
--refresh schedule a refresh of index and meta files

This command logs its output to /var/log/pbid.log. Check this log for errors should you experience any problems with PBI maintenance.

References


  1. http://www.freebsd.org/copyright/freebsd-license.html
  2. http://www.pcbsd.org/~kris/pbi9-slides.pdf
  3. http://www.pcbsd.org/~kris/asiabsdcon2011-pbi9.pdf
  4. http://www.freshports.org/ports-mgmt/pbi-manager
  5. http://trac.pcbsd.org/brower/pcbsd/current/src-sh/pbi-manager/
  6. 6.0 6.1 http://lists.pcbsd.org/mailman/listinfo/pbi-dev
  7. http://en.wikipedia.org/wiki/Xdg
  8. http://www.freshports.org/devel/ccache/
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox