Create an ipk package from sources
From Openmoko
This is a small tutorial to help people that want to write new software for Openmoko, eg. to port existing programs.
Contents |
Theory
OpenMoko uses OpenEmbedded, a package and root image build system.
OpenEmbedded provides patches and Bitbake Recipes, that are similar to .spec files to build RPM packages. Bitbake is used to process recipes and obtain packages and FS images. For example, bitbake can fetch sources, apply patches, configure, make and install compiled sources. Other steps are user definable, but this is not the scope of this tutorial.
A standard use of bitbake is to download an existing "vanilla" package, ie "gcc" or "binutils" or "linux kernel", apply platform patches (adding bugfixes, porting fixes, drivers, etc...) then compile and package the resulting binaries.
Software versionning
Versions management is a very wide topic, there are tons of different opinions about it, I'll try to be quick.
Packages versions are important in a distribution building mechanism, and package versions are a complete part of the full package name. They are obviously used to track important changes made to your package, and isolate each version. Here we must be a little aware of basic version management. Current (most advanced) version is named the "trunk". Additions, fixes, etc. are done to the trunk.
Once a certain level of functionality is achieved, a copy of the trunk (a "snapshot") is taken, and named with a version number. As a result, current development continues on the trunk, but users can download well defined "releases", with known bugs and functionnality level. This is called "branching". If version specific bugs are found in version a.b, then we can update this branch independently.
In bitbake, we must specify which release of our software we are going to package. This may not be an habit, especially if you only develop small programs, not maintained by more than one or two person. If your software is named "myprogram" but has no real version number attached, you can just rename your project directory to "myprogram-1.0".
This is a good practice; If you are packaging a software, then it will be used by other people, so when you modify it next time, put it in a folder named "myprogram-1.1" :) This brings clarity to users, ease of management to you, and ability to plan future releases to your team.
Example
Say you have a working software package, ie a file system directory with sources, Makefile, etc... Because this is a common method, I will base my presentation on a project generated using autotools. This implies that the software can be installed somewhere by following the magical steps ./configure && make && make install.
Prerequisites
You need a configured openmoko development tree. All details are in Building OpenMoko from scratch
The following commands will give you one, they are copied from the above page. Just copy and paste in a shell.
#prepare cd $HOME mkdir OM cd OM #get openembedded wget http://www.openembedded.org/snapshots/OE.mtn.bz2 bunzip2 OE.mtn.bz2 mtn --db=OE.mtn pull monotone.openembedded.org org.openembedded.dev mtn --db=OE.mtn checkout --branch=org.openembedded.dev -r e2dbb52fe39df7ef786b6068f6178f29508dfded openembedded #get openmoko svn co http://svn.openmoko.org/ openmoko #create local config mkdir -p sources build/conf cat <<EOF >build/conf/local.conf MACHINE = "fic-gta01" DISTRO = "openmoko" BUILD_ARCH = "`uname -m`" EOF #fixes (didnt try without them) ln -s openmoko/trunk/oe ./oe cd sources mkdir -p ../build/tmp/stamps/armv4t-linux wget http://ftp.mozilla.org/pub/mozilla.org/js/older-packages/js-1.5.tar.gz touch ../build/tmp/stamps/armv4t-linux/js-1.5-r0.do_fetch wget http://us4.samba.org/samba/ftp/stable/samba-3.0.14a.tar.gz touch ../build/tmp/stamps/armv4t-linux/samba-3.0.14a-r15.do_fetch perl -pi.orig -e 's/ *$//;s/\r//g' ../openembedded/packages/gcc/gcc-4.1.1/gcc-4.1.1-pr13685-1.patch
Then, you need the environment variables described here: Building_OpenMoko_from_scratch#Environment_variables Put the given lines in a script, then go to your build/ dir and source it:
cat <<EOF >doenv.sh #!/bin/sh export OMDIR=`pwd` export BBPATH=$OMDIR/build:$OMDIR/openmoko/trunk/oe:$OMDIR/openembedded EOF . ./doenv.sh
After this, your tree will look like that:
$HOME/OM +- build | +- conf | | +- local.conf +- openmoko +- openembedded +- sources +- env.sh
etc...
Proposed directory layout for our new packages
Bitbake recipes are single files describing how to build a package. I recommend to have a recipe for each package version (myprogram-1.0.bb, myprogram-1.1.bb, etc.) all stored in a directory named "myprogram"
Our personal tree can be everywhere else:
$HOME +- $OMDIR (contains the official openmoko tree) | +- ... +- My_personal_OpenMoko_Packages | +- packages | +- myhello | +- myhello-1.0.bb +- My_Personal_Projects_Folder +- myhello +- myhello-1.0 +- configure.ac +- Makefile.am +- ...
To do a simple test, pack your sources in a tar, and put this tar on a web server:
cd My_personal_Projects_Folder/myhello tar cvfz myhello-1.0.tar.gz myhello-1.0/
This is correct, ie the tarball will have a NAME-VERSION subdirectory, that's what we need.
The recipe
In the bitbake file, you need to give
- a description for your package, variable name is DESCRIPTION
- a package name (PN)
- a package version (PV)
- a source URI, I'll give details later
- a source version, to checkout a particular revision of versionned trees
here's an example:
#package description DESCRIPTION = "An example application for openmoko, depending on a custom source repository" SECTION = "squalyl/myhello" #this is optional LICENSE = "GPLv2" PN = "myhello" PV = "1.0" #use "now" to force svn updates at each build SRCDATE_${PN} = "now" SRC_URI = "svn://repo/path/myhello;proto=https;module=${PN}-${PV}" #for a tarball use: #SRC_URI = "http://server/path/package/${PN}-{$PV}.tar.gz" #include standard rules to build a autoconf/automake package inherit autotools
Expressions starting by ${} are substituted, this allows you to download a tarball named "mypackage-1.0.tar.gz" for example
Source URIs
bitbake can get the source code for your project from a variety of locations including tarball, subversion and git.
All of this is explained in the bitbake manual, particulary here.
The package build directory, the one where bitbake will run "configure" et al, will follow the name template: "PACKAGE_NAME-PACKAGE_VERSION", so
- tarballs must extract to a subdirectory with this name
- svn top directory must have this name
It's very important. I spent 3 days blocking on this!
Tell bitbake to build our package
The only and one thing to do is edit $OMDIR/build/local.conf to tell bitbake to search our folders for .bb files:
BBFILES+="${HOME}/MypersonalOpenMokoPackages/packages/*/*.bb"
There is no need to change BBPATH or something else, this is only needed to add new configuration files, not recipes.
Running
$ cd $OMDIR/build $ bitbake mypackage
will bring all the necessary operations to create a package in $OMDIR/build/tmp/deploy/ipk/armv4t/myhello .....
The following is an example build log:
OE Build Configuration: BB_VERSION = "1.6.9" OE_REVISION = "<unknown>" TARGET_ARCH = "arm" TARGET_OS = "linux" MACHINE = "fic-gta01" DISTRO = "openmoko" DISTRO_VERSION = ".dev-snapshot-20070723" TARGET_FPU = "soft" NOTE: This BitBake doesn't support revision fetching, falling back to current date NOTE: preferred version 2.4 of glibc not available NOTE: preferred version 2.4 of glibc-intermediate not available NOTE: preferred version 2.4 of glibc not available NOTE: package myhello-1.0: started NOTE: package myhello-1.0-r0_0_200707231759: task do_fetch: started NOTE: Fetch svn://svn.unsads.com/squalyl/om/src/myhello;proto=https;module=myhello-1.0 A myhello-1.0/configure.ac A myhello-1.0/include A myhello-1.0/include/config.h.in A myhello-1.0/cleanall.sh A myhello-1.0/AUTHORS A myhello-1.0/bootstrap.sh A myhello-1.0/ChangeLog A myhello-1.0/src A myhello-1.0/src/hello.c A myhello-1.0/Makefile.am A myhello-1.0/NEWS A myhello-1.0/README Révision 2116 extraite. NOTE: package myhello-1.0-r0_0_200707231759: task do_fetch: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_unpack: started NOTE: Unpacking /home/squalyl/OM/sources/myhello-1.0_svn.unsads.com_.squalyl.om.src.myhello__now.tar.gz to /home/squalyl/OM/build/tmp/work/armv4t-linux/myhello-1.0-r0_0_200707231759/ NOTE: package myhello-1.0-r0_0_200707231759: task do_unpack: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_patch: started NOTE: package myhello-1.0-r0_0_200707231759: task do_patch: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_configure: started NOTE: package myhello-1.0-r0_0_200707231759: task do_configure: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_compile: started NOTE: package myhello-1.0-r0_0_200707231759: task do_compile: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_install: started NOTE: package myhello-1.0-r0_0_200707231759: task do_install: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_package: started NOTE: package myhello-1.0-r0_0_200707231759: task do_package: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_package_write: started Packaged contents of myhello-dbg into /home/squalyl/OM/build/tmp/deploy/ipk/armv4t/myhello-dbg_1.0-r0_0_200707231759_armv4t.ipk Packaged contents of myhello into /home/squalyl/OM/build/tmp/deploy/ipk/armv4t/myhello_1.0-r0_0_200707231759_armv4t.ipk NOTE: Not creating empty archive for myhello-doc-1.0-r0_0_200707231759 NOTE: Not creating empty archive for myhello-dev-1.0-r0_0_200707231759 NOTE: Not creating empty archive for myhello-locale-1.0-r0_0_200707231759 NOTE: package myhello-1.0-r0_0_200707231759: task do_package_write: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_populate_staging: started NOTE: package myhello-1.0-r0_0_200707231759: task do_populate_staging: completed NOTE: package myhello-1.0-r0_0_200707231759: task do_build: started NOTE: package myhello-1.0-r0_0_200707231759: task do_build: completed NOTE: package myhello-1.0: completed NOTE: build 200707231952: completed Build statistics: Attempted builds: 1
Get this package included in the rootfs image
bitbake openmoko-devel-image is used to build the entire distribution, but it does not include your new package in the rootfs image.
To do this, use:
bitbake openmoko-devel-image -crebuild
This will actually rebuild the rootfs, including your new packages.