MokoMakefile

From Openmoko

(Difference between revisions)
Jump to: navigation, search
(Another testimonial)
m (added hint to Toolchain)
 
(34 intermediate revisions by 17 users not shown)
Line 1: Line 1:
{{Languages|MokoMakefile}}
+
{{Languages|MokoMakefile (OBSOLETE)}}
  
 +
NOTE: According to the [http://mokomakefile.projects.openmoko.org/  source repository for MokoMakefile], this tool is obsolete. See [[Toolchain]] on how to develop for Openmoko.
  
 
MokoMakefile is a Makefile which saves lots of work when setting up an Openmoko build environment.
 
MokoMakefile is a Makefile which saves lots of work when setting up an Openmoko build environment.
Line 6: Line 7:
 
It brings the same repeatability to build environment creation and maintenance as that which the BitBake scripts bring to [[OpenEmbedded]] ease and standardize the process of building OpenEmbedded.
 
It brings the same repeatability to build environment creation and maintenance as that which the BitBake scripts bring to [[OpenEmbedded]] ease and standardize the process of building OpenEmbedded.
  
Unlike the manual process described at [[Building Openmoko from scratch]], MokoMakefile does not install anything into your system (it can and should be started as normal user).
+
MokoMakefile does not install anything into your system (it can and should be started as normal user). MokoMakefile is a wrapper around all that is required to make it easy to set up and maintain a development environment that fully complies with the setup instructions published by Openmoko.
MokoMakefile is a wrapper around all that is required to make it easy to set up and maintain a development environment that fully complies with the setup instructions published by Openmoko.
+
  
MokoMakefile is developed by [[User:RodWhitby|Rod Whitby]] - it is not an official product of Openmoko (although I would be happy for them to pick it up and use it internally).  If there is any discrepancy between the [[Openmoko2007.2#How_to_build|official Openmoko build instructions]], and the operation of the MokoMakefile, then you should consider the official instructions to be correct.
+
MokoMakefile is developed by [[User:RodWhitby|Rod Whitby]] - '''it is not an official product of Openmoko''' (although I would be happy for them to pick it up and use it internally).  If there is any discrepancy between the [http://downloads.openmoko.org/build/|official Openmoko build instructions], and the operation of the MokoMakefile, then you should consider the official instructions to be correct.
  
The MokoMakefile is able to build both OM-2007.2 and QTopia-X11 images.
+
The MokoMakefile is able to build the stable, testing and unstable versions of the FSO distribution images.
  
MokoMakefile also builds the QEMU-based Neo1973 emulator as part of the build process and has make targets to install the Openmoko images into it and run it. These commands can also be used without downloading and building the whole Openmoko OpenEmbedded distribution. This part is described in [[Using QEMU with MokoMakefile]].
+
MokoMakefile no longer builds the QEMU-based Neo1973 emulator.
  
 
== Requirements for building Openmoko ==
 
== Requirements for building Openmoko ==
Line 26: Line 26:
 
=== Required software ===
 
=== Required software ===
  
Some distribution specific hints on preparing your build host for building OpenEmbedded are on http://wiki.openembedded.net/index.php/OEandYourDistro but they may be outdated, incomplete and do not cover everything which Openmoko needs to build.
+
Some distribution specific hints on preparing your build host for building [[OpenEmbedded]] are on http://wiki.openembedded.net/index.php/OEandYourDistro but they may be outdated, incomplete and do not cover everything which Openmoko needs to build.
  
 
A good guide is [[Building Openmoko from scratch#Build host prerequisites|the section on build host prerequisites]] in [[Building Openmoko from scratch]]
 
A good guide is [[Building Openmoko from scratch#Build host prerequisites|the section on build host prerequisites]] in [[Building Openmoko from scratch]]
Line 34: Line 34:
 
==== Package requirements by distribution ====
 
==== Package requirements by distribution ====
 
Your distribution needs to provide these commands in order for OpenEmbedded to start building:
 
Your distribution needs to provide these commands in order for OpenEmbedded to start building:
subversion texi2html texinfo help2man
+
subversion texi2html texinfo help2man
  
 
Openmoko needs the development packages (with header files, development libraries and tools) in order to finish building:
 
Openmoko needs the development packages (with header files, development libraries and tools) in order to finish building:
ncurses zlib (or libz) OpenSSL GTK++
+
 
 +
<pre>ncurses zlib (or libz) OpenSSL GTK++</pre>
  
 
Because there are bugs in the interaction of QEMU and GCC-4, you'll need a copy of gcc-3.x installed as well.
 
Because there are bugs in the interaction of QEMU and GCC-4, you'll need a copy of gcc-3.x installed as well.
  
 
===== Debian / Ubuntu =====
 
===== Debian / Ubuntu =====
  sudo apt-get install subversion build-essential help2man diffstat texi2html texinfo cvs gawk zip unzip cogito
 
  
  sudo apt-get install libncurses5-dev zlib1g-dev libssl-dev libgtk2.0-dev  
+
<pre>sudo apt-get install subversion build-essential help2man diffstat texi2html texinfo cvs gawk zip unzip cogito bzip2
  # To prevent errors in host validation
+
sudo apt-get install libncurses5-dev zlib1g-dev libssl-dev libgtk2.0-dev
  sudo apt-get install ca-certificates
+
# To prevent errors in host validation
  # For Openmoko 2007.2 using BitBake-1.8.8:
+
sudo apt-get install ca-certificates
  sudo apt-get install python-pysqlite2 sqlite3 sqlite3-doc python-pysqlite2-dbg
+
# For Openmoko 2007.2 using BitBake-1.8.8:
  # For building faster
+
sudo apt-get install python-pysqlite2 sqlite3 sqlite3-doc python-pysqlite2-dbg
  sudo apt-get install quilt python-psyco ccache
+
# For building faster
  # For qemu, install a second compiler for bug avoidance; MokoMakefile knows to look for it.
+
sudo apt-get install quilt python-psyco ccache
  sudo apt-get install gcc-3.4 g++-3.4 libsdl1.2-dev lynx netpbm dosfstools
+
# For qemu, install a second compiler for bug avoidance; MokoMakefile knows to look for it.
 +
sudo apt-get install gcc-3.4 g++-3.4 libsdl1.2-dev lynx netpbm dosfstools
  
  # On Ubuntu 8.04, the following was required instead of cogito
+
# On Ubuntu 8.04 and and Debian testing/unstable, the following was required instead of cogito
  sudo apt-get install git-core
+
sudo apt-get install git-core
  # Ubuntu links /bin/sh to /bin/dash, but some scripts fail because they use pushd and popd, which dash does not support
+
# Ubuntu links /bin/sh to /bin/dash, but some scripts fail because they use pushd and popd, which dash does not support
  sudo rm /bin/sh
+
sudo dpkg-reconfigure dash
  sudo ln -s /bin/bash /bin/sh
+
# Select No when it asks you to install dash as /bin/sh.
  
  # This may be needed only for ASU
+
# This may be needed only for ASU
  sudo apt-get install libxtst-dev
+
sudo apt-get install libxtst-dev</pre>
  
 
===== SuSE =====
 
===== SuSE =====
 
For building Openmoko on 10.3, you need
 
For building Openmoko on 10.3, you need
gcc-c++ ncurses-devel zlib-devel libopenssl-devel gtk2-devel subversion diffstat texinfo help2man
+
 
 +
<pre>gcc-c++ ncurses-devel zlib-devel libopenssl-devel gtk2-devel subversion diffstat texinfo help2man</pre>
 +
 
 
For MokoMakefile to not fail on compiling qemu-user, you need to use gcc33:
 
For MokoMakefile to not fail on compiling qemu-user, you need to use gcc33:
  wget download.opensuse.org/repositories/devel:/tools:/gcc/openSUSE_Factory/i586/{cpp,gcc}33-3.3.3-41.8.i586.rpm
+
 
 +
  wget [http://download.opensuse.org/repositories/devel:/tools:/gcc/openSUSE_Factory/i586/{cpp,gcc}33-3.3.3-41.8.i586.rpm http://download.opensuse.org/repositories/devel:/tools:/gcc/openSUSE_Factory/i586/{cpp,gcc}33-3.3.3-41.8.i586.rpm]
 
  rpm -Uhv {cpp,gcc}33-3.3.3-41.8.i586.rpm
 
  rpm -Uhv {cpp,gcc}33-3.3.3-41.8.i586.rpm
  
Line 73: Line 77:
  
 
10.1 and 10.2: same packages as 10.3, but install <code>openssl-devel</code> instead of libopenssl-devel.
 
10.1 and 10.2: same packages as 10.3, but install <code>openssl-devel</code> instead of libopenssl-devel.
 +
 +
===== Mandriva =====
 +
 +
<pre>urpmi git help2man diffstat texi2html texinfo</pre>
 +
 +
(Tested with Mandriva 2009.0. <tt>kernel-desktop-devel</tt> of the current kernel is installed in advance, which installs compilers, binutils and many other general compilation tools.)
  
 
==== For all distributions ====
 
==== For all distributions ====
 +
 
As the QEMU-based neo1973 emulator is also built as part of the build process started by MokoMakefile, so you need gcc-3.3 and other packages for building QEMU installed. See [[Using QEMU with MokoMakefile#Build requirements|the build requirements section]] in [[Using QEMU with MokoMakefile]] for information on the required software.
 
As the QEMU-based neo1973 emulator is also built as part of the build process started by MokoMakefile, so you need gcc-3.3 and other packages for building QEMU installed. See [[Using QEMU with MokoMakefile#Build requirements|the build requirements section]] in [[Using QEMU with MokoMakefile]] for information on the required software.
  
 
If you are having problems compiling QEMU and do not need it, you can disable building of QEMU by adding following line to build/conf/local.conf file:
 
If you are having problems compiling QEMU and do not need it, you can disable building of QEMU by adding following line to build/conf/local.conf file:
ENABLE_BINARY_LOCALE_GENERATION = "0"
+
 
 +
<pre>ENABLE_BINARY_LOCALE_GENERATION = "0"</pre>
  
 
===== Proxy =====
 
===== Proxy =====
 +
 
Users behind a proxy should ensure that their proxy is configured in their wgetrc file as the http_proxy environment variable is unset by MokoMakefile.
 
Users behind a proxy should ensure that their proxy is configured in their wgetrc file as the http_proxy environment variable is unset by MokoMakefile.
  
Line 86: Line 99:
  
 
=== Preparations ===
 
=== Preparations ===
Create your $OMDIR directory (note that you can change ~/moko to any directory you like. If you have your ~moko dir mounted from a different file system, be sure to edit /etc/mtab to add the 'exec' permission to the file system, else bitbake will fail with error messages stating that /usr/bin/env is an invalid interpreter):
+
Create your ${OMDIR} directory (note that you can change ~/moko to any directory you like):
  mkdir ~/moko ; cd ~/moko
+
 
 +
<pre>mkdir ~/moko ; cd ~/moko</pre>
 +
 
 +
If you have your ~/moko dir mounted from a different file system, be sure to edit /etc/mtab to add the 'exec' permission to the file system, else bitbake will fail with error messages stating that /usr/bin/env is an invalid interpreter.
 +
If you ever want to move your $OMDIR to another location, you must run
 +
 
 +
<pre>make clobber ; make setup</pre>
 +
 
 +
to reinitialize the environment.  Yes, this will take a long time, so choose $OMDIR carefully.
  
 
=== MokoMakefile ===
 
=== MokoMakefile ===
Grab MokoMakefile:
 
  wget http://www.rwhitby.net/files/openmoko/Makefile
 
  
If that doesn't work, try
+
Grab MokoMakefile:
  
  wget http://svn.projects.openmoko.org/svnroot/mokomakefile/trunk/Makefile
+
wget [http://downloads.freesmartphone.org/Makefile http://downloads.freesmartphone.org/Makefile]
  
{{note|MokoMakefile uses BitBake 1.8.8 which requires python-sqlite2 and sqlite-3.3 or later. Users of SUSE Linux 10.1 can update to [http://download.opensuse.org/pub/opensuse/distribution/10.2/repo/oss/suse/i586/sqlite-3.3.8-14.i586.rpm the version of openSUSE 10.2]}}
+
{{note|MokoMakefile uses BitBake 1.8.x which requires python-sqlite2 and sqlite-3.3 or later. Users of SUSE Linux 10.1 can update to [http://download.opensuse.org/pub/opensuse/distribution/10.2/repo/oss/suse/i586/sqlite-3.3.8-14.i586.rpm the version of openSUSE 10.2]}}
  
 
=== Environment ===
 
=== Environment ===
If you want to make the ASU or FSO instead of the OM2007.2, see [[Distributions]], you need to edit your Makefile.  Change OM_GIT_BRANCH to one of the following:
 
  OM_GIT_BRANCH := org.openmoko.dev        # OM2007.2 (enabled by default)
 
  OM_GIT_BRANCH := org.openmoko.asu.dev    # ASU
 
  OM_GIT_BRANCH := org.openmoko.asu.stable # ASU stable
 
  OM_GIT_BRANCH := org.openembedded.dev    # FSO
 
  
 
Set up the environment with:
 
Set up the environment with:
  
  make setup
+
<pre>make setup</pre>
 
+
If you are building for the Neo1973 GTA01, then also do this
+
  make setup-machine-neo
+
If you are building for the Freerunner GTA02, then also do this
+
  make setup-machine-freerunner
+
  
 
=== Building ===
 
=== Building ===
Before starting a lengthy make process, check the [[#Tips|Tips]] section below for how to make Make multicore aware. You may want to modify the build/conf/local.conf file for your target (emulation/chroot) environment:
 
  make openmoko-devel-image
 
  (or use "make openmoko-qtopia-x11-image" if you're building the ASU)
 
  (or use "make fso-image" if you're building the FSO)
 
  
''Hint:'' The images build with this command can be found under {$OMDIR}/build/tmp/deploy/images/neo1973
+
Before starting a lengthy make process, check the [[#Tips|Tips]] section below for how to make Make multicore aware. You may want to modify the build/conf/local.conf file for your target (emulation/chroot) environment.
  
This will set up the recommended directory structure as described in [[Building Openmoko from scratch]], will download all the required software (from the right places with the right versions), and will immediately start building an image.
+
<pre>make fso-gta02-testing-image</pre>
  
Once you have done this, you can choose to continue using the MokoMakefile to initiate your subsequent builds, or you can go into the build directory and run bitbake commands manually.  The choice is yours.
+
or
 +
 
 +
<pre>make fso-gta01-testing-image</pre>
 +
 
 +
''Hint:'' The images build with this command can be found under ${OMDIR}/fso-testing/tmp/deploy/glibc/images/
 +
 
 +
This will set up the recommended directory structure, will download all the required software (from the right places with the right versions), and will immediately start building an image.
 +
 
 +
Once you have done this, you can choose to continue using the MokoMakefile to initiate your subsequent builds, or you can go into the fso-testing directory and run bitbake commands manually.  The choice is yours.
 +
 
 +
With luck - you should now have images in your ${OMDIR}/fso-testing/tmp/deploy/glibc/images/ directory. You can then [[Flashing the Neo FreeRunner|flash these]].
  
 
==Updating the environment==
 
==Updating the environment==
 +
 
For easy maintenance of your build environment the following commands are available.
 
For easy maintenance of your build environment the following commands are available.
  
 
1 - To update the MokoMakefile to the latest version:
 
1 - To update the MokoMakefile to the latest version:
  make update-makefile
 
  
2 - To make sure that any recent changes to the build directory structure have been applied:
+
<pre>make update-common</pre>
  make setup
+
  
3 - To update the Openmoko repository checkout and the MokoMakefile patches to the latest version:
+
Be aware that any changes you previously made to the Makefile are lost when executing ''update-makefile''.
  make update
+
  
A quick way to rebuild a new image with the latest updates:
+
2 - To make sure that any recent changes to the build directory structure have been applied:
  make update-makefile && make setup update openmoko-devel-image
+
  
==Build issues==
+
<pre>make setup</pre>
First, make sure that the problem is reproducible after running
+
  
make update-makefile && make setup && make update
+
3 - To update the Openmoko repository checkouts and the MokoMakefile patches to the latest version:
  
then run
+
<pre>make update</pre>
  
make clean-package-<foo>
+
A quick way to rebuild a new image with the latest updates:
  
(where you replace <foo> with the name of the package which is failing)
+
<pre>make update-common && make setup update fso-gta02-testing-image</pre>
  
and finally
+
==Build issues==
  
make openmoko-devel-image
+
First, make sure that the problem is reproducible after running
  
With luck - you should now have images in your $omdir/build/tmp/deploy/glibc/images/neo1973/ directory.
+
<pre>make update-common && make setup && make update</pre>
You can then [[Flashing openmoko|flash these]].
+
  
 +
If not, and you can get the error to occur three times in a row after running that sequence of commands (including the update and setup steps) three times, then feel free to report it in the Openmoko development Trac at https://docs.openmoko.org/trac/wiki/NewTicket or in #openmoko on [[Development_resources#IRC|IRC]].  If bitbake has started to run, then it's almost guaranteed that the problem is in the OE build recipes, *not* in the MokoMakefile itself (which is just a convenience wrapper around the OpenEmbedded build system).  Please do not report build recipe errors to the MokoMakefile author.
  
If not, and you can get the error to occur three times in a row after running that sequence of commands (including the update and setup steps) three times, then feel free to report it to rwhitby in #openmoko on [[Development_resources#IRC|IRC]].
+
If you have chronic build problems, cleaning your environment variables and re-running the build from scratch may help. The following script should remove all non-essential variables; you should also manually trim your path to the minimum.
  
If you have chronic build problems, cleaning your environment variables and re-running the build from scratch may help.  The following script should remove all non-essential variables; you should also manually trim your path to the minimum.
+
<pre>`env | grep -Ev '^(SHELL|TERM|OLDPWD|USER|USERNAME|PATH|EDITOR|LANG|HOME|DISPLAY|_)=' \
`env | grep -Ev '^(SHELL|TERM|OLDPWD|USER|USERNAME|PATH|EDITOR|LANG|HOME|DISPLAY|_)=' \
+
| sed 's/^\(.*\)=.*/unset \1/'`</pre>
| sed 's/^\(.*\)=.*/unset \1/'`
+
  
 
(you must include the backticks).
 
(you must include the backticks).
 +
 
Note that on some boxes it is necessary to call "make" by typing "/usr/bin/make" afterwards.
 
Note that on some boxes it is necessary to call "make" by typing "/usr/bin/make" afterwards.
  
 +
On current Ubuntu/Debian unstable/testing boxes another trick is necessary. glibc_2.6.1 fails to build since it is unable to generate the necessary locales until you type in the following line:
 +
 +
<pre>sudo sysctl -w vm.mmap_min_addr=0</pre>
  
On current Ubuntu boxes another trick is necessary:
 
glibc_2.6.1 fails to build since it is unable to generate the necessary locales until you type in the following line:
 
sudo sysctl -w vm.mmap_min_addr=0
 
 
Note that this temporarily disables a new security vulnerability check that has entered /etc/sysctrl - but that makes problems with loads of applications.
 
Note that this temporarily disables a new security vulnerability check that has entered /etc/sysctrl - but that makes problems with loads of applications.
 +
 
To re-enable this check do the following:
 
To re-enable this check do the following:
sudo sysctl -w vm.mmap_min_addr=65536
+
 
 +
<pre>sudo sysctl -w vm.mmap_min_addr=65536</pre>
 +
 
 
See the talk page of this article for more details on this solution.
 
See the talk page of this article for more details on this solution.
  
 
===Known MokoMakefile errors ===
 
===Known MokoMakefile errors ===
 +
 
==== SVN Server Certificate Errors ====
 
==== SVN Server Certificate Errors ====
 +
 
If you get an error like the following:
 
If you get an error like the following:
<pre>
+
 
NOTE: Fetch svn://svn.projects.openmoko.org/svnroot/;module=assassin;proto=https
+
<pre>NOTE: Fetch svn://svn.projects.openmoko.org/svnroot/;module=assassin;proto=https
 
Error validating server certificate for 'https://svn.projects.openmoko.org:443':
 
Error validating server certificate for 'https://svn.projects.openmoko.org:443':
- The certificate is not issued by a trusted authority. Use the
+
- The certificate is not issued by a trusted authority. Use the
  fingerprint to validate the certificate manually!
+
fingerprint to validate the certificate manually!
 
Certificate information:
 
Certificate information:
- Hostname: projects.openmoko.org
+
- Hostname: projects.openmoko.org
- Valid: from Thu, 05 Jun 2008 01:02:42 GMT until Tue, 02 Dec 2008 01:02:42 GMT
+
- Valid: from Thu, 05 Jun 2008 01:02:42 GMT until Tue, 02 Dec 2008 01:02:42 GMT
- Issuer: http://www.cacert.org, Root CA
+
- Issuer: http://www.cacert.org, Root CA
- Fingerprint: a5:7d:4e:37:e8:94:ee:ba:c1:e8:e9:4b:33:cb:34:91:a9:6d:d3:84
+
- Fingerprint: a5:7d:4e:37:e8:94:ee:ba:c1:e8:e9:4b:33:cb:34:91:a9:6d:d3:84
(R)eject, accept (t)emporarily or accept (p)ermanently? svn: OPTIONS of 'https://svn.projects.openmoko.org/svnroot/assassin': Server certificate verification failed: issuer is not trusted (https://svn.projects.openmoko.org)
+
(R)eject, accept (t)emporarily or accept (p)ermanently?
</pre>
+
svn: OPTIONS of 'https://svn.projects.openmoko.org/svnroot/assassin':
 +
Server certificate verification failed: issuer is not trusted (https://svn.projects.openmoko.org)</pre>
  
 
It means you haven't yet accepted the server certificate for the SVN server.  To remedy this, execute the following commands, and hit "p" (to permanently accept the certificate) when prompted:
 
It means you haven't yet accepted the server certificate for the SVN server.  To remedy this, execute the following commands, and hit "p" (to permanently accept the certificate) when prompted:
<pre>
+
 
cd /tmp
+
<pre>cd /tmp
 
svn co https://svn.projects.openmoko.org/svnroot/assassin
 
svn co https://svn.projects.openmoko.org/svnroot/assassin
 
rm -rf assassin
 
rm -rf assassin
cd -
+
cd -</pre>
</pre>
+
  
 
==== exposure_svn and Subversion 1.5.0 ====
 
==== exposure_svn and Subversion 1.5.0 ====
Line 207: Line 223:
 
If you get an error like this:
 
If you get an error like this:
  
<code>
+
<code>"NameError: global name 'log' is not defined"</code>
"NameError: global name 'log' is not defined"
+
</code>
+
  
 
There's a bug with subversion 1.5.0 -- downgrade to 1.4.x (or check out the latest subversion sources and use those as they're fixed) and re-fetch the sources.
 
There's a bug with subversion 1.5.0 -- downgrade to 1.4.x (or check out the latest subversion sources and use those as they're fixed) and re-fetch the sources.
  
===Fixes for distribution/environment-specific or isolated issues===
+
=== Fixes for distribution/environment-specific or isolated issues ===
  
 
Work-arounds for temporary or isolated problems can be found and should be added to the [[Talk:MokoMakefile|Discussion page]] which is associated with this page.  As they are fixed, they will be removed from that page.
 
Work-arounds for temporary or isolated problems can be found and should be added to the [[Talk:MokoMakefile|Discussion page]] which is associated with this page.  As they are fixed, they will be removed from that page.
  
 
==Tips==
 
==Tips==
*Make sure umask is set to 0022 before you run "make setup"
 
  
*You can reduce the amount of consumed disk space significantly by adding
+
* Make sure umask is set to 0022 before you run "make setup"
  INHERIT += "rm_work"
+
* If a certain package does not build due to corrupted download or some such try to remove the sources and rebuild it.
in your local.conf (e.g. ~/moko/build/conf/local.conf). This will remove the contents of each build/tmp/work/*/<package> directory after the corresponding package builds correctly. As of 10/16/07, this appears to be present in local.conf by default.
+
<pre>rm sources/<package>*
 +
cd fso-testing
 +
. ./setup-env
 +
bitbake -crebuild <package></pre>
 +
After that your build might just work again.
  
*If a certain package does not build due to corrupted download or some such try to remove the sources and rebuild it.
+
* For people with multiple CPU's (or dual-core ones) this small patch might be useful to build things faster.
rm sources/<package>*
+
cd build
+
. ../setup-env
+
bitbake -crebuild <package>
+
after that your build might just work again.
+
 
+
*For people with multiple CPU's (or dual-core ones) this small patch might be useful to build things faster.
+
 
Edit the local.conf and add the following lines:
 
Edit the local.conf and add the following lines:
PARALLEL_MAKE = "-j 4"
+
<pre>PARALLEL_MAKE = "-j 4"
BB_NUMBER_THREADS = "4"
+
BB_NUMBER_THREADS = "4"</pre>
  
 
Change the PARALLEL_MAKE and BB_NUMBER_THREADS values to something that suits better if it chokes your machine.
 
Change the PARALLEL_MAKE and BB_NUMBER_THREADS values to something that suits better if it chokes your machine.
  
* I have some problem with the parallel building, mkfs.jffs failed to build the image because the directory was modify while the building of the fs. [[User:Patrick.hetu|patrick.hetu]] 02:19, 27 May 2008 (UTC)  
+
* I have some problem with the parallel building, mkfs.jffs failed to build the image because the directory was modify while the building of the fs. [[User:Patrick.hetu|patrick.hetu]] 02:19, 27 May 2008 (UTC)
  
 
* If you encounter an error related with the qemu-native package and not compiling for the qemu, you can edit the build/conf/local.conf file and add ENABLE_BINARY_LOCALE_GENERATION = "0" line to avoid the error.
 
* If you encounter an error related with the qemu-native package and not compiling for the qemu, you can edit the build/conf/local.conf file and add ENABLE_BINARY_LOCALE_GENERATION = "0" line to avoid the error.
  
 
* To prevent building tons of locales, add a line like this to local.conf:
 
* To prevent building tons of locales, add a line like this to local.conf:
GLIBC_GENERATE_LOCALES = "en_US.UTF-8 nl_NL.UTF-8"
+
<pre>GLIBC_GENERATE_LOCALES = "en_US.UTF-8 nl_NL.UTF-8"</pre>
  
 
* To not build any binary locales at all, add this to local.conf:
 
* To not build any binary locales at all, add this to local.conf:
ENABLE_BINARY_LOCALE_GENERATION = "0"
+
<pre>ENABLE_BINARY_LOCALE_GENERATION = "0"</pre>
  
* If you want to rebuild the package indexes (for instance, after compiling a new version of a package) without building <code>openmoko-devel-image</code>, run <code>make build-package-package-index</code>.
+
* If you want to rebuild the package indexes (for instance, after compiling a new version of a package) without building the whole <code>image</code>, run <code>make build-package-package-index</code>.
  
 
[[User:Wurp]] will update this to be a little more comprehensible, but maybe it can help someone as-is in the meantime:
 
[[User:Wurp]] will update this to be a little more comprehensible, but maybe it can help someone as-is in the meantime:
Line 256: Line 266:
  
 
=== Useful commands ===
 
=== Useful commands ===
 +
 
Here is the list (not complete) of useful MokoMakefile commands (actually make targets) and a short description of each one. These should be run inside $OMDIR directory.
 
Here is the list (not complete) of useful MokoMakefile commands (actually make targets) and a short description of each one. These should be run inside $OMDIR directory.
  
; make openmoko-devel-image : build the full development image
+
; make fso-gta02-testing-image : build the full development image
  
; make openmoko-devel-tools : build statically-linked [[dfu-util]] and [[openocd]] binaries
+
; make fso-gta02-unstable-image : build the unstable bleeding edge image (it often will not build or run)
  
; make openmoko-toolchain : builds [[toolchain]]
+
; make update-common : updates MokoMakefile
 
+
; make qemu : builds qemu, downloads flash images from official repository, flashes qemu emulated Neo1973 and runs emulator (see also [[Openmoko under QEMU]]).
+
 
+
; make qemu-local : same as ''make qemu'', but uses locally built images instead of official ones.
+
 
+
; make run-qemu : runs qemu with usb keyboard ''(note: qemu must first be built!)''. This target starts the emulator with some predefined set of options. If you need to start it with other options (enable telnet, etc), see [[QEMU_command_line]].
+
 
+
; make build-package-<package name> : buids ''<package name>'' package
+
 
+
; make update-makefile : updates MokoMakefile
+
 
+
; make clean-package-<foo>: cleans the "<foo>" package
+
  
 
; make setup : to make sure that any recent changes to the build directory structure have been applied
 
; make setup : to make sure that any recent changes to the build directory structure have been applied
 
 
; make update : to update the Openmoko repository checkout and the MokoMakefile patches to the latest version
 
  
; make setup-machine-neo : setup for GTA01 version of Neo1973
+
; make update : to update the Openmoko repository checkout and the MokoMakefile patches to the latest version
 
+
; make setup-machine-freerunner : setup for GTA02 version of Freerunner
+
 
+
==Developing with MokoMakefile==
+
 
+
{{note|If using MokoMakefile with OM2007.2 then references to $OMDIR/openmoko should be replaced with $OMDIR/openembedded.  Also references to tmp/work/armv4t-linux should be replaced with tmp/work/fic-gta01-angstrom-linux-gnueabi}}
+
 
+
For the following explanations $OMDIR is the directory where there Makefile puts all the stuff.
+
 
+
To make in-tree changes and have them built and used by qemu:
+
 
+
  cd $OMDIR/openmoko
+
  quilt new descriptive-patch-name.patch
+
  quilt add trunk/src/name-of-file-to-change # do this for every file you are about to modify
+
  ...make the changes...
+
  quilt refresh # this creates a file in $OMDIR/patches/openmoko-HEAD/ and updates the quilt series file there
+
 
+
Note: Do '''NOT''' use absolute paths as this confuses quilt and will get you a diff of the file against /dev/null!
+
 
+
To build the changes and have them used by qemu:
+
 
+
  make build-qemu
+
  make flash-qemu-local
+
  make run-qemu
+
 
+
If you want to modify applications instead of the openmoko toolchain, this is what you have to do (example: openmoko-messages):
+
 
+
  cd $OMDIR/build
+
  . ../setup-env
+
  bitbake -c patch openmoko-messages
+
  cd tmp/work/armv4t-linux/openmoko-messages-0.0.1+svnnow-r2_2276/openmoko-messages/
+
  ''...make the changes...''
+
  cd -
+
  bitbake -c build openmoko-messages
+
 
+
Note: if you just use <tt>bitbake openmoko-messages</tt> it will remove the sources after building!
+
 
+
Then continue with MokoMakefile usage.
+
 
+
If you want to add an application to your openmoko distribution, do this:
+
All file edits should be done using quilt as described above. That way a patch can easily be submitted to the openmoko project.
+
First, create a directory that will correspond to your package and edit a '''.bb''' file in there:
+
  cd $OMDIR/openmoko/
+
  quilt new mycoolpackage.patch
+
  mkdir trunk/openembedded/packages/mycoolpackage
+
  quilt add trunk/openembedded/packages/mycoolpackage/mycoolpackage_1.bb
+
  quilt edit trunk/openembedded/packages/mycoolpackage/mycoolpackage_1.bb
+
 
+
The file should have the following content:
+
  DESCRIPTION = "This is a cool package"
+
  SECTION = "username/mycoolpackage"
+
  PV = "1"
+
 
+
  inherit autotools
+
 
+
  SRC_URI = "http://www.example.com/download/mycoolpackage-1.tar.gz"
+
 
+
Explanation:
+
* DESCRIPTION - Just a short text explaining the package
+
* SECTION - I have no clue, but I'll use username/mycoolpackage for now
+
* PV - Package Version
+
* inherit autotools - The package can be compiled by './configure && make && make install' so we tell MokoMakefile to do it this way.
+
* SRC_URI = ... - This is the download location of the package source. It's imperative that the tar.gz contains a directory called '''packagename-packageversion''' (in this case: mycoolpackage-1) so that MokoMakefile can find it automatically or the build will fail.
+
 
+
This is not all. We also need to tell MokoMakfile that it needs to build and include the package in the image. To do this, do
+
  $OMDIR/openmoko# quilt edit trunk/openembedded/packages/tasks/task-openmoko.bb
+
Here, increase the value '''PR''' by one and add '''mycoolpackage \''' (with the backslash!) just before the line reading '''#  update-alternatives \'''.
+
 
+
Now run
+
  quilt refresh
+
  cd ..
+
  make update openmoko-devel-image
+
  
And if everything's alright you should now have an OpenMoko image to flash to your phone or run in qemu as described above.
+
== Developing with MokoMakefile ==
  
=== Hello World application ===
+
You are not advised to use MokoMakefile for application development.  Openmoko distributes a stand-alone toolchain for that purpose.
  
There is a [[Building a hello world application|Hello World!]] tutorial available too.
+
== Testimonials ==
  
==Testimonials==
 
 
MokoMakefile is recommended by 4 out of 4 new developers on #openmoko, with testimonials such as "For some reason last night I couldn't get my manual install of everything to work (bb complained about my bbpath I think) ... but with your makefile, it works great!", "MokoMakefile rocks!", "Wow this build system is nice - it just seems more polished than my gumstix toolchain buildroot system", "make openmoko-devel-image :) - no magic there", and "I have never been able to get a build to complete in the pure OE environment, whereas Mokomakefile for some reason builds fine."
 
MokoMakefile is recommended by 4 out of 4 new developers on #openmoko, with testimonials such as "For some reason last night I couldn't get my manual install of everything to work (bb complained about my bbpath I think) ... but with your makefile, it works great!", "MokoMakefile rocks!", "Wow this build system is nice - it just seems more polished than my gumstix toolchain buildroot system", "make openmoko-devel-image :) - no magic there", and "I have never been able to get a build to complete in the pure OE environment, whereas Mokomakefile for some reason builds fine."
  
 
Project page:
 
Project page:
http://mokomakefile.projects.openmoko.org/
+
*http://mokomakefile.projects.openmoko.org/
  
[[Category:Openmoko]]
+
[[Category:Developer resources]]
[[Category:Applications]]
+
[[Category:Application Developer]]
+

Latest revision as of 21:27, 3 September 2009


NOTE: According to the source repository for MokoMakefile, this tool is obsolete. See Toolchain on how to develop for Openmoko.

MokoMakefile is a Makefile which saves lots of work when setting up an Openmoko build environment. By automating the setup process of a new Openmoko build environment, it provides an environment which is configured the same for all the existing developers and should therefore be preferred over manual procedures or individual setup procedures. It brings the same repeatability to build environment creation and maintenance as that which the BitBake scripts bring to OpenEmbedded ease and standardize the process of building OpenEmbedded.

MokoMakefile does not install anything into your system (it can and should be started as normal user). MokoMakefile is a wrapper around all that is required to make it easy to set up and maintain a development environment that fully complies with the setup instructions published by Openmoko.

MokoMakefile is developed by Rod Whitby - it is not an official product of Openmoko (although I would be happy for them to pick it up and use it internally). If there is any discrepancy between the Openmoko build instructions, and the operation of the MokoMakefile, then you should consider the official instructions to be correct.

The MokoMakefile is able to build the stable, testing and unstable versions of the FSO distribution images.

MokoMakefile no longer builds the QEMU-based Neo1973 emulator.

Contents

[edit] Requirements for building Openmoko

Independent on whether MokoMakefile or a manual process is used to setup an Openmoko build environment, there are several requirements which must be fulfilled in order for the Openmoko build to succeed:

  • RAM: The build host needs to have at least 512MB of RAM, and about the same amount of swap. Some packages built by OpenEmbedded like busybox are built by compiling all source files into one binary which causes gcc to grow beyond 300MB of size and no part of this memory may be on swap for the compile to finish in predictable time. For busybox, this can be turned off, but turning this off means that busybox will not as well optimized by gcc.
  • Disk space: You need about 12 GB of available disk space for the Openmoko build to succeed (see below for a tip on how to reduce this).
  • Time: The initial build takes at least 5 hours (on 2GHz core2duo without multiprocessor optimization) and may take several days on slower machines.

[edit] Required software

Some distribution specific hints on preparing your build host for building OpenEmbedded are on http://wiki.openembedded.net/index.php/OEandYourDistro but they may be outdated, incomplete and do not cover everything which Openmoko needs to build.

A good guide is the section on build host prerequisites in Building Openmoko from scratch

If you forgot anything which OE needs itself, OE will tell you shortly after you start building, but it does not check build dependencies of Openmoko, so you either have to install them before starting or install them after the build failed. OpenEmbedded will continue where it stopped when you restart the build afterwards.

[edit] Package requirements by distribution

Your distribution needs to provide these commands in order for OpenEmbedded to start building: subversion texi2html texinfo help2man

Openmoko needs the development packages (with header files, development libraries and tools) in order to finish building:

ncurses zlib (or libz) OpenSSL GTK++

Because there are bugs in the interaction of QEMU and GCC-4, you'll need a copy of gcc-3.x installed as well.

[edit] Debian / Ubuntu
sudo apt-get install subversion build-essential help2man diffstat texi2html texinfo cvs gawk zip unzip cogito bzip2
sudo apt-get install libncurses5-dev zlib1g-dev libssl-dev libgtk2.0-dev
# To prevent errors in host validation
sudo apt-get install ca-certificates
# For Openmoko 2007.2 using BitBake-1.8.8:
sudo apt-get install python-pysqlite2 sqlite3 sqlite3-doc python-pysqlite2-dbg
# For building faster
sudo apt-get install quilt python-psyco ccache
# For qemu, install a second compiler for bug avoidance; MokoMakefile knows to look for it.
sudo apt-get install gcc-3.4 g++-3.4 libsdl1.2-dev lynx netpbm dosfstools

# On Ubuntu 8.04 and and Debian testing/unstable, the following was required instead of cogito
sudo apt-get install git-core
# Ubuntu links /bin/sh to /bin/dash, but some scripts fail because they use pushd and popd, which dash does not support
sudo dpkg-reconfigure dash
# Select No when it asks you to install dash as /bin/sh.

# This may be needed only for ASU
sudo apt-get install libxtst-dev
[edit] SuSE

For building Openmoko on 10.3, you need

gcc-c++ ncurses-devel zlib-devel libopenssl-devel gtk2-devel subversion diffstat texinfo help2man

For MokoMakefile to not fail on compiling qemu-user, you need to use gcc33:

wget http://download.opensuse.org/repositories/devel:/tools:/gcc/openSUSE_Factory/i586/{cpp,gcc}33-3.3.3-41.8.i586.rpm
rpm -Uhv {cpp,gcc}33-3.3.3-41.8.i586.rpm

See also the Talk page on Building on SuSE Linux 10.3-AMD64

10.1 and 10.2: same packages as 10.3, but install openssl-devel instead of libopenssl-devel.

[edit] Mandriva
urpmi git help2man diffstat texi2html texinfo

(Tested with Mandriva 2009.0. kernel-desktop-devel of the current kernel is installed in advance, which installs compilers, binutils and many other general compilation tools.)

[edit] For all distributions

As the QEMU-based neo1973 emulator is also built as part of the build process started by MokoMakefile, so you need gcc-3.3 and other packages for building QEMU installed. See the build requirements section in Using QEMU with MokoMakefile for information on the required software.

If you are having problems compiling QEMU and do not need it, you can disable building of QEMU by adding following line to build/conf/local.conf file:

ENABLE_BINARY_LOCALE_GENERATION = "0"
[edit] Proxy

Users behind a proxy should ensure that their proxy is configured in their wgetrc file as the http_proxy environment variable is unset by MokoMakefile.

[edit] Building Openmoko with MokoMakefile

[edit] Preparations

Create your ${OMDIR} directory (note that you can change ~/moko to any directory you like):

mkdir ~/moko ; cd ~/moko

If you have your ~/moko dir mounted from a different file system, be sure to edit /etc/mtab to add the 'exec' permission to the file system, else bitbake will fail with error messages stating that /usr/bin/env is an invalid interpreter. If you ever want to move your $OMDIR to another location, you must run

make clobber ; make setup

to reinitialize the environment. Yes, this will take a long time, so choose $OMDIR carefully.

[edit] MokoMakefile

Grab MokoMakefile:

wget http://downloads.freesmartphone.org/Makefile
NOTE: MokoMakefile uses BitBake 1.8.x which requires python-sqlite2 and sqlite-3.3 or later. Users of SUSE Linux 10.1 can update to the version of openSUSE 10.2


[edit] Environment

Set up the environment with:

make setup

[edit] Building

Before starting a lengthy make process, check the Tips section below for how to make Make multicore aware. You may want to modify the build/conf/local.conf file for your target (emulation/chroot) environment.

make fso-gta02-testing-image

or

make fso-gta01-testing-image

Hint: The images build with this command can be found under ${OMDIR}/fso-testing/tmp/deploy/glibc/images/

This will set up the recommended directory structure, will download all the required software (from the right places with the right versions), and will immediately start building an image.

Once you have done this, you can choose to continue using the MokoMakefile to initiate your subsequent builds, or you can go into the fso-testing directory and run bitbake commands manually. The choice is yours.

With luck - you should now have images in your ${OMDIR}/fso-testing/tmp/deploy/glibc/images/ directory. You can then flash these.

[edit] Updating the environment

For easy maintenance of your build environment the following commands are available.

1 - To update the MokoMakefile to the latest version:

make update-common

Be aware that any changes you previously made to the Makefile are lost when executing update-makefile.

2 - To make sure that any recent changes to the build directory structure have been applied:

make setup

3 - To update the Openmoko repository checkouts and the MokoMakefile patches to the latest version:

make update

A quick way to rebuild a new image with the latest updates:

make update-common && make setup update fso-gta02-testing-image

[edit] Build issues

First, make sure that the problem is reproducible after running

make update-common && make setup && make update

If not, and you can get the error to occur three times in a row after running that sequence of commands (including the update and setup steps) three times, then feel free to report it in the Openmoko development Trac at https://docs.openmoko.org/trac/wiki/NewTicket or in #openmoko on IRC. If bitbake has started to run, then it's almost guaranteed that the problem is in the OE build recipes, *not* in the MokoMakefile itself (which is just a convenience wrapper around the OpenEmbedded build system). Please do not report build recipe errors to the MokoMakefile author.

If you have chronic build problems, cleaning your environment variables and re-running the build from scratch may help. The following script should remove all non-essential variables; you should also manually trim your path to the minimum.

`env | grep -Ev '^(SHELL|TERM|OLDPWD|USER|USERNAME|PATH|EDITOR|LANG|HOME|DISPLAY|_)=' \
| sed 's/^\(.*\)=.*/unset \1/'`

(you must include the backticks).

Note that on some boxes it is necessary to call "make" by typing "/usr/bin/make" afterwards.

On current Ubuntu/Debian unstable/testing boxes another trick is necessary. glibc_2.6.1 fails to build since it is unable to generate the necessary locales until you type in the following line:

sudo sysctl -w vm.mmap_min_addr=0

Note that this temporarily disables a new security vulnerability check that has entered /etc/sysctrl - but that makes problems with loads of applications.

To re-enable this check do the following:

sudo sysctl -w vm.mmap_min_addr=65536

See the talk page of this article for more details on this solution.

[edit] Known MokoMakefile errors

[edit] SVN Server Certificate Errors

If you get an error like the following:

NOTE: Fetch svn://svn.projects.openmoko.org/svnroot/;module=assassin;proto=https
Error validating server certificate for 'https://svn.projects.openmoko.org:443':
- The certificate is not issued by a trusted authority. Use the
fingerprint to validate the certificate manually!
Certificate information:
- Hostname: projects.openmoko.org
- Valid: from Thu, 05 Jun 2008 01:02:42 GMT until Tue, 02 Dec 2008 01:02:42 GMT
- Issuer: http://www.cacert.org, Root CA
- Fingerprint: a5:7d:4e:37:e8:94:ee:ba:c1:e8:e9:4b:33:cb:34:91:a9:6d:d3:84
(R)eject, accept (t)emporarily or accept (p)ermanently?
svn: OPTIONS of 'https://svn.projects.openmoko.org/svnroot/assassin':
Server certificate verification failed: issuer is not trusted (https://svn.projects.openmoko.org)

It means you haven't yet accepted the server certificate for the SVN server. To remedy this, execute the following commands, and hit "p" (to permanently accept the certificate) when prompted:

cd /tmp
svn co https://svn.projects.openmoko.org/svnroot/assassin
rm -rf assassin
cd -

[edit] exposure_svn and Subversion 1.5.0

If you get an error like this:

"NameError: global name 'log' is not defined"

There's a bug with subversion 1.5.0 -- downgrade to 1.4.x (or check out the latest subversion sources and use those as they're fixed) and re-fetch the sources.

[edit] Fixes for distribution/environment-specific or isolated issues

Work-arounds for temporary or isolated problems can be found and should be added to the Discussion page which is associated with this page. As they are fixed, they will be removed from that page.

[edit] Tips

  • Make sure umask is set to 0022 before you run "make setup"
  • If a certain package does not build due to corrupted download or some such try to remove the sources and rebuild it.
rm sources/<package>*
cd fso-testing
. ./setup-env
bitbake -crebuild <package>

After that your build might just work again.

  • For people with multiple CPU's (or dual-core ones) this small patch might be useful to build things faster.

Edit the local.conf and add the following lines:

PARALLEL_MAKE = "-j 4"
BB_NUMBER_THREADS = "4"

Change the PARALLEL_MAKE and BB_NUMBER_THREADS values to something that suits better if it chokes your machine.

  • I have some problem with the parallel building, mkfs.jffs failed to build the image because the directory was modify while the building of the fs. patrick.hetu 02:19, 27 May 2008 (UTC)
  • If you encounter an error related with the qemu-native package and not compiling for the qemu, you can edit the build/conf/local.conf file and add ENABLE_BINARY_LOCALE_GENERATION = "0" line to avoid the error.
  • To prevent building tons of locales, add a line like this to local.conf:
GLIBC_GENERATE_LOCALES = "en_US.UTF-8 nl_NL.UTF-8"
  • To not build any binary locales at all, add this to local.conf:
ENABLE_BINARY_LOCALE_GENERATION = "0"
  • If you want to rebuild the package indexes (for instance, after compiling a new version of a package) without building the whole image, run make build-package-package-index.

User:Wurp will update this to be a little more comprehensible, but maybe it can help someone as-is in the meantime:

  • If MokoMakefile always fails on some fetch in zlib, just find the binary somewhere, add it to the downloads or sources directory manually (create the .md5 matching file if necessary), and retry.
  • Make sure you do not have . in your PATH, this causes an "unrecognized option -Qy" error when building g++spec

[edit] Useful commands

Here is the list (not complete) of useful MokoMakefile commands (actually make targets) and a short description of each one. These should be run inside $OMDIR directory.

make fso-gta02-testing-image 
build the full development image
make fso-gta02-unstable-image 
build the unstable bleeding edge image (it often will not build or run)
make update-common 
updates MokoMakefile
make setup 
to make sure that any recent changes to the build directory structure have been applied
make update 
to update the Openmoko repository checkout and the MokoMakefile patches to the latest version

[edit] Developing with MokoMakefile

You are not advised to use MokoMakefile for application development. Openmoko distributes a stand-alone toolchain for that purpose.

[edit] Testimonials

MokoMakefile is recommended by 4 out of 4 new developers on #openmoko, with testimonials such as "For some reason last night I couldn't get my manual install of everything to work (bb complained about my bbpath I think) ... but with your makefile, it works great!", "MokoMakefile rocks!", "Wow this build system is nice - it just seems more polished than my gumstix toolchain buildroot system", "make openmoko-devel-image :) - no magic there", and "I have never been able to get a build to complete in the pure OE environment, whereas Mokomakefile for some reason builds fine."

Project page:

Personal tools


MokoMakefile is a Makefile which saves lots of work when setting up an Openmoko build environment. By automating the setup process of a new Openmoko build environment, it provides an environment which is configured the same for all the existing developers and should therefore be preferred over manual procedures or individual setup procedures. It brings the same repeatability to build environment creation and maintenance as that which the BitBake scripts bring to OpenEmbedded ease and standardize the process of building OpenEmbedded.

Unlike the manual process described at Building Openmoko from scratch, MokoMakefile does not install anything into your system (it can and should be started as normal user). MokoMakefile is a wrapper around all that is required to make it easy to set up and maintain a development environment that fully complies with the setup instructions published by Openmoko.

MokoMakefile is developed by Rod Whitby - it is not an official product of Openmoko (although I would be happy for them to pick it up and use it internally). If there is any discrepancy between the official Openmoko build instructions, and the operation of the MokoMakefile, then you should consider the official instructions to be correct.

The MokoMakefile is able to build both OM-2007.2 and QTopia-X11 images.

MokoMakefile also builds the QEMU-based Neo1973 emulator as part of the build process and has make targets to install the Openmoko images into it and run it. These commands can also be used without downloading and building the whole Openmoko OpenEmbedded distribution. This part is described in Using QEMU with MokoMakefile.

Requirements for building Openmoko

Independent on whether MokoMakefile or a manual process is used to setup an Openmoko build environment, there are several requirements which must be fulfilled in order for the Openmoko build to succeed:

  • RAM: The build host needs to have at least 512MB of RAM, and about the same amount of swap. Some packages built by OpenEmbedded like busybox are built by compiling all source files into one binary which causes gcc to grow beyond 300MB of size and no part of this memory may be on swap for the compile to finish in predictable time. For busybox, this can be turned off, but turning this off means that busybox will not as well optimized by gcc.
  • Disk space: You need about 12 GB of available disk space for the Openmoko build to succeed (see below for a tip on how to reduce this).
  • Time: The initial build takes at least 5 hours (on 2GHz core2duo without multiprocessor optimization) and may take several days on slower machines.

Required software

Some distribution specific hints on preparing your build host for building OpenEmbedded are on http://wiki.openembedded.net/index.php/OEandYourDistro but they may be outdated, incomplete and do not cover everything which Openmoko needs to build.

A good guide is the section on build host prerequisites in Building Openmoko from scratch

If you forgot anything which OE needs itself, OE will tell you shortly after you start building, but it does not check build dependencies of Openmoko, so you either have to install them before starting or install them after the build failed. OpenEmbedded will continue where it stopped when you restart the build afterwards.

Package requirements by distribution

Your distribution needs to provide these commands in order for OpenEmbedded to start building:

subversion texi2html texinfo help2man

Openmoko needs the development packages (with header files, development libraries and tools) in order to finish building:

ncurses zlib (or libz) OpenSSL GTK++

Because there are bugs in the interaction of QEMU and GCC-4, you'll need a copy of gcc-3.x installed as well.

Debian / Ubuntu
 sudo apt-get install subversion build-essential help2man diffstat texi2html texinfo cvs gawk zip unzip cogito
 sudo apt-get install libncurses5-dev zlib1g-dev libssl-dev libgtk2.0-dev 
 # To prevent errors in host validation
 sudo apt-get install ca-certificates
 # For Openmoko 2007.2 using BitBake-1.8.8:
 sudo apt-get install python-pysqlite2 sqlite3 sqlite3-doc python-pysqlite2-dbg
 # For building faster
 sudo apt-get install quilt python-psyco ccache
 # For qemu, install a second compiler for bug avoidance; MokoMakefile knows to look for it.
 sudo apt-get install gcc-3.4 g++-3.4 libsdl1.2-dev lynx netpbm dosfstools
 # On Ubuntu 8.04, the following was required instead of cogito
 sudo apt-get install git-core
 # Ubuntu links /bin/sh to /bin/dash, but some scripts fail because they use pushd and popd, which dash does not support
 sudo rm /bin/sh
 sudo ln -s /bin/bash /bin/sh
 # This may be needed only for ASU
 sudo apt-get install libxtst-dev
SuSE

For building Openmoko on 10.3, you need

gcc-c++ ncurses-devel zlib-devel libopenssl-devel gtk2-devel subversion diffstat texinfo help2man

For MokoMakefile to not fail on compiling qemu-user, you need to use gcc33:

wget download.opensuse.org/repositories/devel:/tools:/gcc/openSUSE_Factory/i586/{cpp,gcc}33-3.3.3-41.8.i586.rpm
rpm -Uhv {cpp,gcc}33-3.3.3-41.8.i586.rpm

See also the Talk page on Building on SuSE Linux 10.3-AMD64

10.1 and 10.2: same packages as 10.3, but install openssl-devel instead of libopenssl-devel.

For all distributions

As the QEMU-based neo1973 emulator is also built as part of the build process started by MokoMakefile, so you need gcc-3.3 and other packages for building QEMU installed. See the build requirements section in Using QEMU with MokoMakefile for information on the required software.

If you are having problems compiling QEMU and do not need it, you can disable building of QEMU by adding following line to build/conf/local.conf file:

ENABLE_BINARY_LOCALE_GENERATION = "0"
Proxy

Users behind a proxy should ensure that their proxy is configured in their wgetrc file as the http_proxy environment variable is unset by MokoMakefile.

Building Openmoko with MokoMakefile

Preparations

Create your $OMDIR directory (note that you can change ~/moko to any directory you like. If you have your ~moko dir mounted from a different file system, be sure to edit /etc/mtab to add the 'exec' permission to the file system, else bitbake will fail with error messages stating that /usr/bin/env is an invalid interpreter):

  mkdir ~/moko ; cd ~/moko

MokoMakefile

Grab MokoMakefile:

  wget http://www.rwhitby.net/files/openmoko/Makefile 

If that doesn't work, try

  wget http://svn.projects.openmoko.org/svnroot/mokomakefile/trunk/Makefile
NOTE: MokoMakefile uses BitBake 1.8.8 which requires python-sqlite2 and sqlite-3.3 or later. Users of SUSE Linux 10.1 can update to the version of openSUSE 10.2


Environment

If you want to make the ASU or FSO instead of the OM2007.2, see Distributions, you need to edit your Makefile. Change OM_GIT_BRANCH to one of the following:

 OM_GIT_BRANCH := org.openmoko.dev        # OM2007.2 (enabled by default)
 OM_GIT_BRANCH := org.openmoko.asu.dev    # ASU
 OM_GIT_BRANCH := org.openmoko.asu.stable # ASU stable
 OM_GIT_BRANCH := org.openembedded.dev    # FSO

Set up the environment with:

  make setup

If you are building for the Neo1973 GTA01, then also do this

 make setup-machine-neo

If you are building for the Freerunner GTA02, then also do this

 make setup-machine-freerunner

Building

Before starting a lengthy make process, check the Tips section below for how to make Make multicore aware. You may want to modify the build/conf/local.conf file for your target (emulation/chroot) environment:

  make openmoko-devel-image
  (or use "make openmoko-qtopia-x11-image" if you're building the ASU)
  (or use "make fso-image" if you're building the FSO)

Hint: The images build with this command can be found under {$OMDIR}/build/tmp/deploy/images/neo1973

This will set up the recommended directory structure as described in Building Openmoko from scratch, will download all the required software (from the right places with the right versions), and will immediately start building an image.

Once you have done this, you can choose to continue using the MokoMakefile to initiate your subsequent builds, or you can go into the build directory and run bitbake commands manually. The choice is yours.

Updating the environment

For easy maintenance of your build environment the following commands are available.

1 - To update the MokoMakefile to the latest version:

  make update-makefile 

2 - To make sure that any recent changes to the build directory structure have been applied:

  make setup 

3 - To update the Openmoko repository checkout and the MokoMakefile patches to the latest version:

  make update

A quick way to rebuild a new image with the latest updates:

  make update-makefile && make setup update openmoko-devel-image

Build issues

First, make sure that the problem is reproducible after running

make update-makefile && make setup && make update

then run

make clean-package-<foo>

(where you replace <foo> with the name of the package which is failing)

and finally

make openmoko-devel-image

With luck - you should now have images in your $omdir/build/tmp/deploy/glibc/images/neo1973/ directory. You can then flash these.


If not, and you can get the error to occur three times in a row after running that sequence of commands (including the update and setup steps) three times, then feel free to report it to rwhitby in #openmoko on IRC.

If you have chronic build problems, cleaning your environment variables and re-running the build from scratch may help. The following script should remove all non-essential variables; you should also manually trim your path to the minimum.

`env | grep -Ev '^(SHELL|TERM|OLDPWD|USER|USERNAME|PATH|EDITOR|LANG|HOME|DISPLAY|_)=' \
| sed 's/^\(.*\)=.*/unset \1/'`

(you must include the backticks). Note that on some boxes it is necessary to call "make" by typing "/usr/bin/make" afterwards.


On current Ubuntu boxes another trick is necessary: glibc_2.6.1 fails to build since it is unable to generate the necessary locales until you type in the following line:

sudo sysctl -w vm.mmap_min_addr=0

Note that this temporarily disables a new security vulnerability check that has entered /etc/sysctrl - but that makes problems with loads of applications. To re-enable this check do the following:

sudo sysctl -w vm.mmap_min_addr=65536

See the talk page of this article for more details on this solution.

Known MokoMakefile errors

SVN Server Certificate Errors

If you get an error like the following:

NOTE: Fetch svn://svn.projects.openmoko.org/svnroot/;module=assassin;proto=https
Error validating server certificate for 'https://svn.projects.openmoko.org:443':
 - The certificate is not issued by a trusted authority. Use the
   fingerprint to validate the certificate manually!
Certificate information:
 - Hostname: projects.openmoko.org
 - Valid: from Thu, 05 Jun 2008 01:02:42 GMT until Tue, 02 Dec 2008 01:02:42 GMT
 - Issuer: http://www.cacert.org, Root CA
 - Fingerprint: a5:7d:4e:37:e8:94:ee:ba:c1:e8:e9:4b:33:cb:34:91:a9:6d:d3:84
(R)eject, accept (t)emporarily or accept (p)ermanently? svn: OPTIONS of 'https://svn.projects.openmoko.org/svnroot/assassin': Server certificate verification failed: issuer is not trusted (https://svn.projects.openmoko.org)

It means you haven't yet accepted the server certificate for the SVN server. To remedy this, execute the following commands, and hit "p" (to permanently accept the certificate) when prompted:

cd /tmp
svn co https://svn.projects.openmoko.org/svnroot/assassin
rm -rf assassin
cd -

exposure_svn and Subversion 1.5.0

If you get an error like this:

"NameError: global name 'log' is not defined"

There's a bug with subversion 1.5.0 -- downgrade to 1.4.x (or check out the latest subversion sources and use those as they're fixed) and re-fetch the sources.

Fixes for distribution/environment-specific or isolated issues

Work-arounds for temporary or isolated problems can be found and should be added to the Discussion page which is associated with this page. As they are fixed, they will be removed from that page.

Tips

  • Make sure umask is set to 0022 before you run "make setup"
  • You can reduce the amount of consumed disk space significantly by adding
  INHERIT += "rm_work"

in your local.conf (e.g. ~/moko/build/conf/local.conf). This will remove the contents of each build/tmp/work/*/<package> directory after the corresponding package builds correctly. As of 10/16/07, this appears to be present in local.conf by default.

  • If a certain package does not build due to corrupted download or some such try to remove the sources and rebuild it.
rm sources/<package>*
cd build
. ../setup-env
bitbake -crebuild <package>

after that your build might just work again.

  • For people with multiple CPU's (or dual-core ones) this small patch might be useful to build things faster.

Edit the local.conf and add the following lines:

PARALLEL_MAKE = "-j 4"
BB_NUMBER_THREADS = "4"

Change the PARALLEL_MAKE and BB_NUMBER_THREADS values to something that suits better if it chokes your machine.

  • I have some problem with the parallel building, mkfs.jffs failed to build the image because the directory was modify while the building of the fs. patrick.hetu 02:19, 27 May 2008 (UTC)
  • If you encounter an error related with the qemu-native package and not compiling for the qemu, you can edit the build/conf/local.conf file and add ENABLE_BINARY_LOCALE_GENERATION = "0" line to avoid the error.
  • To prevent building tons of locales, add a line like this to local.conf:
GLIBC_GENERATE_LOCALES = "en_US.UTF-8 nl_NL.UTF-8"
  • To not build any binary locales at all, add this to local.conf:
ENABLE_BINARY_LOCALE_GENERATION = "0"
  • If you want to rebuild the package indexes (for instance, after compiling a new version of a package) without building openmoko-devel-image, run make build-package-package-index.

User:Wurp will update this to be a little more comprehensible, but maybe it can help someone as-is in the meantime:

  • If MokoMakefile always fails on some fetch in zlib, just find the binary somewhere, add it to the downloads or sources directory manually (create the .md5 matching file if necessary), and retry.
  • Make sure you do not have . in your PATH, this causes an "unrecognized option -Qy" error when building g++spec

Useful commands

Here is the list (not complete) of useful MokoMakefile commands (actually make targets) and a short description of each one. These should be run inside $OMDIR directory.

make openmoko-devel-image 
build the full development image
make openmoko-devel-tools 
build statically-linked dfu-util and openocd binaries
make openmoko-toolchain 
builds toolchain
make qemu 
builds qemu, downloads flash images from official repository, flashes qemu emulated Neo1973 and runs emulator (see also Openmoko under QEMU).
make qemu-local 
same as make qemu, but uses locally built images instead of official ones.
make run-qemu 
runs qemu with usb keyboard (note: qemu must first be built!). This target starts the emulator with some predefined set of options. If you need to start it with other options (enable telnet, etc), see QEMU_command_line.
make build-package-<package name> 
buids <package name> package
make update-makefile 
updates MokoMakefile
make clean-package-<foo>
cleans the "<foo>" package
make setup 
to make sure that any recent changes to the build directory structure have been applied
make update 
to update the Openmoko repository checkout and the MokoMakefile patches to the latest version
make setup-machine-neo 
setup for GTA01 version of Neo1973
make setup-machine-freerunner 
setup for GTA02 version of Freerunner

Developing with MokoMakefile

NOTE: If using MokoMakefile with OM2007.2 then references to $OMDIR/openmoko should be replaced with $OMDIR/openembedded. Also references to tmp/work/armv4t-linux should be replaced with tmp/work/fic-gta01-angstrom-linux-gnueabi


For the following explanations $OMDIR is the directory where there Makefile puts all the stuff.

To make in-tree changes and have them built and used by qemu:

 cd $OMDIR/openmoko
 quilt new descriptive-patch-name.patch
 quilt add trunk/src/name-of-file-to-change # do this for every file you are about to modify
 ...make the changes...
 quilt refresh # this creates a file in $OMDIR/patches/openmoko-HEAD/ and updates the quilt series file there

Note: Do NOT use absolute paths as this confuses quilt and will get you a diff of the file against /dev/null!

To build the changes and have them used by qemu:

 make build-qemu
 make flash-qemu-local
 make run-qemu

If you want to modify applications instead of the openmoko toolchain, this is what you have to do (example: openmoko-messages):

 cd $OMDIR/build
 . ../setup-env
 bitbake -c patch openmoko-messages
 cd tmp/work/armv4t-linux/openmoko-messages-0.0.1+svnnow-r2_2276/openmoko-messages/
 ...make the changes...
 cd -
 bitbake -c build openmoko-messages

Note: if you just use bitbake openmoko-messages it will remove the sources after building!

Then continue with MokoMakefile usage.

If you want to add an application to your openmoko distribution, do this: All file edits should be done using quilt as described above. That way a patch can easily be submitted to the openmoko project. First, create a directory that will correspond to your package and edit a .bb file in there:

 cd $OMDIR/openmoko/
 quilt new mycoolpackage.patch
 mkdir trunk/openembedded/packages/mycoolpackage
 quilt add trunk/openembedded/packages/mycoolpackage/mycoolpackage_1.bb
 quilt edit trunk/openembedded/packages/mycoolpackage/mycoolpackage_1.bb

The file should have the following content:

 DESCRIPTION = "This is a cool package"
 SECTION = "username/mycoolpackage"
 PV = "1"
 
 inherit autotools
 
 SRC_URI = "http://www.example.com/download/mycoolpackage-1.tar.gz"

Explanation:

  • DESCRIPTION - Just a short text explaining the package
  • SECTION - I have no clue, but I'll use username/mycoolpackage for now
  • PV - Package Version
  • inherit autotools - The package can be compiled by './configure && make && make install' so we tell MokoMakefile to do it this way.
  • SRC_URI = ... - This is the download location of the package source. It's imperative that the tar.gz contains a directory called packagename-packageversion (in this case: mycoolpackage-1) so that MokoMakefile can find it automatically or the build will fail.

This is not all. We also need to tell MokoMakfile that it needs to build and include the package in the image. To do this, do

 $OMDIR/openmoko# quilt edit trunk/openembedded/packages/tasks/task-openmoko.bb

Here, increase the value PR by one and add mycoolpackage \ (with the backslash!) just before the line reading # update-alternatives \.

Now run

 quilt refresh
 cd ..
 make update openmoko-devel-image

And if everything's alright you should now have an OpenMoko image to flash to your phone or run in qemu as described above.

Hello World application

There is a Hello World! tutorial available too.

Testimonials

MokoMakefile is recommended by 4 out of 4 new developers on #openmoko, with testimonials such as "For some reason last night I couldn't get my manual install of everything to work (bb complained about my bbpath I think) ... but with your makefile, it works great!", "MokoMakefile rocks!", "Wow this build system is nice - it just seems more polished than my gumstix toolchain buildroot system", "make openmoko-devel-image :) - no magic there", and "I have never been able to get a build to complete in the pure OE environment, whereas Mokomakefile for some reason builds fine."

Project page: http://mokomakefile.projects.openmoko.org/