More on Yocto Terminology – recipes and packages
As I mentioned in my previous blog, much confusion exists around terminology. I think this is especially true when we speak of recipes and packages in the Yocto context. In the early days of Open Embedded (OE) , on which Yocto is based, there was a very simple relationship between recipes and packages. The relationship is still a close one, but it has become more complicated, as everything does over time.
A recipe is a set of instructions that is read and processed by the build engine called BitBake. In its most basic form a recipe is a single file with some lines in it describing the software to be built. In practice, most recipes consist of multiple files, having a .bb (bitbake) file which conveys version information and often upstream source location, and a .inc (BitBake include) file which has the bulk of the processing instructions for a given software collection. I resisted the urge to use the term ‘software package’ because package is an overloaded term and the goal here is to remove some of the ambiguity surrounding these terms. In fact, most recipes also include other functionality beyond what is defined in the .bb and .inc files. The most common example of this is for a recipe to include the autotools bitbake ‘class’ which provides the well-known functionality of ‘./configure’, ‘make’, and ‘make install’ for many packages based on autoconf and friends.
A package is an archive that most typically contains binary artifacts from a build. The two most popular package formats are still .deb and .rpm, the former from the Debian project and the latter used by Redhat and it’s derivatives. Packages can also contain source code, but certainly in the embedded Linux community, that practice is dwindling. One of the more common package formats found in the embedded space is .ipk, manipulated with the opkg tools found on a Google source repository (http://code.google.com/p/opkg.) This was the default package format for OpenEmbedded-based distributions, while Yocto currently favors the RPM format. (Yes, I’m guilty of using the term Yocto when I really mean “Yocto Project”! See my previous blog on this subject.)
By default, a recipe produces at a minimum, 4 packages. The most obvious one is the binary package itself, which is used to populate the root file system with the artifacts from that recipes’ build output. The other three are the development (-dev) package, the documentation (-doc) package, and the debug (-dbg) package. The development package most typically contains the header files and libraries, if there are any, for a given package. The documentation package should be self explanatory, and for many packages, is empty! Documentation has not always been the top priority for development engineers, and this is no less true for open source developers. The debug package typically contains copies of the binary executables and/or libraries that have been compiled with debug symbols included. The debug packages are what your debugger/IDE reads in order to enable symbolic debugging.
Some recipes produce many more packages. Take a look at the recipe for python for a good example. The current version of that recipe in the poky repository produces seventy packages. As you might expect, this is a pretty extreme example because python has many modules. You can see what packages a given recipe produces using BitBake’s -e (show environment) command line switch:
$ bitbake -e python | grep ^PACKAGES=
PACKAGES="libpython2 python-dbg python-2to3 python-audio python-bsddb python-codecs python-compile python-compiler python-compression python-core python-crypt python-ctypes python-curses python-datetime python-db python-debugger python-dev python-difflib python-distutils-staticdev python-distutils python-doctest python-elementtree python-email python-fcntl python-gdbm python-hotshot python-html python-idle python-image python-io python-json python-lang python-logging python-mailbox python-math python-mime python-mmap python-multiprocessing python-netclient python-netserver python-numbers python-pickle python-pkgutil python-pprint python-profile python-pydoc python-re python-readline python-resource python-robotparser python-shell python-smtpd python-sqlite3 python-sqlite3-tests python-stringold python-subprocess python-syslog python-terminal python-tests python-textutils python-threading python-tkinter python-unittest python-unixadmin python-xml python-xmlrpc python-zlib python-modules python-misc python-man"
When BitBake completes “baking” the python recipe, a package is created for each of the named elements show above in the ‘PACKAGES=’ listing.
Images, Recipes and Packages
This is most confusing when trying to add new packages to your root file system image. A recent example of this was encountered when one of our customers asked how to incorporate the NET-SNMP package into his image. It seemed obvious to him (and to this author) that because there is a nice, tidy recipe called net-snmp_5.7.1.bb, all he needed to do was add this to his image. He tried the obvious, based on general instructions in the various manuals which describe adding packages to images:
IMAGE_INSTALL += "net-snmp"
However, that failed because while there is a recipe called net-snmp, there are no packages by that name. Root file system images are created from packages, and not recipes. Looking at the packages produced by the net-snmp recipe, using BitBake’s -e switch, we have:
$ bitbake -e net-snmp | grep ^PACKAGES=
PACKAGES="net-snmp-dbg net-snmp-doc net-snmp-dev net-snmp-staticdev net-snmp-static net-snmp-libs net-snmp-mibs net-snmp-server net-snmp-client"
Looking at this output, it becomes clear that the packages desired for installation on the root file system are net-snmp-server, net-snmp-client, and possibly the supporting packages such as the -libs and -mibs. So your image must be modified to have these packages installed. In summary, the correct way to specify this is as follows:
IMAGE_INSTALL += "net-snmp-server net-snmp-client net-snmp-libs net-snmp-mibs"
BitBake -e is your friend. Use it often – it can really help you understand what’s going on when things don’t work out as you expect.
- strace: My Favorite Tool for Debugging and Profiling Linux Applications
- Achieving Faster Linux Boot Time: 2013 Version
- BeagleBone and 6502 – The Ultimate Geek Toys
- Fun with toolchain versions
- More on Yocto Terminology – recipes and packages
- Yocto versus Poky versus Angstrom, etc
- Thoughts on Embedded Linux Terminology
- Welcome to my little corner of the world