sbuild on a tmpfs

As some already know, is that I use sbuild to build all my packages in clean chroot environments. For this, I use the ‘aufs’ “mode” of schroot, that allows you to setup a chroot with one read-only base directory and one writeable overlay where changes are written to.

One thing I had problems with was the time required to install build dependencies due to disk I/O. Given that I have 4G RAM in my computer, I decided to use a tmpfs as the writable overlay. For those of you who want to do the same, putting the following in /etc/fstab makes it work:

overlay                     /var/lib/schroot/union/overlay tmpfs         defaults                    0       0

Build times are much faster this way because all write I/O is directed to the RAM

The Three Levels of Package Management

In today’s Linux distributions, there are usually two to three levels of package management. In this blog post, I will explain the three levels of package management.

1. Local (dpkg, rpm)

The first level of package management is the ‘local’ level. This level consists of package management tools that install and/or remove packages via package archives such as .deb or .rpm files and a database of the local’s system state.

2. Connected (APT, YUM, Smart)

The second level of package management is the ‘connected’ level. In this level, tools are fetching packages and information about them from (remote) locations known as ‘repositories’ or ‘sources’. A level 2 package manager is usually built on top of a level 1 package manager, for example, APT uses dpkg – but there are also cases where one tool is both, such as opkg.

3. Abstract (PackageKit, Smart)

A level 3 package manager is a tool that supports different level 1 package managers via one generic interface. It may be implemented on top of a level 2 package manager (PackageKit), or may be implemented directly in level 2 (Smart).

Conclusion: Merge Level 2 and 3

Most level 2 package managers share a great deal of tasks, such as solving dependency problems. Merging level 2 package management and level 3 package management would enable us to reduce the number of dependency problem solvers and combine our efforts into a few package managers. Such an implementation would need to be written in C and support all common level 1 package managers in order to be successful. As some might have guessed, this is what I plan to do with the project codenamed APT2, although work is not progressing very fast.

PS: Back from vacation

I spent the more than the complete last week in (mostly rainy) Corfu, Greece and am now getting back to my usual development stuff. I have already processed my news & blogs backlog (1000+ messages) and will now start with my email backlog, so don’t be angry if the answer to your email takes some time.

APT2 is now UPS

APT2 is now called UPS (Universal Package System). The name is inspired by the company that delivers packages in those brown trucks and from the Image Packaging System (IPS) of OpenSolaris; and mvo writing ups after I proposed upt (über package tool) in IRC. It’s definitely better than my first thought which was ‘moo’ (and libmoo).

Update: OK, let’s cancel this rename insanity.

APT2 – this time in C

As I wrote a few hours ago on deity@l.d.o (see
http://lists.debian.org/deity/2010/08/msg00057.html
), APT2 is back again. The first time, I tried Vala; but this time I wrote it in C (with the help of GLib, but no GObject) and the cache uses GVariant instead of an SQLite database. It’s really basic at the moment (no solver, package installation/removal), but it will improve. Read operation should be faster than with APT, although writes are slower (this will be fixed by reusing unchanged parts of the cache).

See the announcement for further information.

Maemo + Moblin = MeeGo = Failure

Today, Nokia and Intel announced the merge of Maemo and Moblin into the MeeGo project. This is sad, because it will end the era of the Debian-based mobile operating system Maemo and replace it with a system using RPM and probably some other evil stuff as well. In fact, dpkg & apt-get where two of my main reasons to buy the N900.

And another question is why yet another name. Moblin was already a well-known name and they shouldn’t have changed the name just because they switch the servers and add some Nokia developers.

Furthermore, does this all mean that there will be no Maemo 6? What will happen to the Maemo users on the N900, will it be possible for them to use MeeGo?

Maybe this is the time to start a new project, Debian Mobile. Debian Mobile would take the MeeGo experience and apply it to a Debian system. This would probably the best mobile operating system ever, being backed by Debian’s large repositories with thousands of software packages.

Just uninstalled hal

I have just uninstalled hal from my laptop running unstable, and everything still seems to work, including suspend & resume.

python-apt 0.7.93 just hit unstable.

I just uploaded python-apt 0.7.93 to unstable with support for Python 2.6 and Python 3.1, meaning that there is now a single development branch again.

This uploads brings developers the new API with real classes in apt_pkg (you can now use pydoc to view documentation), C++ bindings for making apt-pkg applications scriptable (although they should be considered experimental), a test suite (although aptsources fails in one test for now) and many new context managers for enhanced Python 3 coding fun. And objects are now freed when their reference count reaches 0. A more complete list of news can be found in the What’s New In python-apt 0.7.100 part of the documentation.

For the next releases until 0.7.100 release, the focus is clearly on fixing bugs and improving the documentation. We need more tests of the Python 3 builds, especially in areas dealing with str and unicode stuff.

Have fun, read the documentation, and code.

The previous post

Well, it seems that several news sites (golem.de, pro-linux.de, linux-magazin.de, linux-magazine.com, ubuntu-user.de [the last ones all from the same publishing house]), especially German ones have picked up the last blog post with same false impressions.

First, they stated that I am planning an APT2 release for Christmas. They took the statement

[...] the internal branch has seen a lot of new code[...]. Most of the code will need to be reworked before it will be published, but I hope to have this completed until Christmas.

as a proof for this. But in the context of this paragraph, ‘publish’ was not meant in the term of publishing a release, but in the term of publishing the code (of the internal branch) in the public repository. The code is public now and lives in a ‘temp’ branch and will be reworked there for inclusion in the master branch.

Secondly, those news sites called me an Ubuntu developer. While I do contribute to Ubuntu, and am an Ubuntu Member, I am NOT an Ubuntu Developer, as I am NOT a member of the relevant ubuntu-dev team at launchpad.

Thirdly, those who stated that speed is a goal (mostly pro-linux, at least from my understanding): It is not, at least not now. It is just a coincidence caused by using a relational database. Furthermore, the test was not really fair, since the other package managers both provide more information than capt; information which has yet to be made accessible in APT2. It was just an initial conclusion that SQLite is quite fast.

Please note that APT2 is a free time project in development, and the programming language used is still in development as well; as well as some other reverse dependencies. I should also state that neither Debian nor Ubuntu have any plans to drop apt at the moment; and APT is still actively developed.

APT2 progress report for the 1st half of December

This week was successful. I have pushed some changes from November to the repository which change the license to LGPL-2.1+ (which makes bi-directional sharing of code with other projects easier, since most Vala code is under the same license) and implement HTTP using libsoup2.4 directly, instead of using GIO and GVFS for this. I also added a parser for the sources.list format which uses regular expressions to parse the file and is relatively fast. The code needs a current checkout of Vala’s git master to work correctly; as released versions had a bug which I noticed today and Jürg Billeter fixed in Vala 25 minutes later; thank you Jürg.

While nothing else happened in the public repository, the internal branch has seen a lot of new code; including SQLite 3 caches; Acquire text progress handling; and capt; the command-line advanced package tool. Most of the code will need to be reworked before it will be published, but I hope to have this completed until Christmas. It will also depend on Vala 0.7.9 or newer, which is yet to be released.

The decision to use SQLite 3 as a backend means that we won’t see the size limitations APT has and that development can be simplified by using SQL queries for filtering requests. It also means that APT2 will be very fast in most actions, like searching; which currently happens in 0.140 seconds (unstable,experimental and more repositories enabled), whereas aptitude takes 1.101 seconds, cupt (which has no on-disk cache) 1.292 seconds, and apt-cache 0.475 seconds. Searching is performed by one SQL query. I also want to thank Jens Georg <mail@jensge.org>, who wrote Rygel’s Database class which is also used with minor modifications (like defaulting to in-memory journals) in APT2 as well. Rygel.Database is a small wrapper around sqlite3 which makes it easier to program for Vala programmers.

The command-line application ‘capt’ provides a shell based on readline with history (and later on command completion) as well as direct usage like ‘capt config dump’ or ‘capt search python-apt’. Just as with Eugene’s cupt, ‘capt’ will be the only program in the core APT2 distribution and provide the same functionality currently provided by apt-get, apt-config and friends. The name is not perfect and can be easily confused with ‘cupt’, but it was the closest option for now; considering that the name ‘apt’ is already used by Java (for its “Annotation Processing Tool”).

That’s all for now, I’ll tell you once all those features have passed my QA, and there is really something usable in the repository. In the meanwhile, you can discuss external dependency solvers, database layouts and other stuff in their threads on deity@lists.debian.org.

And a ‘screenshot’ from capt:

jak@hp:~/Desktop/APT2:temp$ capt
apt$ help
APT2 0.0.20091213 command-line frontend

Commands:
  config dump               Dump the configuration
  config get OPTION         Get the given option
  config set OPTION VALUE   Set the given option
  search EXPRESSION         Search for the given expression
  show PACKAGE              Show all versions of the given package
  sources list              Print a list of all sources
  version                   Print the version of APT2
apt$ search python-apt
build-depends-python-apt - Dummy package to fulfill package dependencies
python-apt - Python interface to libapt-pkg
python-apt-dbg - Python interface to libapt-pkg (debug extension)
python-apt-dev - Python interface to libapt-pkg (development files)
python-aptdaemon - Python module for the server and client of aptdaemon
python-aptdaemon-gtk - Python GTK+ widgets to run an aptdaemon client
apt$ 

Ubuntu Software Center coming to Debian

I just uploaded aptdaemon 0.11-1 and software-center 1.1debian1 to Debian unstable. They are currently waiting in NEW, and will hopefully pass it in a short time. I plan to replace gnome-app-install with software-center for Squeeze, but you can currently have both installed.

Ubuntu Software Center (or just ‘Software Center’) is a new graphical user interface for installing and removing applications; replacing gnome-app-install. Under the hood, it uses aptdaemon which exposes an interface to APT via D-Bus; i.e. something in the direction of PackageKit. At a later stage, the Software Center shall replace Synaptic, Update Manager and various other programs related to package management.

The aptdaemon package is completely compatible to the Ubuntu one, and could thus be synced directly to Ubuntu without any change (if Ubuntu supports “3.0 (quilt)” source packages now, I have not looked into this). The software-center package is based on the latest Ubuntu lucid package; and contains some generalization (e.g. replacing ‘Ubuntu Software Center’ with ‘Software Center’) at some more places. It still needs some work in the documentation and some parts of the program will have to be adjusted for Debian aswell. We also do not have a debianized icon yet; this will be worked on later.