Python Speed: Getting the first part of a string

If you want to get the first part of a string (in the following example a), you should use the partition method of the string, as it is the fastest way (and it also gives you the delimiter in [1] and the other part of the string in [2]):

In [1]: a = ‘a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z’

In [2]: timeit a.partition(’/')[0]
1000000 loops, best of 3: 362 ns per loop

In [3]: timeit a[:a.find('/')]
1000000 loops, best of 3: 443 ns per loop

In [4]: timeit a.split(’/', 1)[0]
1000000 loops, best of 3: 697 ns per loop

In [5]: timeit a.split(’/')[0]
1000000 loops, best of 3: 1.45 µs per loop

These tests were made with IPython 0.8.1 (Python 2.5.1 (r251:54863, Oct 5 2007, 13:50:07) ) on an Ubuntu 64 bit system.

debimg news: disk splitting, new data lists, PowerPC

The next release of debimg (0.0.3) will be the first release supporting splitting packages over multiple disks. I worked a lot on the base technology today, and the basics are working.

In order to get support for disk splitting, I introduce three new classes: Media, MediaSet and MediaSetCollection (all in libdebimg.media).

The Media class is responsible for adding files to an image (incl. copying/linking), generating extra files (like MD5SUMS or documentation) and for building the image. It knows the space which is free, the size of each file and the md5sum of each file. It also writes all data inside dists/

The MediaSet class contains one or more Media instances. It has the functionality to split the packages into the disks.

MediaSet knows the dependencies of all packages and tries to add it to the disk with smallest number (e.g. 1) if there is enough space left for the package and all dependencies. If a dependency is on disk2, the package will be added to disk 2 or later. (If one dependency is on the last disk, we don’t check where the others are, instead, we put it on the same disk [faster])

The MediaSetCollection class contains multiple MediaSet instances (e.g. for DVDs and CDs), and calls their functions. This makes it possible to build multiple MediaTypes at the same time in an efficient way, because we don’t need to call debimg multiple times or run in multiple loops.

This means that many methods of libdebimg.packages.MirrorBuilder move into the new module. For example, the writePackagesFiles() and writeReleaseFiles() will be part of media.Media. (and can be called from MediaSet [for every Media] and MediaSetCollection [for every MediaSet], too)

From boot, run_genisoimage() will be removed. It is replaced by the createImage() method of media.Media.

In future, debimg will be able to run in three modes: simple, set and sets. The simple mode only creates one disk and is used for netinst and businesscard images. The set is used when only one MediaType has to be generated. The recommended mode for building e.g. weekly-builds is sets, as it has support for multiple MediaTypes.

Another change is the change to the format of data/*.list. I remove the keys called Priority and Tasks. Instead of these, I will add support for these directly into Depends. This means that you will be able to use “Priority:standard” just like “pkgname”. This makes sorted lists possible, which are needed for disk splitting. I will also rename the package list to match the names used in debian-cd (mostly) and add a new field called Single-Disk: yes, which disables sorting of the list, and is a bit faster than the other one.

This feature code is unreleased at the moment, but will hopefully be ready next week. Please note that I will never build complete sets with debimg. Therefore, I added a mode to create a list of all packages and build the image without them. (I have no local mirror)

Next week will also be the start of the first daily-builds of PowerPC images and businesscard images available from jak-linux.org/cdimage.

If you want to use the current release (0.0.2), a good point to start is a mail I wrote:

http://lists.debian.org/debian-cd/2008/03/msg00116.html

debimg 0.0.2 builds netinst in less than 4 seconds

debimg 0.0.2 has a new high speed dependency resolver and support for local repositories.

With these new features, debimg is now able to produce netinst images in less than 4 seconds.

BTW, you can easily build your own disk. Simply unpack the tarball, change the mirror to your one, and run ./debimg debimg.cfg. This should give you an i386 netinst disk.

Get debimg 0.0.2 from http://users.alioth.debian.org/~jak-guest/debimg_0.0.2.tar.gz

Fetch daily-built images at http://jak-linux.org/cdimage/daily-builds/testing/.

For more information, see http://lists.debian.org/debian-cd/2008/03/msg00114.html

Ubuntu stuff: gnash 0.8.2 for gutsy, ndisgtk, aufs

A backport of gnash 0.8.2 is available in my PPA at

deb http://ppa.launchpad.net/juliank/ubuntu gutsy main

I wanted to try it, but I had no time to compile it, so I uploaded it to the PPA. The next day, I had a compiled backport.

I don’t use it actively because it does not support some sites I visit.

Another news is the recent addition of ndisgtk to the ship and ship-live seeds, which means it will be available on the disk. http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/ubuntu.hardy/revision/1223.

BTW, recent Ubuntu Hardy images (including Alpha 6) have support for using aufs instead of unionfs, simply add union=aufs to the kernel options. It may help if you have problems with unionfs and should be faster and more stable.

cdimage for debimg is online now!

debimg’s cdimage is now online at http://jak-linux.org/cdimage/, and contains daily-builds of Debian testing netinst disks for i386 and amd64 architectures.

debimg is a new software which aims to replace debian-cd. No code has been released yet, but will be in the next days.

Please note that I can only provide jigdo images, because of bandwith and storage.

I will soon add build logs, and files listing the differences compared to original netinst images. I will also add support for more architectures.

debimg homepage at jak-linux.org

gimmie (ITA) and the new menu policy + building with pristine-tar

While working on Bug#460620: ITA: gimmie — elegant desktop organizer, I noticed that I need to upgrade debian/menu to the new policy.

Currently, gimmie uses Apps/Tools, but what should I use now? It does not fit into the other sections. Recommendations?

Other important changes:

• I am the new maintainer
• Gmenu and sexy extensions are not built, instead the python-gmenu and python-sexy packages are used
• Gimmie’s extensions are compiled for Python 2.4 and Python 2.5
• debian/copyright uses the format defined at http://wiki.debian.org/Proposals/CopyrightFormat
• The suspend and hibernations buttons are back again (Commit)

The package is maintained in a git repo in the collab-maint project, and can be viewed online using Gitweb. You can get the source from its repo at git://git.debian.org/git/collab-maint/gimmie.git

To create the orig.tar.gz tarballs, you need to have pristine-tar installed. To build the package, install git-buildpackage (from unstable) and pristine-tar and simply run ‘git-buildpackage –git-pristine-tar’

The same instructions also apply to building readahead-list, which can be found at git://git.debian.org/git/collab-maint/readahead-list.git