Android's roaming detection & its implementation

I know I wrote about Android already today, but there is another thing that concerns me right now. I am owner of an Android-based phone (an HTC Dream) and recently switched my mobile network provider. The problem is that my new provider is a virtual provider and as such there is no real network of that provider. Now Android has a feature to turn off broadband connections when in roaming mode, which itself is a great idea and can save you from paying quite a lot of money when the phone connects to 3G abroad, but this feature also turns off broadband connections when roaming locally. All this is being discussed in bug report #3499.

After noticing this problem I became curious on how Android detects that it is roaming and I found the GsmServiceStateTracker.isRoamingBetweenOperators method to be responsible for that magic, but soon noticed that the method is not only inefficient, but also doesn't work as intended. This is hardly related to the bug mentioned above, but let's have a look at the code in question:

/**
* Set roaming state when gsmRoaming is true and, if operator mcc is the
* same as sim mcc, ons is different from spn
* @param gsmRoaming TS 27.007 7.2 CREG registered roaming
* @param s ServiceState hold current ons
* @return true for roaming state set
*/
    private
    boolean isRoamingBetweenOperators(boolean gsmRoaming, ServiceState s) {
        String spn = SystemProperties.get(PROPERTY_ICC_OPERATOR_ALPHA, "empty");

        String onsl = s.getOperatorAlphaLong();
        String onss = s.getOperatorAlphaShort();

        boolean equalsOnsl = onsl != null && spn.equals(onsl);
        boolean equalsOnss = onss != null && spn.equals(onss);

        String simNumeric = SystemProperties.get(PROPERTY_ICC_OPERATOR_NUMERIC, "");
        String operatorNumeric = s.getOperatorNumeric();

        boolean equalsMcc = true;
        try {
            equalsMcc = simNumeric.substring(0, 3).
                    equals(operatorNumeric.substring(0, 3));
        } catch (Exception e){
        }

        return gsmRoaming && !(equalsMcc && (equalsOnsl || equalsOnss));
    }

Okay, let me summarize what this piece of code does wrong, at least from my understanding:

• It takes both the network operator alphanumeric identifier and alphanumeric long identifier and compares both to the alphanumeric identifier coming from the SIM card, whilst...


• ... it could simply use the network and SIM card numeric identifiers and compare those, which should be a lot cheaper than comparing those strings


• Then it takes the first three characters/digits of the numeric identifiers (which indicate the country) and compares those

Now in my case my SIM card doesn't seem to provide the phone with a alphanumeric identifier, so the first two comparisons always fail for obvious reasons and, looking at the inline-if in the last line of that method my phone will always indicate that I am in roaming mode, even when I am not.

The problem is not only the logic which seems to be wrong, but I rather see the inefficient comparisons used there to be a major problem in embedded systems like mobile phones. This is the first piece of Android code I have had a look at, but if all other code is as ugly and inefficient as these few lines Android really needs some major fixes. Related to this I have reported bug #4590 and forked the git repository in question over at github, to fix this method, should be a matter of 5 minutes.

Android, Mythbusters and openness

I have been reading a great many posts about Android lately, some consisting of criticism, some of praise and some simply addressing issues in the Android "community". Let's have a look at those.

Matt Porter's Android Mythbusters presentation and Harald Welte's reaction


I haven't seen the presentation live, but I had a look at the slides. Impressing work done by Matt putting all this information together. However, we all knew that Android only (ab-)uses Linux, without making use of the GNU userland for a long time, didn't we?

In his presentation Matt has shown things such as Android's udev "replacement" that uses hardcoded values for device node creation and (on his blog) Harald has then come up with a statement I have found to be very strong:

The presentation shows how Google has simply thrown 5-10 years of Linux userspace evolution into the trashcan and re-implemented it partially for no reason. Things like hard-coded device lists/permissions in object code rather than config files, the lack of support for hot-plugging devices (udev), the lack of kernel headers. A libc that throws away System V IPC that every unix/Linux software developer takes for granted. The lack of complete POSIX threads. I could continue this list, but hey, you should read those slides. now!

Now both of these statements target technical details, but the root of the problem seems to be elsewhere.

Where is my Android 2.0?

Okay, that heading might not be making any sense in the context of this post at a first glance, but let me elaborate on that. Google and the Open Handset Alliance refer to Android as being an "Open Source" operating system, but the project is different from "real" Free Software projects: development takes place in a closed group and the results are shared with the community later on, when they are deemed to be ready.

This means that innovation also takes place behind closed curtains and that the community is not involved in the actual development process at all. Lately we have seen the result of that, as Motorola is bragging about working close with Google on Android 2.0 ("Eclair"), but the AOSP source trees, open for everyone to have a look at, show no signs of version 2.0. In fact no changes that might even remotely suggest the release of a new major version have been made public in the past few weeks. So where is the openess there?
Actually, the Motorola Droid has already shipped with Eclair on 6th, but still, there is no indication that Eclair will be made available to the broader public.

In short Android seems to be developed behind closed curtains, with hardly (read no) community input whatsoever and is sometimes released as Free Software, not what I would describe as an open development process.

The Android Market problem

As we have seen in the past Google is enforcing their copyright on proprietary applications that ship with pretty much every Android device, such as the Android Market. This has become really clear when Steve Kondik received  a cease and desist letter when packing the Google-proprietary applications into his ROMs. Okay, it's Google's right to enforce their copyright and there is nothing wrong with actually doing so, the thing I really have a problem with is something else: the Market is proprietary.

Now what this means should become rather clear. You can have an Android device without Google's proprietary bits, but with default settings you just do not have any way of installing additional software. In my opinion the Market should be freed by Google themselves, or the community has to react and come up with a free replacement to overcome the vendor lock-in. Oh, you might know a replacement called SlideMe (or Mobentoo) already. Well, that bugger is proprietary too, so not a solution at all.

Nokia and Maemo to the rescue

In most discussions about the openness of Android someone throws in Nokia and Maemo, as a solution to the dilemma. Reading all those positive comments I simply had to give it a try, but all my hopes were destroyed within a few minutes.

Let's start with the good news and let alone the reason why my hopes were destroyed for another minute or two. Maemo is based on Debian GNU/Linux and various Free Software components, such as GTK+, gstreamer, esd and friends. Most of the system is Free Software which is a good thing(tm) and reading all of this really got me into Maemo. Okay, some applications seem to be proprietary, but I am sure that could be fixed rather easily, so I could once for all use a truly open phone.

...and then came the SDK installer shell script:

#!/bin/sh
# Copyright (C) 2006-2009 Nokia Corporation
#
# This is proprietary software owned by Nokia Corporation.
#
# Contact: Maemo Integration <integration@maemo.org>
# Version: $Revision: 1110 $

Now there is one question you should ask yourself: Why would someone trying to promote his platform as being open make the *installer* script for its SDK proprietary? Come on, it's an installer script, how much of your secret juice could be in there? What's the problem with people modifying it and working on this installer script in an open development environment?

I had high hopes for Nokia actually doing a bit better than Google, but it seems they've failed to do so. It may be me overreacting, but a proprietary SDK installer shell script scares me enough not to install the SDK and have a look at it for now nor to think about buying a Maemo-based device in the near future. Please Nokia, either get the facts straight or provide us with a free SDK to your free & open platform.

So, in short, Google is bad at working with the community and creating a truly open development process, and Nokia simply fails in terms of not scaring off prospective developers for their open platform with the proprietary SDK installer. Do you have any solutions in terms of an open phone environment, apart from what OpenMoko has come up with?

How to move panels in Gnome 2.28

I just installed Ubuntu Karmic Koala on my workstation and came across the problem of not being able to move/drag Gnome panels around in order to have the panels on my primary monitor.
On the Debian system that was powering the workstation before this was a non-issue as I could simply click, hold and drag both the upper and the lower panel, but this didn't work.

So, after a few minutes of googling I came across an entry at answers.launchpad.net[0] and a blog post, but I cannot seem to remember the URL to that one. I can imagine that some of you might be having the exact same problem, so the solution is holding down the ALT, whilst dragging as usual.

[0] https://answers.launchpad.net/ubuntu/+source/gnome-panel/+question/264

Summer Of Code 2009: Working for Debian

Yesterday Google announced the students and projects that have been accepted for Google Summer Of Code 2009 and guess what: my project was accepted. This means I will be working full-time on FOSS this summer.

So I guess it's about time to introduce my project to you: Distribution-independent update manager, mentored by Michael Vogt (mvo).

Okay, I believe some of you might wonder what this project is all about, as update-manager is in the Debian package archive already. There is a problem with update-manager though. As you see in the package's version number (it contains ".debian") update-manager has been adapted for use in Debian. Also, Debian contains update-manager 0.68 right now, whilst upstream (Ubuntu in this case) has released 0.111.6 (actually there were quite a few upstream versions meanwhile). The reason Debian is nowhere near being up-to-date with upstream is that right now a lot of effort has to be put into porting update-manager to Debian every time a new upstream release is made, because certain Ubuntu-specific functionality breaks update-manager in more or less severe ways on Debian.

This leads me directly to what my project is about: making update-manager (Ubuntu-) distribution-independent, but not package manager independent.
There are 6 main goals for this project, which I will be working on in the order below.

• Analyzing the code and identifying Ubuntu-specific parts.


• Creating a distribution-plugin interface and moving the Ubuntu-specific parts into a distribution-plugin, creating a core package that is distribution-independent.


• Creating a special notification for important/security related updates and providing the code that handles updates from security.debian.org as such.


• Creating a backend-plugin interface, moving the synaptics backend into a backend-plugin and optionally create a python-apt based plugin.


• UI redesign, providing a simpler interface to average joe, whilst allowing more experienced users to optionally display more information.


• Automatic downloading & installation of updates. This is still up to discussion, as automatic downloading is already provided by software-properties (-gtk and -kde) and automatic installation can be handled by unattended-upgrades. Both packages are part of Debian already.

Please note that this list should not be considered final and may be extended or modified over time. It exists to give you an overview of what exactly my project is about and how I am planning on carrying out the tasks.

Finally I wanted to let you know that I will keep you posted on the progress I am making, via this blog. Alternatively a blog aggregator for Debian's GSoC students has been set up over at http://soc.alioth.debian.org/feeds/blogs/, where you can not only find my posts, but those of all of Debian's students.

Python everywhere: computer games

This is the second article in my series Python everywhere and covers the use of Python for in computer games. The first article of this series covered the use of Python for the conficker worm scanner tool and can be found here.

Python everywhere: extending applications with Python

Extending applications with Python: gimp, Evolution, Inkscape, Paint Shop Pro, [...]

Python everywhere: A Python Operating System called cleese

Cleese....

Python everywhere: conficker scanner

This article is the first in my new series "Python everywhere".

As this is the first article in this series I would like to explain what the series is all about.
As an avid Python user and developer I want to share my observations whenever I find Python applications doing not-so-unusual things, Python applications running on embedded devices. In the end I want to point out just what the name of this series suggests: Python is everywhere and can be used for everything.

So, straight ahead to the first issue: the conficker scanner.

Introducing pyttpd

In this article I would like to inform you about my newest pet-project: pyttpd.

pyttpd is my effort of implementing a webserver in Python, with a focus on security (through privilege separation), extensibility and scalability.

I started this project because I was not entirely happy with the lack of flexibility and support for privilege separation by popular webservers. Whilst both lighttpd and Apache httpd provide means of running processes under different users these usually require hacks like suexec. Additionally I am somehow curious about how a fully-fledged webserver implemented in Python would perform compared to the mentioned daemons.

How using proprietary software can affect system security

There has been a lot of discussion on whether Free Software is more secure than proprietary software, but I have an additional argument that shows how the use of Free Software can improve system security.

Now you probably expect me to come up with a pure technical reason showing superiority of Free Software, but I am taking another path this time: let's talk about user trust.

python-argvalidate 0.9.0 released

Even though I planned providing a release candidate first, which can be seen in the project's Mercurial changelog I have released python-argvalidate 0.9.0 today. Tarballs can be obtained from the Python Package Index (pypi), as usual.

EuroparlTV for everyone? No, only for users of proprietary software!

The European Parliament (EP) has just recently started a new service: EuroparlTV. A web-TV service which should give citizens of the European Union (actually everyone around the world)  a way to inform themselves about how the EP works, what it does, and so on.

After I first read these news over at heise (german) I was impressed, but started to fear that yet again some sort of government has invested in proprietary software and is able to bring its services only to users of such software. Seconds later my fears became reality.

EuroparlTV seems to work only for users of either Adobe's proprietary Flash player (via the proprietary Adobe Flash file format) or users of Microsoft's Windows Media Player (via the proprietary WMV file format).

What this means to an open web, that is usable for everyone, should be clear.

Basically this is a service all citizens of the European Union pay for, but some cannot use. Is this really how governments (and the EP is some sort of government) should treat their citizens? Rather not.

On the one hand the European Commission is fighting vendor lock-in and monopoles, but on the other hand it directly helps these vendors by creating such services. Not a smart move in my opinion, neither is it understandable.

What I am asking myself though is why the EP was unable to create such a service, which itself could be quite interesting, without having all users of that service use proprietary software?
Is it so hard to deliver the service in a free (as in freedom), standardized format?
I will let answering these questions to you, but keep in mind that there are alternatives to this whole proprietary mess, like Ogg, which are completly free.

Personally I am pretty disappointed by this move. However, I hope that I at least informed people that there is a problem with EuroparlTV.
Putting it simple and short this way the EP does a great deal with helping vendor lock-in whilst fighting the freedom of its own citizens. Even though it should be the other way round.

sptest - a Python unittest extension

Even though this is meant to be an introduction to sptest, I want to start off by letting you know why I wrote this extension to the Python unittest module.

I am currently working on a (still private) project that uses Python's unittest module and the underlying framework. Even though unittest is a great utility for creating unit tests I found that the output it generates is unusable for me. I wanted something different though. Maybe a bit more aesthetic than the simple command line output unittest provides.

So I started off writing a class extending unittest.TestResult to fit my needs. I soon realized that interfacing with this part of unittest is not as easy as it could be, but I still continued to write my class.
After two hours of hacking I noticed that this class had become a monster. It was huge and I felt uncomfortable having such a huge class lying around somewhere in a "runtests.py" file for the only reason of having that pretty output.

This was the point when I decided to move all that code into a separate project and try to come up with a more intuitive API. This was the second when sptest was born, about 5 hours ago.

What I did come up with is a small Python module that makes customizing the way unit test results are presented (or stored) easier. It currently includes two output handler classes. One providing fancy CLI output on ANSI terminals and the other one providing XML output.

Additional output handler classes could store the result of the unit tests in a database or send it to a central point on the network, but implementing that is up to someone else, for now.

Running unit tests with sptest is as simple as calling:

sptest.TestMain(TestSuite).run()

By default the FancyCLIOutput handler class will be invoked and you will see why the handler is called the way it is immediatly.

In order to generate an XML file containing the test results one just has to modify the call to sptest to look like this:

sptest.TestMain(TestSuite, output_class=sptest.output.XMLOutput).run()

sptest also provides support for preparation and cleanup functions. The only thing you have to do is define these functions and adjust the arguments passed to TestMain accordingly.

Most of the code is already documented and a doxygen configuration file for generating the html documentation comes with the code. Also, two examples are included that show how to use sptest.

UPDATE: Google Chrome: Good or evil? -- GOOD!

UPDATE: You can find the update to this article at its bottom.

Even though Google's slogan is "don't be evil" I am not entirely sure whether this also applies to their newest development: the Google Chrome browser.

The announcement over at the Official Google Blog tells us that Google is about to release a Free Software-based browser. When I first read the announcement I wasn't too impressed reading that Google has actually built a browser, this was logical and I have been expecting this move for years. Also, reading that they based their browser on Free Software didn't impress me too much either, but then I found the comic.

Autoconf and Python: checking for modules

I am currently writing a Python application that makes use of GNU Autotools as build system and noticed that determining whether a specific Python module is installed is not that easy and no usable Autoconf macro exists. So I came up with my own solution, which I would like to share with you.

The AC_CHECK_PYTHON_MODULE macro takes two arguments: The module name and optionally the variable name holding version information. This way it is not only possible to determine whether a module is installed (ie. loads in Python) on the current system, but also retrieve version information from that module.

The following examples checks whether the Crypto module is installed and retrieves its version information from Crypto.__version__:

AC_CHECK_PYTHON_MODULE(Crypto, __version__)

The macro itself does never report and error, but rather only a found/not found result. Error checking is up to the user and can be done via these two Autoconf variables:

• PYTHON_<MODULE_NAME>


• PYTHON_<MODULE_NAME>_VERSION

PYTHON_<MODULE_NAME> is set to "1" if the module is present and "0" if not present.
PYTHON_<MODULE_NAME>_VERSION is only set when the version variable argument has been set and contains the version information of the module, if the module been found. If the module is not present this variable is also set to "0".

The version variable argument is optional as I wrote, so the following invocation works too and only checks whether the distutils module is present:

AC_CHECK_PYTHON_MODULE(distutils)

As I wrote earlier in this article I would like to share this macro with you. You can download it here.

Debian GNU/Linux 5.0 ("lenny") on a Samsung P55-Pro T8100 Sevesh

I have recently bought a new laptop, a Samsung P55-Pro T8100 Sevesh. As I was not able to find an installation report for this model anywhere on the internet I thought writing one myself is a good idea. This way people interested in getting this laptop or installing GNU/Linux on it can get some information.

The article covers both the hardware configuration of the laptop itself, a list of which features of the laptop do work and which don't (do not be afraid, most things work perfectly well out of the box) and finally a short installation report.

Why are hardware manufacturers keeping specs to themselves?

This is one question I have been interested in ever since I started using GNU/Linux.

Just think about it for a moment. About 20 years ago you got specifications for pretty much every piece of hardware you bought. You were given exact instructions on how to use the hardware you just bought, not only how to install it. Things have changed since then.

If you buy any piece of hardware today you actually have to expect not to get any documentation on how to "talk" to your new toy. You are only given a CD (sometimes even only a link to a homepage) containing drivers for a few specific operating systems, usually only Microsoft Windows.

Now I am no driver hacker and so I probably wouldn't be able to implement a driver for anything on my own anyways, but the Free Software community would largely benefit from hardware documentation, as there are a lot of capable driver hackers out there.

This is not a problem that only affects the Free Software community though. There are a lot of pieces of hardware which do not work on recent proprietary operating systems anymore due to lack of support by its manufacturers.
At least this problem would not exist for Free Software operating systems, such as GNU/Linux, if hardware makers would publish documentation of their hardware. The people still using devices which are well beyond their end-of-life could implement drivers on their own, not being dependent on anyone.

What I am really wondering about in this case is why hardware companies are unable to coin standards for accessing devices of the same class. It works perfectly well for USB (take USB mass storage devices as an example) and I do not understand why there can't be standardized interfaces to other hardware, such as network adapters, as well. On a very-low level these standardized interfaces do work. Just think of PCI, PCI Express or AGP.

Actually, if you think about this for a few more seconds you should realize one thing: Having standardized interfaces for devices of the same class would cut a lot of costs for hardware makers. Why? Oh well, if they design a brand new networking chip and still implement the given standard there would be no need of writing a new driver. Wait, there would be no need for per-device drivers at all. Implementing a common driver that accesses the standardized interface would be enough, for a whole range of devices.

So what am I asking of hardware makers? I would love to see companies creating devices of the same class to get together, create standardized interfaces, publish them and implement them in their new devices.
I know, this is not likely to happen anytime soon, so a more realistic approach is asking for Free Software drivers and/or documentation.

Personally I have stopped buying hardware which "works" with GNU/Linux, I have come to the point where I try only to buy hardware which either comes with Free Software drivers from the manufacturer or documentation which allows implementation of Free Software drivers.
This is probably the best way of showing these companies what you demand: Freedom.

Free Software Supporter

I was quite stunned when I noticed that the Free Software Foundation (FSF) has recently started a new monthly-published newsletter, called the Free Software Supporter.

The reason I was amazed is not the fact that the FSF is now publishing such a newsletter, but rather the fact that I did not hear about that yet. Basically, the Supporter is about informating the Free Software enthusiasts about recent happenings and the work of the FSF, the GNU project and the global Free Software community.

It seems as if I am not the only person that is excited about the supporter, as Joshua Gay, who apparently is writing the Supporter, also seems to like it, as he writes in a blog post:

I hope that you enjoy the Supporter. I am looking forward to reflecting each month upon the work of the FSF, the GNU project, and the global free software community. I only hope that the number of highlights I add each month will continue to grow as quickly as the community is growing. In either case, we hope to keep it short and we hope to keep you informed.

You can sign up to receive the Supporter via email on a monthly basis at http://lists.gnu.org/mailman/listinfo/info-fsf and you can read the first issue online at http://lists.gnu.org/archive/html/info-fsf/2008-03/msg00000.html.

Also, if the Supporter looks like an interesting read to you, you may as well enjoy the monthly newsletter the FSF Europe publishes. The FSFE Newsletter can either be read online or you can sign up for the FSF Europe press-release mailing list.

Personally I believe both newsletters are worth reading and give you a great overview of what has happened in the past month, what is going to happen and the work done by the FSF and FSF Europe.