update-manager weekly update #6

So finally I have the time to provide you with a weekly update, instead of my usual bi-weekly ones.

Unfortunately I did not work on anything on last week's TODO list, but found other issues I worked on and corrected.  So let's have a look at what I've done.

Debian packaging update

I have done some work on the Debian packaging, which allows update-manager to be built using dpkg-buildpackage now. The way packages are splitted is not finalized yet and not up-to-date with my (and my mentor's) idea of how we should do that. You can expect an update to that soonish.

Automatically invoking package list reloading / update check

There is a command line switch (namely -c, or --check) now, that automatically performs an update check on startup. This gives other programs, like software-properties, a way of forcing a check when, for example, the package list sources have changed.

Checking/unchecking all updates in Gtk frontend

Finally the small feature of selecting or deselecting all updates works in the Gtk frontend. Special cases like "all updates already checked" or "no updates checked" yet are handled too, meaning that you can only use one of these methods if it actually makes sense.

Package dependencies in python-apt backend and Gtk frontend

Both the python-apt backend and the Gtk frontend are now aware of package dependencies. This means that when you select an upgrade that depends on another one that other update is selected too. The same works vice-versa too. Additionally the UI now lists all dependencies and dependencies on packages that are not installed yet and automatically deselects all updates that would requires new packages to be installed.

Displaying of overall download size in Gtk frontend

There has been a missing feature (ok, maybe a bug) so that the displayed download size would not be updated in the Gtk frontend. This has been fixed.

Install button being set sensitive correctly in Gtk frontend

In the past the install button would be set to either sensitive or insensitive at startup and not updated afterwards. That means if there were no packages to update when starting update-manager, then checking for updates where new updates are found, the install button would not be set sensitive again. I fixed that too.

Sorting of packages in Gtk frontend

In the Gtk frontend packages were not sorted at all, which meant that finding a specific package was rather hard. I added code that sorts the update list by package name now, which solves this issue.

Bugfixing humanize_size

The humanize_size method, which is responsible for human-readable size displaying in the Gtk frontend contained a major bug so that sizes were rounded. Again, I was able to solve this.

Next week's TODO list

As I didn't find time to work on last week's TODO list my new TODO list is in fact my old one, with additional "Bugfixing" and "Debian packaging" tasks:

• Downloading and installing of updates


• Bugfixing (?)


• Debian packaging


• Checking that everything is documented


• Even more unit tests


• Pylint checking


• If time permits and everything else works correctly: working on an aptdaemon backend

The next thing you can expect me to update is the Debian packaging and the documentation, which are my highest priority tasks for now, followed by support for downloading and installing updates.

Happy hacking!

update-manager weekly update #5

Firstly I have to apologize again for not providing you with weekly update #4, but again I didn't have the time to write one, so this post is going to sum up everything that happened since my last update.

Let's have a look at my previous TODO list:

Documentation

Even though my TODO list entry contained a more detailed entry I have updated the UpdateManager documentation as a whole, leaving only a few blank spots right now.

Ubuntu distribution specific code

I implemented changelog fetching for Ubuntu, which works just as fine as its Debian counterpart now.

More unit tests

There are plenty of unit tests now, but not everything is being tested yet. I am especially proud of my Python interface validation code, that is being used in unit tests to check if handlers implement an interface correctly.

Update list downloading

Checking for updates is what caused me major trouble in the past few days. Basically I had all the code ready, but for some reason the UI froze, with no apparent reason.
However, today I was able to finally identify and fix the problem. As I expected my code was just fine, but python-apt was messing up. I am going to discuss the exact problem and its solution later on, but first: a screenshot. :-)

Note: As you probably noticed I replaced the default progressbar with a pulsating one, because we cannot get exact information on how many items/bytes to fetch and would likely get a progress bar moving backwards, which isn't beautiful.

Further changes

The TODO list was rather short and I did a lot of other work, which I want to elaborate on.

Dynamic selection of frontend, backend and distribution specific modules

Even though this is probably not of any interest to John Doe, it helps a great deal when debugging code as all three components can be selected via separate command line switches now.
Additionally some magic has been put in place that automatically detects the system's distribution and loads the corresponding distribution specific module. This is done via lsb_release and the newly introduced code in UpdateManager.Util.lsb.

Pylint cleanup

Just out of curiosity I decided to start a pylint run on the codebase and quite a few problems were detected, which I then fixed. To be honest though I added quite some code afterwards that probably needs pylint checking and fixes again.

update-manager IPC

My original plan and IPC design involved using callback functions and passing them between the different modules. Even though this worked out fine I had the feeling this wasn't clean enough and decided to ditch this approach and replace it with handler classes.
The handler base classes now provide an interface of methods that are called on certain events and their implementations act accordingly. The main benefit was that I could easily drop a lot of enums and rather have different methods handling different events.

Gtk, threads and python-apt

With the new IPC approach it became easier to use threads that do the actual work in the background, which I had implemented in next to no time, but a few problems showed up.
Whilst cache reloading from within a thread worked just fine checking for updates did not, and until today I didn't know why. I spent a good amount of time debugging this issue, even using python profiling, but nothing obvious showed up. The background process was running, whilst the UI froze.
Today I finally found the root of the problem: python-apt. Even though I assumed that the python-apt worker threads must be stealing CPU time from the thread running gtk.main I wasn't sure how this could be happening, having two completely independent threads.

Now, the cause of all this mess was that Python has a global threading lock and it seems as if this one is *LOCKED* when running C-code, such as the one python-apt comes with. The solution lies in calling Py_BEGIN_THREADS_ALLOW and Py_END_THREADS_ALLOW from within the C code, to release the global lock and let the Python interpreter do some work every now and then.

As with the python-apt acquire code I was able to allow other threads to work as soon as the fetching code starts working and only disallow threads when actually modifying Python objects or calling methods and/or functions. Surprisingly python-apt already made use of this in its cache loading code, but not the fetch progress code.
Fixing this problem took me less than half an hour and you probably can't believe how glad I was to finally get things working again.

UI updates & other changes


Some details in the UI were anything but optimal, like horizontal scrollbars in a few places, which I removed. Additionally I saw the need to move some code out of the Gtk frontend's __init__.py file and to a separate ui.py file.
A full list of all changes I made is available from the bzr changelog at bzr.debian.org.

A few more screenshots

Finally, I would like to provide you with two more screenshots (don't worry about my system being insecure because of not applied updates - this is a testing machine that is  not up-to-date on purpose):





TODO list

My TODO list for next week:

• Downloading and installing of updates


• Checking that everything is documented


• Even more unit tests


• Pylint checking


• If time permits and everything else works correctly: working on an aptdaemon backend