Posts Tagged ‘yum’

The Shiny New DNF, and Why I Prefer Yum!

You certainly already know about DNF, the new package manager in Fedora which is available in Fedora repositories since Fedora 18 while Yum is still the default (command line) package manager. However, DNF is expected to replace Yum as the default command line package manager in Fedora 22. And so, every Fedora user should consider it seriously.

DNF has some interesting properties that I really liked about it. Probably the most visible one, and certainly the most exciting one for me, is speed. It is really much faster than Yum. Yum spend much time specially when starting up. For example, I ran a similar “list” command using both DNF and Yum for a few times. Yum took around 25 to 29 seconds to do it, while DNF did it in about 5 seconds (no downloads for both). I don’t know what does do during this period, but my HDD is busy most of the time.

Surprisingly, Yum downloads compressed SQLite metadata, but DNF downloads compressed XML metadata. The former is considerably larger than the latter, but Yum selected it for speed purposes! AFAIK, DNF (actually libsolv) doesn’t use XML metadata internally but converts them to its own format; and IMHO this is actually the right thing to do. Yum could also use a similar approach (I’d prefer), but maybe they preferred to waste bandwidth rather than client’s processing power (and yum wouldn’t be faster than today anyway).

It was great news for me; not only DNF is way faster than Yum, but also it downloads much smaller metadata (about half the size of compressed sqlite dbs used by Yum)… However, I was wrong; The amount of metadata downloaded by DNF is actually more than that of Yum! I saw that Yum downloads about 20M of metadata for fedora-updates repository while DNF get around 30M! Why? Because DNF is too lazy and downloads all metadata, including list of all files in all packages which is around 15M; while Yum only downloads list of files when they are actually needed. And in my system, Yum has never downloaded the list of files while DNF downloads them every time the repository is updated. I always hated Yum for downloading such a large metadata over and over again, and DNF is just horrible. Unfortunately, DNF author has no will to change the situation.

Actually, I use two plugins with Yum which I’d like to have in DNF too: yum-plugin-local and my own Yum Fast Downloader. The former is not that important, and I can live without it; and I hope to be able to create an equivalent of the latter, while it will need a completely new approach.

Currently, I use DNF for some operations like search and list commands, but use Yum for most other tasks. And I almost always run DNF from cache, except when I have free Internet access! I’ve also disabled automatic cache download by DNF, which can make DNF much more hostile!

DNF can eat your Internet credit silently!

Let’s assume that you pay for your Internet connection per GiB. And you install Fedora with DNF, and use it for one month. You never use DNF during this month. Now, you might get shocked that DNF has used about 1GiB of your internet credit, while you have not used it at all! Even if you decide to use DNF after one month, what you get is the latest metadata. So, 30MiB of that 1GiB is what you actually used. Yes, this is currently the default settings, which I consider insane. If Fedora 22 is released with DNF configured like this, it means that: Fedora 22 consumes about 1 GiB of your internet connection by default just for downloading repository metadata! Surprisingly, even if you update your Fedora regularly, the amount of actual data you download to update your Fedora might be less than that (thanks to delta RPMs).
Its true that the repository metadata format is currently very inefficient for updates and you should get all metadata again even if one package is added, and it is not DNF’s fault, but the decision to download metadata automatically, and downloading unnecessary metadata, is what DNF has made. That makes sense for people who have access to cheap internet, or don’t pay for it per GiB; but IMHO this should not be the default settings for a distribution like Fedora.
And, 1 GiB per month is just an estimate. I assumed that updates are pushed to Fedora updates repository once per day, and the metadata is around 30MiB. However, if, for example, updates are pushed two times a day, the estimate would be around 2GiB. Notice that with the default settings, DNF refreshes metadata every 3 hours. So, if there are regular updates in Fedora updates repository, DNF would re-get metadata up to 8 times a day, which will be around 8*30=240MiB per day, which will be 8GiB per month…

Anyway, if you happen to install DNF, make sure that you check metadata_timer_sync option in “man dnf.conf” and put an appropriate option in your dnf.conf. I use “metadata_timer_sync=0” as I don’t want it to waste my money for no good!


Nowruz Gift: Yum Fast Downloader 0.6.0 Released!

Happy Nowruz!

Gladly, I was able to devote some time to update yum fast downloader plugin to work much better with recent yum versions in Fedora 19/20. The plugin effectively disabled delta rpm support since the integration of presto into yum. To fix the issue, I reworked the plugin using the new download framework of yum/urlgrabber. The result is yum-fast-downloader plugin for Fedora 20, which not only fixes the problem but also brings better integration. As a result, the plugin is now responsible for almost all downloads including downloading drpm packages.
Besides, it is now possible to specify arbitrary command line arguments (e.g. -q) for aria2c in the plugin’s configuration file. Unlike previous versions, in this version download of multiple files in parallel are handled by multiple instances of aria2c; but I’d like to change it back to a single aria2c instance in future. I’ve added the new version for Fedora 20 in the repo, but not for Fedora 19. It should work on Fedora 19 too, but I have not tested it. If you tried YFD 0.6.0 in Fedora 19 successfully, please let me know so that I’ll add the new version to F19 repo too. Thanks!

What about DNF?
DNF looks promising. It is much faster than yum, and even uses xml metadata rather than sqlite ones which are smaller. However, it’s internal structure is completely different from yum, so it needs a completely new plugin. I’m still not sure if it is possible, but I’ll try in future. 🙂