Ext4 defragmentation with e4defrag

[ Saturday, 31 May 2008, optimizationkit ]

Lately I have described ok_defrag — a simple tool for defragmenting Linux file systems. Today, I want to describe a specialized program designed to defrag the new Ext4 file system.

(For all who would like to say that Linux file systems don’t require defragmenting: Please redirect your great ideas to the Ext4 developers. They are the authors of this program — I do not have anything in common with it. As it will turn out, Ext4 is indeed quite different than other Linux file systems. Thanks in advance for your understanding :) )

To begin playing with e4defrag we must start compile of new kernel with patches needed to carry out the defragmentation. Below you will find a short description of that process, and if you would not like to use all those experimental patches for Ext4, then you should be interested in these seven only:

ext4-online-defrag-exchange-blocks-between-two-inodes.patch
ext4-online-defrag-relocate-file-data.patch
ext4-online-defrag-alloc-contiguous-blks.patch
ext4-online-defrag-ext4-defrag.patch
ext4-online-defrag-for-relevant-files.patch
ext4-online-defrag-check-for-freespace-fragmentation.patch
ext4-online-defrag-move-victim-files.patch

In first step we start from creation of directory, grabbing the kernel source, and downloading and unpacking needed patches.

mkdir linux-2.6-ext4
cd linux-2.6-ext4/
ketchup -G 2.6-git
wget http://www.kernel.org/pub/linux/kernel/people/tytso/\\
ext4-patches/2.6.26-rc2-git5-ext4-1/broken-out.tar.bz2
tar xjvf broken-out.tar.bz2
mv broken-out patches

Then we apply the patches on the sources.

quilt push -a

Then we start a process of configuring/building our new system kernel.

cp /our/beloved/.config .
cp /our/beloved/script/for/building/system/kernel/build.sh .

(on my PC it look like this:)

#!/bin/sh 

VER=`ketchup -m` 

make
sudo make modules_install
sudo cp arch/x86/boot/bzImage /boot/vmlinuz-$VER
sudo cp System.map /boot/System.map-$VER
sudo /sbin/new-kernel-pkg --mkinitrd --depmod --install $VER
)
make oldconfig
./build.sh

Meanwhile, when our new kernel compiles, we may open file: linux-2.6-ext4/patches/ext4-online-defrag-command.patch in our text editor of choice and we delete all lines from 1 to 18, and then from 1932 to 1935. Then, we copy our source code to other location.

cp linux-2.6-ext4/patches/ext4-online-defrag-command.patch e4defrag.c

And compile…

gcc -Wall e4defrag.c -o e4defrag

It is worth to grab some tools to handle the Ext4 file systems (the very latest version from the code repository).

git-clone git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git
cd e2fsprogs
./configure
make
make install

After running our system with brand new kernel for the first time, we can finally start to test e4defrag. First we create and mount a test file system.

dd if=/dev/zero of=fs.img bs=256M count=1
/sbin/mkfs.ext3 fs.img

Now we must allow the use of experimental code on that partition.

/sbin/debugfs -w fs.img
debugfs 1.41-WIP (27-Apr-2008)
debugfs:  set_super_value s_flags 4
debugfs:  quit

Now our new file system is ready to be mounted

mount -t ext4dev -o loop,extents fs.img /mnt/loop0/

How to cause fragmentation at Ext4? - I have not yet tested any good way for fast fragmenting Ext4. This file system contrary to other Linux file systems is exceptionally resistant to fragmentation.

 /sbin/fsck.ext3 -nfv fs.img
e2fsck 1.41-WIP (27-Apr-2008)
First  course: checking i-nodes, blocks and sizes
Second course: checking directory structure
Third  course: checking communications directories
Fourth course: checking counters appeals
Fifth  course: checking the summary information about groups

     368 inodes used (0.56%)
       1 non-contiguous inode (0.3%)
         number i-nodes with blocks ind/dind/tind: 136/129/129
  202885 blocks used (77.39%)
       0 bad blocks
       0 large files 

     351 regular files
       8 directories
       0 character device files
       0 block device files
       0 fifos
       0 links
       0 symbolic links (0 fast symbolic links)
       0 sockets
--------
     359 files

It seems that Ext4 is a tough row to hoe, but its developers still deliver a defragmentation tool for it. Take a peep!

With use e4defrag we can defrag single files, for example:

e4defrag /patch/to/file

or all directories:

e4defrag -r /patch/to/directory/

also, there is a way to defrag a partition:

e4defrag /dev/sda1
e4defrag file_system_image.img

E4defrag contrary to other solutions - Con Kolivas defrag or ok_defrag is a true professional program designed to defrag Ext file system — it does not use a simple tricks with moving files, but it checks where on a drive the free blocks are located and then moves the data into those places.

A user, who will be capable to fragment Ext4, surely will be satisfied with the results of e4defrag. I am fully satisfied with the Ext4, for now I have no need to run e4d or even ok_defrag :).

Subscribe to RSS feed for this article!

8 Comments

fold this thread Aaron  Sunday, 1 June 2008 o godz. 8:49 am #  Add karma Subtract karma  +1

As mentioned in my previous comment in your previous thread, fsck is only reporting inode fragmentation, *NOT* block fragmentation. Also, e4defrag is designed specifically for this use. Taking a look at one of the key features of ext4- extents, you will have an exceptionally difficult time fragmenting the inodes on this filesystem.

 
fold this thread Giordano  Sunday, 1 June 2008 o godz. 6:48 pm #  Add karma Subtract karma  +2

In fact, if you have a look at the paper linked in the wikipedia ext4 article you can read these lines at the end:

To reduce system administrator’s effort, it is nec-
essary to automate defragmentation. This can be
realized by the following procedure.
1. The kernel notifies the occurrence of frag-
ments to user space.
2. The user space daemon catches the notifica-
tion and executes the defragmentation com-
mand.

This could really be a cool feature to have ^_^.
The effort required to the user to maintain an efficient system under the fs aspect would be basically cut down to 0 :)

fold this thread ok  Sunday, 1 June 2008 o godz. 7:23 pm #  Add karma Subtract karma  +0

I will give you good advice - _never_ talk about Linux file systems fragmentation (unless you want to be burned at the stake).

 
 
fold this thread Giordano  Sunday, 1 June 2008 o godz. 6:49 pm #  Add karma Subtract karma  +1

There’s a spurious “in fact” in the beginning of my previous comment :$

 
fold this thread Ben  Tuesday, 3 June 2008 o godz. 1:17 pm #  Add karma Subtract karma  +3

Nice article; thanks.
Regardless of whether the file systems fragment, at least you talk about it, and from what I can see everywhere, ext4 really is very good with not having much fragmentation.

 
fold this thread scorp123  Tuesday, 3 June 2008 o godz. 10:05 pm #  Add karma Subtract karma  --3

LOL, more unqualified nonsense ….

 
fold this thread EdwardOCallaghan  Tuesday, 10 June 2008 o godz. 2:56 pm #  Add karma Subtract karma  +0

Errrm ..

ZFS .. OpenSolaris, OSX and FreeBSD Kernels have it..
Kernel land ZFS in Linux, get with the times Linus..

Give a crack;
http://www.opensolaris.com

Regards.

 
fold this thread PolishJoke  Wednesday, 2 July 2008 o godz. 3:46 am #  Add karma Subtract karma  +0

You people are so very very high. Don’t even know where to start. RAID arrays die twice over before linux filesystems fragment. Desktops? User space? WTF are you people smoking.

 
Name (required)
E-mail (required - never shown publicly)
URI

Adjust field size: shrink | enlarge)

You can use simple HTML in your comments. Some examples are as follows:
  • A hyperlink: <a href="polishlinux.org">GNU/Linux for everyone!</a>,
  • Strong text: <strong>Strong text</strong>,
  • Italic text: <em>italic text</em>,
  • Strike: <strike>strike</strike>,
  • Code: <code>printf("hello world");</code>,
  • Block quote: <blockquote>Block quote</blockquote>

Sponsors

RSS: Comments

You can follow the comments to this article through a special channel RSS 2.0 .

Related articles: CLI

 more »

PolishLinux Top Content

Select distros:

vs

Shopping Ads

Google Ads