Comments on: Ext4 defragmentation with e4defrag http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/ All About GNU/Linux and BSD - reviews, comparisons, articles Sun, 13 May 2012 17:43:19 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: cmol http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-162751 cmol Mon, 20 Dec 2010 14:54:52 +0000 http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-162751 @agaida, nice trick for small filesystems but i can't imagine doing that with my 6TB data. i'd like a defrag tool :-) unless i do it sequentially, folder by folder eg 1TB at a time... will that work? ps. i'm just an enthusiast/hobbyist, so forgive my ignorance :-) interesting reading on the harddrive technologies. @agaida, nice trick for small filesystems but i can’t imagine doing that with my 6TB data. i’d like a defrag tool :-) unless i do it sequentially, folder by folder eg 1TB at a time… will that work?

ps. i’m just an enthusiast/hobbyist, so forgive my ignorance :-)

interesting reading on the harddrive technologies.

]]> By: agaida http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-143161 agaida Sun, 13 Jun 2010 11:13:01 +0000 http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-143161 Yesterday if figured it out to me. Size does matter. It make sense to rebuild the partitioning under Ubuntu. Finaly i got a 10 G '/', a 10G '/usr', 100G '/home' and '/tmp' in tempfs. Then 'defragment' the system with a few copys like this: -build a rescue system -make a 100G partition 'dirty', mount as '/defrag' - cp -ax /source/* /defrag - rm -rf /source/* - cp -ax /defrag/* /source - rm -rf /target/* In the end i have no fragmentation left and it speeds up the system a lot. But it will be nice to have a ext4defrag. Yesterday if figured it out to me. Size does matter. It make sense to rebuild the partitioning under Ubuntu. Finaly i got a 10 G ‘/’, a 10G ‘/usr’, 100G ‘/home’ and ‘/tmp’ in tempfs. Then ‘defragment’ the system with a few copys like this:
-build a rescue system
-make a 100G partition ‘dirty’, mount as ‘/defrag’
- cp -ax /source/* /defrag
- rm -rf /source/*
- cp -ax /defrag/* /source
- rm -rf /target/*

In the end i have no fragmentation left and it speeds up the system a lot. But it will be nice to have a ext4defrag.

]]> By: NoName http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-139651 NoName Thu, 27 May 2010 23:11:38 +0000 http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-139651 David I totally agree with you! When I want to get my shiny new Ubuntu I only use Transmission to save bandwidth on the server side and to be sure I got the error-free file (due to internal hash checking). I noticed that my ext4 performance goes down to 15-20% of nominal one when using - lets call it - non-linear download or video editing. When I make a copy of file after downloading and remove the original the fragmentation rate goes down, but I don't feel like doing it every time I download something. Ext4 essentially needs e4defrag! David I totally agree with you! When I want to get my shiny new Ubuntu I only use Transmission to save bandwidth on the server side and to be sure I got the error-free file (due to internal hash checking). I noticed that my ext4 performance goes down to 15-20% of nominal one when using – lets call it – non-linear download or video editing. When I make a copy of file after downloading and remove the original the fragmentation rate goes down, but I don’t feel like doing it every time I download something. Ext4 essentially needs e4defrag!

]]> By: David Turner http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-136841 David Turner Fri, 16 Apr 2010 15:03:13 +0000 http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-136841 I use a certain P2P system, and at some point I noticed that copying files to a flash drive would run at about 2MB/s, whereas copying side-to-side between two flash drives would run at about 20MB/s. This was a mystery until I tried running filefrag on various large data files. The number of extents was in the hundreds of thousands. After some cleaning up, the speeds improved to normal. So claiming that Linux filesystems aren't affected by fragmentation is, frankly, bullshit. Of course this was ext3 - I've since upgraded to ext4 and am waiting with interest to see how it handles my usage patterns. I use a certain P2P system, and at some point I noticed that copying files to a flash drive would run at about 2MB/s, whereas copying side-to-side between two flash drives would run at about 20MB/s. This was a mystery until I tried running filefrag on various large data files. The number of extents was in the hundreds of thousands. After some cleaning up, the speeds improved to normal. So claiming that Linux filesystems aren’t affected by fragmentation is, frankly, bullshit. Of course this was ext3 – I’ve since upgraded to ext4 and am waiting with interest to see how it handles my usage patterns.

]]> By: Frank http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-135081 Frank Wed, 10 Mar 2010 13:03:00 +0000 http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-135081 you are my hero you are my hero

]]> By: JohnDoe http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-133381 JohnDoe Wed, 27 Jan 2010 01:52:00 +0000 http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-133381 You also have to take cell size into account, since its contents can only be written all at ones. Imagine the ssd-cell is 16k and your file system sector size is 4k. Means you have four logical blocks on a single ssd-cell. If you want to write to only one of these sectors, the ssd-controller would have to read the cell into a cache, modify it's content in-memory and flush it back to the cell afterwards. You would have needed to do en extra read and actually re-write 3 data 3 times the size of your payload. A Sequential write is of course much simpler: Just write it directly to disk! Wear-Levelling is actually making these things somewhat more complicated, especially since manufacturers don't disclose their individual algorithms. You also have to take cell size into account, since its contents can only be written all at ones. Imagine the ssd-cell is 16k and your file system sector size is 4k. Means you have four logical blocks on a single ssd-cell. If you want to write to only one of these sectors, the ssd-controller would have to read the cell into a cache, modify it’s content in-memory and flush it back to the cell afterwards. You would have needed to do en extra read and actually re-write 3 data 3 times the size of your payload.

A Sequential write is of course much simpler: Just write it directly to disk!

Wear-Levelling is actually making these things somewhat more complicated, especially since manufacturers don’t disclose their individual algorithms.

]]> By: Marco http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-133321 Marco Sat, 23 Jan 2010 20:40:11 +0000 http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-133321 I just switched from ext3 to ext4 and 'benchmarked' a little. Here are the results before the transition: Boot time reported by Bootchart: 45 seconds Using VCD gear to export a 700 MB MPEG of a VideoCD image: 51 seconds Then I did that online conversion with e2fstune: boot: 43s VCD: 31s I was impressed by the write speed for large files but the boot time was unsatisfying. I tried e4defrag, but it did NOT add the 'e' attribute as reported by lsattr to all my files. So I made a backup of my data and formated the root partition. With a fresh ext4 file system and all data copied back with rsync I got this result: boot: 34s (It was 26% slower before) VCD: 30s Now that's how I like it! Wether there WAS considerable fragmentation on my ext3 fs or some feature of ext4 wasn't active after the online conversion, I don't know and don't want to join the flame war. But if someone asked me I would advise them to reformat. I just switched from ext3 to ext4 and ‘benchmarked’ a little. Here are the results before the transition:
Boot time reported by Bootchart: 45 seconds
Using VCD gear to export a 700 MB MPEG of a VideoCD image: 51 seconds

Then I did that online conversion with e2fstune:
boot: 43s
VCD: 31s
I was impressed by the write speed for large files but the boot time was unsatisfying. I tried e4defrag, but it did NOT add the ‘e’ attribute as reported by lsattr to all my files. So I made a backup of my data and formated the root partition.

With a fresh ext4 file system and all data copied back with rsync I got this result:
boot: 34s (It was 26% slower before)
VCD: 30s

Now that’s how I like it! Wether there WAS considerable fragmentation on my ext3 fs or some feature of ext4 wasn’t active after the online conversion, I don’t know and don’t want to join the flame war. But if someone asked me I would advise them to reformat.

]]> By: Alex Atkin UK http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-133261 Alex Atkin UK Fri, 22 Jan 2010 17:39:33 +0000 http://polishlinux.org/apps/cli/ext4-defragmentation-with-e4defrag/#comment-133261 I'm pretty sure when SSD specs claim sequential and random access they are referring to accessing a large file vs accessing a lot of smaller ones. The reason there is any difference at all between requests is that accessing a large file is considerably less IO requests than accessing a lot of small ones, and its those requests for read/write operations that seem to clog up SSDs. Fact is, data is not allocated how you think on an SSD anyway. They are designed to fragment the data intentionally due to the wear levelling process. The REAL killer for SSDs is low disk space, as the wear levelling demands that every write operation can be allocated to a different part of the drive (the least used). So if you have very little free space, it struggles to operate correctly. But that is also why defragmenting an SSD is pointless, you would just be jumbling the data at random around the drive wasting write cycles of the flash chips. I’m pretty sure when SSD specs claim sequential and random access they are referring to accessing a large file vs accessing a lot of smaller ones.

The reason there is any difference at all between requests is that accessing a large file is considerably less IO requests than accessing a lot of small ones, and its those requests for read/write operations that seem to clog up SSDs.

Fact is, data is not allocated how you think on an SSD anyway. They are designed to fragment the data intentionally due to the wear levelling process. The REAL killer for SSDs is low disk space, as the wear levelling demands that every write operation can be allocated to a different part of the drive (the least used). So if you have very little free space, it struggles to operate correctly. But that is also why defragmenting an SSD is pointless, you would just be jumbling the data at random around the drive wasting write cycles of the flash chips.

]]>