As a control it might be worthwhile to run your fragmentation script, run the system normally for a period, say a week or so, then check the fragmentation level to see if the fragmentation has been decreased by the normal filesystem operation.

I still wonder if it would not be easier to just copy a whole partition to another partition/harddrive, wipe the original data and then copy it back. (make two copies and checksums for added security)

Only real problem I see is, that you need way more diskspace to do that. Is there something more that I miss?

How about ZFS in opensolaris ?

I tried to launch ok_defrag but failed -no such command).

Previously, I downloaded the python-dialog packet and I did an image disk of my partition which needs defrag after four successive upgrades.

Could there be some more explicit information about it ?

Thanks for help.

“When is there no need to perform a defragmentation? Generally speaking, if fsck returns non-contiguous inode below 10%.”

Why? Not “Why is higher fragmentation *possibly* worse than lower fragmentation” - I already know this.

I’m asking: why did you pick the nice, round, base-10 friendly number “10%” as your recommended threshold?

Could it be because you’re picking numbers out of thin air? If not - if you’ve actually performed some real-world, performance-oriented benchmarks on filesystems with different fragmentation levels - then post these results. They’d be useful.

Having read your article, I’m quite confident you haven’t. It lacks any kind of rigour. For instance, why say that “35.2% of the inodes are not placed continuously — the fragmentation degree of files in our image is very high”? Surely “99.9%” is “very high”, and “35.2%” is rather lower?

It’s *relative* and, depending on the access patterns of the files that are fragmented, may be utterly unimportant in determining the performance of the containing system.

TL;DR Don’t give up the day job.

“Also, many users will never exceed 10%.”

Sounds like another fact, pulled from thin air!

If they won’t generally exceed 10%, and this method can’t take you much below 10%, and a newly created FS is at ~10% defragmentation, what does that tell you?

Perhaps that “10%” is *meaningless*! The only thing that matters is the performance of the system that contains the FS.

Professionally, as a *nix sysadmin who maintains many hundreds of Linux, *BSD and Solaris boxes for a tier-2 UK ISP, I’ve never defragmented any system. Ever. The time it’ll take to do, coupled with the risk inherent in moving *all* your data off one partition, onto another, and back again, coupled with the fact that this *only* helps with frequently read, infrequently written data, makes me doubt its overall usefulness in anything other than very very specific circumstances.

YMMV.

In terms of native Linux defrag support:
Ext4 will have (online) defragmentation built in. Currently XFS has defrag support. I think you have to take the disk offline first.

Just as a sidenote, XFS comes with a tool for performing on-line defragmentation by means of copying files: xfs_fsr

Hi guys I have some problem during the procedure. Actually when I try to launch the command /sbin/fsck.ext3 -nfv filesystem.img there are several errors. In particular: /sbin/fsck.ext2 -nfv filesystem.img
e2fsck 1.40.2 (12-Jul-2007)
Impossible to find the block ext2 trying backup blocks…
/sbin/fsck.ext2: Bad magic number in super-block during filesystem.img opening

Super block is not readable, or is not describing filesystem ext2
If the device is valid and contains a filesystem ext2
(e non swap, ufs o altro),the super block is corrupted, you could try to execute e2fsck with an alternativ super block:
e2fsck -b 8193

what am I supposed to do now? thanks for your help

Yes with this command:

/sbin/mkfs.ext3 filesystem.img

I never defrag my linux boxes, but I think about ‘defraging’ sometimes, since I’m using windows alot

It’s funny how so many people can take very simple comparisons and completely screw them up to make their case. Anyone looking at the graphs for comparisons will quickly see differences, bu if you look at the numbers, the scales are different, and maybe the numbers aren’t quite so different.

Can we see the graphs with identical scales for both situations?

Thanks

IMO there isn’t much that this blog post can make conclusion of.

And that’s not only because I feel that the difference in performance is so unsubstantial.

The very fact that the test is conducted on a loop device makes all the numbers invalid. There is high likelyhood that the .img file itself is defragmented on the host fs. Before and after moving data, the performance of the host fs may in its way affect the data collected for this “test”.

For example, there’s chance that before defrag, the files in the loop device somehow were written across defragmentation points of the img file itself and luckily after defragmentation the files ended up on a continuous part of the img on the host device. This definitely affects throughput measured.

If you ask me, Jon had a point. The recommendations were given pretty baselessly, and needless to say, without any substantial evidence or reasoning to back up the claim that those were the optimal numbers. optimizationkit tried to defend by throwing more numbers around but again, they were to be marked “citation needed”.

Possibly if anyone has free time, do some tests on fs on real disks, with accountability for caches, begin/end of disk, max/min, types/sizes of files, relation to everyday usage of disk etc. Now that will more likely help us server administrators. =)

So far I have not noticed any difference between speeds of my own fragged/defragged every-day filesystems (I used the copy out all, verify, delete all, copy in method), but hey, that’s just me and only yours truely.

“This seems like a rediculous idea”

“I have never seen the levels of fragmentation mentioned in this article in real world experience using ext2/3.”

et all

Publication of this article was a _bad_ idea, even worse idea was to translate it into English.

Why? Guess what - I _really_ _don’t_ _care_ about your opinion.

I wrote this tool for myself - like an other OK tools. I did not want to cause fscking holly wars. Since I do not want to cause further flame wars, I will not publish any future OK daemon/tools versions or articles about it.

Now I’m no Linux guru (I do know some however) but it does seem that this article is bent on fabricating a problem with Linux that does not cause problems on the level of a Windows system. The deciding factor should be performance not the fact that fragmentation exists.

If you are only interested in the fragmentation of a specifically small set of files on an ext3 system you can use the filefrag command as root to find out how many extents a file is taking up.

if you’re using XFS - you can do all that ONLINE:
-xfs_fsr -v /mountpoint
((defrags the filesystem while mounted))
-echo “frag\n” | xfs_db -r /dev/sda
((displays fragmentation ONLINE))

you can clearly see by that, that not every fragmentation is bad:
If a huge file has 2 extents instead of 1, no big deal.
If there are a total of 200.000 files in /, what’s the
problem - if 10000 of them are non-ideal ?
etc. etc.

I don’t think fragmentation plays any significant role nowadays, where you have megabytes of disk-cache and an efficient block-buffering in (gigabytes of) RAM.

It’s only a myth if someone knows nothing about disk storage.
The real advantage Linux, or any U**X derivative, has is that the defrag process is inherentaly faster than Windows. While Windows attempts to move around a bunch of files and place the contents in the correct order, any U**X operating system needs only to move a set of the files to another drive, then move back. You don’t need to copy the entire drive at once, only large chunks…say the size of your biggest file or 10% of drive space.
Even with a 300GB drive, this can be accomplished in a matter of hours, while a Windows system would probably take a couple of days.

the / partition on my laptop is 21 G, contains 245089 files, is 89.61% used (in terms of blocks), and has been heavily used for many months (since last reformatted), with many files changed and thrown around (like by a dist-upgrade). The fragmentation in term of non-contiguous inodes is 4.4%.

I am not convinced that I need to worry about fragmentation on ext3.

Very nice article.

I just did not understand what the contiguous inodes mean.

Keep up the good work.

Thank you very much for the information.

Refusal of the ext filesystem developers to accept that defragmentation is needed reminds me of the refusal of Apple to bundle in a mouse with more than one button.

It says above:

> Some systems have a functionality to remove files located in /tmp
> during bootup. The best thing would be to turn it off for the time of
> defragmentation.

Why not make the defrag script use /var/tmp instead?

/var/tmp, according to the Linux Filesystem Hierarchy Standard (FHS), is preserved between boots.

This article is idiotic, sorry to say so. In 12 years of UNIX + Linux administration I have never ever seen a system suffering from fragmentation. This entire “Linux needs a defrag tool as well” nonsense is just that: nonsense.

I could be COMPLETELY wrong here, but my understanding of the difference between linux and Windows filesystems here (ext2/3 vs FAT/NTFS) is that linux writes an entire file in a contiguous block whenever possible whereas Windows will write a little piece of a file in scattered blocks all over the hard drive until it’s finished writing the file. This means that, by default, linux is FAR less prone to file fragmentation than Windows. Windows fragments by default. Linux does not.

So fragmentation on linux would only become a problem if an insanely huge number of tiny files were written and deleted on a regular basis over a very long period of time. (To the point where all the contiguous blocks have been used and now the filesystem has to write to blocks freed up by deleted files.) At THAT point a defrag would be beneficial to the system … but not until the entire disk had essentially been filled.

With terabyte drives now on the market for a couple hundred bucks … how often is that going to happen?

That is MY understanding and I could be wrong. So take it with a grain of salt.

Why does this issue keep coming up despite the fact that numerous sources and numerous debated going back more than a decade show that degramentation is not a problem? First, start with understanding what “non-contiguous inode” means in ext3. Then understand how that number will rarely, if ever exceed 10%. Then look back at the hundreds of debates on this issue and see that each time it has been proven defragmentation in Linux is not a problem. This article is irresponsible and ignortant, you should remove it and offer a retraction.

B.S.