2014 – January CLI command of the month
This month we are checking out fallocate. This can replace dd as a superfast file creation tool. Here is a example:
fallocate -l 1G /tmp/test.2
For a quick benchmark (time output abbreviated for brevity):
/usr/bin/time -vvv fallocate -l 1G /tmp/test.2
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
Maximum resident set size (kbytes): 2192
Voluntary context switches: 1
Involuntary context switches: 1
/usr/bin/time -vvv dd if=/dev/zero of=/tmp/test.2 bs=1G count=1
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:26.15
Maximum resident set size (kbytes): 4197616
Voluntary context switches: 5023
Involuntary context switches: 864
(UPDATE)
I wanted to test fallocate on some non-standard filesystem like Lustre and finally got the chance and, like I had feared, it did not like it
fallocate -l 1G 1Gtest
fallocate: 1Gtest: fallocate failed: Operation not supported
but it did like the `dd if=/dev/zero of=/tmp/test.2 bs=1G count=0 seek=1` command given below in the comments section.
I have some ideas on why this does not work but I will save that for later….
INFO on Lustre:
$>lfs getstripe stripe8/
stripe8/
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
If anyone has a distributed filesystem like Gluster installed and can do some fallocate testing I would be interested in seeing if its broken there as well (Iwager it is). If not I might just get it going and play around with it.
I personally just use the “sync” flag on dd:
/usr/bin/time -vvv dd if=/dev/zero of=/tmp/test.2 bs=1G count=1 seek=1G
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.70
Maximum resident set size (kbytes): 1049348
Voluntary context switches: 2
Involuntary context switches: 10
It may not be quite the same, but it works great for me.
Sorry, that should be count=0 seek=1, not count=1 seek=1G, unless you want a 1.1 exabyte file instead of a 1 gigabyte file.
/usr/bin/time -vvv dd if=/dev/zero of=/tmp/test.2 bs=1G count=0 seek=1
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.19
Maximum resident set size (kbytes): 752
Voluntary context switches: 2
Involuntary context switches: 37
Great example, thanks for sharing!
I did want to add that there is a slight different between these two commands and what they are doing (which to most folks is likely irrelevant).
fallocate -l 1G /tmp/test.3
stat /tmp/test.3
Size: 1073741824 Blocks: 2097160 IO Block: 4096 regular file
dd if=/dev/zero of=/tmp/test.2 bs=1G count=0 seek=1
stat /tmp/test.2
Size: 1073741824 Blocks: 0 IO Block: 4096 regular file
The blocks are pre-allocated with fallocate.
This makes me think of some tests that I can do to dig deeper into this….thanks again !
(UPDATE)
I wanted to test fallocate on Lustre and finally got the chance and, like I had feared, it did not like it
fallocate -l 1G 1Gtest
fallocate: 1Gtest: fallocate failed: Operation not supported
but it did like the `dd if=/dev/zero of=/tmp/test.2 bs=1G count=0 seek=1` command. I can go into some deep FS voodoo here on why fallocate does not work on Lustre but I dont want to bore most readers…
INFO on Lustre:
$>lfs getstripe stripe8/
stripe8/
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
Normally I just like to use dd command to create big files or sparse file…