Adventures with DD

In my humble opinion when it comes to CLI tools `dd` ranks pretty high in my toolkit. Right there with `nc` and a few other ‘ole favorites.

Today I wanted to show a quick hack on how to determine information about a block device using dd. This goes along the same path as  previous post about using `dd` to get LVM configuration off devices.

Well enough talk lets get down to the CLI:


# Thanks to file(1) magic(5)
# Example 1 - CentOS 6 LVM2
>dd if=/dev/sda3 of=/tmp/info bs=512 count=1000
1000+0 records in
1000+0 records out
512000 bytes (512 kB) copied, 0.0190352 s, 26.9 MB/s
You have new mail in /var/spool/mail/root
> file /tmp/info
/tmp/info: LVM2 (Linux Logical Volume Manager) , UUID: M7pUi7daFBsEQ95UNJUd5pN604qSa0Z

#Example 2 - CentOS 7 LVM2
> dd if=/dev/sda2 of=/tmp/info bs=4k count=100
100+0 records in
100+0 records out
409600 bytes (410 kB) copied, 0.00487443 s, 84.0 MB/s
> file /tmp/info
/tmp/info: LVM2 PV (Linux Logical Volume Manager), UUID: fYcfT0-3Oyk-J87h-A6SV-rYCc-q1of-zhx7Fd, size: 119508303872

#Example 3 - CentOS 7 XFS
> dd if=/dev/sda1 of=/tmp/info bs=4k count=100
100+0 records in
100+0 records out
409600 bytes (410 kB) copied, 0.00990939 s, 41.3 MB/s
> file /tmp/info
/tmp/info: SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs)

#Example 4 - CentOS 7 BTRFS
> dd if=/dev/vdb2 of=/tmp/info bs=4k count=100
100+0 records in
100+0 records out
409600 bytes (410 kB) copied, 0.00324626 s, 126 MB/s
> file /tmp/info
/tmp/info: BTRFS Filesystem (label "----_fs", sectorsize 4096, nodesize 16384, leafsize 16384)

#Example 5 - CentOS 6 - EXT4
> dd if=/dev/sda1 of=/tmp/info bs=4k count=100
100+0 records in
100+0 records out
409600 bytes (410 kB) copied, 0.0132642 s, 30.9 MB/s
> file /tmp/info
/tmp/info: Linux rev 1.0 ext4 filesystem data (needs journal recovery) (extents) (huge files)

I always forget to test this when I have more “esoteric” filesystems in my home lab. If anyone can and post a comment with the results from other filesystems’s or distros I would appreciate it!

Yum has error on primary.xml.gz or similar when behind squid. (clearing and reloading a URL in Squid)

Verify the entry is getting hit in your cache by watching your logs

TCP_MEM_HIT/200 1597 GET – HIER_NONE/- application/x-gzip (or similar)

To reload this entry run

squidclient -r


To clear this entry

squidclient -m PURGE

You can also whitelist this in  several ways if you do not want this to be cached or do not want it cached for long (ie 24 hours) but I will save that for another post.

Caching packages and yum related information in squid can be helpful if you have a large number of machines behind a company/personal proxy and have limited (either in speed or monthly amounts) bandwidth at your location. While running a local mirror is preferred for many reasons this could fill that role till such time as you feel you need the complexity of local mirror.

Linux Brain Teaser – Fall 2014

This was a fun problem exchanged with a coworker. The problem is simple run this :

ssh $USER@$HOST history 

and have it return the remote history. Lots of ways to solve this one. If you want to confirm you solved it without giving it away just email/PM me. Otherwise show your work in the comments section. Since I have such ridiculously low traffic I will post the answer in 2015!

Quick Hack to use DD to get LVM configs

This is more of a exercise than a real world use example. I have used something similar in a disater recovery situation so it does have some merit ..but good planning can prevent needing hacks like this.


for i in $(pvdisplay | awk '/PV Name/ {print $3}'); do
dd if=$i of=/tmp/lvm_config_for_$(echo $i |sed 's/\//_/g') bs=$(stat -f -c %s $i) count=10;

KVM Local Network VMs TCP Congestion Algorithm on CentOS 6.5

First off I have no idea what made me want to test this…likely because it was easy to script and let run a few times while I went off and did other things.

What I am doing is testing network bandwidth between two KVM VMs on a single system using a bridge, the virtio network driver, and CentOS 6.5 as the Client and Host OS.

A little info on how to reproduce this. To find what your system supports you run

ls /lib/modules/`uname -r`/kernel/net/ipv4/

and look for tcp_* to see what to populate in the script….

Then take 30 seconds to write a little for loop(s) with these ….


#Clear Log....
echo "" > $LOG

for i in cubic bic highspeed htcp hybla illinois lp scalable vegas veno westwood yeah; do     
    for s in {1..10}; do
        echo "Testing with : $i" >> $LOG
        echo "Run $s" >> $LOG
        iperf -Z $i -c sec01 >> $LOG

Results are all in Mbps and are based off 20 total runs almost back to back. These are VERY inconclusive but at least warrants more testing.

UPDATE – I am doing more testing and found that the longer run times (120 seconds vs 10) is showing MUCH more consistent numbers.

It appears that vegas and Yeah dont like the way these local bridged networks handle cwnd’s and ssthresh among other things.

It also might be worth further testing to see how/if these affect RTT among other network variables.


Fixes for named (bind) errors

I ran into a few errors during load testing on my bind server the other day and found ways to quickly fix them. Your mileage may vary but for me these helped. Note these are just the configuration names with no settings ! I did this so you can evaluate whats best for your system!

I will make this my default post for these errors and the config changes I did to fix them:

ISSUE: named[1698]: dispatch 0x7fb0180cd990: open_socket( -> permission denied: continuing
FIX: raise the range of ports in `use-v4-udp-ports` make sure that range does not overlap with existing UDP services.

ISSUE: named[932]: clients-per-query increased to 20 (or other number)
FIX: Raise `max-clients-per-query` and `clients-per-query` a “0” will set to unlimited. Be careful of this due to resource exhaustion!

ISSUE:DNS queries timeout/lost under load (dnsperf or other tool can show this).
FIX: (OS) set “net.core.rmem_max” and “net.core.rmem_default”

2014 – January CLI command of the month

This month we are checking out fallocate. This can replace dd as a superfast file creation tool. Here is a example:

fallocate -l 1G /tmp/test.2

For a quick benchmark (time output abbreviated for brevity):

/usr/bin/time -vvv fallocate -l 1G /tmp/test.2
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
Maximum resident set size (kbytes): 2192
Voluntary context switches: 1
Involuntary context switches: 1

/usr/bin/time -vvv dd if=/dev/zero of=/tmp/test.2 bs=1G count=1
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:26.15
Maximum resident set size (kbytes): 4197616
Voluntary context switches: 5023
Involuntary context switches: 864

I wanted to test fallocate on some non-standard filesystem like Lustre and finally got the chance and, like I had feared, it did not like it

fallocate -l 1G 1Gtest
fallocate: 1Gtest: fallocate failed: Operation not supported

but it did like the `dd if=/dev/zero of=/tmp/test.2 bs=1G count=0 seek=1` command given below in the comments section.

I have some ideas on why this does not work but I will save that for later….

INFO on Lustre:
$>lfs getstripe stripe8/
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1

If anyone has a distributed filesystem like Gluster installed and can do some fallocate testing I would be interested in seeing if its broken there as well (Iwager it is). If not I might just get it going and play around with it.


cron and its limited enviroment….

If you ever see this from a crron script :

$FILE_RUN_FROM_CRON: line $LINE: $COMMAND: command not found
Error: (CLI:043) Invalid shell command syntax.



Then you likely:

  • Forgot to setup a proper PATH in your script
  • You declared a variable but didnt use it or the file isnt in the setup PATH (ie set VAR1=/usr/local/bin/crazybinary but then just used  crazybinary in the script)
  • messed up something in a /etc/profile.d/ file
  • Likely more but ….

I run across this from time to time due to typos I make and thought the internet at large might benefit. Don’t forget cron has a limited environment by default ! Lots of ways on the net to fix this but this error was a little more hidden so I thought I would help it get some google love! For those that wanted to know about how different the cron environment is here is a (near) default RHEL6 environment:


Quite Spartan !

Quick CLI Hack to set writecache on 3w-9xxx 9650SE on the cli

First off this hack requires you have the tw_cli tool installed. If so just modify this to suite your needs and have fun. I ran into the need for this after I noticed that the following log messages :

Jan 13 11:09:44 pbnj kernel: 3w-9xxx: scsi0: AEN: INFO (0x04:0x0055): Battery charging started:.
Jan 13 11:09:45 pbnj kernel: 3w-9xxx: scsi0: AEN: INFO (0x04:0x0056): Battery charging completed:.

would cause the write cache to get turned off. Likely a GUI way or flag to trip but I like a little CLI challenge on my lunch break ;)


Not using a battery with write cache on is dangerous…sometimes silly..and sometimes down right deadly to your data. Use with caution! I am not responsible if you use this and it harms your system in anyway!

UPDATE : A more powerful version of this code now lives at My Gitlab Page for this project . I am going to be updating there moving forward.

SYSLOG="/usr/bin/logger -t LSI"
CONTROLLER=$(tw_cli show | awk '/c./ {print $1}')
for i in $($TWCLI /$CONTROLLER show unitstatus | egrep -o 'u[0-9]'); do
    case $($TWCLI /$CONTROLLER/$i show wrcache | cut -d= -f2 | sed 's/^[ \t]*//;/^$/d') in
	$SYSLOG "WriteCache is on for /c2/$i"
	$SYSLOG "WriteCache is off for /c2/$i ..turning on"
	$TWCLI /$CONTROLLER/$i set wrcache=on quiet
	$SYSLOG "WriteCache or script is fubar please fix"

Quick BASH check for slow DNS queries

This little script (sorry for the formatting..I really need to clean up these scripts in worldpress) can start the debug process for slow DNS queries. It could also be used, witn a little work, to populate a graph if you see frequent issues.

DOMAIN=Domain your testing with
for i in {1..100} ; do 
   dig @$NS $DOMAIN | awk -F: '/Query/ {print $2}';
   sleep 1 ; 

A server under load will be all over the place. I recently helped someone with this issue where a nameserver was going into swap and was causing VERY slow (900+ ms) NS lookups. I start with a domain the server is auth for as that should be fastest and have the lowest network load but if you dont know any be prepared for a slow response or two as the server populates its cache.
Here is what I saw from a test on a non-auth domain for a server that is local:

151 msec
0 msec
0 msec
1 msec

and for a domain that the server is auth for…

0 msec
0 msec

and against a remote DNS server at google…

101 msec
26 msec
27 msec
25 msec
24 msec

I have begun building on this to help troubleshoot further as to where the latency exists. Just a quick 5 min hack I did that helped someone that might help someone else.