Pipe Viewer – A Unix Utility You Should Know About

A little more typing, but I find dd present on most systems already, so I tend to do this:

  tar ... | dd status=progress | ...

I've used pv longer than dd had this option for, but that's fair! I also don't use find options, for example, since find piped into the tool everyone already knows anyway - grep - is much easier

Sadly, dd will not give you an estimated time or allow you to limit the transfer rate, which are two features I use a lot in pv

dd conv=swab is a cool and useful option. swab stands for swap bytes, iirc. guess what it is used for, those who don't already know.

> guess what it is used for, those who don't already know.

Changing the endianness of the data?

Exactly!

That's just what I used it for, early in my career, in a software troubleshooting case.

I was a newbie dev, tasked with converting some data from another machine / OS format on tape to a Unix machine format.

Was fairly new to Unix too.

Looked through man pages, found dd, could read and copy the data from tape drive to disk using it. But the data, while in ASCII, and English letters, seemed garbled. Seemed like actual words, but not quite, if you know what I mean. Something seemed off a bit.

I puzzled over it for a while, then checked the dd man page again. I saw the conv=swab option and its meaning, which is to swap adjacent bytes. (Typical Unix cryptic syntax, and I say that as a long term Unix guy). I probably was new to endianess at that time. This was in a pre-Internet time, so I could not google to check what endianness meant.

I added that option to my dd command line, and what was my shock and pleasure to see all the words coming out as proper English words, now, on the screen!

Then I knew that the tape contained actual data, not garbage. I saved it to disk under some file name.

Job done.

One problem I've noticed with status=progress is that systems can sometimes have gigabytes of buffer space waiting to be filled, so the transfer spends most of its time in a "nearly done" state while (for instance) the SD card gets slowly filled at its real speed.

That's slowish, bottlenecking disk based IO. (yes you can improve it with dd options, if you are versed in the language...)