Pipe Viewer

Posted by & filed under BASH, Programming, Server Admin.

pv – Pipe Viewer – is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.


Updating OpenWRT packages on a USB stick

Posted by & filed under Uncategorized.

With opkg.conf configured with a usb destination I can install packages with:

opkg -dest usb install <<package>>

But sometimes, because files are not where the system expects the packages will not run once installed. A simple way to fix this is to run the following command after installing something to USB:

echo "Making the etc files, libs and bins happy..."

ln -s /mnt/usb/etc/* /etc/
ln -s /mnt/usb/usr/lib/* /usr/lib/
ln -s /mnt/usb/usr/bin/* /usr/bin/
ln -s /mnt/usb/usr/sbin/* /usr/sbin/
ln -s /mnt/usb/usr/share/* /usr/share/

echo "Done... errors saying the file already exists are fine..."

I made it a quick script so I can just run it to sync everything up.

Data Recovery: Finding VHD Files

Posted by & filed under Data Recovery, Server Admin.

SITUATION: Server had what appears to be a raid controller failure complete with it blowing away the data on the drives. The only backup available is corrupt.

I began by mounting one of the drives on my local machine. Windows Disk Manager would not recognize the device as having a partition table. I then ran TestDisk. It was unable to recover a partition table. TestDisk also comes with a nice utility called PhotoRec. Photorec looks at the raw data on the device attempting to extract files. PhotoRec can bea bit of a pain when recovering a lot of files because it cannot recover filenames. Fortunately, this particular server was a virtual host, and the only files I cared about were the 10Gb+ VHD virtual disk files.

I initially just ran PhotoRec just to see if and or what it could even find. After running for several hours, I was happy to see it found thousands of files. Upon closer inspection of the files I found that they were actually files that were inside the actual VHD’s. This makes sense because of how PhotoRec searches for files. I needed to create a custom PhotoRec file signature extension to handle the VHD files. www.cgsecurity.org/wiki/Add_your_own_ext…

Using a hex editor I opened several VHD files from my archive and noted thay all started with “conectix”. I then created photorec.sig in the PhotoRec directory with the following inside:

vhd 0 "conectix"

I then re-ran PhotoRec, this time using the options to only select my custom signature extension. It then began recovering VHD files. It found around 6 header signatures, and after some time had dumped them all out. I was able to mount one using diskmgmt.msc on my Windows7 laptop. The other VHDs were all unable to mount due to corruption or other factors.

Back to TestDisk
I decided to try TestDisk again, this time though when it asked what kind of partition, I selected none, then had it scan. It found the NTFS partition! I was able to navigate into the drive’s structure and instruct TestDisk to copy the VHD I needed to my local machine…

…After some hours, the dump froze. I was monitoring the file copy progress and it was making good progress (over 100G recovered) but then abruptly restarted. Or appeared to do so. Before I started this process I notices TestDisk was indicating that there were multiple VHD files with the same name. I am guessing it completed, then moved on the the next, overwriting the existing recovered file since no sane file system allows two files of the same name in the same directory. Eventually it froze as mentioned.

Data Dump
This should of been one if the first things I did, but I did it now since I wanted to simply attempt to repair/rebuild the NTFS partition table which should then give me direct access to the files rather than trying to extract them thru TestDisk. Obviously this should be the first thing done, but as I was confident in the physical drive’s health, I bypassed this at first.

dd if=/dev/sda of=/home/disk1.img

I did the backup thru TestDisk but I believe it just used dd emulated thru cygwin anyway so it’s the same thing.

Rebuilding the NTFS Boot Sector On An NTFS Partition
I dumped the MBR and sadly it was all zeros thanks to the RAID controller nuking it.

Both NTFS boot sectors (the primary and the backup) are corrupted so we need to rebuild the NTFS boot sector.

TestDisk searches the MFT (Master File Table: $MFT) and its backup ($MFTMirr). It reads the MFT record size, it computes the cluster size, and it reads the size of the Index Allocation Entry in the root directory index. Using all these values, TestDisk can provide a new boot sector.

Finally it lets the user list the files before writing.

Update: That failed. The sector count is different now too. I suspect the data got hosed. I can easily recover with my dd command:

dd if=/home/disk1.img of=/dev/sda

The Solution
Loaded up a copy of my dd into TestDisk, Created a new log file and selected the drive. For partition type I selected “None”. Then select Analyse. It located the NTFS partition and the I was able to hit (f or p — not sure typing this from memory) to recover files. I attempted this exact method previously, but tried to restore the entire directory. This time I simply had it restore the individual VHD I was interested in. Set the destination path (make sure there is space!!) and a few hours later I had a VHD… working flawlessly.

11-17-2012 Note: This was mentioned by the CG Security authors here: www.cgsecurity.org/mw/index.php?title=In…

Flashing OpenWRT without a GUI

Posted by & filed under Uncategorized.

I have successfully completed my custom firmware build and have cross compiled it. I now have bin files ready for flashing. Since I did not opt to select the LuCi web interface that is packaged with OpenWRT, I deemed it wise to research my other options for reflashing the firmware, since once I flash my custom image, there will be no web interface available to reflash again thru.

Two methods — TFTP and JTAG.
TFTP is the way to go as there is a tiny TFTP server built right into the devices boot loader. This means that I can recover from most hosed flashes. The other option is to populate the empty JTAG pins on the board with a header and go at it that way, although this should only be needed in a worst case scenario.

Before flashing via TFTP, I need to make sure that the nvram variables are set correctly to allow the boot loader to pause searching for the TFTP server for long enough. This is done by:

nvram set boot_wait=on
nvram set boot_time=10
nvram commit && reboot

Source: wiki.openwrt.org/toh/linksys/wrt54g#inst…

Once the bootloader has been configured to wait fo rthe TFTP server, we can proceed witht he standard OpenWRT TFTP flashing instructions: wiki.openwrt.org/doc/howto/generic.flash…

NHP Theme Options Framework

Posted by & filed under Programming, Web Development.

The NHP Theme Options Framework has an exhaustive list of features, which include…
Simple, Easy to use

Simple Theme Options Framework for WordPress, Leaving you to concentrate on creating beutiful WP Themes.
Built In Field Types

Contains many built in field types for easy use (all basic field types like text, checkbox, select, upload are built in).
Built In Validation Methods

Contains many built in validation methods for clean data saving (email, url, numeric, escape js, no html, html).
Custom Error/Warning Handling

Using the Validation methods you can create custom errorand warning messages for each field type.
Multiple Option Sections

Have as many, or as little sections as you need.
Custom Tab Sections

Want to show additional info in your options panrl? Its there for you to use and abuse.
Extendable Field Classes

Easily create custom field classes for use in your theme options. View WIKI
Extendable Validation Classes

Want custom validation? Just use the extendable Validation Class methods. View WIKI
Fully Customisable

Many Hook points to customise the Framework to your requirements. View WIKI


gnokii — phone interface

Posted by & filed under Server Admin.

gnokii provides tools and a user space driver for use with mobile phones under various operating systems (most testing is done under Linux but also Solaris, *BSD family and MS Windows families are known to work). gnokii allows you to communicate with the phone over the serial cable connection, usb connection (support depends mostly on the operating system level support), infrared connection and bluetooth connection.

You can send SMS, receive them and save them in the phone, read and write to the phonebook, initiate and answer calls and more. Pretty cool and just what I need for a a GSM alerting project at work.


Kerbal Space Program

Posted by & filed under Gaming.

What is better than sitting in my backyard watching rockets blast off from KSC? Building my own of course. Ok maybe not AS cool as the real thing, but hey its interesting nonetheless.