Yikes. Rather than deciphering it symbol-by-symbol, let’s experience the idea (hear the song then read the sheet music, dig?).
Let’s start with a plain-English metaphor:
What does the Fourier Transform do? Given a smoothie, it finds the recipe.
How? Run the smoothie through filters to extract each ingredient.
Why? Recipes are easier to analyze, compare, and modify than the smoothie itself.
How do we get the smoothie back? Blend the ingredients.
Next, we’ll refine the analogy into “math-English”:
The Fourier Transform extracts each “cycle ingredient” from a time-based signal (the cycle strength, delay & speed), resulting in a final “cycle recipe”
UPDATE 11/17/15: Another nice command to auto purge old kernels is: sudo apt-get autoremove
Also, removing old kernels is easy with sudo dpkg -r linux-image-3.2.0-83-generic
Recently I wanted to install a new package on a Ubuntu server. Typically this is as simple as issuing a
But this time around, I got a interesting error:
I started poking around and found that the /boot partition is full:
Ok, that is starting to make a bit more sense now… so we need to purge old kernel packages to free up space on the /boot partition. The first step is to identify the kernel version we are currently on so we do not delete that. Secondly, it was recommended to me that you keep the oldest kernel as it was the one the system was installed with. We can see the kernel version with:
So we are on 3.0.0-25-server and need to make sure not to delete that. A handy command to get a list of all the kernels you are not using is:
I attempted to remove old kernels the “nice” way — by letting apt handle the removal:
But it failed. Following the above instructions to run sudo apt-get -f install, it failed saying that there was not enough disk space on /boot (duh!). So much for being nice.
Inside the /boot partition there are three “types” of files — abi-kernel-version, config-kernel-version, initrd.img-kernel-version, System.map-kernel-version, vmcoreinfo-kernel-version, and vmlinuz-kernel-version. There will be a file for each of these for each version of the kernel you have installed. For example: vmlinuz-3.0.0-28-server. Leaving the earliest kernel version and the version I am running (reported by uname -a), I moved the other kernel files off to another location where there was ample space. It looked something like this:
As you can see this is moving the files for kernel version 3.0.0-16 off the /boot partition.
If the boot partition just needed a bit of space freed up, you can now likely use apt-get to purge the other kernels “cleanly”. What I mean by that is apt-get also removes the kernel files from /lib/modules. You could do this by hand as well. I am not sure if it does anything beyond cleaning up /boot and /lib/modules, but I do not believe it does.
Ran into a issue where I wanted to do a mysqldump of a database in order to transfer it to a new server.
This failed saying that three of the tables were corrupted. I ran the mysqlcheck utility to see if it could be repaired:
It outputted the following errors among checking the rest of the tables successfully:
I was strongly suspecting that these tables were old remnants of a old software version or something along those lines.
I tried to re-run the command, telling it to repair the tables. It kicked out the same errors about the tables not being found.
Went ahead and issues a drop command for the three tables, as I suspect this is unused and leftover from a previous upgrade.
After each of the drop statements, MySQL reported a error that it was unable to delete as it could not find the table. I re-ran mysqlcheck and found that it actually did remove them, and it reported no issues. I was then able to go ahead and re-run my mysqldump command and completed extracting the database.
StatsD is a simple NodeJS daemon (and by “simple” I really mean simple — NodeJS makes event-based systems like this ridiculously easy to write) that listens for messages on a UDP port. (See Flickr’s “Counting & Timing” for a previous description and implementation of this idea, and check out the open-sourced code on github to see our version.) It parses the messages, extracts metrics data, and periodically flushes the data to graphite (which I previously wrote about here).
The Etsy blog post: codeascraft.etsy.com/2011/02/15/measure-…
Popcorn Maker makes it easy to enhance, remix and share web video. Use your web browser to combine video and audio with content from the rest of the web — from text, links and maps to pictures and live feeds.
The X-Ray Goggles make it easy to see and mess around with the building blocks that make up the web. Activate the Goggles to inspect the code behind any web page, from the New York Times to your own blog. Then remix elements with a single click, swapping in your own text, images and more.
It’s like editing a site’s markup with the developer tools, but it works better and is easier to use. I’m planning on trying this out for prototyping new changes.
Recently I needed to recursively count the number of lines of code in each of the specific file types. In this instance I wanted to count the number of lines of code in my PHP files. The below command worked flawlessly. In addition to breaking down the line count in each directory, it gives a overall total at the end as well.
I took a look at Internet Traffic report while the storm was hitting the upper northeast, and the results weren’t surprising. North American packet loss went up ~5%, while the traffic index went down. Again, not surprising, but cool to see nonetheless.
Sometimes a situation pops up where I need to retrieve the contents of a DNS zone but do not have access to the DNS servers. This is typically a situation where a client’s IT person is no longer available and before moving the name servers, we need to recreate the DNS zone on our name servers. If the current host of the DNS zone cannot release the records, we have a few options.
1. Try a zone transfer. I previously wrote about this. This is highly unlikely to work, but if the DNS server is poorly configured, it’s a possibility. This works rarely but if it does work will be the most accurate.
2. Brute force the zone. It sounds bad, but the reality of it is that most sysadmins don’t log or throttle DNS requests, and therefore with a decent enough dictionary of words it is possible to enumerate a large majority of the dns zone. I have mirrored the zip file containing bfdomain and the dictionaries here. (original source)
UPDATE: One other thing that I noticed later on is the fact that this is seemingly only capturing A records, so things like the MX record would not be tried. The python script could be modified easily to add this functionality. Also, the nmap version of this may already do this.
More dictionaries and wordlists:
Sometimes it is useful to know if a page has been included by another page or not, and then take action based on the result. A quick and handy way to accomplish this is to use a tenary operator to do something like:
Which will return true if it is included, and false if it is running as itself. Now, we are able to do something like:
Recently I had a issue where I needed to clean up some disk utilization on a linux server. In order to find a list of larger files, I used the following find command from the root directory I want to recurse through:
As you can see, the -size switch is setting the minimum size to find at 50Mb. Another issue that I ran into was deleting a large amount of files at once using something like:
“Argument list too large” was the error. Apparently the list of files is too large for rm to handle. I found that there are a variety of methods to solve this, from using loops to split up the files into smaller groups, to recompiling the kernel. One of the simplest is to use the find command to delete the files it finds:
The list of files to get deleted can also be tuned so it does not delete all the files in the path: