Xymon script testing/debugging

Posted by & filed under Server Admin.

This is a follow-up to a older post I made about Hobbit/Xymon and testing alerting (natesbox.com/blog/?p=378)

I have needed to write a few Xymon scripts and in order to test them outside of Xymon, but with the Xymon environment variables which most scripts depend on. In order to do this we use the xymoncmd command like so:

This works for a client script, but I wanted the server env vars, so I added one additional option:

Also one note on older compatibility: As of the (slightly outdated) version I am currently running, xymoncmd is just a alias of bbcmd, So this should apply to older hobbit stuff too.

PHP mail() from sender incorrect

Posted by & filed under PHP, Programming, Server Admin.

Today I ran into a issue where a client’s server was sending mail via php’s mail function. Everything was working great, except the from field was showing as “user@the.server.name.net“.

This was perplexing partly due to the fact that examination of the SMTP headers shows the From: field being correctly populated with the seding address.

After some further digging in the headers, I found that the envelope-from was being set as user@the.server.name.net which is where my issue was coming from.

Header excerpt:

Received: from username by web1.server.net with local (Exim 4.69)
(envelope-from )
id 1Qbeva-00060p-9K
for user@testaddress.com; Tue, 28 Jun 2011 16:30:34 -0400
To: "test test"
Subject: testing 3
From: "E-mail Testing"

It seemed that the smoking gun was somewhere with PHP. I tried setting the php.ini value sendmail_from parameter, but found that this is a windows-only parameter and therefore does not apply to my issue.

Finally I found that the mail function did not have the 5th parameter defined. Once I added the 5th parameter, it all started working correctly:

return mail($to, $subject, $message, $from);

return mail($to, $subject, $message, $from, '-f user@testaddress.com');

Problem solved!

git setup and init

Posted by & filed under Programming, Server Admin.

new git project initialization

In your project’s directory where projectname is the name of your project:

git init
git add .
git commit -am ‘Initial codebase commit’
git remote add origin git@monitor.tnsc.net:projectname
git push origin master

create new git branch
Get a initial copy of the git repo
git clone…

Switch to the new branch
git checkout newbranchname

Make changes and commit
git commit -am “Commit…”

Push branch to the remote
git push origin newgranchname

Managing git user access and projects thru gitosis
on monolith edit /home/gitosis/gitosis.conf

Chmod a+rx the repo inside /home/git/repositories so gitweb can view

Git CheatSheet help.github.com/git-cheat-sheets/

Installing, Configuring and using Git

Posted by & filed under Programming, Server Admin.

I’ve been brushing up in git as of late.

Basic git use: http://progit.org/book/ch2-0.html
git refrence cheetsheet: http://gitref.org/

Server Install/Config: http://progit.org/book/ch4-.html

Branching operations: http://progit.org/book/ch3-0.html

git videos: http://chacon.blip.tv/posts?view=archive&nsfw=dc

Controlling git user access: gitosis (or a branch)
gitosis config: http://progit.org/book/ch4-7.html
gitosis config: http://www.ivankuznetsov.com/2010/05/setting-up-your-own-git-server-on-ubuntu.html

accessing git over ssh with windows and public key auth: http://serverfault.com/questions/194567/how-to-i-tell-git-for-windows-where-to-find-my-private-rsa-key

git web ui’s:
Indefero (looks promising): http://www.indefero.net/open-source/
Viewgit (standard kit)
–> Install Guide (down the page): https://help.ubuntu.com/community/Git
–>Website: viewgit.sourceforge.net/

Datacenter Management Tools

Posted by & filed under Networking, Server Admin.

I have been researching datacenter management tools as of late that will allow us to track and manage our racks/ipspace/network connections etc.

  • NetDotnetdot.uoregon.edu/trac/ – Netdot is an open source tool designed to help network administrators collect, organize and maintain network documentation.
  • OpenNetAdminopennetadmin.com/ – OpenNetAdmin provides a database managed inventory of your IP network. Each host can be tracked via a centralized AJAX enabled web interface that can help reduce tracking errors. A full CLI interface is available as well to use for scripting and bulk work. We hope to provide a useful Network Management application for managing your IP subnets and hosts. Stop using spreadsheets to manage your network! Start doing proper IP address management!
  • Rack Monkeyflux.org.uk/projects/rackmonkey/ – RackMonkey is a web-based tool for managing racks of equipment such as web servers, video encoders, routers and storage devices. Using a simple interface you can keep track of what’s where, which OS it runs, when it was purchased, who it belongs and what it’s used for. No more searching for spreadsheets or scribbled notes when you need to find a server: RackMonkey can quickly find any device and draw a rack diagram of its location. RackMonkey is free and open source (licensed under the GPL).

RFC 1918

Posted by & filed under DNS, Networking, Server Admin.

I recently setup a BIND dns server and after monitoring the logs for some time found lines like this:

RFC 1918 response from Internet for

This means one of two things… either the bind server itself is querying the internet for local subnets and leaking info the the internet, or a DNS client queried them. Since the logs indicate the source IP, I know it is not the BIND server.

To remedy this, I enabled RFC1918 zones on the server to catch the queries before the leak to the internet. It ended up looking something like this:

zone "10.IN-ADDR.ARPA" {
type master;
file "empty";

zone "16.172.IN-ADDR.ARPA" {
type master;
file "empty";


zone "31.172.IN-ADDR.ARPA" {
type master;
file "empty";

zone "168.192.IN-ADDR.ARPA" {
type master;
file "empty";

@ 10800 IN SOA . . (
1 3600 1200 604800 10800 )
@ 10800 IN NS .

nginx Load Balancer

Posted by & filed under Server Admin.

nginx [engine x] is a HTTP and reverse proxy server, as well as a mail proxy server written by Igor Sysoev. It has been running for more than five years on many heavily loaded Russian sites including Rambler (RamblerMedia.com). According to Netcraft nginx served or proxied 4.70% busiest sites in April 2010. Here are some of success stories: FastMail.FM, WordPress.com.


Configure Xymon Client (BBWin)

Posted by & filed under Server Admin.

I periodically need to configure BBWin clients for our Xymon monitoring system.

Config Steps:
–>Install BBWin Client
–>Copy bbwin/etc/bbwin.cfg from an existing install
–>Set HKLM/Software/BBWin/hostname
–>Start Service

Kick off a test alert with Xymon

Posted by & filed under Server Admin.

If you want to test how your alert configuration handles a specific host, you can run xymond_alert in test mode – you give it a hostname and servicename as input, and it will go through the configuration and tell you which rules match and who gets an alert.


osiris:~ $ cd server/
osiris:~/server $ ./bin/xymoncmd xymond_alert –test osiris.hswn.dk cpu
Matching host:service:page ‘osiris.hswn.dk:cpu:’ against rule line 109:Matched
*** Match with ‘HOST=*’ ***
Matching host:service:page ‘osiris.hswn.dk:cpu:’ against rule line 110:Matched
*** Match with ‘MAIL henrik@sample.com REPEAT=2 RECOVERED COLOR=red’ ***
Mail alert with command ‘mail -s “XYmon [12345] osiris.hswn.dk:cpu is RED” henrik@sample.com

BigBrother is slightly different:
$ bbcmd hobbitd_alert –test ns2.whatever.net conn

Both can optionally be appended with a duration to meet any requirements:

dev@monitor:/usr/lib/hobbit/server/bin$ bbcmd hobbitd_alert --test ns2.whatever.net conn --duration=501
2011-07-08 10:37:05 Using default environment file /usr/lib/hobbit/client/etc/hobbitserver.cfg
00013395 2011-07-08 10:37:05 send_alert ns2.whatever.net:conn state Paging
00013395 2011-07-08 10:37:05 Matching host:service:page 'ns2.whatever.net:conn:' against rule line 26
00013395 2011-07-08 10:37:05 Failed 'HOST=$INTIDEAS' (hostname not in include list)
00013395 2011-07-08 10:37:05 Matching host:service:page 'ns2.whatever.net:conn:' against rule line 39
00013395 2011-07-08 10:37:05 *** Match with 'HOST=*' ***

The hobbitd_alert and xymond_alert (depending on if it is xymon or bb) command calls the module:

Update 07/25/18: The Xymon syntax is a bit off, it should be:

See www.xymon.com/xymon/help/manpages/man8/x… for additional details on the parameters.

Activating Server 2008 without MAK using KMS

Posted by & filed under Server Admin.

slmgr.vbs (Vista and Server 2008)Software Licensing Management Tool (C:\windows\system32\slmgr.vbs)
slmgr [MachineName [Username Password]] [Option]
Key -dli Display the current license information with activation
status and partial product key.

-dlv Verbose, similar to -dli but with more information.

-dti Display Installation ID for offline activation

-ipk Key Enter a new product key supplied as xxxxx-xxxxx-xxxxx-xxxxx-xxxxx

-xpr Show the expiry date of current license (if not permanently activated)

-upk Uninstall current installed product key and return license status back to trial state.

-ato Activate Windows license and product key against Microsoft’s server.

-atp Confirmation_ID Activate Windows with user-provided Confirmation ID

-skms activationservername
-skms port
-skms activationservername:port
Set the KMS server and the port used for KMS activation
(where supported by your Windows edition)

-rearm Reset the evaluation period/licensing status and activation state of the machine

-ckms Clear the name of KMS server used to default and port to default.

-cpky Clear product key from the registry (prevents disclosure attacks)

-ilc License_file Install license

-rilc Re-install system license files

machinename The machine to administer, by default the current local machine.

username An administrator equivalent user account for the computer.

password The password for the user account.
Running slmgr.vbs requires elevated administrator privileges.
C:\windows\system32\slmgr.vbs wkstn0064 administrator password1 -dli
slmgr.vbs -skms
slmgr.vbs -skms KMSServer:8090
“One resolution I have made, and try always to keep, is this: To rise above little things” – John Burroughs

Enable RDP Remotely via RPC/Remote Registry

Posted by & filed under Server Admin.

Two good methods for enabling rdp:

You enable remote desktop over the network via regedit if you have administrator rights to the remote machine:
1.Run Regedit
2.Select File –> Connect Network registry
3.Enter the name of the remote computer and select Check Name
4.At the bottom of the registry tree you will see 2 Hives appear Hkey_Local_Machine and
Hkey_Users (under the remote computer’s name)
5.Goto hklm\system\currentcontrolset\control\terminal server\FdenyTSConnections=1
6.Change the FdenyTSConnections to 0
7.Attempt to Re-Login

Or with psexec magic:

psexec \\remotecomputername netsh firewall set service remoteadmin enable
psexec \\remotecomputername netsh firewall set service remotedesktop enable

Debugging STOP Errors / Reading Crash Dumps

Posted by & filed under Server Admin.

1. Configure for a large dump model — A complete memory dump, which is written to the %SystemRoot%\Memory.dmp folder.

The paging file on the boot volume must be large enough to hold all of the physical RAM plus 1 MB.

The default type of memory dump is the small memory dump. To change or view the settings for the type of memory dump, follow these steps:
1. Click Start, and then click Control Panel.
2. In Control Panel, double-click System, and then click the Advanced tab.
3. Click Settings in the Startup and Recovery area.
4. View or change the type of memory dump under Write debugging information.

2. Download and install the debugging tools for Windows 32-bit: www.microsoft.com/whdc/devtools/ … llx86.mspx

3. Download and install, (Important: install to the folder C:\Symbols), the Appropriate Symbol package: www.microsoft.com/whdc/DevTools/ … olpkg.mspx

4. To make life easier, download and install DebugWiz: windowsbbs.com/debugwiz.zip
Install this to C:\Windows so that it can be found in your Path.

Now the next time your computer shows a BSOD:

For non-BSOD crashes, the program crash dump is probably located in C:\Documents and Settings\All Users\Application Data\Microsoft\Dr Watson\user.dmp

For BSOD events, “Blue Screens of Death”: the dump file will be found at c:\MEMORY.DMP ; or (for Mini-dumps) in C:\WINDOWS\Minidump\

* Find your copy of DebugWiz.exe and run it.
* Select a dump using the tool and ‘generate log’
* Open the file c:\debuglog.txt in notepad

You can paste the log back to the Forum.

Additional Reading:

See the 3-Part discussion by Ken Schaefer on 0x0A errors due to driver issues. The methods apply to any driver-related BSOD event. The discussion shows how to use the Windows debugger, as well as driver utilities such as Driver Verifier and the use of Special Pools.

www.adopenstatic.com/cs/blogs/ke … art-1.aspx

www.adopenstatic.com/cs/blogs/ke … art-2.aspx

www.adopenstatic.com/cs/blogs/ke … /1005.aspx