Posted by & filed under Server Admin, Virtualization, VMWare.

Ran into some issues with the ssl certs on the vCenter server when trying to run the Migration Assistant. Notes on the will follow, but first links to articles on the actual upgrade:

The issues I ran into with the migration assistant complained of the SSL certs not matching. Upon inspecting the certs I found all were issues for domain.lan except for one which was issued to domain.net. I followed the following articles to generate a new vCenter cert and install it:

  • Generate SSL cert using openssl: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2074942
  • Install and activate cert: https://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2061973

As the Appliance Installed reached Stage 2 of the install where it copies the data to the new VCSA, I received the following error (note the yellow warning in the background along with the details in the foreground):

To resolve this error, I followed the following articles:

  • Upgrading to VMware vCenter 6.0 fails with the error: Error attempting Backup PBM Please check Insvc upgrade logs for details (2127574): https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2127574
  • Resetting the VMware vCenter Server 5.x Inventory Service database (2042200): https://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2042200#3

Which essentially had me reset the inventory service’s database due to corruption. I had noticed the vSphere client slow in recent weeks, this could be a side effect.

  • Additional more generic docs for tshooting vCenter upgrades: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2106760

 

Posted by & filed under Active Directory, Server Admin, Virtualization, VMWare.

Attempting to join a freshly deployed VCSA server to a AD domain can be problematic if SMB1 is disabled. In my case it was 5.5 but I believe this issue persists in 6.x. SMB1 was disabled on the DC as it should be as it is broken and insecure. The problem lies in the fact that VCSA doesn’t support SMB2 and this causes the error. The VAMI (web interface) might report something like the following when attempting to join the domain:

Error: Enabling Active Directory failed.

Additionally, on the VCSA, /var/log/vmware/vpx/vpxd_cfg.log contains entries like the following:

2017-08-16 14:30:07 26987: ERROR: Enabling active directory failed: Joining to AD Domain:   domain.lan
With Computer DNS Name: vcenter-server.domain.lan


Error: ERROR_GEN_FAILURE [code 0x0000001f]
2017-08-16 14:30:07 26987: VC_CFG_RESULT=302

Of course DNS resolution of the VCSA’s hostname should be validated before continuing, but assuming everything else is in working order, the fix is to enable SMB2 on the VCSA.

Verify SMB2 is disabled (note the Smb2Enabled key is 0:

vc-01:~ # /opt/likewise/bin/lwregshell list_values '[HKEY_THIS_MACHINE\Services\lwio\Parameters\Drivers\rdr]'
   "EchoInterval"     REG_DWORD       0x0000012c (300)
   "EchoTimeout"      REG_DWORD       0x0000000a (10)
   "IdleTimeout"      REG_DWORD       0x0000000a (10)
   "MinCreditReserve" REG_DWORD       0x0000000a (10)
   "Path"             REG_SZ          "/opt/likewise/lib64/librdr.sys.so"
   "ResponseTimeout"  REG_DWORD       0x00000014 (20)
   "SigningEnabled"   REG_DWORD       0x00000001 (1)
   "SigningRequired"  REG_DWORD       0x00000000 (0)
   "Smb2Enabled"      REG_DWORD       0x00000000 (0)

Enable SMB2:

vc-01:~ # /opt/likewise/bin/lwregshell set_value '[HKEY_THIS_MACHINE\Services\lwio\Parameters\Drivers\rdr]' Smb2Enabled 1

Restart the lwio service:

vc-01:~ # /opt/likewise/bin/lwsm restart lwio

Log out of VAMI web interface, log back in and retry joining to the domain.

Posted by & filed under Hardware.

Using the sas2ircu utility from LSI, we can blink the drive LED to help ID the failed drive correctly. Of course this requires a LSI card. Some LSI cards may need to use the sas3ircu utility instead. There have been some reports from the interwebs that this utility failed to blink the correct drive, but I have not experienced this myself.

As always use the supercomputer between your ears to ensure the physical serial and the serial reported by the system match, etc etc.

[root@jetstore] ~# sas2ircu list
LSI Corporation SAS2 IR Configuration Utility.
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved.


         Adapter      Vendor  Device                       SubSys  SubSys
 Index    Type          ID      ID    Pci Address          Ven ID  Dev ID
 -----  ------------  ------  ------  -----------------    ------  ------
   0     SAS2308_2     1000h    87h   00h:06h:00h:00h      1000h   3020h

         Adapter      Vendor  Device                       SubSys  SubSys
 Index    Type          ID      ID    Pci Address          Ven ID  Dev ID
 -----  ------------  ------  ------  -----------------    ------  ------
   1     SAS2308_2     1000h    87h   00h:81h:00h:00h      1000h   3020h
SAS2IRCU: Utility Completed Successfully.

Back to the sas2ircu utility in a moment. We need to first acquire the serial number of the failed disk. For a system that is multipath, we can find the actual dev names by running the following to locate a disk in the fail state:

[root@jetstore] ~# gmultipath list | grep -i -B 10 fail
Consumers:
1. Name: da43
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
2. Name: da16
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r1w1e1
   State: FAIL

Now we can see da16 is failed. Time to get the serial number of that disk. Or da43. they are the same just multipaths.

[root@jetstore] ~# smartctl -a /dev/da16 | grep Serial
Serial number:        WMC1F0D5T1DF

Save that serial number for the next step.

Smartctl also outputs other useful information about the drive, statistics, etc. Worth checking out, but not relevant here.

Next, we can display the disks attached to one of those controllers. Be sure to input the correct serial number in the grep command:

[root@jetstore] ~# sas2ircu 0 display | grep -C 10 WMC1F0D5T1DF

Device is a Hard disk
  Enclosure #                             : 3
  Slot #                                  : 20
  SAS Address                             : 50000c0-f-01f9-f6eb
  State                                   : Ready (RDY)
  Size (in MB)/(in sectors)               : 2861588/5860533167
  Manufacturer                            : WD
  Model Number                            : WD3001FYYG-01SL3
  Firmware Revision                       : VR08
  Serial No                               : WDWMC1F0D5T1DF
  GUID                                    : N/A
  Protocol                                : SAS
  Drive Type                              : SAS_HDD

Get the enclosure and slot # of the failed drive and turn the led on:

sas2ircu 0 locate 3:20 ON

Turn the led off:

sas2ircu 0 locate 3:20 OFF

NOTE: If you are replacing a disk that is multipath, e.g. you see something like the following when you offline and remove a disk, ensure that the LED above is OFF or GEOM_MULTIPATH will not pickup the new disk as multipath. See the below log for what happens when a disk is inserted with the LED blinking Vs not blinking:

----------start drive detach event (already offline)------------
Aug 14 14:05:31 jetstore mps1: mpssas_prepare_remove: Sending reset for target ID 27
Aug 14 14:05:31 jetstore da43 at mps1 bus 0 scbus10 target 27 lun 0
Aug 14 14:05:31 jetstore da43: <WD WD3001FYYG-01SL3 VR08> s/n         WMC1F0D5T1DF detached
Aug 14 14:05:31 jetstore GEOM_MULTIPATH: da43 in disk17 was disconnected
Aug 14 14:05:31 jetstore mps1: GEOM_MULTIPATH: all paths in disk17 were marked FAIL, restore da16
Aug 14 14:05:31 jetstore Unfreezing devq for target ID 27
Aug 14 14:05:31 jetstore GEOM_MULTIPATH: da16 is now active path in disk17
Aug 14 14:05:31 jetstore GEOM_MULTIPATH: da43 removed from disk17
Aug 14 14:05:31 jetstore (da43:mps1:0:27:0): Periph destroyed
Aug 14 14:05:31 jetstore mps0: mpssas_prepare_remove: Sending reset for target ID 38
Aug 14 14:05:31 jetstore da16 at mps0 bus 0 scbus2 target 38 lun 0
Aug 14 14:05:31 jetstore da16: <WD WD3001FYYG-01SL3 VR08> s/n         WMC1F0D5T1DF detached
Aug 14 14:05:31 jetstore GEOM_MULTIPATH: da16 in disk17 was disconnected
Aug 14 14:05:31 jetstore mps0: GEOM_MULTIPATH: out of providers for disk17
Aug 14 14:05:31 jetstore Unfreezing devq for target ID 38
Aug 14 14:05:31 jetstore GEOM_MULTIPATH: da16 removed from disk17
Aug 14 14:05:31 jetstore GEOM_MULTIPATH: destroying disk17
Aug 14 14:05:31 jetstore GEOM_MULTIPATH: disk17 destroyed
Aug 14 14:05:31 jetstore (da16:mps0:0:38:0): Periph destroyed
----------end detach event-------------

----------start insert with LED BLINKING - note no GEOM_MULTIPATH----------
Aug 14 14:10:27 jetstore da16 at mps0 bus 0 scbus2 target 50 lun 0
Aug 14 14:10:27 jetstore da16: da43 at mps1 bus 0 scbus10 target 39 lun 0
Aug 14 14:10:27 jetstore syslog-ng[1426]: Error processing log message: <WD WD3001FYYG-01SL3 VR08> Fixed Direct Access SPC-4 SCSI device
Aug 14 14:10:27 jetstore da43: da16: Serial Number         WMC1F0D9UX1U
Aug 14 14:10:27 jetstore syslog-ng[1426]: Error processing log message: <WD WD3001FYYG-01SL3 VR08> Fixed Direct Access SPC-4 SCSI device
Aug 14 14:10:27 jetstore da16: 600.000MB/s transfersda43: Serial Number         WMC1F0D9UX1U
Aug 14 14:10:27 jetstore da43: 600.000MB/s transfersda16: Command Queueing enabled
Aug 14 14:10:27 jetstore da16: 2861588MB (5860533168 512 byte sectors)
Aug 14 14:10:27 jetstore da43: Command Queueing enabled
Aug 14 14:10:27 jetstore da43: 2861588MB (5860533168 512 byte sectors)
Aug 14 14:10:27 jetstore ses3: da43,pass47: Element descriptor: 'Slot 21'
Aug 14 14:10:27 jetstore ses0: da16,pass18: Element descriptor: 'Slot 21'
Aug 14 14:10:27 jetstore ses3: da43,pass47: SAS Device Slot Element: 1 Phys at Slot 20
Aug 14 14:10:27 jetstore ses0: da16,pass18: SAS Device Slot Element: 1 Phys at Slot 20
Aug 14 14:10:27 jetstore ses3:  phy 0: SAS device type 1 id 0
Aug 14 14:10:27 jetstore ses0:  phy 0: SAS device type 1 id 1
Aug 14 14:10:27 jetstore ses3:  phy 0: protocols: Initiator( None ) Target( SSP )
Aug 14 14:10:27 jetstore ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
Aug 14 14:10:27 jetstore ses3:  phy 0: parent 50030480003c273f addr 50000c0f0137b686
Aug 14 14:10:27 jetstore ses0:  phy 0: parent 50030480003c27bf addr 50000c0f0137b687

-------end insert with LED BLINKING-------


------start insert with LED off----------------

Aug 14 14:28:53 jetstore da16 at mps0 bus 0 scbus2 target 50 lun 0
Aug 14 14:28:53 jetstore da43 at mps1 bus 0 scbus10 target 39 lun 0
Aug 14 14:28:53 jetstore da16: da43: <WD WD3001FYYG-01SL3 VR08> Fixed Direct Access SPC-4 SCSI device
Aug 14 14:28:53 jetstore syslog-ng[1426]: Error processing log message: <WD WD3001FYYG-01SL3 VR08> Fixed Direct Access SPC-4 SCSI device
Aug 14 14:28:53 jetstore da16: Serial Number         WMC1F0D9UX1U
Aug 14 14:28:53 jetstore da43: Serial Number         WMC1F0D9UX1U
Aug 14 14:28:53 jetstore da16: 600.000MB/s transfersda43: 600.000MB/s transfers
Aug 14 14:28:53 jetstore da16: Command Queueing enabled
Aug 14 14:28:53 jetstore da43: Command Queueing enabled
Aug 14 14:28:53 jetstore da16: 2861588MB (5860533168 512 byte sectors)
Aug 14 14:28:53 jetstore da43: 2861588MB (5860533168 512 byte sectors)
Aug 14 14:28:53 jetstore ses3: da43,pass47: Element descriptor: 'Slot 21'
Aug 14 14:28:53 jetstore ses0: da16,pass18: Element descriptor: 'Slot 21'
Aug 14 14:28:53 jetstore ses3: da43,pass47: SAS Device Slot Element: 1 Phys at Slot 20
Aug 14 14:28:53 jetstore ses0: da16,pass18: SAS Device Slot Element: 1 Phys at Slot 20
Aug 14 14:28:53 jetstore ses3:  phy 0: SAS device type 1 id 0
Aug 14 14:28:53 jetstore ses0:  phy 0: SAS device type 1 id 1
Aug 14 14:28:53 jetstore ses3:  phy 0: protocols: Initiator( None ) Target( SSP )
Aug 14 14:28:53 jetstore ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
Aug 14 14:28:53 jetstore ses3:  phy 0: parent 50030480003c273f addr 50000c0f0137b686
Aug 14 14:28:53 jetstore ses0:  phy 0: parent 50030480003c27bf addr 50000c0f0137b687
Aug 14 14:29:07 jetstore GEOM_MULTIPATH: disk17 created
Aug 14 14:29:07 jetstore GEOM_MULTIPATH: da16 added to disk17
Aug 14 14:29:07 jetstore GEOM_MULTIPATH: da16 is now active path in disk17
Aug 14 14:29:07 jetstore GEOM_MULTIPATH: da43 added to disk17

------end insert with LED off----------------

 

Posted by & filed under Adruino, Hardware, Hardware Development, Programming.

PlatformIO is an open source ecosystem for IoT development
Cross-platform build system. Continuous and IDE integration. Arduino and ARM mbed compatible

 

Came across this cool IDE, built on top of Atom for dev of iot. There is also a commercially supported offering. http://platformio.org/

Posted by & filed under Linux.

Scenario:

We need to scp a file between two hosts. The problem is that the two hosts (A & C) cannot directly communicate. We can solve this using a SSH tunnel and an intermediate host (B) that can communicate with both. This also means, the command for Host B needs to run first, then the scp command for host A.:

 

Host A (source)

This will scp to localhost on port 3000 which is actually our tunnel to host c — /destination_file is the path on host C

scp -P 3000 /source/file username@localhost:/destination_file

Host B (intermediate)

ssh -R 3000:ip.of.host.a:22 ip.of.host.c

Host C (destination)

 

 

Also, if you have spaces in the paths make sure to escape the space with \ e.g.

scp -P 3000 "/source/file/some\ directory/" username@localhost:/destination_file

 

Posted by & filed under Server Admin.

I have a PKCS12 .pfx export of a cert that I need to import into a Tomcat keystore in order to update an expiring certificate.

 

Need to know a few things beforehand:

  • Tomcat keyfile path
  • Source store password for the pfx file
  • Source alias for the pfx
  • Dest source passwd
  • Dest source alias
keytool -importkeystore -srckeystore wildcard_2016.pfx -srcstoretype pkcs12 -srcstorepass changeit -srcalias 4b84576db-35ca-8dc45b92a -destkeystore C:\ibi\ssl\.keystore -deststoretype jks -deststorepass changeit! -destalias tomcat

In order to get the source alias from the new pfx file:

keytool -v -list -storetype pkcs12 -keystore wildcard_2016.pfx > output.txt

If you need to get the alias from the existing Tomcat keystore:

keytool -list -v -keystore C:\ssl\.keystore > tomcatkeystore.txt

Additionally, the above command can be used to verify the certificate, expiry date, etc.

Lastly, if you restart Tomcat and it throws errors like the following in the catalina log, you may need to reset the keystore password:

SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-nio-443"]
java.security.UnrecoverableKeyException: Cannot recover key
...more stack trace here...

Reset to the correct password as defined in the servver.xml keyStorePass parameter using the following command. You may need to adjust alias to your needs. You will be prompted for the new password, which should match the previously mentioned keyStorePass parameter.

keytool -keypasswd -new changeit -keystore C:\ssl\.keystore -storepass changeit -alias tomcat

You can also reset the password for the keystore itself (www.ibm.com/support/knowledgecenter/en/S…):

PS C:\> .\keytool.exe -keypasswd -new REDACTED -keystore C:\.keystore -alias tomcat

 

 

EDIT FROM THE FUTURE:

Additional note — when trying to run the import command I was getting the following error:

Existing entry alias 2 exists, overwrite? [no]:  yes
keytool error: java.lang.Exception: Alias <2> does not exist

I ran the following to verify the alias is correct:

PS C:\> .\keytool.exe -list -keystore C:\server2017.pfx -storetype pkcs12        
Enter keystore password: 

Keystore type: PKCS12
Keystore provider: SunJSSE

Your keystore contains 1 entry

2, Mar 13, 2017, PrivateKeyEntry,
Certificate fingerprint (MD5): RE:DA:CT:ED:DE:AD:BE:EF

Key ID of 2 is displayed correctly here as well as a more verbose output also showed the same:

.\keytool.exe -list -v -keystore C:\server_2017.pfx

I then took the same .pfx file and checked it on a linux machine based on a hint from this stackoverflow on binary chars: http://stackoverflow.com/questions/15301005/keytool-cant-find-alias

nate@beef:~/$ keytool -list -keystore server2017.pfx -storetype pkcs12
Enter keystore password:

Keystore type: PKCS12
Keystore provider: SunJSSE

Your keystore contains 1 entry

1, Mar 13, 2017, PrivateKeyEntry,
Certificate fingerprint (SHA1): RE:DA:CT:ED:DE:AD:BE:EF

And lo’ and behold it shows the alias is actually 1!

 

..Back in Windows land:

PS C:\> ./keytool -importkeystore -srckeystore C:\server2017.pfx -srcstoretype pkcs12 -srcstor
epass REDACTED -srcalias 1 -destkeystore C:\.keystore -deststoretype jks -deststorepass REDACTED -destalias tomcat
Existing entry alias 1 exists, overwrite? [no]:  yes

It accepted alias 1 instead and the cert imported correctly. I love Tomcat -_-

 

 

 

 

Posted by & filed under Linux, Server Admin.

I had a old server I brought up and it was unable to complete it’s boot due to a missing drive in fstab. Editing the fstab in recovery mode is not a option since the filesystem gets flagged as read only.

In order to make the FS writable and therefore be able to successfully edt the fstab, the following command will remount the FS in read/write mode:

mount -o remount,rw /

 

Posted by & filed under Server Admin.

I recently had a Windows XP laptop crash. Windows would not boot to safe mode or anything, and just displayed the following error message:

Windows XP could not start because the following file is missing or corrupt: \WINDOWS\SYSTEM32\CONFIG\SYSTEM

I could not afford to simply wipe the laptop and reinstall windows as it had some old software that was no longer available.I located the following article which details a procedure to recover from this issue using the MS recovery console and using the System Restore: https://support.microsoft.com/en-us/kb/307545

As this laptop did not have a optical cd-rom, it was a difficult proposition to make a XP bootable USB stick to complete this procedure since I do not have the media handy. Additionally, it seemed like a pain to go thru all the steps when it could be simplified quite a bit with a functioning OS like linux. I decided to attempt to recover using a linux live cd:

  1. Create a bootable USB stick with Ubuntu on it using uNetBootin
  2. Boot to the USB stick.
  3. Make backups of any critical files (just in case)
  4. Backup registry files at C:\windows\system32\config to usb stick:
    c:\windows\system32\config\system
    c:\windows\system32\config\software
    c:\windows\system32\config\sam
    c:\windows\system32\config\security
    c:\windows\system32\config\default
  5. Access the System Volume Information which should contain restore points for the system. See Part 2 Steps 7 through 10 in above MS article for details, but in a nutshell you want to access C:\System Volume Information. There will be one or more folders inside and their names will be similar to “_restore{D86480E3-73EF-47BC-A0EB-A81BE6EE3ED8}”. Inside these folders, look for RPx folders. There may be more than 1, and x would be a number. Look at the created dates of these folders to identify a fairly recent restore point. For example I found one that was two weeks old in RP47.
  6. Access the snapshot folder to retrieve registry backups. Example:
    C:\System Volume Information\_restore{D86480E3-73EF-47BC-A0EB-A81BE6EE3ED8}\RP1\Snapshot
  7. Inside the snapshot directory, copy the registry files to a temp location, and make a backup of them:
        _REGISTRY_USER_.DEFAULT
        _REGISTRY_MACHINE_SECURITY
        _REGISTRY_MACHINE_SOFTWARE
        _REGISTRY_MACHINE_SYSTEM
        _REGISTRY_MACHINE_SAM
  8. Copy the snapshots to C:\windows\system32\config.
  9. Delete the old crashed registry files:
    c:\windows\system32\config\system
    c:\windows\system32\config\software
    c:\windows\system32\config\sam
    c:\windows\system32\config\security
    c:\windows\system32\config\default
  10. Rename the backup registry files to replace the ones you just deleted:
        Rename _REGISTRY_USER_.DEFAULT to DEFAULT
        Rename _REGISTRY_MACHINE_SECURITY to SECURITY
        Rename _REGISTRY_MACHINE_SOFTWARE to SOFTWARE
        Rename _REGISTRY_MACHINE_SYSTEM to SYSTEM
        Rename _REGISTRY_MACHINE_SAM to SAM
  11. Cross your fingers and reboot! If it does not work, and you still receive the same error message, you may need to try a older registry snapshot. Simply follow the above steps to try a different registry snapshot.

Good luck!

Posted by & filed under Linux, Server Admin.

This all started with WordPress timeouts. I was trying to activate some premium plugins, and the license activation was timing out. I started doing some digging and found they use the WordPress core library WP_http which in turn uses curl to make the request. I wrote my own code to use WP_Http and it failed in the same way with a timeout. I added a timeout parameter to the wp_remote_get() call, and it was able to complete without a timeout. I then used a IP address in place of the domain name and it worked without the need for the timeout parameter.

<?php
$response = wp_remote_get('http://google.com');
print_r($response);
echo wp_remote_retrieve_body( $response );
?>

With that info in hand, I decided it must be on the server. I started doing some tests:

web@web:~$ time curl "http://google.com"
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

real    0m5.565s
user    0m0.007s
sys     0m0.000s

I then did the same test from another server that uses the same DNS servers in resolv.conf:

dev@web1 [~]# time curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

real    0m0.121s
user    0m0.000s

After much googling, I found a few number of suggested solutions:

  • Disable IPv6
  • Ensure /etc/nsswitch.conf is set correctly (hosts: files dns)

Neither of these worked for me. Finally, I added the following directive into my resolv.conf and it fixed the issue!

options single-request

Apparently, this is actually somewhat related to ipv6 — from the resolv.conf manpage:

single-request (since glibc 2.10)
                     Sets RES_SNGLKUP in _res.options.  By default, glibc
                     performs IPv4 and IPv6 lookups in parallel since
                     version 2.9.  Some appliance DNS servers cannot handle
                     these queries properly and make the requests time out.
                     This option disables the behavior and makes glibc
                     perform the IPv6 and IPv4 requests sequentially (at the
                     cost of some slowdown of the resolving process).

Now, I get good response times when I curl:

web@web:~$ time curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

real    0m0.170s
user    0m0.007s
sys     0m0.000s

Looks like the resolver sends parallel requests, fails to see the IPv6 response, waits 5 sec and sends sequential requests because it thinks the nameserver is broken. By adding the options single-request, glibc makes the requests sequentially be default and does not timeout.

I found some good info and hints on this issue here: https://bbs.archlinux.org/viewtopic.php?id=75770

Lastly, to bring this whole thing full circle, the WprdPress plugins now are able to get out and communicate successfully. Woohoo!

Posted by & filed under Uncategorized.

MassMine allows you to easily datamine Twitter, Google, Wikipedia, and soon Facebook for data. Pretty cool! From the official site:

MassMine is a social media mining and archiving application that simplifies the process of collecting and managing large amounts of data across multiple sources. It is designed with the researcher in mind, providing a flexible framework for tackling individualized research needs. MassMine is designed to run both on personal computers and dedicated servers/clusters. MassMine handles credential authorizations, rate limiting, data acquisition & archiving, as well as customized data export and analysis.

 

www.massmine.org/

Posted by & filed under Firewalls, Security, Server Admin.

cPanel WHM’s cpHulk system manages iptables blocks against IP addresses that fail to authenticate repeatedly. While the settings are fairly lenient and shouldn’t result in legitimate users being blacklisted, occasionally it can happen. The following command will reset the blocklist completely. While this is akin to using a shotgun when a scalpel is required, the blocks are time based and any malicious addresses would get quickly re-blocked.

 

iptables -F cphulk && mysql -e "Delete from cphulkd.login_track;"

There is a method to remove specific addresses, but I do not have the commands handy at present, and if I remember correctly it entails connecting to the mysql console, running a query to find the IP in the block table and issuing a drop query.

Posted by & filed under Software.

First step is creating the network.

# Command
/network add -nick twitchusername nameofnetwork
# Example
/network add -nick billpaxton Twitch

Second step we are going to add a server to that network. Generate oauth password here http://www.twitchapps.com/tmi/

# Command
/server add -auto -network nameofnetwork irc.twitch.tv 6667 oauth:password
# Example
/server add -auto -network Twitch irc.twitch.tv 6667 oauth:asgsdftgwe5tq45t134ra

In case you’re wondering, the above -auto tag is optional. What this means is when you connect to this network, it will automatically connect to this server.

Third step is where we add the channel to the network you created in the first step. And in case you’re wondering, the channel is just your Twitch username.

# Command
/channel add -auto #twitchusername nameofnetwork
# Example
/channel add -auto #billpaxton Twitch

Again, the -auto tag is optional.

And that does it. All you need to do now is connect to that network. Which is accomplished simply by the following:

# Command
/connect nameofnetwork
# Example
/connect Twitch

One thing that I would suggest you go ahead and do once you get that sorted out, is ignore the user jtv. It will ping you information that you simply don’t need or care about. Of course, feel free to leave it. But if you do want to ignore it, just type:

/ignore jtv

Saved from the void via Google’s cache. Woohoo

Posted by & filed under Security, Web Development.

A few handy commands to cut to the chase and find the crap spammers/skiddies have added to a WP install:

Find files containing text recursively:

 grep -ri "string to search" .

A good use of this is to search for the below. It can return false positives, but finds a function commonly used to obsfucate code:

grep -ri "base64_decode" .

Diff two installations. If you have a clean copy of WP, you can compare it to a compromised version to find the differences. Here I am excluding the error_log file, and sending the output to diff.txt for review:

diff --exclude "*error_log*" -r /path/to/wp /path/to/other/wp > diff.txt

Find php files (and other filetypes that should not be present in the uploads directory. This is typically one if the first places things are placed:

find /wp-content/uploads -name "*.php" -type f

Grep the DB. Sometimes things get hidden in the database in an effort to hide malware. Considering that a WordPress database is tiny in the grand scheme of things, a simple way to quickly review what is in the database is to use mysqldump, phpmyadmin or whatever tool you would like to export the entire database to SQL. Then you can review the contents easily. Be on the lookout for base64 encoded strings, they are a good giveaway.

Find recently modified PHP files:

find . -name \*.php -mtime -2

 

 

Posted by & filed under Linux, Server Admin.

Quick and dirty way to pull out the key and crt from a pkcs12 file:

openssl pkcs12 -in filename.pfx -nocerts -out filename.key

openssl pkcs12 -in filename.pfx -clcerts -nokeys -out filename.crt 

If you are using this for Apache and need to strip the password out of the certificate so Apache does not ask for it each time it starts:

openssl rsa -in /path/to/originalkeywithpass.key -out /path/to/newkeywithnopass.key