New Skydrive App for Android

The new app is awesome!

Working With WSUS (Windows Update Server) from the Client Side

WSUS (Windows Software Update Server) is what us admins use (usually) to push out the Windows patches every month. I've had some occasions where my clients were behaving oddly in the past and there are some tricks I've learned over the years on how to deal with this. So, here goes:

If you are imaging machines, it will be easy for the WSUS ID to get stuck in the registry and then propagated out to your computers. You won't notice this unless you compare your real inventory to the computer listed in the WSUS Admin Console. When an ID is used by more than one computer, then your reporting is off. What happens is that ComputerA will check in with ID#4 (The ID's a lot longer than that, but bear with me). ComputerA will get its updates and be happy. Now ComputerB will check in with ID#4. To WSUS, this looks like the computer changed its name. ComputerB will get its updates, too. The problem isn't that the computers won't update. Run that scenario again, but this time let's say ComputerA had errors and patching failed for one or many of the patches. If ComputerB checked in before you ran a patching report, you'll never know that ComputerA had issues.
To combat this, I like to delete the ID from the registry. It's one of the very last things I do when I build/image a new computer. You can do this while the computer's running safely.
reg delete HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate /v SusClientId /f

After the key is deleted, you should restart the "Windows Update" service. It's important to note that the name of this service is different between Windows XP and Windows 7.
In Windows XP you'd run:
net stop "automatic updates" && net start "automatic updates"
In Windows 7 you'd run:
net stop "Windows Update" && net start "Windows Update"

Or, in Powershell:
restart-service "Windows Update"

That's much easier, but I've been messing with WSUS since before I became Powershell savvy, and old habits die hard....

There's a command to regenerate the SUS ID we deleted from the registry (that's not available in Powershell), and it uses a switch within THE command that Windows manages its updates with: wuauclt.exe.

This is kind of a weird little CLI program, because entering wuauclt.exe /? won't get you anything. The commands I use are as follows:
wuauclt.exe /detectnow - forces the computer to check with it's WSUS server for new updates.
wuauclt.exe /resetauthorization - this is the command that re-registers the computer with WSUS, and generates the new SUS ID.
Note that you can combine the switches on a single line, like this:
wuauclt.exe /resetauthorization /detectnow

Miscellaneous How To's So I Don't Forget!!

I'm using this blog as much for me as for my readers (which I only have a handful of, but whatever). It's another source of documentation, as far as I'm concerned. So today, I've got some neat tricks that I perform fairly often, and I'm sick of searching the web for them every time I need them.

Disk Cleanup in Windows 2008 R2

Did you know that Windows 2008 R2 doesn't come with the Disk Cleanup app installed? I have NO idea what Microsoft is thinking, but it's still on the system; you just have to copy some files and make a shortcut to use it.

• Move cleanmgr.exe from %systemroot%\winsxs\amd64_microsoft-windows-cleanmgr_31bf3856ad364e35_6.1.7600.16385_none_c9392808773cd7da to%systemroot%\System32
• Move cleanmgr.exe.mui from %systemroot%\winsxs\amd64_microsoft-windows-cleanmgr.resources_31bf3856ad364e35_6.1.7600.16385_en-us_b9cb6194b257cc63 to %systemroot%\System32\en-US
• Now go back to %systemroot%\system32 and send-to --> desktop (create shortcut), or you can simply type cleanmgr.exe into the run dialog box since system32 is listed in your system path variable.

Bit and Byte Conversion

I found this handy website to convert bits to bytes and vice-versa

Creating Test Files of a Certain Length

Here's a handy command to create test files for testing copy speeds, for instance. Note that you have to run the command prompt as administrator if you have UAC turned on.
C:\> fsutil file createnew <filename> <filesize_inbytes>

For Example (This creates a 1GB file):
C:\> fsutil file createnew C:\Temp\Test_File_1GB.txt 1073741824

Removing Old Patch Installation Files in Windows 2008 R2

Back in Windows Server 2003, one way to reclaim disk space was to delete hidden folders with the $Uninstall prefix from the C:\Windows folder. This would make it impossible to uninstall updates, so you had to be relatively sure that you wouldn't need to do that. There are ways around that: Build a 2003 Test machine, install the update that you need the uninstall files for, and copy the $Uninstall folder for that patch back to the server.

This process was altered in Windows Server 2008 and again in R2. For Windows Server 2008 R2, the command you need to run (as an administrator) is:
C:\> dism /online /cleanup-image /spsuperseded.

Things get complex, but don't forget to check the easy stuff first

So my Veeam backup solution wasn't performing as I thought it should. Veeam had direct access to my SAN via a 4Gb FiberChannel HBA, and was only pulling around 40MB/s. I was pulling my hair out and finally posted on a forum for assistance. The advice? Check the driver on the HBA card. Lo and behold, the QLogic HBA card was using a MICROSOFT driver. That's just no good. After updating the driver, my speeds went from 40MB/s to anywhere between 130-250 MB/s. Now that's a speed boost!

I think it's important that we as technical people strive to maintain good troubleshooting skills. Sure, the infrastructure gets complicated, but don't ever forget that the best fix might be the easiest. Your internet isn't working? Start from the bottom and work your way up. Sure, it might be a routing issue, but put traceroute away for a minute and check that the network cable is plugged in. Keep it simple, stupid.

Ice Cream Sandwich and my battle with backups (Backup Exec, Veeam, and SQL Oh My!!!)

I've been using Firefox for, well, a very long time - IE6 era, I think. I've been hanging on because Firefox could sync my bookmarks and passwords between all of my devices. My Samsung Galaxy S2 was on Android version 2.3.3 (Gingerbread) and I couldn't run Chrome on it. I was pretty close to rooting it, but I fix computing devices often enough, and my phone is a device that just has to work. My life is too busy to have it go down and immediately be able to sit down and fix it. Aaaanyway, I got the Ice Cream Sandwich update from Sprint this past week (which was out in, what, March?) and NOW I can run Chrome!!!! My favorite thing so far is Chrome-to-Phone, and my biggest pet peeves are the lack of a decent FTP program like FireFTP on Firefox, and the absence of any command to sort my bookmarks alphabetically. I've also installed Google Cloud Print, but haven't needed to use it yet. The extensions are pretty amazing:

• I can post to this blog, my blog on Tumblr (The LAG), Twitter, Facebook, etc with AddThis
• X-Notifier checks all of my email accounts on a regular basis
• Facebook, Google Phone, and Skype all have good add-ons as well

I really like Ice Cream Sandwich, too; I'm just more excited about being able to switch the default browser on all of my computers to Chrome. My notifications are much more descriptive, and it looks like I've gotten a battery boost.

At work, I've rolled out PRTG for network monitoring. Having a good network monitoring solution in place saves the IT department A LOT of headaches, especially with software as robust and flexible as PRTG. Having the ability to see all of your problems (with some exceptions) at a glance saves a ton of time. I turn off monitoring while I run the monthly Microsoft updates, then when I'm done rebooting the servers I can flip PRTG back on and see almost immediately if anything didn't come back on correctly. The BEST thing about something like PRTG though, and the most indispensable feature, as far as I'm concerned, is the tracking of sensors over time. Now, I can look at some graphs and predict when we might need to add drive space, or if someone copies a giant 20GB file to my server I can detect it immediately and possibly head things off before they bring down the server (this has happened more than once).

Also, I'm completely redesigning the backup situation. We were using Symantec Backup Exec 2010 R3 exclusively for backups, and we weren't using it right. We needed a lot more licenses to comply with the licensing requirements for myriad SQL servers, and the backup process just wasn't working well. Backup Exec is.... difficult to use. For me, anyway. Maybe with some training.... oh never mind. So I sold the boss-man on plunking down about $14,000 for the Veeam Management Studio, which includes Veeam ONE monitoring for our VMware Infrastructure, which plays into the previous topic. So I've finally got Veeam Backup and Replication 6.1 in place and it's doing it's first backup as we speak. Well, technically it's the second, but the first backup was so slow I had to scrap it. You see, Veeam B&R can talk directly to the SAN over a Fiber Channel HBA, instead of trying to move 5 Terabytes (in my case) over a gigabit network link. I misconfigured it the first time and was in network mode. Veeam also does Reverse Incremental backups, Change-Block Tracking (only backs up things that have changed), and is really good at deduplication (REALLY good). I'm hoping to cut my storage needs in half, and shrink my backup window big time. For example, my current backup has now read 3.4 TB, and only written 1.9 TB to disk. A little over half. Monday's backup will only back up what's changed from this one, so it should run pretty fast. I'll post some more stats once I get the process going and smoothed out. When Veeam's done, it will automagically launch my Backup Exec job, which will write the Veeam backup to tape. So, here's how to launch a backup exec job with Veeam: Edit the job, select storage, click the advanced button, and click on the advanced tab. The bottom section allows you to run commands at the end of a backup job. The command to launch a backup exec job looks like this: "C:\Program Files\Symantec\Backup Exec\bemcmd.exe" -o1 -jJobName. Use the quotes in there, by the way. Here's a handy web page that outlined the rest of the details.

While I was auditing what was being backed up, I kept finding little SQL Express instances everywhere. Symantec requires a special agent to properly back these up. Instead of plunking down around $9,000 more for the appropriate licenses to back up around 15 more SQL Express servers with Backup Exec, I'm skipping the SQL instance in my backup selections, and will just run a scheduled SQL script as a scheduled task to back up the database to a .bak file before the regular file backups run. Here's the page that gave me the instructions on putting this together. It's talking about backing up your VMware vCenter database (if you store it in SQL Express), but the same technique applies to other SQL Express instances.

Working with Scheduled Tasks

So I've been getting the IT "general purpose" server up and running. I have all of my management tools installed, and now it's time to automate some reporting and maintenance functions. Right now, I have 3 scheduled tasks:

• Windows Service Monitor: This Powershell script looks at many of my servers and the services that I deem "important". It emails me if it finds any that aren't running. It's a terrible script; I pretty much did a Get-WMIObject query, ran some logic with a where-object statement and then dumped it out to a file. The beast is 900 lines long or something ridiculous (done through copying and pasting, not by hand). I KNOW that there's a more elegant way to do it, but I haven't got the time to spend on it. It does what it needs to do.
• Website Monitor: This script downloads the html code from a couple of web pages, then runs select-string to return a true or false based on whether certain text appears in the code. If the expected code isn't there, the script emails the admins, because something's not right.
• The "Start Patching" script: We run this script before we start patching to turn off our monitoring software. First, it disables our SNMP monitoring software, then it stopd the Spiceworks service to take spiceworks offline. We don't want a bunch of emails that so-and-so system is down. The last thing it does is to disable the above two scheduled tasks, and let me tell you that this took me a bit to figure out. Yeah, you use powershell, but you use it to call a CLI program called schtasks.exe, which I think is pretty weird for Windows Server 2008 R2. You'd think they'd have some powershell commands for the Tasl Scheduler, but no. So here's the syntax for disabling a script:

 $CommandDisable = 'schtasks.exe /Change /S ComputerName /TN "Windows Service Monitor" /Disable'
Invoke-Command $CommandDisable

So first, I put the command into a string variable. Using single quotes forces Powershell to interpret the double quotation marks literally (as part of the string itself). Then I can use Invoke-Command to run the string as a command. Schtasks.exe has a ton of syntax options. Way back when I remember using it to set up scheduled tasks from scratch from a batch file; it's got really great functionality (too bad it hasn't been ported to powershell (grumble grumble). Every Windows computer has schtasks.exe, so if you're curious pop into a cmd shell and type schtasks /?. The help function is layered, so you can type schtasks /Change /? and get help specific to the subset of the command structure. Fun stuff.

Just some links

While I've learned a lot, I really haven't had much time to actually write about any of it. I'm in the process of rolling out Spiceworks as a helpdesk and inventory solution. It's kind of clunky to work with, honestly. My employer wants layers of helpdesk categories, and that functionality isn't built-in, so I have to rely on volunteer developers for a solution. I LOVE that people like to code, and I love free stuff (who doesn't) but in a year when I have to upgrade this those people might have moved on, and the project abandoned. I just don't have the time to get intimate with a solution right now, so I'm taking the position that if we want all the features of a for-pay helpdesk system, then we pay for it. If you want something nice and flat, then sure, we can do Spiceworks and save a few grand.

I'm also working on learning about our SAN, and our vendor (Mass Mountain) gave me a flash drive with their software on it (it's a 60 day trial once you create a volume group). You just boot from the flash drive and run it on whatever hardware you have lying around, which is pretty cool. I wanted to run it in VMware Workstation 8, where I have a VMware lab - 3 ESXi hosts in a cluster with High Availability and Distributed Resource Scheduling configured, a domain controller, a vCenter server, a FreeNAS iSCSI server, an OpenFiler NFS server, and a Server running Veeam that's backing up 3 servers running ON the ESXi cluster. Unfortunately, I had some problems getting VMware Workstation to boot from the USB drive. I found this workaround using a boot manager ISO called Plop, which did the trick.