Cookies and Perl

I have a client who is using and old (outdated and unsupported) php/perl CMS as an intranet site. In implementing a DRP (Disaster Recovery Plan) they switched all the server references in the code and on the server from the server IP address to a DNS name. This was done using a search and replace :-(
This effectively broke the intranet site. They were able to get most of it back up and running with the exception of logins and how sessions were managed. For some reason, once this CMS saw a domain name instead of an IP address, it changed the path in the cookie to something like this:
mywebserver/http://mywebserver/somesite
from what worked before which was:
mywebserver/somesite
We searched through config files and didn't find anything conclusive to begin with. Then we found some code where cookies were being created and modified it so that after the cookie string was created we did some string manipulation on it like this:
$setcookie =~ s/http:\/\/mywebserver//m
This resets the setcookie var by searching through the setcookie var (it's a multiple line string so we needed to use /m) for the string 'http://mywebserver', replacing it with nothing.
It turned out that the CMS was creating more than one cookie, so we found this bit of code in two more files, added our hack there, and sessions worked again!
I have to confess, I was surprised that this little hack worked.

IIS default app pool proc terminated

Here's a great link to error codes for w3wp (event id 1009) and what they likely mean:
http://blogs.iis.net/brian-murphy-booth/archive/2007/03/22/how-to-troubleshoot-an-iis-event-id-1009-error.aspx
My current issue is error 0x0 which is documented in this link but fairly sparse. I checked the debugger flags with gflags and everything was unchecked. I'm still looking for a resolution to my issue.

***

I did find a resolution to this issue. I went through all the configs and our DRP (Disaster Recovery Procedure) document to see if a could figure it out. No dice there. I downloaded and hooked up procmon to see if it would tell me which registry key or file was causing the issue. No dice there either. It turned out that since our app had a dependency on SoftArtisan's FileUpEe, the fileUpEe dll's were registered in the wrong order! I had registered them 'com' folder first, 'dotnet' folder second (per the DRP). They needed to be registered 'dotnet' first and 'com' second. That made my IIS appPool stop cratering (it was definitely the issue).

Tools for working with SVN & Visual Studio

We're moving to Subversion as our code repo. Tools that we are using are:
- VisualSVN Server on the server box for maintaining the svn server (managing users, creating/importing new repos, etc)
- TortoiseSVN on client boxes. This works in conjunction with windows explorer to tell you the status of files in your local copy of the repo. We've found that the icons don't change status immediately - you need to be patient with them.
- Collabnet AnkhSVN - Subversion plug-in for visual studio. Allows you to see the status and check files in and out inside of VS.
- Collabnet SVN command line client. We found we needed this (particularly svn.exe in the PATH environment variable) if we wanted to run the SVN steps in a Visual Build file.

Monitor the right things

I started re-reading Release It! by Michael Nygard this morning on the commute into work. In his first chapter he talks about a (very small) issue that turns into a colossus and takes down an airline's check-in system. The system had a monitor configured and performing checks on it, but it turned out that it wasn't checking the right things (it was looking at the http port on transactional servers when it should have been looking at the RMI port).
It totally reminded me of something that happened about a month ago. We have a bunch of web applications that run on our production server. After fine tuning our monitoring to look at pages that the application has to apply logic to to server up (rather than a static home page) we found that our monitoring corresponded much closer to complaints from users.
Think twice about what you want to monitor and where to point it.

MS Sql Server bug

We ran into an Sql Server bug today that was rather interesting. We were implementing synonyms across a number of views, tables, and stored procs in a couple of DB's. Everything was working fine until another team did a deploy and changed the index on a view that was referenced by one of our synonyms. It turns out that there is a documented bug which requires that any time the DDL has changed on a view that is referenced by a synonym, that synonym looses it's connection to the view. This includes just updating the index to the view.

Handy Utilities for Windows

I don't want to forget about these handy software utilities for doing troubleshooting and debugging on the windows platform.

SpaceMonger is a tool that graphically displays how data is utilized in you hard drive.

SysInternals software...

TcpView essentially provides a graphical view of a netstat on a windows box with a few extra handy features (counting the number of sockets in time_wait, for example).

Procmon allows you to get information about what is happening with a specific process id.

windows account security

The domain (active directory) policy will override any local security setup for a particular account on a server. If the domain security policy says that an account will lock after 5 failed attempts and the local security policy says the account will lock after 3 attempts, the account will lock out after 5 attempts.
Here's the interesting thing:
The account lockout counter is reset every 24 hours or with every passed login attempt. So if you have two services using the same account, one with a correct password and one with an incorrect password, you can likely run indefinitely before the account will lock out. One service will never work and you should get a lot of errors in the security event log.