IIS default app pool proc terminated

Here's a great link to error codes for w3wp (event id 1009) and what they likely mean:
http://blogs.iis.net/brian-murphy-booth/archive/2007/03/22/how-to-troubleshoot-an-iis-event-id-1009-error.aspx
My current issue is error 0x0 which is documented in this link but fairly sparse. I checked the debugger flags with gflags and everything was unchecked. I'm still looking for a resolution to my issue.

***

I did find a resolution to this issue. I went through all the configs and our DRP (Disaster Recovery Procedure) document to see if a could figure it out. No dice there. I downloaded and hooked up procmon to see if it would tell me which registry key or file was causing the issue. No dice there either. It turned out that since our app had a dependency on SoftArtisan's FileUpEe, the fileUpEe dll's were registered in the wrong order! I had registered them 'com' folder first, 'dotnet' folder second (per the DRP). They needed to be registered 'dotnet' first and 'com' second. That made my IIS appPool stop cratering (it was definitely the issue).

Tools for working with SVN & Visual Studio

We're moving to Subversion as our code repo. Tools that we are using are:
- VisualSVN Server on the server box for maintaining the svn server (managing users, creating/importing new repos, etc)
- TortoiseSVN on client boxes. This works in conjunction with windows explorer to tell you the status of files in your local copy of the repo. We've found that the icons don't change status immediately - you need to be patient with them.
- Collabnet AnkhSVN - Subversion plug-in for visual studio. Allows you to see the status and check files in and out inside of VS.
- Collabnet SVN command line client. We found we needed this (particularly svn.exe in the PATH environment variable) if we wanted to run the SVN steps in a Visual Build file.

Monitor the right things

I started re-reading Release It! by Michael Nygard this morning on the commute into work. In his first chapter he talks about a (very small) issue that turns into a colossus and takes down an airline's check-in system. The system had a monitor configured and performing checks on it, but it turned out that it wasn't checking the right things (it was looking at the http port on transactional servers when it should have been looking at the RMI port).
It totally reminded me of something that happened about a month ago. We have a bunch of web applications that run on our production server. After fine tuning our monitoring to look at pages that the application has to apply logic to to server up (rather than a static home page) we found that our monitoring corresponded much closer to complaints from users.
Think twice about what you want to monitor and where to point it.

MS Sql Server bug

We ran into an Sql Server bug today that was rather interesting. We were implementing synonyms across a number of views, tables, and stored procs in a couple of DB's. Everything was working fine until another team did a deploy and changed the index on a view that was referenced by one of our synonyms. It turns out that there is a documented bug which requires that any time the DDL has changed on a view that is referenced by a synonym, that synonym looses it's connection to the view. This includes just updating the index to the view.

Handy Utilities for Windows

I don't want to forget about these handy software utilities for doing troubleshooting and debugging on the windows platform.

SpaceMonger is a tool that graphically displays how data is utilized in you hard drive.

SysInternals software...

TcpView essentially provides a graphical view of a netstat on a windows box with a few extra handy features (counting the number of sockets in time_wait, for example).

Procmon allows you to get information about what is happening with a specific process id.

windows account security

The domain (active directory) policy will override any local security setup for a particular account on a server. If the domain security policy says that an account will lock after 5 failed attempts and the local security policy says the account will lock after 3 attempts, the account will lock out after 5 attempts.
Here's the interesting thing:
The account lockout counter is reset every 24 hours or with every passed login attempt. So if you have two services using the same account, one with a correct password and one with an incorrect password, you can likely run indefinitely before the account will lock out. One service will never work and you should get a lot of errors in the security event log.

More IIS automated deploys

Been working more with IIS automated deploys and found a couple of good posts in David Wang's Blog along with some other good blog posts.

This one is related to how to manipulate IIS list data. He provides a handy vbs script to do this that worked nicely for me in adding a new application mapping to IIS.

This one talks about IIS App pool crashes - fatal communication errors between the Application Pool and IIS.

This one talks about app pool recycling and IIS availability. There's some great conversations past David's article further down the page.

This one (not David Wang) shows how to change COM+ MSDTC settings programmatically. This script worked great for me as well.

This one (Ian Morrish) shows how to manipulate COM+ Security launch and activation permissions using DComPerm from the windows SDK.

With all of these scripts, I found I had to carefully read the directions and make sure I had the parameters that were being passed in correct. I was often passing in erroneous parameters on my first couple of tries.