EC2 and the AWS (Amazon Web Services) Free Tier - My First Experience

The Amazon Web Services Management Console

I had my first go-round with EC2 in AWS this last week in a real-life context.  I was teaching a class on Content Management Systems at SAIT, and I wanted my students to experience what its like to install WordPress and MySQL on a Linux VM.  I also wanted to personally get some experience with AWS, so I thought 'why not kill two birds with one stone?'

About 6 weeks ago I had purchased Amazon Web Services IN ACTION, written by Andreas and Michael Wittig and published by Manning.  It gave me a great primer on setting up my AWS account, my Billing Alert, and my first couple of VM's.  I leveraged that experience, crossed my fingers, spun up 18 VM's for my students, and hoped I wouldn't get charged a mint for having them running 24/7 for a few days.  It was a Friday around 1pm when I created them and gave them to my students to use.  Imagine my surprise when I checked my billing page in AWS on Monday and discovered they had only charged me $2.81!

I had clearly reached some kind of threshold as after that day I got charged, on average, about $8/day - for all 18 VM's.  They were charging me 2 cents per hour per VM, and some small data transfers.  Granted, I used a 'T1 Micro' - with 1 CPU, .613 Gib memory and 8 GiB storage.  Still, I was quite happy.

On the last day of class, I split the students up into two teams and gave them large web projects to do, and spun up a 'T1 Micro' for each team.  I gave them some scripts they could run to create some swap memory if they needed it.  They ran those scripts right away, and within an hour (with 9 people uploading files and content into those systems continuously) those T1 Micros CPUs pinned.  So I quickly imaged them over lunch and spun up an 'M3 Large' VMs (2 CPU's, 7.5 GiB memory) for each team and threw their images on.  I ran into one issue spinning up the new team VM's - I had to spin down the original Team VM's first before I could start the new ones because there is a limit/quota of 20 VMs on the 'free-tier' in AWS.   Aside from that and the changed IP addresses, the transition was seamless.  I was a happy camper and now that the students had responsive VM's, so were they.

My total bill for the week - $38!  A colleague pointed out that I probably could have added auto-scaling to those team VM's and increase the CPU and memory in place, without losing the current IP's.  He's right, I probably could have, but I didn't have the experience and didn't want to waste class time (and potentially lose the student's work) by trying something I didn't know how to do.  All in all, I was very impressed with my first run of EC2 in AWS.  It was very reasonable, responsive, and easy to use.  I'd definitely do it again.

Random Thoughts About the DevOps Movement in IT - Part 2

The Phoenix Project (image of the book on the left) was recommended to me over a year ago as a good read about DevOps.  It emphasizes the benefits of Dev Ops to management, shareholders, and a company.   Impressions I had of the book were:

• Its a great, fun read, however it skims over the difficulties and trials of the actual automation of the technical systems and software development process - where the rubber really hits the road.  It seems to practically romanticize the idea of automation in IT a bit - like DevOps is the goose that will lay your golden egg.  Unfortunately, there's significantly more work to get that golden egg in my experience.
• It also glosses over how to get the Security team on board with what DevOps wants to do.  In many companies, the Security team holds the trump card and if they decide to change all your certs from 128 bit encryption to 2048 bit encryption (don't laugh, I've seen it happen and we had to regenerate certs for all applicable servers in all envs).  Their wish is your command unless you can convince someone influential that that kind of encryption is overkill in a non-prod environment.

Finding and Cracking That Golden Egg
Hurdles I've encountered enroute to the DevOps 'golden egg' are:

• Silo-ed Application Projects.  
• Lack of consistent naming conventions for deployment artifacts and build tags/versions is majorly detrimental to the automation process. 
• Lack of understanding of the dependencies between application projects (API contracts, or library dependencies).  If you don't understand your dependencies between applications, they may not compile corectly, or they may not communicate properly with each other.
• Lack of consistent development methodology and culture between teams.  If you have one team that doesn't get behind the new culture, and they insist on manually deploying and 'tweaking' their code/artifacts after deployment, they risk the entire release.  Getting everyone on board with culture is challenging.  
• Overusing a tool.  When you have a hammer (a good DevOps tool), everything is a nail.  Generally, many of the DevOps tools out there all have their niche.  You could potentially use Chef, Puppet, or Ansible to do all your provisioning and automation.  Is that the best solution though?  I'm inclined to say no.  How much hacking and tweaking are you having to do to get all your automation to work with that one tool?  Use the tools for what they are best at, what they were originally made for.  Many of these tools have Open Source licenses and new functionality is being added to them all the time.  While it might seem that the new functionality is turning your favourite tool into the 'one tool to rule them all',  you might have a big headache getting there, trying to push a square peg in a round hole.  
• Lack of version control.  Everything must be stored in a repository.  The stuff that isn't will bite you.  VM Templates, DB baselines, your automation config - it all needs to go in there.
• DB Management.  All automated DB scripts should be re-runnable and stored in a repo.  
• Security.  How to satisfy the Security Team?  How to automate certificate generation when they are requiring you to use signed certs?  How to manage all those passwords in your configuration?
• Edge Cases.  Any significantly sized enterprise is going to have/require environments that land outside your standard cookie cutter automation.  Causes I've seen for this are:
• Billing cycles - we needed a block of environments that were flexible with moving time so QA didn't have to wait 30 days to test the next set of bills and invoices.  
• Stubbing vs. Real Integration - Depending on the complexity of integration testing required, there may be many variations of how your integration is set up in various environments - what end-points are mocked vs. which ones point to some 'real' service. 
• New Changes/Requirements - Perhaps new functionality requires new servers or services.  This can make your support environments look different that your development environments.
• Licensing issues.  When everything is automated, it can be easy to loose track of how many environments you have 'active' versus how many you are licensed for.  License compliance can be a huge issue with automation - check this interesting post out 'Running Java on Docker?  You're Breaking the law!'
• Downstream Dependencies   This is where the Ops of DevOps comes into play.  Any downstream dependencies that your automated system might have need to be monitored and understood.  You can't meet your SLA with your client if your downstream dependencies can't meet that same SLA.  Important systems to consider here are: LDAP, DNS, Network, your ISP, and other integration points.
• YAML files.  Yes, perhaps they are more terse than storing your deployment config in XML.  However, I'm at a bit of a loss to see how they are better than a well named and formatted CSV file.  Sure, you can 'see' and manage a hierarchy with them.  But you can do the same in a CSV file with the proper taxonomy.  YAML files utilize more processing power to parse and have extraneous lines in them because their trying to manage (and give a visual representation of) the hierarchy of the properties.  I've seen YAML files where these extra lines with no values account for a significant percentage of the total lines in the file, making the file more difficult to maintain and prone to fat fingers.  Several major DevOps tools use these files and I really can't see a good reason why except that they were the 'new, cool thing to do.'

Random Thoughts About the DevOps Movement in IT - Part 1

DevOps related books I have in my library currently

Almost 5 years before DevOps was given the name it has today, I helped implement a modern DevOps solution at a company I was consulting with.  While our enterprise wasn't huge, and we didn't have a lot of integration (compared to some) it was a pragmatic solution that used Kickstart and Ant with a PHP/Mysql database.  We had Continuous Integration (CI) and Continuous Delivery (CD) - members of our QA team could select an environment, select a build, press a button, and the deployment happened.  Our environments at the time consisted of JBoss containers on Unix machines, and IIS web-apps and services on Windows machines.

Software development methodologies and practices have evolved a lot since then.  It's been a challenge to keep up.  I'm a pragmatic guy, and I thought one could get pretty much all the functionality needed for an automated build and deployment stack just using Ant, CruiseControl, Kickstart, VMWare, and some helpful API's.  Clearly I was wrong.

Cause for Pause
Without a doubt, the DevOps movement has captured the imagination of many a software developer.  Just look at all the tools out there now!  Chef, Puppet, Bamboo, RunDeck, Octopus, Docker, TeamCity, Jenkins, RubiconRed, Ansible, Vagrant, Gradle, Grails, AWS...  I could go on.  I'm beginning to question whether or not the polarization and proliferation of these tools has been helpful.  I find many companies looking to fill a Devops role are asking for resources who have experience with the specific tool stack they are using.  That must make human resource managers pull their hair out.  Even personally, I'm concerned about hitching my wagon to the wrong horse.  If I decide to accept a position with a company who is using a Chef/Docker stack (for example), but the industry decides that Ansible/Bamboo is the holy grail,  have I committed professional suicide?  Probably not, but it does make one consider job opportunities carefully.

Looking Forward
This P and P (proliferation and polarization) of DevOps tools makes me wonder what the future is going to look like for the DevOps movement.  Consider what Microsoft did with C#.  Instead of continuing to diverge and go their own way with the replacement for VB, they created a language with a syntax that essentially brought the software development industry back together (in a way).  Brilliant move.  Java developers quickly ported their favourite tools/frameworks over to C# and suddenly developing with Microsoft tools was cool again.  It would sure be nice if something like that happened with the DevOps tools.

Something else to consider for the future...  so far in my experience with overseas, outsourced teams, they have not yet embraced the DevOps movement.  As a result, they aren't as efficient or as competitive as they could be.  When they do jump on the DevOps bandwagon and truly tap into it's potential, it could be a game changer for people like me....

Tools to Watch

• Perhaps Amazon is the 'new Microsoft' with its expansive and ever expanding AWS tool stack?  There's definitely some momentum and smart thinking going on there.  They appear to have automated provisioning all wrapped up in a bow. 
• Atlassian is another company (I didn't realize they were based out of Australia) that is putting together DevOps tools that have a lot of momentum in the industry.  Their niche is collaboration tools.
• Puppet and Chef both have a strong foothold in the DevOps community.  Both of them are adding new features all the time, enabling them to automate the deployment and provisioning of more products and systems.  Many people use Puppet and Chef in conjunction with RunDeck or Docker to get all the automation they are looking for.

Books I want to read soon...

Here's two books I want to read soon:

The Nomadic Developer: Surviving and Thriving in the World of Technology Consulting







And

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley Signature Series (Fowler))

Some good links (to blog articles) on Configuration Management

Here's some good links to blog articles on Configuration Management.  The first is about the importance of a CM person in a company.  I would tend to agree with one of the comments that talks about the size of the company making a difference.  Obviously a smaller startup doesn't need a dedicated CM person.  However, larger enterprises like the one I'm at now with numerous development environments, multiple servers, a GIS infrastructure, a document management infrastructure, and a beomoth of a mainframe legacy system needs a dedicated CM team.  Here's the link:
http://evgeny-goldin.com/blog/2010/08/21/configuration-management-position/

 The other link considers adding database automated deployment to you CM system.  I would highly recommend when I contrast my experience over the last two years with my previous release management experience.  Previously we automatically deployed database schema and data changes. In my more recent experience the DBA group has flatly declined any encouragement and offer to do this.  The difference in the amount of time to do database refreshes and debug issues with them is stark - like hours vs. days.  Here's the link:

http://www.simple-talk.com/sql/t-sql-programming/when-database-source-control-goes-bad/

.Net Self Serve Automated deploy

There are a couple of prerequisites that need to be in place in order for a self serve automated deployment system to work.
1. You should have a Nant, MsBuild, Visual Build, or a similar type of scripting file that is doing deploys for you already - see my previous posts on doing that here
2. Your folder structure for every project you want to deploy should be similar. To have a good Self-Serve Deploy tool, it needs to work on the assumption that directory structures are consistent for certain types of projects. This could be set up using your repository tags directory, or automatically copying deployment files to the required folder structure after the build is successful. A folder structure I've used in the past is b:\deployment\projectname\version\ My build file(s).  Any other required folders and files reside in the version folder with a consistent folder structure.
3. You need either a properties file or a DB table to manage which tag/version has been deployed to which environment. We've used a symlink in the past to denote the current 'tip' build. This example will use a file. The file is structured like:

<versions>

   <serverType>

       <server name="blahboxType" version="1.2" />

       <server name="fooboxType" version="1.0" />

   </serverType>

   <blahboxType>

       <project name="MyProject" version="3.1.0.23" />

       <project name="MyProject2" version="3.1.4.11" />

   </blahboxType>

   <fooboxType>

      <project name="fooProj1" version="6.2.123"/>

      <project name="fooProj2" version="6.3.92"/>

   </fooboxType>

</versions>

Once the deploy is finished to a particular environment, logic in the deployment file automatically updates the respective project in the appropriate version file.

Here's a bit of sample code for a client side asp page for a Self Serve app.  It passes some values to a server side asp page (deploy.asp) so that page will know which version of which project to deploy into which environment.  The onchange event of the env select control called an AJAX function that retrieves the currently deployed version of the project from the versions.xml file. 

<form action="deploy.asp" method="get" name="MyProject">

  <input name="project" type="hidden" value="MyProject" />

  <input name="serverType" type="hidden" value="blahboxType" />

  <select name="version">

     <option value="">--Select--</option>

     <option value="" & version & "">" & version & "</option>

  </select>

<select id="MyProject" name="env" onchange="getVersions('MyProject',this.value)">

    <option value="">--Select--</option>

    <option value="Test1">Test 1</option>

    <option value="Test2">Test 2</option>

    <option value="SB">Sandbox</option>

    <option value="STG">Staging</option>

    <option value="PRD">Production</option>

</select>

<div id="MyProjecttarget" style="color: green;">

-</div>

<input type="submit" value="Deploy" />

</form>

Then on my deploy.asp page, the request parameters are set to variables which are then used to dynamically build the path to the deploy file (in this case a visual build file called Deploy-<serverName>-<component>.bld)  This could be a nant, msbuild, or ant file as well, you'd just have to change which executable you're using in the executeString.  The command is then executed in a Windows Shell and if it has a return code of '0' it was successful.  Successful or not, we do some very simple logging to show who tried to deploy what, where, and when.
Here's the code:

 ' need a longer server timeout setting so our deploy can finish

     Server.ScriptTimeout=1200

'instantiate some vars

    env = ""

    version = ""

    project = ""

    severType = ""

   

'set the vars

    set version = Request.queryString("version")

    set env = Request.queryString("env")

    set project = Request.queryString("project")

    set serverType = Request.queryString("serverType")

    'gets the user

    Set WSHNetwork = CreateObject("WScript.Network")

    user = WSHNetwork.UserName

    if version="" then

        Response.Write("Please ensure you chose a version <br/>")

    elseif env="" then

        Response.Write("Please ensure you chose an environment")

    elseif project="" then

        Response.Write("error occured - project required")

    elseif LCase(env)="PRD" then

        Response.Write("We don't want to deploy to Production from this app")

    else

        Response.Write(version & " " & env & " " & project)

'set up a dynamic path here to the deployment file (nant or ant or visual build file)  This string assumes the

'build file is taking a parameter called Env - the environment to deploy to

        execString = "<pathToVisualBuildExecutable>\VisBuildCmd.exe /b " & Chr(34) & version & "\visual build\Deploy-" & serverType & "-" & component & ".bld" & Chr(34) & " Env=" & env

        Response.Write("<br/>" & executeString)

        set oFs = server.createobject("Scripting.FileSystemObject")

            set oTextFile = oFs.OpenTextFile("<pathOfFileToLogTo>\deploylog.txt", 8, True)

   

        Dim WshShell

        Set WshShell = CreateObject("WScript.Shell")

        proc = WshShell.Run(executeString,6,true)

        if proc="0" then

            response.write("<br/><br/><div style='color:green;font-size:16px;'>Deploy of build " & component & " " & version & " to " & env & " as " & user & " was successful")

            dt = now()

            logtext = dt & ":  Deploy of build " & component & " " & version & " to " & env & " was successful" & vbCrLf

            oTextFile.Write logtext

            oTextFile.Close

        else

            response.write("<br/><br/><div style='color:red;font-size:16px;'>Deploy of build " + component + " " + version + " to " + env + " failed using user " + user)

            dt = now()

            logtext = dt & ":  Deploy of build " & component & " " & version & " to " & env & " failed as user " & user & vbCrLf

            oTextFile.Write logtext

            oTextFile.Close

        end if

        response.write("<br/>Deploy times and results are logged at <pathOfFileToLogTo>\deploylog.txt")

        set oTextFile = nothing

        set oFS = nothing

        set WshShell=nothing

    end if

        response.write("<br/><a href='javascript:history.back()'>Back to deploy page<a/>")

Automatic Deployments taken to a new level

I was doing some research for an article I was thinking about writing on Automated Deployment and came across this fantastic blog post by Grig Gheorghiu. He talks about how OpenX uses Amazon EC2 (virtual machine cloud) to automatically provision and terminate servers of different types, different monitoring solutions they use, and in another post he talks about the different automated deployment technologies that can be used (push vs. pull, ruby, python, etc).
I personally liked the comments about 'never having to log onto a production server', and "There's nothing like real-world network traffic, and I mean massive traffic -- we're talking hundreds of millions of hits/day -- to exercise your carefully crafted system infrastructure."
All the comments at the bottom are really good as well.

The importance of Process (adodb.dll and SSRS)

We have a number of different development environments. Deployments follow a natural progression through these environments before they get put into production. We were getting reading for a release and doing deploys and testing in our Staging environment (our env just before prod in our progression). One morning recently, our Nunit tests failed after a run, complaining about not finding adodb.dll registered.

We were surprised because adodb.dll is a Microsoft dll and we certainly weren't deploying any dll's like that - for that matter, we hadn't done any deploys in the last day or two. So why were our Nunits failing suddenly?

Quickly looking into the event logs, we discovered that SSRS (Sql Server Reporting Services) had been (unbeknowst to us) uninstalled on our transaction server during the evening. I went and spoke with the infrastructure group who did the uninstall and they got very concerned as they were ready to uninstall SSRS in production the next evening.

In the end, it appears that uninstalling SSRS did, in fact, remove adodb.dll from the Gac on our transaction server. Our infrastructure group cancelled the uninstall from production. We are trying to impress upon them and others the importance of communicating changes like this even if the assumption is there is no impact. Fortunately, our process of percolating changes up in different environments managed to catch this change before it blew us up in production. However, it wasn't caught in the lower environments because it was unadvertised. Communication is key!

Musing on Automated Deployments

I have been a key player in big automated deployment strategies in two significantly sized organizations now. One used ant with a java code base, the other used Visual Build with a VB code base. Both of these implementations deployed multiple dependent projects onto a variety of server types into development, testing, staging, and production environments. With the exception of prod, each environment had more than one instance of the environment running.

Some of my earlier musings on automated builds and deploys can be found by clicking here.

One would think that an automated deployment would be deterministic. In other words, given the logic in the deployment file(s), it should deploy the same every time. Surprisingly, we have found this not always true. Since many of these deploys are pushed to remote boxes, hiccups in the network end up throwing a proverbial wrench into things. And (again) surprisingly, these can occur more often than I would've though. We actually blamed these hiccups on increased solar activity for a while. I have no solutions to getting around these network hiccups, except to say that if you see your deployments failing consistenly at a certain time during the day, schedule them for another time. Our Sunday evening deploys lately have always been failing. Yet when we kick them off Monday morning (with no changes to deployment logic) everything this fine. We're thinking that there's possibly a weekly batch job or two that are running during our Sunday deploy that is bogging the network down....
I've also seen automated deploys act inconsistently (only with Windows) with registering dll's in the assembly. We can deploy and Gac things fine onto our bare metal, VM servers with no problem. Yet, when we deploy the same software onto a legacy hardware server where the dll's are already gac'ed (our deployment logic un-gac'ing and re-gac'ing the dll's) they fail to register it seems. I've wondered if perhaps the deployment moves through all the logic too fast? The command is definitely correct. Sometimes we'll even see the dll's in the assembly folder in the GUI, but the application can't. Manually registering them from the command line fixes the problem, but we shouldn't have to do that.

Something else to consider when implementing automated deploys - do you want to deploy everything from scratch (bare metal deploy) or do you want to deploy onto an already working image or server (overlay deploy)? I have tossed this question around a number of times. I think the correct answer for you depends on how you answer the following questions:
Are you thinking about deploying to a system that's already running in production? Are all the configurations that make that production system work documented? Are you confident that you could rebuild the production server and getting it running without any major problems? If you answer 'yes' to all of these questions, then you could probably save some time and implement overlaying automated builds. If you are starting work on a greenfield (new) application or you aren't confident that you could rebuild you production server, then you should probably consider bare metal deploys. Bare metal deploys done properly essentially become self documenting DRP's.

More IIS automated deploys

Been working more with IIS automated deploys and found a couple of good posts in David Wang's Blog along with some other good blog posts.

This one is related to how to manipulate IIS list data. He provides a handy vbs script to do this that worked nicely for me in adding a new application mapping to IIS.

This one talks about IIS App pool crashes - fatal communication errors between the Application Pool and IIS.

This one talks about app pool recycling and IIS availability. There's some great conversations past David's article further down the page.

This one (not David Wang) shows how to change COM+ MSDTC settings programmatically. This script worked great for me as well.

This one (Ian Morrish) shows how to manipulate COM+ Security launch and activation permissions using DComPerm from the windows SDK.

With all of these scripts, I found I had to carefully read the directions and make sure I had the parameters that were being passed in correct. I was often passing in erroneous parameters on my first couple of tries.

Windows bug with adding users to a group

I was using a script to try and add users to a local group today in an automated build script. I was sent a script that worked, but when I plugged in my values, it didn't. I always got a script syntax error, like one of my parameters was not in the right place, or had spaces or something. Well the group name did have spaces, but that wasn't my issue.

It turns out that there is a documented Windows bug (http://support.microsoft.com/kb/324639) that limits the username to be added to only 20 characters. Any more and it won't run.

XML manipulation in Visual Build (and vbscript)

We've been working on basically turning our Visual Build deployment files into DRP (Disaster Recovery Plan) scripts. Essentially, everything required to get our applications running on a 'blank' server box is documented as a config, permission, or push in our Visual Build files. This has required updating then machine.config and web.config files that reside under windows/microsoft.net/framework/v1.1.4322 or v2.0.50727/config at times. Sometimes we're adding attributes to existing element, sometimes we're adding entirely new elements.

It seems to me that adding entirely new elements in Visual Build (using the Write XML Action) has a bug in it. It will add new attributes to existing elements no problem. But it won't add new elements. So we hacked around that and used the 'Run Script' Action with vbscript to add elements to our files. To do this in vbscript it looks something like this:

Dim objXMLDoc, objNewNode, objText, strXPath, objParentNode, objChildNode, objCaseExist

Set objXMLDoc = CreateObject("Microsoft.XMLDOM")
objXMLDoc.async = False

' load the XML file - make sure to include the fully qualified path
fSuccess = objXMLDoc.load("\\%TARGET_SERVER%\c$\WINDOWS\Microsoft.NET\framework\v2.0.50727\config\web.config")
If Not fSuccess Then
wscript.echo ("error loading XML file")
Else
wscript.echo ("XML file loaded")
End If

' set to proper node
Set nodeList = objXMLDoc.getElementsByTagName("configuration/system.web/compilation/buildProviders")
If nodeList.length > 0 Then

Set objParentNode = nodeList(0)

' add the SGAS SecurityDisabled element/node
Set objNewNode = objXMLDoc.createElement("add")
objNewNode.setAttribute "extension", ".uplx"
objNewNode.setAttribute "type", "System.Web.Compilation.PageBuildProvider"
objParentNode.appendChild(objNewNode)
objParentNode.appendChild objXMLDoc.createTextNode (vbCrLf)

set objNewNode = Nothing

else

wscript.echo ("node list empty")

end if

' save the XML file
objXMLDoc.save("\\%TARGET_SERVER%\c$\WINDOWS\Microsoft.NET\framework\v2.0.50727\config\web.config")

Automated Windows Deploy scripts and tricks

Permissions (domain is optional):
cacls drive:\folder /T /E /G "domain\username:C" - :C modify permissions
cacls drive:\folder /T /E /G "domain\username:R" - :R read permissions
cacls drive:\folder /T /E /G "username:F" - :F full permissions

net localgroup "IIS_WPG" "domain\username" /add - adds user to IIS_WPG localgroup

IIS configs:
Set the friendly name for an IIS website:
cscript drive:\path-to\adsutil.vbs SET W3SVC/WEB_SITE_Number/ROOT/AppFriendlyName IIS_WEB_SITE_Name
Set the port of a website:
cscript drive:\path-to\adsutil.vbs SET W3SVC/WEB_SITE_Number/ServerBindings ":8080:"
Install an IIS Application pool:
cscript drive:\path-to\adsutil.vbs CREATE w3svc/AppPools/AppPoolName MyApplicationPool
Set identity type on AppPool
cscript drive:\path-to\adsutil.vbs SET W3SVC/AppPools/WebSiteName/AppPoolIdentityType 3 (2 is the predefined network service account)
Set appPool username
cscript drive:\path-to\adsutil.vbs SET W3SVC/AppPools/WebSiteName/WAMUserName username
Set appPool password
cscript drive:\path-to\adsutil.vbs SET W3SVC/AppPools/WebSiteName/WAMUserPass password
Delete a web site:
cscript c:\windows\system32\iisweb.vbs /delete "WebSite name"\
Create a web site:
cscript c:\windows\system32\iisweb.vbs /create "IIS File Path" "WebSite name" /dontstart
Set server bindings
cscript drive:\path-to\adsutil.vbs SET W3SVC/WEB_SITE_Number/ServerBindings :80:myWebSite :80:myWebSite.com
Set virtual directory permissions
cscript drive:\path-to\adsutil.vbs SET W3SVC/WEB_SITE_Number/ROOT/vDir_Name/AccessFlags 513
Set enable anonymous user for vDir
cscript drive:\path-to\adsutil.vbs SET W3SVC/WEB_SITE_Number/ROOT/vDir_Name/AuthAnonymous TRUE
Set vDir anonymous username
cscript drive:\path-to\adsutil.vbs SET W3SVC/WEB_SITE_Number/ROOT/vDir_Name/AnonymousUserName username
Set vDir anonymous password
cscript drive:\path-to\adsutil.vbs SET W3SVC/WEB_SITE_Number/ROOT/vDir_Name/AnonymousUserPass password
Set vDir to use integrated windows authentication
cscript drive:\path-to\adsutil.vbs SET W3SVC/WEB_SITE_Number/ROOT/vDir_Name/authNTLM TRUE
Set vDir ASP.NET version
C:\WINDOWS\Microsoft.NET\Framework\version_number\aspnet_regiis -s W3SVC/WEB_SITE_Number/root/vDir_Name

Windows and Automated builds

We've run into a couple of interesting situations with Windows with our automated build scripts that I wanted to document. Both of them have to do with inconsistencies with what the Windows GUI is saying about a configuration versus what the operating system is actually using for a configuration.
Both of these situations were discovered while we were trying to do an expansive upgrade of our code to use the new .NET framework. In the first instance we were manually trying to install/uninstall dll's from the gac on a particular box. We manually dragged and dropped a particular dll into the c:\windows\assembly folder and the GUI would say everything was successful with registering the dll in the gac. However, check it from the command line, or try to run something in the application against the newly gac'ed dll and the OS didn't seem to see it. The only way we could get the OS to successfully see the change was to gac the dll from the command line.
In the other instance (which is sort of similar in nature), our automated deploy scripts are creating a Virtual Directory underneath an IIS web site on a remote server. After the Vdir is created, we had a 'step' that automatically updated the ASP.NET version on the Virtual Directory fo 2.0.50727 (C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis -s W3SVC/%WEB_SITE_NUMBER%/Root/%WebSiteName%). This script ran successfully, and the ASP.NET tab in the VDir on the web site displayed the change. However, IIS didn't actually see the change and continued to run the web site on the old 1 .1.4322 version - even though the GUI in IIS said it was. We knew this because the C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files folder didn't have the new corresponding virtual directory folder (this is where IIS keeps the compiled pages - it's like the 'working' folder in Tomcat). It seems the IIS metabase was updated, but the registry was not. I have a hunch that this might be because during that particular deploy, we left IIS running while we did the deploy. Perhaps shutting it down would make the update work properly through to the registry.

Don't assume anything

We are doing a big upgrade of our components in our upcoming release. The security structure of our organization is fairly complex with users, roles, user groups assigned to roles, boxes in the domain, boxes outside of the domain, and a whole other layer of users and roles at the database level. Managing security for this release has been a hassle. It seems in every environment there are differences with who is assigned to what group, which users and which groups have 'these' permissions on 'this' folder, etc.
Thinking we had most of the issues documented, automated and swept under the rug, we deployed into our staging environment. Surprise, surprise, there were permissions issues there. The really interesting thing was these turned out to be new issues. Well, we spend the good part of the day trying to see if we missed a permission on a folder somewhere. In the end, it turned out that our architecture group (who is responsible for the shared components code and the overall enterprise architecture) had some code for applying special permissions that was hard-coded to be applied against ONLY our staging and production environments.
Shock.
I guess it goes to show, don't assume anything - no matter who is responsible for code or where the error seems to be, or how stable you think your configuration are.... We also discovered that that particular componenet had a different security setting for the COM+ in the Staging env compared to the Testing env.

Automatically deploying IIS web apps

We've been working on automated builds and deploys at work. All of our web apps run on IIS and configuring 'good' automated deploys for these applications has been challenging. But I think we're seeing the light at the end of the tunnel now.

Microsoft provides a good number of support scripts (in vbs) to configure IIS, App Pools, and Virtual Directories from the command line. Some of them come with the IIS installation, some of them we had to download - if I remember correctly.

There are two 'sets' of these CLI vbs scripts that I'm aware of that work with IIS. One is found in the ../inetpub/adminscript folder. There's a bunch of vbs scripts in there. The other set is found in c:/windows/system32/iis*

Here's some examples of what you can do with these scripts:

• create a virtual directory: cscript c:\windows\system32\iisvdir.vbs /create webSiteName virDirName virDirPath /s serverName
• set perms on a virtual directory: c:\inetpub\adminscripts\adsutil.vbs SET w3svc/webSiteNumber/Root/virDirName/AccessFlags permNum (like 513)
• set ASP.NET version for the virtual Directory c:\windows\Microsoft.net\framework\v2.0.50727\aspnet_regiis -s w3svc/webSiteNumber/root/virDirName
• and tons more...
  

You can view all the website numbers in the IIS manager. They show up in a column.

Visual Build

I've been working with Visual Build in a .NET environment off and on for almost the past year. When we decided to go forward with Visual Build, I thought it would be a more painful process than it turned out to be. Our technology stack includes ClearCase, Subversion, Ms SQL, PSExec, VMWare, VB.NET, IIS, CruiseControl, Nant, Nunit, and a bunch of 3rd party tools and servers related to document management.
Visual Build provides a bunch of example build files that are quite helpful in getting all kinds of different functionality going. We automatically compile, unittest, deploy (to remote boxes), and run verification tests using Visual Build. There a numerous other interesting smaller tasks that we've got Visual Build to manage for us like setting perms on remote boxes, stopping and starting servers, services and com+ objects, performing baselines, checkins/outs, and updates of clearcase streams and views, etc.
Some of the gotchas we've discovered in our work with Visual Build:
- managing windows (child build files that get spawned in a complex build process) and logs is not so trivial. If they aren't managed correctly, your build will stop without much notification as to why. We found that piping the output from the build to a separate file worked for us with logging. With windows, we found that matching the build context, waiting for completion, running the GUI app in silent mode, and not closing the GUI app on failure were what made things tick.
- psexec needs to be on the path on the box you're running Visual Build on.
- CruiseControl integration was relatively easy - we just used the command line with options:
<tasks>
<executable>C:\PathToVisualBuildInstallation\VisBuildCmd.exe</executable>
<baseDirectory>D:\pathToBuildFile</baseDirectory>
<buildArgs>/b "XXX.bld" -other args to pass to buildfile go here</buildArgs>
<buildTimeoutSeconds>1000</buildTimeoutSeconds>
</tasks>