Books I want to read soon...

Here's two books I want to read soon:

The Nomadic Developer: Surviving and Thriving in the World of Technology Consulting







And

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley Signature Series (Fowler))

Thoughts on Design

I recently re-read The Last Lecture by Randy Pausch and was impressed with what he did with his classes that focused on User Interface Design.  The first day of class he would bring in an old VCR, put it on a table in front of class, and without saying a word he'd pull a sledgehammer from behind his desk and destroy it.  Needless to say it got the class's attention.  The message Randy was trying to convey in this demonstration is that poor end user design is detrimental to the product.  End User Design is critical.

There is another person who is a strong proponent of excellence in End User Design.  He is still alive today and he's be a driving force in perfection in Design in the IT industry.  He designs have been flying off of store shelves for the last 25 years or so.  If you haven't guessed yet, I'm thinking of Steve Jobs from Apple. I ran across a very interesting article about him in Bloomsberg Businessweek.  It's basically an interview with John Sculley, the CEO of Apple from the mid-80's to the mid-90's, and what it was like 'Being Steve's Boss'.  He talks a lot of Steve's philosophy and background in design, and Steve's influence on Apple in this area.  It's a great article.

It got me thinking...  In software I have not been super stringent about software design on the applications I've written.  Let me clarify - object oriented design is a must.  The train of thought I'm seeking to follow here is 'is it a requirement to implement design patterns in your code everywhere possible?' A major reason for this is I've worked with a number of people who think their job is refactoring and renaming classes and packages in a code base - to the determent of the project.  I often tell the students in my classes that there are numerous ways to 'skin a cat' as it were, when coming to a working solution in code.  I wonder though, is there always one perfect way?  Is this something to strive for?  Does it matter as long as the application works the way the client wants it to?  What do you think?

Personally, I think that the answers to these questions depends on a lot of things, not the least of which are your client, the project, and your team.  Elegant solutions look and feel so good when your done - both for you and for you client.  Perhaps the motivation and context for the project need to be analyzed before a decision is made whether or not you should 'shoot for the best designed, most elegant solution' possible?  Obviously if your project has constraints like budget, resources, and time, there's a limit to how much effort you can put into evolving your project into the best design in can be.  That is one big reason why design patterns and frameworks are so prevalent (and pretty much a requirement) in software engineering.

I like to read/sign out Fast Company from the Library - I'm actually thinking about getting a subscription.  Anyway, they have an annual 'Masters of Design' issue that is a lot of fun to read.  In it they highlight different world class designs and their designers.  Here's a link to their MOD tag cloud.

Windows OS Performance Guidelines

These are notes from a workshop I was able to be a part of a couple of weeks ago.  The workshop was on Windows Performance Monitor and monitoring vital signs on Windows Servers.  The windows performance monitor has a multitude of counters that a person could potentially monitor on a server.  I'm just going to point out a few critical ones that were highlighted in the workshop and what their tolerances are.

Some points to note before I get into these values:  

1. Don't always look at the graph first. The counter graphs can be scaled right out to lunch so they aren't necessarily a good first glance indicator.  For a particular counter, your focus should first be on the Minimum and Maximum values in perfmon just below the graph.
2. In the past, I would tend to keep an eye out for sympathetic counter relationships on the graphs.  However, keeping an eye out of inverse counter relationships is also a good idea (where one counter could be decreasing in value while another is increasing)
3. Counters can get corrupted.  Apparently this happens more often than one would think.  They can be rebuilt - directions are in this KB post.
4. You can attach a PID (process ID) to a perfmon counter in an OS older than Windows Server 2008 by modifying reg keys. Details are in this kb article. Versions of perfmon on 64bit OS's come with this already set up.

On to the objects/counters of note:

• Process Object
• Handle Count - greater than 500 handles may point to a problem.  
• Private Bytes - greater than 250MB could be a problem.  I've seens procs over a Gig and they definitely were a problem (they can get that high).               
• Working set - greater than 250MB could be a problem
• Thread Count - Greater than 500 threads needs to be watched to ensure they aren't increasing over time
•  Processor Object
• Processor Time - all core instances.  _Total can get you an overall trend.  Greater than 91% utilization is potentially an issue.
• Network Interface Object - you need to know what the spec is for your network interface to determine it's capacity.  Anything over 80% of capacity could point to a problem
• Current Bandwidth - help you determine the NIC's capacity
• Output Queue Length - greater than 2 is an problem
• Bytes Total - greater then 65% of capacity utilized is past the warning threashold (blinking read with siren)
•  Memory Object
• Free System Page Table Entries - the higher the better here.  Lower than 5000 is considered critical.  I've seen boxes 'run' (aka hobble) around 2500.
• Available Megabytes - again higher here is desirable.  Less than 100 MB or 5% free is very problematic
• Pool Non Paged Bytes - greater than 80% consumed is out of spec (not at all good)
• Pool Paged Bytes - Same as non-pages bytes.  Anything between 60-80% should be watched.
• Logical and Physical disk Objects - they have the same critical counters so I've put them together here
• Idle percentage - 19 to 0 percent is critical. Anything over 55 is warning.
• Current or Avg Disk Queue Length - 3 to 31 in the queue - you need to keep an eye on it.  greater than 32 is an issue
• %Avg Disk Sec Read or Write - 25ms and above is critical

 Other interesting points of note:
- Mark Russinovich was the original developer of Perfmon.  His blog is apparently pretty good and our facilitator was very impressed with him.
- Windows Server 2008  does processor/core parking.  This means the server will 'retire' (effectively turn off to save power) CPU's when there isn't a heavy load on the box.  Our facilitator told us that one ISP moved all their boxes to Windows Server 2008 for this reason and their power bills were 15% less a month. 

Newbie Advice

Having gone through an IT technical school, I realize they push a lot of information and technology at you. There's a lot to assimilate in less than a year (depending on the school). On this page I'd like to point out some technologies that perhaps weren't covered much in classes that are widely used by good shops in the industry, and also offer some career advice for the future. Before I delve deeper, let me say that this information is based on my experience, which is mostly on the Java technology side of the fence.

Technical Heads Up

1. Design Patterns. One thing many of the technical schools don't have time to cover in their curriculum is code design patterns. It doesn't matter if you doing .NET development or Java development, design patterns can really make a difference in organizing, maintaining, and the reusability of your code. A great book to pick up for an introduction on design patterns is published by O'Reilly and called Head First Design Patterns. If you don't have access to the book, a good place to start is to google some popular patterns that are being used in code now. Dependancy Injection is a pattern that is really big right now. Two frameworks that use it a lot are Spring and Google's Guice. Other pattern I've seen used frequently are the decorator pattern, the abstract factory pattern, factory pattern, the singleton pattern, and the proxy pattern.

2. Using a Repository. Using a repository, or a version control system, should be required by every development shop. Frankly, if you're in a company where they aren't using one, either push to get one installed, or start looking for another place to code. A repository is basically an application that functions as the premier source/bank/datahub/repository for all code. This application should be on a server with lots of disk space and accessable to all developers in the company. Repositories allow developers to manage code changes at many different levels allowing many developers to work on the same code base at the same time. Some repositories I've used in the past are Subversion, CVS, StarTeam, and Perforce. Googling any of these will give you more information.

3. Unit Testing. Unit testing is the development practice of coding 'tests' that validate individual classes or 'units' of code, to ensure they are functioning as required. This practice has developed over the past 4-5 years or so and has been found to be very effective in giving developers the confidence to refactor their code and ensure that it's functionality stays consistent over time. Some unit testing technologies are JUnit, JWebUnit, HttpUnit, DBUnit, and NUnit. 

4. Continuous Integration. Continuous integration is a process that automatically rebuilds and tests (with unit tests) an application or code base. Continuous integration employs an application like CruiseControl or Anthill Pro which is configured to automatically check out the code base from the repository, compile it, run it's unit tests, and sometime even deploy it to a test environment. It allows developers to check for integration problems in the code, and warns developers of broken code, allowing and reminding them to fix it ASAP.

Soft Skills and Career Advice

1. Change is GOOD. Once you've gotten your feet wet in the industry, being a Java developer in one company can mean something quite different that being a Java developer in another company. There is such a vast variety of Java technology that two Java developers can have quite different experiences in developing applications. Whole development processes and methodologies can be very different in differenct companies. One company could be flying by the seat of their pants, developing with not backups or repository, and manually pushing out code, while another is totally immersed in Agile methodology and outsourcing everything but development.... etc. Keeping this in mind, my advice is to not get comfortable in any one company but move around a bit - that is, after 18 months or two years at one place, get a different job. You'll be put into a position where you'll either learn a lot of new stuff, or you'll be mentoring. Either way, it'll be an experience that will likely teach you more then if you had stayed put. It'll also give you some motivation to brush up on your resume writing, interview skills, and expand you network. 

2. Walk & Talk While software developers, for the most part, end up working in cube farms, it should not mean that they need to stay in their cubby hole all day and do work. This may be just my style, but I find when I'm working on a big integration project or on a project with a lot of departments or teams involved, getting up and talking to people really helps things move along. Whether it's getting socializing a new means of managing your server configuration in the code base, or talking to a DBA about a script that you'll need to release with you new code, talking face to face is far more effective than emails, I've found. I generally use emails if I can't find the person, or I want to make sure there is hard documented proof about a decision that is being made, or if I want to include a bigger audience in my communication. Successful software development is more about relationships that a lot a people realize. Relationships are more apt to be cultivated face to face... plus it's good exercise.

 

Career Paths in IT

I've been teaching at SAIT again this week.  Generally before every class gets going in the morning I like to do a little 'blurb' on concepts or thoughts that aren't covered in the course the students get.  One of the things I find is that the vast majority of students (and the population at large for that matter) don't realize how many different career paths there are in just the software side of IT.  Guaranteed there are some students in the course that I teach who, by the time I get the chance to teach them, are very concerned they made the wrong decision by getting into software programming.  They don't feel like they are cut out for programming.  I try to put their fears at rest by suggesting that there are other opportunities for them - careers that can utilize the coding experience that they are getting, but allow them to use talents that they are more comfortable with.  Here's a list (by no means exhaustive) of some other opportunities in the software field.

- Project Management.  Project managers who have coding experience or at least a technical background are a step ahead of PM's who aren't techies.  I myself have considered taking some project management courses or getting my PMP, but thought the better of it after seriously thinking about what I like to do at work.  PMs need to have a great soft-skill set.  While I don't think I'd have a problem with that, I would missing troubleshooting/solving problems.  They also have to be politically savvy and have patience for meetings.

- Business Analyst.  BA's also need to have a great soft-skill set, as they need to be talking with everyone (the business, the developers, and the PM) and writing down the requirements.  It also helps if they are technically competent, and they enjoy documenting/writing.  For some reason in Calgary right now, there have been a lot of postings for BAs over the past 6 months. I just found out today that a variety of institutions offer certificates/diplomas for a Business Analyst career path. (NAIT and Mount Royal University are two local options)

- Quality Assurance.  This field has been growing a lot over the past 8 - 10 years, and frankly I don't think projects that I've been on over the past 5 years would have been successful without our QA teams.  QA experts need to have a penchant for detail and process.  They need to be able to understand the business and relevant business rules well.  Most of the QA people I've worked with haven't had a certification, but you apparently can get more info on a software QA certification here.

- Release Management.  This is what I (like to think I) specialize in.  We ensure that teams are using a continuous integration server and QA has the builds they need for testing where and when then need them.  We have two main goals: 1. Keep downtime in production to a minimum and 2. Keep lost man days in QA to a minimum.  I like release engineering because I get to see the whole picture - I have to understand the code and it's dependencies, I need to understand the infrastructure, and I'm responsible for the product getting through all the environments to production.  For me (being a bit proactive), this has also lead to automating deployments of database objects and datascripts, server configurations, and code, as well as installing monitoring tools to ensure that every environment is ready to 'rock'.

- Security Analyst.  These dudes make sure the good people have access to everything they need access to, and the bad people are locked out.  They spend a lot of time combing though logs and configuring IPSec rules.  They also perform software audits on code, all kinds of different tests on web sites, and network audits.  I wouldn't mind moving my career this direction in the future as they also need to have a pretty good idea of the whole system/infrastructure/architecture.  Great web sites for more resources on these types of positions are here and here.

Even more specialized positions:
- GIS (Geographic Information System) developer or architect or analyst.  Working with digital 2d and 3d mapping systems.  Having some limited experience with this at the ERCB (deploying mapping systems that display reservoirs,etc in Alberta) I think this is pretty cool stuff.
- Search Engineer (Search Engine Optimization specialist)  I've dabbled in this off and on for my clients.  You definitely need to be on top of your game and know how the search algorithms are changing.
- Business Process Specialist - there's lots of software to specialize in here.  Tibco, Agile Point, SAP, JDEdwards, etc.
- Document Management Specialist - Sharepoint, Edocs, Documentum, etc.  This involves the automated management of documents - not the writing of them, but their storage.  Government bodies and law firms definitely have a need for these kinds of systems.

Another emergency software fix

A couple of weeks ago I had another client take advantage of my Emergency Software Troubleshooting services.  They had a web site maintained by a popular CMS system.  Unfortunately, their scheduled backups weren't as scheduled or robust as they thought and they were left without a website (or backend database) after their hard disk died.  I was called to help.

After some invesigation, we surmised we were left with:
- a bunch of .myd, .myi file backups for MySql (they were a couple of months old)
- another .sql file backup of the database 12 months old
- a file system backup of the site a couple of months old
- and another file system backup around 12 months old.

We rebuilt the database using a combination of the most recent myd and myi files.  This was a bit of a trick as the .frm files were missing in some cases.  What we did in those cases was create a dummy .frm file, rebuild the tables from the myd/myi files and when we did that we had mysql replace the frm file (it's an option you can enable during this operation).  We then used the data from the older database to populate the tables that didn't have the frm files.

With the file systems, it was a matter of patching together what worked between the two file system backups with the data we had in the database.  Since the old system was pretty much blown away, it was decided to upgrade the OS and the web server at the same time.  This meant reconfiguring the web server and ensuring the majority of the site worked.  In the end, the site was pretty much ready to go after 8.5 hours of work.

Licenses.Licx error with DevExpress in Visual Studio 2010

While refactoring a business solution at work to deploy using Visual Build rather than InstalShield, I ran into an issue with our DevExpress web components not seeing the correct licenses in the licenses.licx file.  After googling for a bit and discovering how the licenses.licx files are created and manged in VS2010, I started troubleshooting to see if I could resolve my issue - seeing the red text on every web page telling me that I'm using a 'trial' edition of the software when I clearly wasn't.
 
I deleted the file to get VS to recreate it.  That didn't seem to work for me.  I renamed it.  I checked permissions on it.  I investigated how it got deployed and whether I needed it on the server or not (if I remember correctly, you don't have to explicitly move it to the server).  In the end, my solution sort of knocked my socks off.  I deleted all of the content in the file and left it blank.  I then built my msi and deployed.  Imagine my surprise when the blank licenses.licx file got rid of my trial license message!  I don't know if this is a bug with VS2010 or it's integration with DevExpress 8.2.2, but my fix seemed like a total hack.  However we've tested this in multiple environments and it's solid.

The Phoenix and Projectors

Two business ideas in this post.  Actually one is a business model, the other is an idea.

The Phoenix
The business model that I ran across I'd like to call 'The Phoenix'.  Basically the idea is to take a dead business or business idea, and resurrect it.  The tale of the Twin Otter, a Canadian built Short Takeoff and Landing Utility plane is a classic example of this business model.  Details of it's life, death, and resurrection can be read on Wikipedia here.

In its first life, production ended on the Twin Otter in 1988.  The aircraft's versatility made it a bush pilot favorite and demand for the planes have increased.  Viking Air of Victoria, BC, picked up on this and bought the rights to build the plane again.  After nearly 20 years of being out of production, new Twin Otters are now being built in Calgary and the company has over 50 orders from all over the world.

Nov 14 update - I just ran into an article in The Globe and Mail about another brand that did something similar.  In fact, some work colleagues and I had been musing about this turn-around in the summer as we found some Pop-Shoppe bottles in a nearby convenience store.  Here's the link to the article in the Globe and Mail.


Projectors
The second idea isn't nearly as sexy.  Watching my project manager try juggle between a spreadsheet of software development stories and another of schedules and tasks, I got to thinking why can't there be a single projector with the ability to support and display like two monitors?  Or even right a button into the software/firmware of the projector so it can flip between screen instances.  For that matter, why hasn't windows come up with the ability to manage screen instances like Linux has? Maybe that functionality is there and I just haven't been paying attention...