Random Thoughts About the DevOps Movement in IT - Part 1

DevOps related books I have in my library currently

Almost 5 years before DevOps was given the name it has today, I helped implement a modern DevOps solution at a company I was consulting with.  While our enterprise wasn't huge, and we didn't have a lot of integration (compared to some) it was a pragmatic solution that used Kickstart and Ant with a PHP/Mysql database.  We had Continuous Integration (CI) and Continuous Delivery (CD) - members of our QA team could select an environment, select a build, press a button, and the deployment happened.  Our environments at the time consisted of JBoss containers on Unix machines, and IIS web-apps and services on Windows machines.

Software development methodologies and practices have evolved a lot since then.  It's been a challenge to keep up.  I'm a pragmatic guy, and I thought one could get pretty much all the functionality needed for an automated build and deployment stack just using Ant, CruiseControl, Kickstart, VMWare, and some helpful API's.  Clearly I was wrong.

Cause for Pause
Without a doubt, the DevOps movement has captured the imagination of many a software developer.  Just look at all the tools out there now!  Chef, Puppet, Bamboo, RunDeck, Octopus, Docker, TeamCity, Jenkins, RubiconRed, Ansible, Vagrant, Gradle, Grails, AWS...  I could go on.  I'm beginning to question whether or not the polarization and proliferation of these tools has been helpful.  I find many companies looking to fill a Devops role are asking for resources who have experience with the specific tool stack they are using.  That must make human resource managers pull their hair out.  Even personally, I'm concerned about hitching my wagon to the wrong horse.  If I decide to accept a position with a company who is using a Chef/Docker stack (for example), but the industry decides that Ansible/Bamboo is the holy grail,  have I committed professional suicide?  Probably not, but it does make one consider job opportunities carefully.

Looking Forward
This P and P (proliferation and polarization) of DevOps tools makes me wonder what the future is going to look like for the DevOps movement.  Consider what Microsoft did with C#.  Instead of continuing to diverge and go their own way with the replacement for VB, they created a language with a syntax that essentially brought the software development industry back together (in a way).  Brilliant move.  Java developers quickly ported their favourite tools/frameworks over to C# and suddenly developing with Microsoft tools was cool again.  It would sure be nice if something like that happened with the DevOps tools.

Something else to consider for the future...  so far in my experience with overseas, outsourced teams, they have not yet embraced the DevOps movement.  As a result, they aren't as efficient or as competitive as they could be.  When they do jump on the DevOps bandwagon and truly tap into it's potential, it could be a game changer for people like me....

Tools to Watch

• Perhaps Amazon is the 'new Microsoft' with its expansive and ever expanding AWS tool stack?  There's definitely some momentum and smart thinking going on there.  They appear to have automated provisioning all wrapped up in a bow. 
• Atlassian is another company (I didn't realize they were based out of Australia) that is putting together DevOps tools that have a lot of momentum in the industry.  Their niche is collaboration tools.
• Puppet and Chef both have a strong foothold in the DevOps community.  Both of them are adding new features all the time, enabling them to automate the deployment and provisioning of more products and systems.  Many people use Puppet and Chef in conjunction with RunDeck or Docker to get all the automation they are looking for.

Teaching an IT class (JavaScript, XML, C#, VB, CMS/PHP)

I've taught part-time at SAIT for 11 years now.  I've had more than 30 different classes with 5 different courses.  Recently I've been challenged by a colleague (also a part-time teacher here at SAIT) to make my courses better and more interactive.  While I thought my courses were pretty interactive to begin with, his ideas have definitely pushed me further in a good way.

All the courses I've taught have been introduction to different code languages or working with code:
Visual Basic, C#, XML, Javascript, PHP, and Content Management Systems like WordPress and Drupal.  Up until recently, my courses had been - at a high level - structured as follows:  I demonstrate a new concept/code idea to the class and I have them follow along.  Once they are reasonably comfortable with implementing it (perhaps an exercise or two of following along like this) I give them an exercise that they have to try and do themselves.  Sometimes I'll add a twist to make them think a bit.  For the most part this has worked great.  Its looked like this (an example with XML):

• Intro to the Language and syntax - I'll go through several power point slides that discuss: 
• What does well formed XML mean?  
• How do you ensure your XML is well formed?  
• What are the keys/rules to well formed XML? 
• I'll ask the class questions as we go along and get them to answer.
• We'll code a simple XML file together.  I'll make intentional mistakes along the way and see if the class is paying attention, or just call them out and ask 'Do you think this is right?'  I will also put comments in my code so the students can refer to them later.  Once done, I'll give this complete exercise to the students so they have it to refer to.
• Then I'll give them an XML file that is not well-formed (completely broken) and get them to try and fix it.  
• After about 15 minutes, I'll try and fix it myself in front of them, getting them to help me.
• Once that's done with my comments, I'll again put it in a shared folder for the students to grab so they can refer to it later.
• I'll give them another broken XML file and get them to fix it. 
• Again, after about 15 minutes, I'll try and fix it myself in front of them, getting them to help me.
• Once that's done with my comments, I'll again put it in a shared folder for the students to grab so they can refer to it later.
• Then, next concept.... and repeat.

This has worked quite well.  I've followed this process with all my courses and it seems to keep students engaged and helps them grasp new concepts.

As I mentioned earlier, I've been challenged by a colleague to make my courses better, even more collaborative, and (hopefully) gets the students more return on their tuition investment.  Here's what I'm changing and improving:

• Using a Repo.  Instead of using a shared folder to give files to students and have them turn in assignments, I'm using BitBucket with SourceTree (and forcing them to use it too).  This is great practice for them for the industry, and so far has worked really well.  My SourceTree client does get bogged down at times (with over 30 repositories) but it hasn't been a major deal.
• Commenting. I continue putting lots of comments in my code for the class to refer to.  They really like this and find it helpful when they review after class.  I'll definitely keep doing this
• Error Challenge.  I tend to fat-finger a bit when I'm coding in front of the class (even when I'm working off of a printed copy of a complete exercise).  Sometimes these errors are hard to catch and I waste some class time while I try and figure out what I did.  So instead of waste our collective time and to help them pay attention, I instituted a little game.  If a student catches an error I make in my code, they get a point.  Points accrue for the duration of the course.  The student at the end of the course with the most points gets a free Starbuck's gift card (or something like it).  This worked really well, and I'm definitely going to do it again.
• Student Journal. Student feedback is key for me.  I teach in 'Fast-Track' programs where the students know that they are going to get inundated hard and fast with new concepts all the time.  Ensuring I'm not bull-dozing them with new stuff is key, so having a feedback loop helps.  Up to this point I've just been asking them in class 'How's the speed - am I going to fast?'  But my colleague suggested getting the students to journal and having them  commit that to their repo so I can see it as another feedback loop.  I did this in my last course and it worked quite well.  I don't have them journal every day - just once a week or so, but the feedback I got from them individually was great. 
• Group Projects.  Out in the industry, most of these students are going to be working on teams, so having them do exercises in class as teams seems logical.  It turns out that students, for the most part, really like having group projects to work on.  They can collaborate together and learn from each other.  It also pushes them to compete a little which makes for an even better result.  I have them create a team repo for the project(s), and this also gives them experience with some BitBucket administration (branching and merging, etc.) which also prepares them for the industry.

IOException - Disk Quota Exceeded error - yet there's lots of disk space

I use a Mac for work.  I've run into an interesting issue with making backups of client web sites and zip files with my Mac...  It seems that whenever I've zipped a file and backed it up to my Mac, I get these wonderful __MACOSX folders - that's 2 underscores at the beginning of the folder name injected into the folder mix.

Restoring a site from one of those backups on a linux machine can have adverse consequences because of those 'injected' __MACOSX folders.  It seems that certain flavours of linux really don't like that folder name and as a result will not write a single file to the machine any more.... at all.  Instead, it throws an a 'Disk Quota Exceeded' error.  You can try touching a file, you can try unzipping some other files - whatever you do, you'll get that error.   The error didn't go away for me until I deleted all __MACOSX folders that got copied up to my machine in the zip file.  Once I did that - it was all good.

The interesting thing about this error was I could have tons of disk space still on the machine, and yet it would still throw that error until I removed those offending __MACOSX folders.

There are various commands out there for removing these folders from zip files created on a Mac... something like this:
zip -d your-archive.zip "__MACOSX*"

Get Hired as an IT Newbie Part 2 - Soft Skills

I participated in a focus group this past week at SAIT where we discussed their 'Web Developer Fast-Track Program' and its relevance to the industry.  The goal behind the meeting was how can we make the course more relevant to the changes and continual evolution of technology that is happening in the industry so students are more prepared for positions when they leave.  The discussion was engaging and took some interesting turns.  Here's what I took away related to Soft Skills:

Soft Skills
Employers like student hires to be reasonably technically adept.  Their technical expectations aren't in the stratosphere.  However, they want their student hires to have more than just technical ability.  Students should be well versed in soft skills.

Soft skills is (to some degree I'm sure) an ambiguous subject for many students.  I can hear you thinking 'What do people mean when they say that??'  In the context of our discussion  yesterday,  it meant prospective IT employers in the focus group yesterday were looking for students to have, in addition to technical skills:

• An ability to communicate well with a client - verbally, directly in front of them or over the phone.  Also having the discernment to know when to send an email versus when to talk verbally.  Poorly written emails are notorious for communicating the wrong things - particularly negative feelings where there were none.
• An ability to know what good work is.  Does the design and function of a web site fit the purpose/company it was designed for?  For example, does a sales/marketing website have lots of good calls to action?  Does the site feel right?  Would 'Joe's Mom' know how to navigate and use the site.
• A can-do attitude.  Students will likely get the soul-crushing repetitive work that senior developers/designers don't want to do (or don't have time to do).  Students should accept this and be prepared to do it with gusto.  If you're in an interview and you don't have a clue about a technology they asked you about, reply: "I don't know about that, but I will know about it tomorrow" and mean it.
• An ability and desire to collaborate and work in a team setting.  Its a fact of the industry - if you want to work in an agency, you're going to need to be able to work on a team and collaborate with team members. 

Also discussed was: What habits should students have (or start to form if they don't have them)?

• Students coming out of school should love learning and know how to learn things/solve problems on their own.
• Students should be able to receive constructive criticism without responding with excuses about bad instructors, poor course material, or the speed at which things were taught.  You don't get much chance for excuses on the job.  Employers are looking to 'break even' economically on their student resource investment within 4 months.  

Get Hired as an IT Newbie Part 1 - Have a Good Portfolio!

I participated in a focus group this past week at SAIT where we discussed their 'Web Developer Fast Track Program' and its relevance to the industry.  The goal behind the meeting was how can we make the course more relevant to the changes and continual evolution of technology that is happening in the industry so students are more prepared for positions when they leave.  The discussion was engaging and took some interesting turns.  Here's what I took away related to portfolios:

Student Portfolios
15 years ago I had gone through a similar IT fast-track program and we had to do portfolios of our work for potential employers.  I wasn't aware of a single employer who looked at my portfolio and as
a result I haven't placed much emphasis on it in my classes.  BIG MISTAKE.

It turns out that prospective employers for SAIT students DO look at their online portfolios.... and generally weren't super impressed.  Here's why:

• Industry attendees said that a student's portfolio should reflect the position or career track that the students are looking to get into.  There should be evidence in the portfolio that the student tried on their own to investigate, explore, and work with technologies and code that interests them.  Posting student projects (with every student having very similar projects) doesn't help anyone in making a hiring decision.
• Prospective employers are not just interested in the technologies that students used, but also: 
• Thought processes that the students went through in completing their projects - why they chose to make certain decisions (for example, use a canned WordPress theme instead of design/develop their own)
• Challenges students encountered in design and development and how they overcame those problems in their journey to the completed project - Employers are interested in soft skills like perseverance,  the ability to google a problem and uncover a solution on your own, the ability to asking for help when you are stumped (not before you've tried googling the problem)... etc.
• Is the student willing to go the extra mile, put in some extra effort and explore interesting tools and technologies outside the classroom?  The industry was quite clear that this was one of the main things they were looking for  - a motivated individual with the right attitude.  This kind of motivation should be clearly seen in a student's portfolio.

There was also some suggestions during the focus group that students should be given the opportunity to provide constructive feedback to their peers on their portfolios.  A good critique is a gift.  Can students accept constructive feedback and use it to improve?  Could providing peer critiques give them a better perspective and more experience with what is good (design, code, functionality, etc.) versus what is not good.  We thought so.

Finally, here's a couple of links to student portfolios that 'make the grade' so to speak, in my humble opinion.  One was a student of mine this past semester, the other is a current student at the University of Calgary:

- Gary S. Jennings (former SAIT student)

- Carrie Mah (U of C student)

Books read to date in 2015

One of the advantages of a good commute is having the chance to get some good reading done.  I've taken a different tact the last few months in my technical reading in that I'm choosing books that I hope I'll enjoy.  For the most part, its been a good experience.  Here's what I've managed to get through so far this year:

The Cuckoo's Egg by Cliff Stoll.  This has been the only 'reread' for me so far this year.  If you're in IT, you might this this book and enjoyable read.  Cliff recounts his adventures back in the late 1980's chasing a hacker internationally on the ethernet through university, military, and other government networks and computers.  If you consider yourself a geek and you haven't read this book, you really should.



The Phoenix Project by Gene Kim, Kevin Behr, and George Spafford.  The team leads at my current contract recommended I read this book.  It's marketed as 'A novel about IT, DevOps, and helping your business win.'  I enjoyed this book (and probably most people do) because it was easy to associate characters in the book with real people I've worked with.  On top of that, the problems the characters confront in their business are taken from 'last month' in many IT shops around the world.  Readers can relate to this book.  In some ways, I could say I've lived this book about 8 years ago, and I'm trying to help implement those solutions again where ever I end up working.

Social Engineering in IT Security by Sharon Conheady.  I enjoyed this book more than I thought I would have.  Sharon doesn't only give a great overview of the tools, tactics and techniques of IT related social engineering, she also gives it a historical context and augments the 'theory' with real life experiences.

Cyber Warfare by Jeffrey Carr.  I found this book to be something of a disappointment.  Even though it was a second edition (2011) I didn't find many updates.  I believe its intended target audience is CIOs and it seemed to me that the author included more legal documents and previously published papers in this book it needed.  I seriously considered returning this book and getting my money back.

Callings by Gregg Levoy.  I really enjoyed reading this book, perhaps because I'm very interested in the topic.  Certain readers might find it a bit academic, but there are many interesting, true stories that keep the narrative engaging.  This is NOT an IT book - rather it more about 'finding and following and authentic life.'  I originally got this book from the library on a whim, but after reading it I bought a copy.  Beware - some editions of this book (like the one I got from the library) have the last 20 pages out of order.  Check the page order from 300 on before you buy your copy.

Enterprise Architecture in Calgary

I went to a CIPS (Canadian Information Processing Society) lunch on Tuesday and the topic of discussion was 'The State of Enterprise Architecture'.  The talk was given by Martin Mysyk, the curator of www.enterprisearchitecture.ca.  Actually, it was more of a discussion lead by Martin.  This was appreciated as there were many practicing architects in the room and it was good to hear the variety of perspectives, challenges, and concerns that these fellows face.

Martin began by posing the question 'What do Enterprise Architects Do?'  Someone responded from the back of the room, 'We slow projects down!'  Everyone laughed and then we got more serious.  Other answers were offered:  'we provide strategic vision, influence, and guidance to the enterprise',  someone read a Wikipedia definition for EA's.  In the end, Martin compared EA's with municipal planners.  I thought this was interesting and decided to google it, and I found this page:  Enterprise Architecture Analogies  Some interesting food for thought.

A slide was displayed about EA As a Career and a short discussion ensued on how does one get their foot in the door (certification, promotion from BA role, promotion from SA role, etc).  EA's require a delicate balance of technical and business expertise, combined with superb soft skills like communication and effective discussion leadership.  Martin recommended a book called the Zoom Factor by Sharon Evans.  I might have to pick this one up as it looks like it might be interesting.

Warming to his subject, Martin moved into the EA Practice.  He reviewed several different flavours of EA methodologies out there today, commenting that TOGAF seemed to be the current favourite.  A slide was displayed with EA Practice at the top, followed by four boxes (focuses of EA?):  Business, Data, Applications, and Technology.  I was surprised that Security didn't count among them, and it didn't seem to arise in the discussion as much as I thought it would.  If TOGAF suggests that a Business architecture is important, doesn't a Security architecture merit some consideration?  Perhaps I missed it in the diagram...

Some important considerations for EA's that came up in the discussion:

• We need to consider growth versus optimization in Calgary right now (considering the price of oil)
• Who in the organization should benefit from EA?   Is I.T. really the right place for every organization's EA practice? Some companies have their EA practice reporting to the Legal dept.  others report to Operations, or even the CEO.  
• Perhaps the question is 'Where can the EA practice provide the most value in organization?'  Answers to these questions will be different for each situation.  Depending on the answers, one may have to reconsider who the EA practice reports to...

And finally our discussion wrapped up with the challenges that EA's face:

• We need to prove our value with a small budget.  How do we do that?
• How do we influence the decision makers without seeming heavy handed?
• How do we provide guidance and structure to development and growth without seeming heavy handed?

I'd love to have some comments!