Thursday, June 3, 2010

CMS systems

I found this good post http://designtutorials4u.com/15-great-content-management-systems-for-designers/ on some of the more popular systems out there.  Unfortunately, it doesn't document specifically the technology they use or differentiate very well which ones are 'free' vs. which ones you have to pay to use.  I've also run into a couple others lately:
Umbraco - an asp.net based cms
and
Alfresco - a java based cms

Thursday, May 13, 2010

More Software Emergency Fixing Experiences

I've recently helped two more clients with 'emergency fix' issues.

The first issue was a problem a national grocery chain had with an export from a SAS program that did a pull from multiple database tables into an excel spreadsheet, and subsequently imported this data into a MS Access database using a SSIS (Sql Server Integration Services) package.  The SAS export seemed to be fine - that is, it executed without errors, but the SSIS package would fail. From my understanding, the client would run this process about every month or so, so there were a fair number of records (1000's).  However, they started having this issue in Dec. and I didnt' get called about the issue until late April, so there was a backup of data.
The client didn't have the SQL BI studio installed, so we had to go through the idx file manually to see if there was anything obviously wrong.  There didn't appear to be.  The database and excel connections tested fine. There didn't appear to be anything riotously wrong with the data formatting or invalid characters in the excel spreadsheet.  So, what do you do?
Well,  for one thing, don't test with the full dataset if you think it might be a data related issue.  Keep making your dataset smaller until you are sure that none of the data works.  In this case, we cut the dataset in half, started from Feb to the end of Apr and the import worked!  This confirmed for us that there were no permissions issues, or software upgrade issues (we were considering that software upgrades might be part of the problem as more than one piece of the software had been upgraded in that time period).  The successful test confirmed for us that we were dealing with a data related issue.  From there it was just a matter of determining exactly which rows of data couldn't be imported, and what it was about the data in those rows that botched things.
Less than 5 hours of work to this point on this fix.

With the second emergency, a telecom company was having issues getting an automated FTP (file transfer protocol) process to work.  They had changed the paths of where the ftp files were residing and they were voip experts, not so much into the bash shell scripting thing.  Their first problem was pathing in the bash file.  That was an easy fix.  Then there was a problem with sending a notification email that apparently they never got to work.  This was a little bit more tricky.  They originally had the email coded similar to this example.

       #!/bin/bash
       telnet smtp.example.org 25 <<_EOF
       HELO relay.example.org       MAIL FROM:<joe@example.org>
       RCPT TO:<jane@example.org>
       DATA       From: Joe <joe@example.org>
       To: Jane <jane@example.org>
       Subject: Hello
       Hello, world!
       .
       QUIT       _EOF

However, after a bit of work I discovered that the server this script was running on was processing all the command too quickly for the smtp server to get the email off.  In the end, I had to do something more like this:
       #!/bin/sh
       ( echo HELO ccielogs.com       
       sleep 2
       echo MAIL FROM:<test@somedomain.com>
       sleep 2
       echo RCPT TO:<test@someotherdomain.com>
       sleep 2
       echo DATA
       sleep 2
       echo Subject:Test-Mail!
       sleep 2
       echo If you can read this, it Works!!!
       sleep 2
       echo .
       sleep 2
       echo QUIT
       ) | telnet XXX.XXX.XXX.XXX  25

Then to set the bash script to run in a windows scheduler (using cygwin) I had to do something like this:
       C:\cygwin\bin\bash.exe --login -c "/myfolder/myscript.sh" 
Less than 5 hours to fix.

Monday, April 26, 2010

Param datatypes with Sql Server Stored Procs

I tried several times to save and post this blog entry earlier today and it would NOT save. I have no idea what is wrong as I tried to email it to myself so I could do it later and I couldn't email it either. I'm thinking there was a bad character in my original text, but I pasted in into (and copied it out of notepad) so...  I'm not sure what is going on.  Sometimes these tools in the 'cloud' seem to be in a 'fog'.

Anyway, what I wanted to say was....
I've been working with stored procs lately in Sql Server.  My stored procs take search parameters that I wanted to be very flexible - strings with wild characters on either end using a LIKE in the SQL.  Originally in my declaration for the parameters I used a char datatype.  What I didn't realize was if the param passed in is shorter than the length you specified, a char datatype will automatically add placeholder characters to fill up the rest of the space. So, if  you declared a char with 9 characters and passed the SP a 3 character string, SQL Server would automatically add 6 more characters to it.  I was then trying to prefix and append the percentage sign (wild character) on either end of my param, but it was getting ignored.

In the end (with the help of my more DB inclined colleague, Karlson) I changed the datatype of my param to varchar and I could happily prefix and append wild characters to it.

Got another interesting business idea over the weekend - it's based on a few assumptions, and it's for the investment/financial industry.  The assumptions are:
- history repeats itself.
- the stock market, like history, is cyclical in nature
- investment funds are just as much marketing as they are statistical science
- people will buy anything that is marketed well, whether it provides them with tangible value or not - (the value of stock is all about perception of value)
The idea is simple - create a fund that is based on the cyclical nature of the stock market.  Predictable things happen when interest rates go up, or when commodities or high.  Convince people that you can harness the 'value' in the history of what happens to the stock market with certain indicators are up, and you'll have a winner fund.

Wednesday, April 21, 2010

Network isolation

For the second time this week, production has gone down for an extended period of time. This time it was because of this bug introduced by McAffee. It was proliferated through all of our environments (apparently) by SSCM. I'm not sure why we are putting McAffee fixes right into production without testing them in a staging environment first. Seems to me like there is a wrong process there somewhere - Either on our side, McAffee's side, or both.
Before we realized this was a McAffee bug, it seemed that this 'virus' got propagated to production from lower environments. Having the production environment on an isolated network from the rest of our development and test environments would have protected us to some degree from this kind of propagation. I have (officially) suggested this more than once, but resourcing and other priorities have gotten in the way of making it a reality.
For future reference though, here's just another reason to have a production environment on an isolated network.

Tuesday, April 20, 2010

Single Points of Failure

Within the last 8 months, all of the production servers for a client of mine were moved out of the building they've been in and into a 'bunker', or co-lo. There was a variety of reasons for doing this. As an architect choosing a Co-location provider, and planning the move, you would want to make sure as many systems there are as redundant as possible - power supply, UPS system, networking, etc.

Well, this week everyone painfully discovered a system that wasn't redundant (or if it was, it wasn't as redundant as it should have been). Apparently the HVac systems went down. This resulted in servers getting too hot and consequently having to be shut down. Production servers.

Everything was resolved in a couple of hours, but it just goes to show, it seems that there is always one system that is forgotten.

I've actually run into another situation like this before. It wasn't nearly as big an issue for our shop as it only affected development environments. However, the Calgary airport(!!) along with most of NE Calgary was without a network connection just as long as we were. Apparently a construction crew was using a backhoe doing some digging at an intersection and accidentally cut a main networking cable that supplied most of NE Calgary with it's network connection. Kind of makes you wonder if municipalities should be considering redundant underground networking, doesn't it?

Networking Architects  work hard to eliminate as many single points of failure in their systems as they can.  Some are hard to control though.  Recently McAfee released a virus library update that wasn't tested properly and as a result shut down a plethora of systems across North America.  It thought a Windows dll was a virus (false positive) and sent thousands of systems into a perpetual reboot.  It would be an interesting bit of process engineering to figure out the best way protect your production systems from the most current viruses AND protect them from bugs like this sent out by the 'big' virus scanning companies at the same time.

Monday, April 19, 2010

Google Tools post

Good link here that lists the top Google Tools for website developers:
http://sixrevisions.com/tools/the-top-15-google-products-for-people-who-build-websites/
Personally, I start to get lost in the plethora of tools out there. Just getting Adwords properly integrated with Analytics is a headache I don't really have time to deal with (when I have so many other higher priority things to manage). I wish some of these tools where a little more straight forward to use.
My hesitation in making the investment to learning one well is that it will change too soon and too much for me to keep up with (I do have a 9-5 job along with at least half a dozen clients on the side). I need to focus my attention on where I get the most return on my time investment - and learning how every single Google Tool works doesn't rate that high.
It is a good post for reference, though. I like knowing what all the different tools are out there.

Wednesday, April 14, 2010

More Ideas

Here's some more business ideas that I've had lately. I want to write them down so I don't forget them as they are fun to discuss with people.
- An automatic window washer for houses. Living in Calgary with all the construction and wind, windows don't stay clean very long. Wouldn't it be nice to have a little 'thing' (like the robot vacuum cleaner) that you could run on your windows?
- A job website specifically targeted to the baby boomers and finding them good part-time work.
- Any kind of inventions/devices that will help baby-boomers stay in the workplace longer. I'm sure their feet would get sore after serving customers for a couple of hours in Tim Horton's. Wouldn't it be interesting if you could install 'chairs' that ran on a 'track' behind the counter in fast food restaurants for the servers to sit on?
- I learned at a conference this past weekend that apparently there is no Canadian company offering servers on demand in 'the cloud' like Amazon's EC2. There is a potential business opportunity here for a smart Canadian company with a bit of infrastructure in place. Eucalytpus is basically a freeware version of the same software that Amazon uses to run it's cloud. Set a few servers up with that and you're ready for Canadian customers. The advantage here is other Canadian businesses are hesitant to use American cloud solutions because of the American Federal government's attitude toward privacy. Host your cloud in Canada and don't worry about the American 'Fed's' keeping an eye on your business data. :-)