Friday, August 31, 2007

Assumptions and Critical Thinking

A couple of weeks ago I programmed a DB load process that ran off a cron job. It ftp-ed data files from a public box to a local one, and then loaded the data based on .dat files that were extracted from the download, and .CTL files already on hand. I assumed this process worked correctly because every time I ran it manually it worked, and when I started it using cron, it 'looked' like it workded. It turned out that when it ran off the cron, it didn't do everything I wanted it to do. I set about debugging this today, and it turned out that my problem was very similar to my PLSQL problem from earlier this week.

Part of the Load process used a java program to created create the .dat files. The java worked fine when I ran it manually because the user I was running it with had JAVA_HOME set in it's profile. Cron runs differently. Even though my cron job su's to the proper user, doing the same thing manually and then checking with a 'who am i' command (which, by the way, gets different results than a 'whoami' command) returns a surprise. It's not my su'ed user. Adding JAVA_HOME into the script that gets run solved the problem.

Been looking at graphing solutions for a little thing I'm working on. GDchart seems usable. I might do some more work with it on Tuesday next week. FusionCharts seems very interesting as well. I've worked with Cewolf and JFreeCharts in the past, but I need something quicker right now.

Picked up a book called Th!nk off the discount shelf at Cole's today. I'm liking it so far. It's something of a rebuttal to another book called Blink!.

3 comments:

Jaye said...

My personal policy with cron scripts is log everything, especially the username, host and date the script is running.

Did you try gnuplot for quick and easy graphs? Very amenable to scripting.

Perry said...
This comment has been removed by the author.
Perry said...

In my poking at different graphing possibilities, I actually did not run into gnuplot. I'll have to take a look.

If you look at my blog from Sept. 5 you'll see I ran into more cron grief. My policy with cron from now on is to assume it knows nothing about anything. :-) Logging is a great idea. I did actually start logging and that's what lead me to realize that I need absolute paths. Something like:
'15 * * * * /opt/app/myscript.sh > /opt/app/myscript.log' works well for me. If I didn't want to overwrite the log file, I'd use two '>>' to append to it instead.

Jaye, I mostly make these notes for my benefit - hope you don't mind the extra detail.