SAS Stored Process: Ignore the Log at Your Own Peril
I love our readers – especially when they have a great idea they want to share. Quentin McMullen, who has been programming in SAS for 15 years, primarily in clinical trials and public health submitted such an idea recently. Several months ago he left a comment asking about checking the log from a stored processes. I didn’t really have a good idea – so he came up with one! Please add your thoughts in the Comments section – he wants feedback from other experts!
Quentin does not have his own blog to plug (yet), but if you want to chat with him about log checking in person, catch him at NESUG this November in Baltimore, where he will be presenting a simple data verification macro, %assert.
Ignore the Log at Your Own Peril
When I was a newbie SAS programmer, I was always afraid of making mistakes. The most embarrassing mistakes were when my boss found an error in some report or dataset I had delivered, and the reason for the error became obvious as soon as I looked at the log and noticed a big red ERROR: line (okay, when I started in v6.12 the log wasn’t actually colored, but you get my point). From those painful experiences, I quickly learned to review the log before I look at my results. And like many programmers, I wrote a %logcheck macro that I invoke at the end of a program to automatically parse the log looking for errors, warnings, and problematic notes.
Now I’m a newbie again, just getting my feet wet with SAS Stored Processes (my learning curve has been greatly facilitated by reading both of Tricia and Angela Hall’s books!). And since I will be building some production tools, I’m back to thinking about log checking, and how to incorporate log checking into the Stored Process framework. Of course, ideally a production Stored Process will contain sufficient error-handling code to trap any errors encountered. But I have yet to see a program with even a moderate level of complexity where I would think that it is bullet-proof enough to feel comfortable delivering results without reviewing the log. This post describes an approach for incorporating automated log review into your stored processes.
There are three steps to the process:
- Run a report, while writing the SAS log to a file.
- Scan the log for errors, warnings, and bad notes.
- If the log is clean, send the report to the user; if the log is dirty, send an error message.
Step 1: Run report while writing SAS log to a file
filename mylog "%sysfunc(pathname(work))/mylog.log" ; proc printto log=mylog new ; run ; ods _all_ close ; ods rtf file="%sysfunc(pathname(work))/myreport.rtf" ; options device=png ; proc print data=sashelp.shoes (obs=5) ; run ; proc gchart data=sashelp.shoes ; pie region ; run ; quit ; ods _all_ close ; proc printto ; run ;
Often the main stored process code sits within a %STPBEGIN/%STPEND sandwich, and %STPBEGIN generates the ods statement opening the output destination. In this case, I do not use %STPBEGIN/%STPEND. Instead, I write my own ODS RTF statement, sending output to an RTF file (myreport.rtf) that is written to the work directory. And that ODS sandwich sits within a PROC PRINTTO sandwich, which sends the log to a file in the work directory, mylog.log. Executing this code does not return any results to the user. It simply writes myreport.rtf and mylog.log. If you use the Stored Process Wizard in Enterprise Guide to create or revise the Stored Process, you will need to turn off the automatic generation of the %STPBEGIN/%STPEND sandwich. Angela has a great post on her blog about another scenario where it useful to write your own ods sandwich, and also a post on this blog that illustrates how to turn off the automatic generation of %STPBEGIN/%STPEND.
The above example shows writing an rtf file, but the same approach works for writing a PDF file or an HTML file. One complexity is that when the report contains plots, the ODS HTML statement needs to point to a catalog where the graphs can be stored, and then replayed. The equivalent ODS HTML statement would look like:
ods html file="%sysfunc(pathname(work))/myreport.html" path=&_tmpcat(url=&_replay) ;
Step 2: Scan the log for errors, warnings, and bad notes.
After running the report, you call your logcheck macro to scan the log file. If you don’t have one, search conference proceedings at lexjansen.com, or just use Tricia’s. This macro should read in the log file, and return results that indicate whether the log had any errors, warnings, or problematic notes. For the sake of this example, we will assume that any bad log messages found in the log are written to a dataset WORK.BADLOGMESSAGES.
If a serious error was encountered during execution of the main code, SAS may have entered syntax check mode, setting system options obs=0 and noreplace. For that reason, it’s a good idea to force the following system options to be in effect prior to running the log scan, to ensure the log scan will run: obs=max replace nosyntaxcheck.
Step 3: If the log is clean, send the report to the user; if the log is dirty, send an error message
If the log is found to be free of bad messages, we will send the report back to the user. The code to stream an rtf file back to the user’s browser looks like:
data _null_ ; file _webout ; infile "%sysfunc(pathname(work))/myreport.rtf" ; if _n_ = 1 then do ; rc = stpsrv_header('Content-type','application/msword') ; rc = stpsrv_header('Content-disposition',"attachment; filename=myreport.rtf") ; end ; input ; put _infile_ ; run ;
Basically, this step reads in myreport.rtf and writes it out to _webout so it will be sent to the user’s web browser. Because we are sending an RTF file, and not HTML, we use the stpsrv_header() function to tell the browser that the file is an attachment named myreport.rtf that is associated with MS Word. When this runs, the user will be prompted with a dialog box to open or save the file:
When streaming HTML back to the user, the stpsrv_header() function is not needed, because the results should be displayed by the browser. When streaming a pdf file, the step is slightly different because you need to stream a binary copy of the file.
If bad messages were found in the log, we will send the user an html report listing the problematic errors/warnings/notes found in the log, followed by a full dump of the log:
ods html file=_webout ; title1 j=l h=2 c=red "Errors occurred during the production of the report!" ; proc report data=badlogmessages nowd ; columns line ; define line/display ; run ; title1 ; /* === screen dump of full log ===*/ data _null_ ; file print ; infile mylog ; input ; put _infile_ ; run ; ods html close ;
To see an example of the above steps, you can download LogCheckSmallDemo.ZIP, which is a full example of this log-checking approach.
The proposed method is simply: write the report to a temporary location; check the log; send either the report or an error message to the user, depending on log check results.
Because this approach relies on streaming the results back to the user, it only works when the stored process is set to stream results, and the consuming application can read streamed results (for example, Stored Process Web Application and Information Delivery Platform can read streamed results, Web Report Studio cannot). An alternative that would work for package delivery of results would be to write the results to an ODS DOCUMENT instead of a file, and then replay them if the log is clear. This alternative also has the benefit of allowing you to use the usual %STPBEGIN/%STPEND sandwich. Unfortunately documents do not store all of the formatting associated with a report (e.g. page breaks), and any errors generated during replay of the report would not be detected by the log check.
Some people might object to presenting log messages back to the naïve user, who may not be familiar enough with SAS logs to understand the messages. Instead of sending the actual log messages to the user, you could deliver a generic error message along the lines of “Errors occurred, please contact support….” An alternative approach would be to present the report to the user, regardless of the log check results, but if the log check fails, email the log to the developer of the stored process.
There is plenty of room for further development of this approach. For example, I am working on a macro which will replicate the functionality of %stpbegin, generating an ods statement that sends results to a file for any user-specified destination, and a macro %StreamReportIfLogClean() that calls %LogCheck and then streams the report (or error message).
What do you think?
But at this point, since I’m new to stored processes, I wanted to ask what readers think of this general idea. Do you agree that log checking should be incorporated into Stored Processes? Does this approach seem reasonable, or have you thought of a good alternative? I would love to hear your thoughts in the comments below. And many thanks to Tricia for teaching me so much about Stored Processes, and encouraging me to write my first ever blog post!
Learn More about SAS Stored Processes and Prompts
You can learn more tips and tricks for creating, debugging, and using SAS stored processes in the 50 Keys to Learning SAS Stored Processes book. It's a complete guide to SAS stored processes. Check Amazon for best pricing and quick shipping!
Latest posts by Quentin McMullen (see all)
- Building a SAS Stored Process Log - 2014-02-24
- Are you looking at the world through BI Glasses? - 2013-11-13
- Taming SAS Stored Process Selection List Prompts - 2013-08-07
- Using an Autocall Macro Library in SAS BI - 2013-07-10
- These are a Few of My Favorite Things… about the SAS Enterprise Guide Program Editor - 2013-03-27