I was just quickly flipping through code on a terminal and got to thinking that it would be pretty handy to be able to syntax highlight when using head or grep without having to open an editor. Luckily, I remembered Pygmentize was pretty handy when syntax highlighting in LaTeX so I wondered if they had an equivalent output for terminal. It turns out they do (in standard or 256 color no less) so in a couple minutes I had a really short script for highlighting code at the terminal saved it to bin/ccat and was ready to go. Pygmentize really is impressive. Here's the script if anyone else is looking to do the same:

It just looks for stdin and if it doesn't find it it looks for a file name or takes direct input. Obviously it requires Pygmentize (which is really easy to install if you already have Python).

LaTeX tends to ignore white space. This is good when you're writing papers but not so good when you're trying to show code where white space is an essential part (e.g. Python). Luckily there's a builtin verbatim environment in LaTeX that is equivalent to html's <pre>. So something like the following should preserve white space.

\begin{verbatim}
for i in range(1, 5):
  print i
else:
  print "The for loop is over"
\end{verbatim}

Unfortunately, you can't use any normal LaTeX commands inside verbatim (since they're displayed verbatim). But luckily there a handy package called fancyvrb that fixes this (the color package is also useful for adding colors). For example, if you wanted to highlight "for" in the above code, you can use the Verbatim (note the capital V) environment from fancyvrb:

\newcommand\codeHighlight[1]{\textcolor[rgb]{1,0,0}{\textbf{#1}}}
\begin{Verbatim}[commandchars=\\\{\}]
\codeHighlight{for} i in range(1, 5):
  print i
else:
  print "The for loop is over"
\end{Verbatim}

If you really want to get fancy, the Pygments package in Python will output syntax highlighted latex code with a command like: pygmentize -f latex -O full test.py >py.tex The LaTeX it outputs is a bit hard to read but it's not too bad (it helped me figure out the fancyvrb package) and it does make nice syntax highlighted output.

Here's an example LaTeX file with the three examples above and the pdf it generates if you're curious.

• codeInLatex.tex
• codeInLatex.pdf

It turns out SINs and credit card number are checked with something called the Luhn algorithm. Basically the algorithm just involves taking each digit, multiplying the second, fourth, sixth and so on digit from the right by 2 and adding up all the resulting digits (e.g. if 7 is multiplied by 2 then the resulting 14 is split into 1+4). If the sum of the digits is a multiple of 10, the number passes.

For example, to check 345678, you'd split it into 3,4,5,6,7,8. Then multiply 3, 5 and 7 by 2 to give 6, 4, 10, 6, 14, 8. Then split all the digits again to give 6, 4, 1, 0, 6, 1, 4, 8. That adds up to 30 so 345678 would be a valid credit card number (if there wasn't a set length).

Just for fun, here's a quick function in R to run the Luhn Algorithm on a number (or tell you the remainder so you can adjust):

So the next time some stupid web form needs a SIN number I'm going with 999999998.

So what is a macro? Macros are a part of SAS that look through your code before the normal part of SAS sees it and writes out your code for you based on a special syntax. If you've ever found yourself copying and pasting code then you've probably been in a situation well suited for macros. They're also great if you need to perform different functions under different conditions. Once I learned macros, SAS seemed a lot more like a usable (although weird) programming language and tasks seemed to get a lot easier (except actually picking the statistical techniques to use).

Probably the easiest way to see what macros do is an example. So say we once again have a data set of tree heights

I already talked about how to find and use the mean and standard deviation for the whole data set. Now what if we wanted to standardize each species by its own seperate mean and deviation? We could cut and paste but once we get a few more species or want to change something later this really becomes a hassle. So this is where macros come in.

The first thing to do is to calculate the mean and standard deviations for each species. We can use proc means again to do this. Since we won't be using the output I'll add the noprint option and since we only want the means for the individual species and not the whole dataset I'll add the nway option. The class name; statement tells SAS to find the statistics seperately for each species and the output line tells SAS to save the mean and deviation in a dataset called meansd.

Now we just need to get the values from the meansd dataset into macro variables. We'll use the _NULL_ dataset and call symput again to create macro variables. This time we need to create seperate macro variables for each species. Luckily SAS automatically numbers each observation in a dataset in a column called _N_. Since each line of the dataset corresponds to a tree species, we can easily use this identifier to create the macro variables by using call symput('mean'||left(_N_), meanheight);. The left() and ()trim functions (numeric variables have extra spaces to the left and string variables have spaces to the right) removes any unnecessary spaces and the || concatenates (connects) the text "mean" with the line number to give give mean1, mean2, etc.. I'll do the same thing for standard deviation and tree name. Once the macro variables are created, there is still one problem remaining. We don't know how many species there were or how many macro variables were created. Luckily SAS will make another column that indicates the last line of the dataset when it sees end=newcolumnname following a set statement. Then we just need to check if SAS is on the last line and if so save the line number (_N_) to know the number of species of trees.

If you ever want to check what macro variables you have in your program, you can use %PUT _USER_; to print them all to the log file. Or if you want to see every macro variable available (SAS has quite a few automatic ones like operating system and date) use %PUT _ALL_;. Inserting %PUT _USER_; here produces:

Now we've set a lot of macro variables but we still haven't created a real macro. In SAS, macros are started with %MACRO macroname; and finished with %MEND; (short for M[acro]END). %'s are used to indicate commands that the SAS macro facility will read and remove before normal SAS sees the code. Anything not with a % will be printed out by the macro facility. Macros don't spit out their code for SAS until they're are called using %macroname.

So I'll call my macro treestandardizer but you can call it whatever you want. I'm going to use a pretty simple and specific macro but if you were going to use this often and for different datasets you would want to program it better. The first thing to do is create the final dataset and set it to the trees dataset. Since we need to loop through each species of tree, we'll need a %DO loop. Everything between %DO and %END will be repeated while i increments from 1 to the number of tree species. If you want to combine text and a macro variable to reference another macro variable, you use the double ampersand && in SAS. For example, we want to get the mean for species 1 by looking in the macro variable &mean1 so we use &&mean&i. I think the macro processing part of SAS ends up running through the code twice, the first time finding the && and replacing it with & and the &i and replacing it with 1 to leave &mean1 and the second time finding &mean1 and pasting in the appropriate value (65). So we'll have the do loop write out a series of if statements to check what the name of the tree is and use the appropriate mean and deviation. Note that when using a string macro variable like &nameX, you need to surround it with double quotes (the macro processor doesn't look inside single quotes) so SAS doesn't think it is a variable name.

The previous code prepared the macro but nothing actually happens until we call it using %treestandardizer. Unlike almost everything else in SAS this line doesn't have to end in a semicolon (although it's pretty unlikely to hurt if you forget and add one). So to call the macro:

If you want to see what happens when you call a macro, you can have SAS print the code generated by the macro to the log file with the option option mprint; (make sure to set it before actually calling the macro). In this case, it gives:

So it worked and we now have the standardized heights in the stheight column of the final dataset. This particular example could be done a few different ways (the easiest and probably better way being to merge the meancv dataset with the trees) but I hope it gives a decent introduction to SAS macros. If you have any specific questions or something wasn't clear, feel free to ask in a comment.

Here is the SAS source code if you don't feel like copying and pasting.

This is how you make text italic: This is italic.

when I actually wanted:

This is how you make text italic: <em>This is italic.</em>

Of course it's not too hard to remember to encode the special characters but this is WordPress and things are supposed to be easy. So I thought I would throw together a simple WordPress plugin to convert HTML special characters inside <code></code> tags. That was a few months ago and I got the plugin most of the way done and then got distracted with other things. But after reading about another person having the same annoyance, I decided I should finish the plugin.

So here is WP_CodeShield. It simply looks for <code></code> tags in posts and comments and makes sure the contents are encoded properly. You just type the code you want inside <code></code> tags and it should appear correctly. So you can type <code><em>This is italic.</em></code> instead of <code>&lt;em&gt;This is italic.&lt;/em&gt;</code>.

If you have any questions, suggestions or requests, please leave them in the comments below. You can also test WP_CodeShield out in the comments if you are curious.

Current Version:

WP_CodeShield v0.4

Installation instructions:

Unzip wp_codeshield.zip. Upload wp_codeshield.php to wp-content/plugins/ and active in the Plugin Control Panel. That's it. Anything inside <code></code> tags will now automatically be converted to proper HTML.

Technical Details

Most people should be OK just using WP_CodeShield naturally but for especially fancy users here are the more technical details:
Nested code tags are ok (I've been using them in this post) but unequal numbers of <code> and </code> will cause the plugin to leave the entire post or comment alone. WP_CodeShield will leave any string which already contains HTML special characters alone. For example <em> will be left alone. This is in case you have previously hand entered the special characters or want to describe something more complex than the plugin can handle. To keep track of these special occurrences, WP_CodeShield adds a small invisible comment tag (<!--formatted-->). WP_CodeShield only looks for <code></code> tags. <code ></code > or <code class="fancy"></code> will not be filtered. One small gotcha is that if a post contains <code class="fancy"></code> no filtering will occur on that post (WP_CodeShield will think that the tags are unbalanced). You can get around this by making sure both the opening and closing tags do not match WP_CodeShield's target like <code class="fancy"></code > (the extra space keeps WP_CodeShield from seeing the tags but are still valid HTML).

Change Log:

• v0.4 02-04-2008:
  • Changed filter to match updated wordpress the_editor_content
• v0.3 2-26-2007:
  • Properly converts when saving and unconverts when editing
  • Doesn't convert contents that are already encoded
  • Corrected wordpress hooks
• v0.2 2-26-2007:
  • Allows nested codes
• v0.1 2-13-2007:
  • A rough filter