I just found an interesting (that's interesting in the 'I just spent an hour debugging that?' sense) characteristic in SAS. If you have a variable, x, and are using the lag of x, do NOT put the lag(x) inside a conditional statement. This can apparently cause some pretty strange results. It is probably easier to see through code than to explain:
- data test;
- input id$1 x;
- cards;
- a 1
- a 2
- a 3
- b 12
- b 13
- b 15
- run;
- data test;
- set test;
- lagid=lag(id);
- if lagid=id then do;
- lagx=lag(x);
- duration=x-lagx;
- end;
- run;
- proc print;
- run;
And here are the results:
- Obs id x lagid lagx duration
- 1 a 1 . .
- 2 a 2 a . .
- 3 a 3 a 2 1
- 4 b 12 a . .
- 5 b 13 b 3 10
- 6 b 15 b 13 2
There are obviously some strange things going on here. I have no idea what is going on in the 2nd row of the results where lagx is missing when it should be 1. And in the 5th row, lagx is the 3 from 2 rows above when it should be 12. Luckily, the solution is easy. Just execute the lag every time:
- data test;
- set test;
- lagid=lag(id);
- lagx=lag(x);
- if lagid=id then do;
- duration=x-lagx;
- end;
- run;
Anyway, it took me two years to run into this so I guess this isn't a common mistake but if you're ever having trouble with a lag, check to make sure it is executed for every line of the data.
fragtal | 19-Sep-07 at 10:03 pm | Permalink
Thank you for the post! I’ve run into the same problem, and it does seem that SAS lag function doesn’t work correctly in conditional statements.
The Norwegian guy | 25-Oct-07 at 11:58 am | Permalink
I’m a new SAS-programmer, and it took me a mere two months to run into this. It fits nicely in a mile-long list of problems and disappointments about the SAS language, but that’s another story.
The Norwegian guy | 25-Oct-07 at 12:00 pm | Permalink
Oh. And the fix won’t work if your lagx-variable already exists, and you only want to fill in the holes where it is missing and not replace the others.
ScottS-M | 25-Oct-07 at 9:18 pm | Permalink
@Norwegian guy
I really hated SAS when I first started with it. I think it took me several months to finally get to the point where I didn’t mind using it. If I remember right getting used to the macro functionality really helped. I still use R for quick or small things but SAS really is quick when you’re working with huge datasets. It seems like Matlab might combine many of the benefits of R with the speed of SAS but perhaps without as robust a statistics package.
For your already existing lagx problem. I wonder if you could do something like:
altlagx=lag(x);if missing(lagx) then lagx=altlagx;
drop altlagx;
That way the lag gets taken outside the conditional and the holes in lagx are filled in with the results.
The Norwegian guy | 26-Oct-07 at 3:24 am | Permalink
Thanks, ScottS-M
I worked around it after posting yesterday, just like you suggested. I’m still stuck with a bad feeling though, as these types of “workarounds” seem to be needed for just about everything I do in SAS. In this case, I wanted to manipulate 20 of my variables, had to create 20 more temporary variables to do the manipulation, and now have to create even twenty more to bypass this “feature” in the lag-function. Needless to say, the code becomes extremely bloated and hard to read and debug from all of this.
Using macro variables still has me puzzled, as it seems to be no picnic using them actively within a data step. I guess I’ll post about that in another thread :)
I’m still hoping that all of my troubles are I’m no seasoned SAS programmer yet, hopefully I will learn how to code more efficiently as time goes.
Thanks again for helping, and sorry if I let out too much steam.
The Norwegian guy | 26-Oct-07 at 3:31 am | Permalink
Oh! Do you know of any SAS forums where users can help each other, discuss problems etc? I’ve been searching around, but I’m having a hard time actually finding any :/
ScottS-M | 26-Oct-07 at 8:01 am | Permalink
Glad you got it working. No problem letting off steam (and that was pretty mild steam). I remember my first half year or so was filled with expletives.
I tend to bang my head against the wall until I solve things (or switch to another language for the problem) so I don’t really frequent any forums. There are the official SAS forums although they don’t seem to have a category for basic questions and there’s the Usenet group comp.soft-sys.sas. Also in the real world, our university had a nice 2-day SAS course on macros which would have been great for a beginner and I also occasionally get emails from a SAS user group in the area.
Also for macro variables there’s my (rather poor) introduction and now that you mention it I’ll try to get a post up about using macros in the next few days (Edit: Now available here).
Kelly LeVoyer | 05-Nov-07 at 1:46 pm | Permalink
SAS employee here…There’s also http://www.sascommunity.org though admittedly it’s still getting off the ground. If you haven’t already you might want to check out Chris Hemedinger’s blog http://blogs.sas.com/sasdummy/ (not to imply you’re a dummy ;) but it’s a good basic resource as well, feel free to comment there with questions. Chris co-authored the SAS For Dummies book.
ScottS-M | 06-Nov-07 at 2:47 am | Permalink
@Kelly LeVoyer
Thanks a lot for stopping by. It’s really cool to see SAS staff involved in user discussions. I think I’ll throw all this ‘where to find help’ into a quick post in the next couple days to make it easier for people to find.