In case you have a stumble that occurs up to 2k rpm that is non constant, sometimes rough idle but mostly not, most noticeable at 2k but fine everywhere above that, here is how I fixed mine.

2003 Highlander 2.4l 4cyl 2wd 170k miles

Replaced the plugs (Densos lasted 70k, I replaced with NGK)
Cleaned the MAF sensor, and thats it. I was afraid it was something much bigger. Plugs looked a little lean with a small amount of build up. Runs great again.

I posted this because I wanted to remind people not to over look the simple things!