The PSP, as you said, does one thing exceptionally. It just also happens to do many other things with mediocrity. It's a ho-hum MP3 player, an impractical movie player (because they have to be UMD format-- the only movie worth getting that's released on UMD format at this time that I know of is
House of Flying Daggers), and a good picture viewer. The double-edged sword of the picture viewer function is this: it reads Memory Stick Duo, so you can take pictures with a digital camera and display them straight from there to the PSP with no preparation, but why would you want to do that since nearly every digital camera released these days has a viewing screen? Sure, it might be larger, but that's no reason to buy a completely new device for a 1 in.² improvement in screen real estate.
The one exceptionally wonderful thing the PSP does is the games. Metal Gear Ac!d, Lumines, and Wipeout Pure are all outstanding games. Ac!d is the type of game one should play on a road trip-- it can take multiple tries at a level to beat it, it takes about 30-45 minutes per try until you beat it, and (to my knowledge) you can only save between levels. Wipeout Pure is a good game for the 20 minute restaurant waits. A race is about 5 minutes long, and it's got some seriously groovin' tunes to enjoy by the likes of Elite Force, Paul Hartnoll (1/2 of the now-extinct electronica legends Orbital), and Aphex Twin. Lumines is a game to play either with the sound down in small increments, like Tetris, or with the lights down low and
a good set of professional-quality circumaural headphones, enjoying the music's reaction to your playing in what can be a very synaesthetic experience, akin to the oft-overlooked PS2 gem
Rez.
Is it bulky? Well, it doesn't have to be, but with such a muck-up-able screen surface, you'll need a case, so yeah, it is, especially if you opt for the spy-case option. Nintendo, despite thinking the average gamer has three hands (evidence: N64's controller, DS's two-handed layout plus touchscreen with stylus), has definitely got one thing right: A portable game system should be foldable, with a durable, scratch-resistant coating on the outside.
And the Game Boy wasn't greyscale-- it had four shades of cabbage green!