I have some electronics with the same issue (shaver, inflatable mattress pump): they only run on battery power, and plugging them in is only useful for charging the battery. It seems that what happened is they saved money by not installing a robust power system on the AC end - only a small one that is capable of (slowly) charging the battery, but not actually running the machine directly. The only part that has a robust power subsystem is the output of the battery, so unless you've got a charged battery you can't use them. And they take much longer to charge, than to discharge through use, which is why I can tell there isn't a high-output power supply coming from the AC end.
So, yeah, it's worth paying for a quality battery (and AC power subsystem to boot) - a laptop becomes way less useful without it, so a few bucks spent up-front can prolong the usable lifetime of your purchase substantially. It's one of these hidden issues that's hard to discern from the spec sheets, so you kind of have to go by the brand's reputation unless you can find really detailed user reviews of the particular model you're interested in. Even then, these kinds of issues won't pop up for 1+ years so the reviews will still be of limited value.
Another good thing to do when it comes to batteries in mobile electronics is to load them only with what you need. Instead of that super high-power workstation laptop with the big processor and tons of memory and stupid graphics card and crazy-powerful screen and ultra high-speed DVD/bluRay drive, consider getting a netbook with none of that stuff, and a lot of intelligent power management capabilities and low-power core components instead. This way, you put way less load on the battery, and so the battery can be smaller, and so you can get a quality reliable battery for a reasonable price, etc. Usage time between recharges will also go up (at least, on a per-dollar basis). Moore's Law has driven computer performance to the point where the lion's share of users don't need anything like the top-end performance. So, resist the urge to drool over impressive specs, and instead get a machine with suitable performance and very high build/component quality. The thing about those high-spec machines is that there's a huge incentive to cut corners to keep costs down, and this invariably means compromises with "under-the-hood" things that aren't apparent on the spec sheet: long-term battery reliability, well-integrated power management, long-lasting touch interface, resistance to general wear-and-tear, etc. If you decide you really need the high-spec machine, be prepared to pay for it or compromise in some other way (get a desktop instead of a laptop, typically).