It goes back to the original introduction of the New Beetle for 1998; in Europe, cassettes were still popular during the planning stages of the New Beetle, next generation Golf and Jetta platforms. You have to remember it often takes 2-5 years of planning ahead to create a new vehicle platform: in the current ads for MB "C" class, MB says it took 8 years of planning and developement for it came to market. Assuming for a minute that this means that the vehicle design started in 1995 or before, cassettes were still fairly popular in Europe: CD's were far more expensive there than in the US. Records (vinyl) was still being produced (and still is by some smaller labels!), while it had all but died in the USA. In Asia, at the time, cassettes and records were still extremely common.. and CD's were expensive.
In the other VW models at the time, cassettes were also standard in most models, with a dealer installed option of cd players or cd changers. It takes time to arrange with vendors to take an existing unit and modify it to adapt to the factory wiring harness that a auto manufacturer has designed for the audio system.
The Beetle had an accessory CD changger that goes into the console, as space is very limited in the trunk (boot). And although the US market is important to VW/Audi, its still only a fairly small market in terms of total sales, so unlike the Asians who build various models for the whatever the market is, the Europeans tend to build one design to their home market and try to adapt it as needed to individual countries. If VW were to sell something like 300,000 vehicles here, over a limited number of platforms, then things would be very different in the way the vehicles are equipped and sold in the US. As it stands, just one Toyota model, the Camry; (sold 473,308 in 2007) outsold VW,Audi, Porsche, & Saab put all together and then some...
I received part of this information from the VW DSM for our area when I first saw the NEW Beetle in 1997 and asked the same question you have... Why no CD in the dash even as an option? I hope this helps, a car nut.