Plenty of places have been mythologized over the years as glamorous bastions of ill repute, but Tangier in the Forties and Fifties was one of the few that actually lived up to its bad name. Part of an international zone on the North African coast that was officially ruled by nine nations, the city was effectively governed by none of them—so its habitués could get away with things they’d never attempt back home. Spies and arms dealers gathered in seedy cafés; aging Englishmen entertained offers from Arab boys; American heiress Barbara Hutton hosted drug-fueled parties at her house in the casbah. But after the city was incorporated into Morocco in 1956, the expats started to drift away, taking much of the excitement with them, and Tangier became known as a shabby port town, a place you hurried through on your way to Fez or Ouarzazate.

Now, however, the city is on the upswing again, luring a new crowd of in-the-know Europeans as well as a massive influx of government spending, courtesy of Morocco’s young king, Mohammed VI (his conservative father, Hassan II, hated Tangier for its decadent reputation and neglected it for decades). And although the talk among expats these days is more likely to be about scoring good hydrangeas than good hashish, the town still retains much of its gritty, idiosyncratic appeal. Longtime locals are hopeful that Tangier will never be like Morocco’s reigning tourist mecca, Marrakesh, which has lately begun to suffer from its own success, as foreign buyers transform every available corner of the souk into neo-Moorish boutique hotels and British partyers arrive en masse via low-cost fares from London. For many Europeans, Tangier is an ideal under-the-radar alternative, straddling the line between boomtown and ghost town.

“Tangier is one of those cities you either love or despise. I, of course, love it,” says Paris social doyenne Betty Lagardère, who has been renting a summer place here for several years and recently decided to buy. Seated in the back of her chauffeur-driven SUV and dressed in a custom linen djellaba, Lagardère explains that she was drawn to Tangier by the same North African light that captivated Matisse and the same cross-cultural mix that drew composer and writer Paul Bowles, who lived here from 1947 until his death in 1999. “It’s such a mysterious place,” she says. “To understand it, you have to look behind its doors.”

Indeed, Tangier’s charms might not be instantly obvious to first-time visitors: Touring its main public areas, which the city is feverishly renovating in a bid to host the 2012 International Exposition, is a bit like meeting an octogenarian who’s just had her first facelift. But the changes completed in the past year alone have made a dramatic difference. “See those gardens? New,” says Lagardère as we speed past a freshly planted town square. “That fountain? New. A year ago this street was covered in garbage.”