Built between 1666 and 1679, the castle -- really a pentagonal fortress typical of the Dutch defense system adopted in the early 17th century -- is the oldest surviving building in South Africa and marks the original shoreline. Once the hub of civilian and administrative life, the long-serving castle is still the regional headquarters of the South African Defence Force, though the most invasive force it's ever dealt with are the tourists ambling through its ramparts (and, in a wonderful exorcism of the ghosts of the past, more than 5,000 camp brides and other gay revelers at "The Wedding," a costume ball that MCQP hosted here in 2002). The fort combined local materials with European imports -- note the slate paving stones, taken from Robben Island in the 17th century, and the bell at the entrance, cast in Amsterdam in 1697. It still looks much as it has for centuries.
Get here at 10am or noon sharp if you want to see the Key Ceremony, a kind of changing of the guard (Mon-Fri only). There are also 30-minute tours departing at 11am, noon, and 2pm (ask about the many ghosts that wander the ramparts), or you can explore on your own. Unless you're fascinated with colonial military might, you can skip the Military Museum, but don't pass on the paintings that form part of the William Fehr Collection -- an ideal way to brush up on your colonial history. Witness the crazy scene, for example, in Thomas Baines's spectacular painting The Greatest Hunt in Africa, which depicts the mindless slaughter of 30,000 animals in honor of the visiting Prince Alfred.
Note that, at press time, the quality of food at the central courtyard restaurant at the Castle is suspect. Tenders are being sought for a new dining enterprise, but until the right people move in, don't eat here. Alongside the Castle in front of the old City Hall (from whose balcony Mandela made his first public address after release from prison), the so-called Grand Parade is being rejuvenated for the 2010 FIFA World Cup, when it will serve as a Fan Park for soccer enthusiasts without match tickets.