This is probably one of the least-loved buildings in Norfolk and while I agree it has seen better days, I would argue for a complete restoration to its 1960 glory rather than demolition.
According to Wikipedia (which you can believe... or not), this is a Morris Lapidus building (think Fontainebleau Hotel in Miami Beach). That alone makes it worthy of saving, especially in a city that has done so much to eradicate interesting architecture in favor of a neo-urban Williamsburg aesthetic.
Have a look at these pictures from a circa 1970 brochure to get a feel for what it once was. This was where the city's movers and shakers came to party, even forming a sort of "rat pack" clique in a notorious "key club" which reatured rabbit-eared and bunny-tailed waitresses. Check out the view of Scope under construction behind the couple having breakfast. And note especially the Lapidus flourishes in the entrance area and lobby.
Rather than tearing it down, I would love to see all that restored. I think it has the potential to become a retro hotel/nightclub/lounge destination for a hipper, more urban traveler who's looking for something a bit edgier than the folks at Westin, Marriott, Sheraton or Hyatt currently offer.
http://i665.photobucket.com/albums/vv14/ny_wahoo/GTBrochureCover.jpg
http://i665.photobucket.com/albums/vv14/ny_wahoo/GTWelcome.jpg
http://i665.photobucket.com/albums/vv14/ny_wahoo/GTAccommodations.jpg
http://i665.photobucket.com/albums/vv14/ny_wahoo/GTNationsRoom.jpg
http://i665.photobucket.com/albums/vv14/ny_wahoo/GTNorfolkTour.jpg
http://i665.photobucket.com/albums/vv14/ny_wahoo/GTPoolside.jpg