The president said the old structure was "never thought of as being much." But once a historic building is gone, it's gone ...