Honestly, I can't get myself to worry about "societal damage"... what would the contact mean for individuals? Medical and technical advancements would extend our lifespans and could add enjoyment to our lives.
A civilization capable of space travel doesn't seem that would be so interested in slaving or torturing humans for the sake of it. Would "our culture" disappear? I still doubt it. It'd be kept as History.
I like visiting museums and learning about the history of ancient civilizations but by no means I'd like to live in the any of those past environments.