Just image the endurance and structural composition of such Dyson sphere not to collapse around a black hole (I'm not an specialist but it came to my mind)
If you're outside the event horizon, your experience is the same as that of being next to any object of equal mass of the black hole, located at its centre. Just don't go in!