A perfect mirror is a materials challenge and potentially a fundamental physics challenge. In comparison we already know roughly how to do the math to figure out how far the surface of your sphere would need to be from the center of a sun or black hole in order to harvest energy (via whatever method) without collapsing, based on the strength of the materials at our disposal. Of course the materials we have might not be strong enough, but it's possible to push the sphere out until we can build it without it collapsing (and then the issue is collecting enough raw material). And given that solar panels work fine on earth, a dyson sphere can only improve solar panel efficiency even if its surface has to be very very far away due to materials challenges.
In comparison, if you can't get your mirror close enough to perfect, the whole idea doesn't work - the laser will be absorbed or diffused too much by the mirror, or the mirror will be damaged from the energy, etc.