What if you did not get the benefit from this public education? Now you're discriminating against the uneducated, who may have their own opinion on education policies that they should have the right to vote on.
While I agree that everyone should have literacy and calculus and all those other skills that we nowadays consider "basic" or "elementary", and everyone should have access to them, if not be required to go to school up to a certain age / level... you can't assume everyone can or does, through no fault of their own.