Yep! Generally speaking singularities only appear because the math is deficient in some way. Other areas of physics successfully got rid of unphysical singularities by employing better math, it's just that we don't know how to fix this aspect of general relativity (maybe quantum gravity will?)