So basically you know that an specific SSH key signed a file, but that's all you know. Not very useful? Who's going to maintain a file with allowed signers per key?
A repository only allows signed commits / merges / pushes and simply stores the file in the repo itself.
If you already rely on forges like github/gitlab/... and trust them to generate a file for you (e.g. from the users ssh keys having write access to the repo).
You maintain this file independently for yourself with a "Trust on first use" mechanism (which will follow in a future patch to git)
A git patch currently in progress will utilize the valid-after/before options that you can set on keys in this file to verify the commits/tags with keys valid at the time of the objects creation. This will allow for key rollover and still being able to verify old signatures. (Theres also the revoke file which will invalidate all signatures for a key). In addition to that a "trust on first use" patch will follow that can automatically add new keys with the committers ident to this file (unless the ident is already present with a different key of course, which should error badly).
disclaimer: i wrote the git integration for this