More information on stackoverflow.

Remove files of the submodule and unregister from .git/config

git submodule deinit -f <name>
rmdir <name>

Remove internal git submodule storage

rm -rf .git/modules/<name>

Optionally delete .gitmodules files if no more submodules left

rm .gitmodules