If a person with a gun attacks you from a distance, then the only defensive weapon that makes sense would be a gun (or other projectile weapon). However, if the attacker is physically close to you, you have more options, and one option to consider is a knife.
Compared to a gun, it is relatively small (or at least less inconveniently shaped) and light, making it less obnoxious to carry and easier to conceal. Plus, it would generally be less restricted by laws, completely silent, less expensive, and not subject to a limited ammunition supply.
However, if the attacker is close to you, having a knife (or even a gun) other than actually in your hand, is not going to be able to protect you. If you intend to defend yourself close in with a weapon, you must first be able to defend yourself without a weapon, at least long enough to allow you to access your weapon. Check out this video to see what an unexpected knife attack could be like: