For example, if someone started a bounty, then deleted their account, would the bounty still be active?
Yes, but the bounty couldn't actually be manually awarded to an answer. The best you could hope for would be half of the bounty getting auto-awarded to the top voted answer posted after the bounty start, provided it has a score of at least 2.
I didn't have a source for this, except experience (I've seen it happen), but bobble found a main meta source for me:
Going forward bounties from deleted accounts will be moved to (owned) by Community and will be awarded at the end of the period as they normally would (e.g. when the bounty starter doesn't award and 7 days is up).
-- Nick Craver