There could be various reasons why subscriber counts have gone up after the initial sync. In most situations, we've seen that the legacy 'Mailchimp for Shopify' integration was previously installed, and contacts were not added to the list. This could've been with failures on submission to Mailchimp with merge tags being required, rejected by the API or just missed entirely. When ShopSync was activated, these customers have finally been added the list. We've seen many examples that the legacy app simply wasn't working correctly.
If you delete a customer from an audience, it removes their history from Mailchimp. Unsubscribing keeps a trace of the customer, and we use it as baseline for our logic. If there's no record of the customer in Mailchimp, we will then add them a subscriber if they've Accepted Marketing in Shopify.
If you're at all concerned that subscribers on a list that shouldn't be or you think it's been too long of period since these customers were contacted, we recommend sending out a light email - maybe something to the effect of "We're updating our customer newsletter software, click here to opt-out if you don't want to receive further messages from us". Another option is to bulk unsubscribe these users based on their last purchase.