What happened to Napoleon's followers after his first and second exile?

When Napoleon Bonaparte was exiled to Elba, what happened to his followers? Were they executed? Also, what happened to his followers after he was exiled again?