Having more than one person on a lead/contact is typical, particularly when an assistant helps/work with said contact. I'd like to see followers get notified of when a contact/lead replies to them in particular.
Currently only the owner gets notified of all communications regardless whether the response was for the assigned "owner" or the "follower." The "owner" should not be getting notified for communication between "follower" and lead/contact. Unless specifically requested by "owner".
In addition, I think he follower should also be able to see the contact in their respective opportunities pipeline/stage as any other "assigned" lead/contact.