Don't overwrite listener references with null during fragment changes

This should be considered a temporary solution until we replace these controllers with ViewModels.

Closes #1697 (closed), #1704 (closed), #1706 (closed), #1707 (closed).