The handlers are always there, but we only get carbons events when the
server sends them to us, which is only when we enabled them.

fix #3104