This tests the behavior for configs with start_action=start during reloads of the config (updates/removal).