Tell you what. I'll start worrying under one of three conditions. One, Donald Trump actually wins in November. Two, he doesn't win and instead those who voted for him actively begin a second US Civil War. Three, some outside group creates conditions that mandate a World War III.
Those right now are, in entirety, the things that are on my SJHTF radar, and they are listed in order of presumed likelihood.