Wouldn't blame him for leaving but I think a season of not finishing top 4, having done so for so many years in a row, would be a pretty simplistic reason for leaving. If that was combined with another summer of under-investment and there's no prospect of us getting back up there then yeah, I wouldn't blame him.
If we show some intent in the summer (big if, I know) then that might sway him top 4 finish or no.