One practical tip from someone who has managed over a dozen VPS accounts look for a provider that handles daily backups without you having to remember.
hostpro.com does exactly that—every service gets backed up daily, and I have used those backups to fix damage from a failed PHP upgrade. Their support staff is also excellent, available 24/7, and they helped me debug a slow database query for free just last week. That kind of ongoing help makes a real difference.