Dreamhost VPS
trevorturk: If you're doing Python, do consider Google App Engine. If you're doing Ruby, Heroku is a great option. Otherwise, I'd probably recommend using Slicehost. They have a great "articles" area on their site that has lots of information about setting up your own server. Plus, you can add backups for $5 a month, which is a good safety net in case something goes wrong.