What kind of server should I choose? Shared hosting? VPS? Or dedicated servers?

Whenever you want to create a website or to start writing a blog, there’s one problem you need to solve: What kind of server should I choose? Shared hosting? VPS? Or dedicated servers? What’s the differences among those servers?