The first thing to look for is if they have the features you need?
Do you need wordpress?
Do you need email accounts?
Do you need FTP access?
Will you need ecommerce for selling things from your website?
Do they offer databases?
Do they offer SSL certificates?
Next, look at their performance.
What is their uptime? You want a host that is reliable and will stay "up" as much as possible.
Do they offer site backups?
Do they have security against hacking/ DDOS attacks?
Are their loadtimes aacceptable?
Next, look at their customer support.
Can you contact someone 24/7?
Call their support line and ask a technical question. Can they answer it accurately? You don't need to have a problem on a weekend and can't get anyone on the phone who knows anything until Monday. VERY frustrating.
Finally, if they have the features, performance, and customer support you want, look at cost. Some people look at cost first, but using a cheap webhost that doesn't have the features you need or has terrible customer service will make you regret saving a few dollars.
Personally, I like NameHero.com, but you may find one you like better.
PS. - If you need a handyman in Delaware, check out www.delhandyman.com