Jonathan Hilgeman

Everything complex is made up of simpler things.

Archive for March, 2008

Improving Email Delivery

Mar-13-2008
email

Recently I was reading a forum post where someone was having a problem with their newsletters not being delivered to most of their recipients. I ended up writing a lengthy response with some of the different e-mail delivery tips and tricks I’ve come across over the years. Some of these are specific to PHP mailing applications. So if you want to get your e-mail into someone’s inbox, read through these items:

1. Limit the number of recipients to 1 per e-mail.

2. Use phpMailer (http://phpmailer.codeworxtech.com/), which is a free PHP application (it just recently was picked up by a company, but it’s still free from them). phpMailer gives you a lot more control over the different options when you’re creating mails, and gives better structure to the e-mail than the built-in PHP mail() function. For example, you can add “friendly” names like “Bob Johnson” to your To/From addresses, and later add HTML content or attachments if you want. You should also change the X-Mailer header within phpMailer to be something like boltMail (just a random name). This should help avoid any sort of spam filters that check for mails that originate from a web programming language like PHP or ASP.

3. Don’t use CC if you can help it. Preferably send to 1 recipient and if someone needs to be copied, use BCC.

4. Check your e-mail content to make sure you’re not using words or phrases that you see in other spam e-mails a lot. I’ve never used this service, but I’ve heard lots of good things about InboxInspector, which is supposed to check your e-mails to see how well they fare against different firewalls and spam filters, and how they’ll look in different clients. http://www.mailchimp.com/add-ons/inboxinspector/ (I don’t know if it costs anything)

5. DNS is becoming more and more important. Specifically “reverse DNS” (or pointer / PTR records) is a type of record set up by your hosting company or by THEIR internet provider that lets them update the IP address of your mail server so that when some 3rd party “asks” to see what domain name an IP address belongs to, they’ll be told that it’s your domain name / mail server. Setting up rDNS isn’t something that is done overnight usually, so put in the request and explain to your hosting company what you’re trying to do. They should be able to help you, and if they can’t, then they’re probably not a very good hosting company, and you can find a hundred better ones. I personally use JaguarPC (jaguarpc.com) and have been with them for about 5 years now (but ask the ISP about rDNS before signing up).

6. Someone on the forum had mentioned DomainKeys and SPF (Sender Policy Framework). What are those? Here’s an explanation: You could write a snail mail letter, put someone else’s “From” address on the envelope and drop it in just about any mailbox in the U.S. and it would probably still reach the recipient. However, that recipient has no way of knowing whether it’s REALLY from you or someone else pretending to be you. If the postmark said it was sent from a post office in Arizona and the recipient knew that you ONLY used the post office next to your house in California, then he/she would know that it probably was not really from you. This same idea, but with e-mail, is where SPF and DomainKeys come in. They are basically lists of mail servers that YOU have said, “Okay, these mail servers can send mail using e-mail addresses with my domain name in them.” Then, on the recipient’s side, when the recipient receives an e-mail, it goes out and looks at the domain of the “From” e-mail address and looks up that list of mail servers. Once it knows what mail servers are the ONLY ones approved to send mail from that domain, it then looks at what mail server DID send that message. If that mail server isn’t in the approved list, then the recipient can say, “This message is probably coming from a spammer who just faked their From address.” Whoever controls your DNS (possibly your hosting provider) should have access to set up these tools. If you have a network administrator that does these things, then have him/her go to http://www.openspf.org/ for some help with SPF and http://domainkeys.sourceforge.net/ for help with DomainKeys.

7. Blacklists are quick ways to stop your e-mail from being delivered. Blacklists are simply services (usually free) that have lists of mail servers that send out spam or for whatever reason, should not be trusted to send “good” mail. Try this tool here to check your mail server’s IP address on all the major different blacklists: http://www.mxtoolbox.com/blacklists.aspx    Unfortunately, blacklists are tricky beasts that all work differently. Some have actually died but the service is still available and simply never updated with real content. Normally, no self-respecting mail administrator would use one of these lists, but you should still make sure you’re not on any of the others. This tool should also give you links that take you to that particular blacklist’s web page so you can see details about if, (sometimes) how, (sometimes) why, and (sometimes) when you were blocked. Most blacklists will give you a way to de-list yourself, and it usually takes them at least 24 hours to process your request. Just make sure you’re not having to de-list yourself multiple times at the same blacklist – that would indicate something evil afoot.

8. The way to get yourself instantly blacklisted is to have an open relay. If you, or someone on your IP address (sometimes even close to your IP address) has one of these, then you need to figure out a way to shut it down. An open relay is simply a term that means a mail server that has no proper security set up, and basically allows ANYONE to send e-mail to anyone else for free. Open relays are used by spammers a LOT because they can send their junk mail through without any worries in the world and it doesn’t affect them or cost them a thing. Spammer heaven. If you’re unlucky enough to be in the same range of IP addresses as an open relay or someone else that has been marked as a spammer, then you need to contact your hosting company and have them deal with that other person – not much you can do.

9. If you have any mail forms on your site, then you may want to try adding captcha onto them. Mail forms, while not exactly open relays, can almost be used as ones if they let the visitor specify the “To” address anywhere where it could be changed (even in a hidden field). Sometimes all it takes is one spam message to somebody who reports it to the blacklists… Fortunately, the blacklists are there to help, not to harm, so most will give you second chances and help you with advice on how to prevent yourself from getting listed again. Usually they don’t mention “open” forms like this, because they’re not too common, so this is a “just in case you have one of these….” comment.

10. Be careful when sending HTML mail (if you do this in the future) – if you send badly-formed HTML, it can count against you in some spam filters. Also when the message contains a high image-to-text-content ratio, that’s also a red flag (remember all those spams with almost no content but one big image that had stock quotes in it?).

11. Don’t use a lot of all-caps words or lots of exclamation marks like GUARANTEED!!!!

12. Don’t use the phrase “You are receiving this because you signed up for…” etc…

13. ALWAYS include a link for unsubscribing and ONLY send to people who have requested the newsletter.

14. As a worst case scenario, there are whitelisting services. Someone mentioned to me that Hotmail has one, but that would probably be limited to Hotmail users (I might be wrong on that). There is a service called Habeas which is a popular whitelist and they have a lot of cool toys for tracking your e-mails. The downside is the high cost. For just about every project I’ve ever come across, they are prohibitively expensive. So you’ll have to weigh the potential ROI of your e-mails against their cost. If you’re not selling anything, or if you’re not a really big company, then chances are it’s not going to be worth it to spend $5k to $20k just to whitelist your newsletter. (The 5k to 20k is just a rounded range of quotes that I’ve received from them – you would need to check with them for more accurate pricing – just trying to say that they are not a one-time $100 – $1000 solution). But the option is there.

15. If you’re sending attachments, make sure you’re not sending .EXE files, password-protected ZIP/RAR/other-compressed files, .BAT, .COM, .PIF, .ELF, or any other sort of executable program file. If you do need to send a program attachment of some sort, put it in an unprotected ZIP file so it can be scanned by the recipient’s antivirus scanner. Even .DOC files are becoming hard to send without putting them into ZIP files. Generally, ZIP files and images are almost always safe to send. I’ve seen filters and scanners choke at least once on just about everything else, from MS Word docs to Powerpoints to proprietary formats to OpenOffice files. PDFs are iffy – it’s a popular format to send, but many filters and scanners have the ability to “understand” PDF files, and if the PDF files are written by a program that doesn’t do it right, then that could cause problems, too (so stick it in a ZIP file to help).

16. If you get blacklisted on Earthlink, don’t bother trying to contact their abuse department. Their abuse department is an unmanned voicemail machine. I was blacklisted once on there, and their automated service kept thinking that I was blacklisted. After several e-mails, voicemails, and finally a BBB complaint, I got nowhere.

I’m sure I’ve left a few things out that I’ll probably slap my head later and say, “D’oh – how could I not include that?” but this should give you a decent head start on getting your e-mails through to your recipients.