We've heard you referred to more than once as "one of the inventors of the Internet." Not to make you blush.
Well, I don't think that's true. Many people today want to be thought of as the Father of the Internet, but I certainly don't qualify.
OK, let's put it another way: The infrastructure and rhythm of communication of today's Internet owe a great deal to your efforts.
There's definitely some engineering I did that helped make the Internet actually work. There's a big step from inventing it to making it work. I didn't design the protocols. I didn't come up with the ideas. I'm an engineer. A damn good one, perhaps, but an engineer.
Could you describe the sendmail team today?
It's like a series of rings. At the core are Greg Shapiro and Claus Assmann, who both work here at Sendmail, Inc. They implement the most lines of code. I'm not implementing many lines of code anymore, but I am talking with them extensively and guiding them through, and that does include going into how things should be implemented. But they're the primary coders right now.
If you extend the ring out a little bit, there are the people in the sendmail.org crew, who help with front line support, and most of whom also act as early testers - I mean, hardcore "we'll pound on it" testers. To a certain extent they'll also pour back in suggestions for bug fixes or new features, often contributing suggested code. So those come back to Greg and Claus and me, and then we actually put it into the base code line. There are rings outside the sendmail.org crew, too: people who are fairly regular contributors, people who may contribute once in their lifetime, people who don't contribute anything, but do report bugs, and so on.
Is it just the three of you who have commit access to the code base?
Theoretically, it's anyone at Sendmail, but as a practical matter, it's Greg and Claus and I who are making actual commits to the sendmail MTA code tree. Of course, there are other code trees for different projects.
Paul Vixie has talked about the mindset and the resources it takes to bring Open Source efforts up to the level of bona fide software engineering - rigorous methodology, commercial-grade development tools, a substantial QA effort, things like that. At what point did sendmail make that transition?
I think it was about the time I was working on version 8. A couple of things happened there. I hadn't been working on sendmail for a number of years, and I had matured. I had also, before version 8, expected something to take over from sendmail, and nothing had. That really surprised me. At that point, it was clear that sendmail was here to stay, and so I wanted to beef it up a lot. There were some things that just weren't that great.
Also, frankly, computers had come along. I started work on sendmail on Version 6 Unix, and things like reliable signals didn't exist back then. And so there was a certain mindset that, well, it's just impossible to do a really rock-solid piece of code on Unix. In fact, with early versions of Unix, that was true. It's not anymore, because the people who have designed the Unix code base, people like CSRG at Berkeley and so forth, really took that as a serious goal: We want to be able to write things that will run solidly forever.
In all fairness, I had had an attitude from early on that sendmail should never, ever lose mail. I mean, it was just essential. If people didn't trust email, email became unusable. So even back in the version 4 and version 5 days, sendmail was very good about not losing messages - for instance, making sure they were committed to disk in such a way that if the system crashed, or sendmail crashed, it could pick up and recover. Yeah, it wasn't perfect, but what is?
And there was another thing, too. At about the time I was working on version 8, Bryan Costales started writing the sendmail book. He was originally going to document version 5, but I said, "Look, I've got this new version coming out," so he started beta testing that. And he's one of these people who, because he was writing the book, tried every bizarre combination of options you could possibly think of. He'd send chapters for me to review that said things like, "Don't do this, because sendmail will do something weird," a core dump or what have you. And I'd look at it and say, "Well, that's pretty stupid." And I'd fix it, and then I'd just scribble "fixed" in the margin so he could take that out. I think it was actually Bryan's documenting it, and his insistence on documenting even the bugs, that made me want a lot of those bugs to go away. It was embarrassing otherwise. [laughs]
One of the interesting things about O'Reilly & Associates is the way a project that began as documentation has become a feedback mechanism for the developers of the software. It's really become an intrinsic part of the Open Source process.
Oh, absolutely. Bryan is really very much responsible for making sendmail version 8 happen. Because of the book, and in two ways. First, as I said, he spurred me to do better. Besides that, though, having a book out describing sendmail made it a lot more accessible to people. And I can only begin to estimate what that was worth in the long run.
Obviously, email has changed people's lives in part by letting people communicate in ways they couldn't before. Some of the consequences have been political: as with the fax machine during the Tienanmen Square uprising, email and the Internet have made it much harder to keep things hidden and keep people from being heard. How do you feel about your part in making that happen?
Oh, I think it's great. I mean, the world is a smaller place than it was before. And we can see that very clearly today. There was the case of that girl in Kosovo who was corresponding with that guy at Berkeley High - that made some news. Email played a part in Tienanmen as well. But even before that, during the Cold War, when I was at Berkeley, I had some guy who wrote me asking for assistance getting sendmail configured. He wanted something really off the wall; I don't remember what it was exactly. So I helped him get this thing configured, and later he came back and said, "Well, I guess I should tell you what this is all about. I'm a teacher in the Chicago public schools, and we've got a pen pal arrangement with a class in the Moscow public schools, and we exchange email." But of course there weren't any direct network links back then, so it had to route through some weird network thing. Sendmail made that possible. And so you had these schoolkids in the Soviet Union exchanging email with schoolkids in the United States. You couldn't do it by telephone because of cost and time zones, and letters are just so slow, and who knows what percentage of them get through? Email was truly an enabling technology for these kids to get to know each other. That, to me, was a pretty incredible thing.
Tim O'Reilly has pointed out how Open Source implementations of standards help keep those standards open by keeping the proprietary players honest. What role does the sendmail team play in developing and reinforcing open standards?
The main answer is the IETF, that being the primary standards-setting body that's relevant for Internet communications. The IETF is an interesting organization. You're not elected; you just go, if you want to. There are people who are active and really do things, and there are the lurkers who go and listen and then go back to their companies and tell people what happened. I've been doing things with the IETF for a number of years now, some more intimately, some more casually. Greg Shapiro is on one or two of the working groups, and I'm also working with a couple of them. This allows us to help direct the standards.
The other big one, though, is sendmail's incredible installed base. It's one thing to have a published standard. It's quite another thing to get people to use that published standard. And in particular, because the IETF doesn't like to make old implementations stop working, most standards now are described as extensions: you may put in this extension to the mail protocols, but you're not required to do it in order to interoperate. There are a few instances where they've said, "You must do this," but mostly it's things that extend existing protocols, often in very important ways. For instance, Delivery Status Notifications were one fairly complex and I think very important standard that went through a few years ago. I made comments on it while it was in the standardization process. In fact, I implemented it before it was passed as a standard - although I don't think technically it is a standard yet; they've got this whole hierarchy of stages that it goes through over time.
It can start looking pretty byzantine sometimes.
Well, they do it for a reason, although sometimes it's hard to remember that. Anyway, DSNs took off, and a lot of that was because sendmail implemented it. Other mailers have as well, although a lot of the vendor versions haven't yet - I don't know why. There are other things that we have not implemented in sendmail yet, mostly because, in my opinion, we didn't get as much leverage from them. Pipelined SMTP is one example. We will implement those in the future. They're good standards. They're just not as critical for making your system run smoothly.
What's the timeline on things like SMTP authentication and LDAP?
SMTP authentication is scheduled to be in the next version of sendmail. We seem to have that together. As for LDAP, we've had limited LDAP support for some time. The next version will have better LDAP support, and the version after that will probably be better still. The LDAP schema for mailing lists has yet to be defined, and rather than picking one of the vendor versions and doing it, we're going to wait for a standard to shake out - and again, we're going to try to drive that standard. I believe Greg Shapiro is working on that group.
One of the changes in sendmail 8.9 that got a lot of attention was turning off relaying by default. Was this the influence of Mr. Vixie and the MAPS project? I suppose this is an interest you guys have in common.
A couple of years ago, Paul came to me and said, "Sendmail shouldn't do relaying by default." And I said, Yeah, but that would break too many things, and I wasn't willing to make an incompatible change like that. And then it became clear that Paul was right. You know, the first and second derivatives of spam were positive, and probably the third derivative as well, and something had to happen. So I came up with a bunch of potential features - I actually worked them out at an IETF meeting in D.C. with John Beck, another one of the people on the sendmail support list; he's also the sendmail guy at Sun. And so we just sat in the hallway and figured out what the defaults should be, and what ways to tweak them. So Paul got me thinking in that direction, although I needed to be inundated with more spam before I finally decided to go ahead and do it. It wasn't as painful as I thought, actually. I had a few people who were annoyed with me about it, but I gave them an easy way to set it back to the way it was before, and most people accepted it. I think spam had gotten bad enough that everyone breathed a collective sigh of relief.
At a practical level, what are the biggest security and network abuse issues for a sysadmin today, especially related to mail systems? If you had to make a short list of "be very afraid" issues, what would they be?
Oh, that's a big, big topic. There are so many security issues going on. If you read the BugTraq list, it's really an incredibly active list. There are a huge number of problems in things like Web servers, and a disturbingly large number of them, probably 60 or 70 percent, are Microsoft operating system-based things. They fall into classes like buffer overflows, and denial of service attacks, and so forth. There's just a huge set of them.
For the email administrator, the thing I'm probably most worried about today are the email-borne viruses. Again, these are much worse on Microsoft platforms than on Unix platforms because of the tendency there to make everything so powerful without thinking about the consequences of that power. There are things like attaching Word documents with macro viruses in them - it's not directly email, but email is how it gets carried around. We saw this with Melissa. We've seen it with a bunch of viruses today.
Viruses bother me for a couple of reasons. One is the potential for a real meltdown. The other, though, is that one of the things I've tried for with sendmail is to make mail be trusted, make it a reliable medium, one you can actually use to communicate. In that sense, spam is a problem, but viruses are even worse. If you get something by email and are afraid to read it because you don't know what it's going to do to your computer, then we have a real problem. It reduces the usefulness of email, and indeed, it reduces the usefulness of the Internet. It's a social process, but it's still a security issue.
Are there features in sendmail that people should be aware of but aren't?
Oh, there are probably dozens of them. One that comes to mind, a very simple one, is the fallback MX option, which lets you redirect mail that has failed the first time to another location. It essentially acts as a lowest possible priority MX record for all hosts. For example, if you've got a mail system that's got a lot of traffic going through it, you have another machine that you dedicate to the slow mail, the stuff that didn't go through the first time, where presumably you're less concerned about how quickly it goes because the other end's being slow. So you set your initial connection timeout to something low - five seconds, ten seconds, whatever's right for your site - and you set the fallback MX on your main site to this fallback host. That way the mail that's going to go through quickly just goes fsssssssst right through your main server, while the stuff that's going to be slow (because the other end is either slow to connect or down) goes off to this other machine and doesn't clog up the main machine. It turns out to be just an amazing win. And these days the price of a PC box running FreeBSD or Linux is close enough to zero that it might as well be zero, so it's not really a problem to do it.
What other features are there? It depends a lot on your environment. There are things for firewall configurations - things like the RunAsUser option, which lets you set things up so sendmail's not running as the root user on your firewall - that I'm not sure people are completely aware of. That kind of thing.
Anything else you'd like to add?
Well, a pet peeve of mine is people who directly edit the .cf file instead of using the m4 configuration files. Don't do it! [laughs] I treat the .cf file as a binary file - you should too.
mark durham
18 october 1999
