qmail and store-and-forward

[faris]

A very curious thing happened today and I'm going nuts trying to figure out what on earth went wrong.

We do email store-and-forward from one of our servers to the IP of a client's Windows SBS server, using qmail's smtproutes (domain.com:mail.domain.com where mail.domain.com = ip of SBS server) facility. This has always works just fine.

Today the client's SBS server ran out of disk space. When qmail tried to deliver messages, it got a deferral - insufficient resources - from the SBS server. qmail then continued to queue messages and retry delivery, just as it should.

The SBS server was then taken offline to check it.

Unsurprisingly, qmail then showed deferral: Sorry,_I_wasn't_able_to_establish_an_SMTP_connection._(#4.4.1) errors in the logs.

Here's when things went wrong and I just don't understand why. When the SBS server came online again, complete with plenty of disk space, qmail continued to generate the deferral: Sorry,_I_wasn't_able_to_establish_an_SMTP_connection._(#4.4.1) errors.

From the qmail server I could telnet to port 25 on the SBS server with no problems and send a test message.

The only way to get things working again was to restart qmail.

Can anyone suggest why on earth this might have happened? It makes no sense to me.

Faris

[faris]

p.s. Hmm.....

Although the vast majority of stored/queued emails are flowing to the SBS server, I'm one or two have failed with deferral: Connected_to_[ip-of-SBS-server]_but_connection_died._Broken_pipe_(#4.4.2)/

They get requeued and eventually delivered, however.

[scott]

did its IP or hostname A record change at all? It could have been cached by qmail.

To the latter, maybe the MTA on the other side got overwhelmed by the incoming messages. Thats what I would expect to see from one when that happens

[faris]

No change in IP or A record or anything.

We just did a manual test by rebooting the SBS machine. Same thing happened - qmail got stuck in the "can't establish" quagmire and didn't come out of it until I restarted qmail.

I don't think it was too much email load on the SBS though -- there were 200 messages in the queue in total, maximum age 8 hours, but they went out really slowly once I restarted qmail - they didn't all go at once. It took about 2 hours to get most of them delivered. Even before that, the "can't establish" deferrals were about one every 30 seconds or so or even longer - not all at once.

And now there are six left which just won't deliver at all: Connected_to_[ip-of-sbs-server]_but_connection_died._Broken_pipe_(#4.4.2)/

They are all 1.4Mb in size and from the same sender.

I'm wondering if a problem with the broadband connection that the SBS machine is on, or a problem with the AV/Anti-Spam on the SBS machine may be partly responsible for this issue.

The thing that bothers both myself and the customer is that we tested about 8 months ago and it worked perfectly - as soon as the SBS machine came up, delivery started again with no problems.

I take it that there are no known bugs in qmail that could cause this? I can't find any reference on Google to the "can't establish" deferral other than various people suggesting "remote machine broken/local machine's DNS broken".

Faris.

[BruceLee]

What do the Exchange logs say?
Do those mails even come to the Exchange Server?
Is ISA running in front of the Exchange Server?
I would check the router in front of it. "I-can-give-you-any-business-feature-for-just-39.99" routers are a nightmare.
I have seen a lot different errors that where solved by a better router.
I would not exclude SBS AV/Antispam but it does not indicate that it's the root of the problem.
And do you know if the SBS Server is set up correctly?
Maybe the error comes from a misconfigured server in general.

[faris]

Thanks BruceLee,

Many great ideas!

I'll check with the customer on the SBS side of things. He's one of our nicest customers - we mostly only have nice ones but he's at the top of the list - to see what's in the exchange logs.

Faris.