Strange qMail issue (google hunt = nothing useful)

[GeekApp]

Okay, so here's the annoying issue; both qmail and postfix are displaying this behavior.

I have one email address @geeklan.com

qMail has the following in the log files:

Aug 27 11:15:06 GeekLAN relaylock: /var/qmail/bin/relaylock: mail from 209.85.212.179:53157 (mail-vw0-f179.google.com)
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4625]: Handlers Filter before-queue for qmail started ...
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4625]: from=*****@gmail.com
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4625]: to=*****@geeklan.com
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4625]: hook_dir = '/usr/local/psa/handlers/before-queue'
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4625]: recipient[3] = '*****@geeklan.com'
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4625]: handlers dir = '/usr/local/psa/handlers/before-queue/recipient/*****@geeklan.com'
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4625]: starter: submitter[4626] exited normally
Aug 27 11:15:06 GeekLAN qmail: 1251396906.607598 new msg 4718644
Aug 27 11:15:06 GeekLAN qmail: 1251396906.607648 info msg 4718644: bytes 1708 from <*****@gmail.com> qp 4626 uid 2020
Aug 27 11:15:06 GeekLAN qmail: 1251396906.613327 starting delivery 9: msg 4718644 to local *****@geeklan.com
Aug 27 11:15:06 GeekLAN qmail: 1251396906.613370 status: local 1/10 remote 0/20
Aug 27 11:15:06 GeekLAN qmail-local-handlers[4627]: Handlers Filter before-local for qmail started ...
Aug 27 11:15:06 GeekLAN qmail-local-handlers[4627]: cannot create temporary file - (13) Permission denied
Aug 27 11:15:06 GeekLAN qmail-local-handlers[4627]: cannot read message from stdin
Aug 27 11:15:06 GeekLAN qmail: 1251396906.616487 delivery 9: failure:
Aug 27 11:15:06 GeekLAN qmail: 1251396906.616528 status: local 0/10 remote 0/20
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4629]: Handlers Filter before-queue for qmail started ...
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4629]: from=
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4629]: to=*****@gmail.com
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4629]: hook_dir = '/usr/local/psa/handlers/before-queue'
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4629]: recipient[3] = '*****@gmail.com'
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4629]: handlers dir = '/usr/local/psa/handlers/before-queue/recipient/*****@gmail.com'
Aug 27 11:15:06 GeekLAN qmail-queue-handlers[4629]: starter: submitter[4630] exited normally
Aug 27 11:15:06 GeekLAN qmail: 1251396906.625173 bounce msg 4718644 qp 4629
Aug 27 11:15:06 GeekLAN qmail: 1251396906.625262 end msg 4718644
Aug 27 11:15:06 GeekLAN qmail: 1251396906.625768 new msg 4719109
Aug 27 11:15:06 GeekLAN qmail: 1251396906.625854 info msg 4719109: bytes 2190 from <> qp 4630 uid 2522
Aug 27 11:15:06 GeekLAN qmail: 1251396906.629426 starting delivery 10: msg 4719109 to remote *****@gmail.com
Aug 27 11:15:06 GeekLAN qmail: 1251396906.629465 status: local 0/10 remote 1/20
Aug 27 11:15:06 GeekLAN qmail-remote-handlers[4631]: Handlers Filter before-remote for qmail started ...
Aug 27 11:15:06 GeekLAN qmail-remote-handlers[4631]: from=
Aug 27 11:15:06 GeekLAN qmail-remote-handlers[4631]: to=*****@gmail.com
Aug 27 11:15:06 GeekLAN qmail-remote-handlers[4631]: hook_dir = '/usr/local/psa/handlers/before-remote'
Aug 27 11:15:06 GeekLAN qmail-remote-handlers[4631]: recipient[3] = '*****@gmail.com'
Aug 27 11:15:06 GeekLAN qmail-remote-handlers[4631]: handlers dir = '/usr/local/psa/handlers/before-remote/recipient/*****@gmail.com'
Aug 27 11:15:09 GeekLAN qmail: 1251396909.467656 delivery 10: success: 209.85.217.45_accepted_message./Remote_host_said:_250_2.0.0_OK_1251396908_25si1202 529gxk.23/
Aug 27 11:15:09 GeekLAN qmail: 1251396909.467723 status: local 0/10 remote 0/20
Aug 27 11:15:09 GeekLAN qmail: 1251396909.467752 end msg 4719109


I'm guessing it has something to do with these lines:

Aug 27 11:15:06 GeekLAN qmail-local-handlers[4627]: Handlers Filter before-local for qmail started ...
Aug 27 11:15:06 GeekLAN qmail-local-handlers[4627]: cannot create temporary file - (13) Permission denied
Aug 27 11:15:06 GeekLAN qmail-local-handlers[4627]: cannot read message from stdin
Aug 27 11:15:06 GeekLAN qmail: 1251396906.616487 delivery 9: failure:


FACTS:
- No server configuration has been changed that deals with mail in any way - ever
- I am an experienced UNIX Admin (~10 years behind me) and this issue has completely and utterly stumped me. I'd strace the pid, but it's impossible to catch it fast enough. That would tell me everything I need to know.
- This issue has, quite literally, popped up on it's own with 0 interaction. Mail was working two days ago and then stopped.
- I am not using any of the AV Applications.
- Hunting google for similar messages comes up with a few posts on the parallels forums, but the one suggestion was to chmod 777 the spool directory - no thanks.

I'll be damned if I can figure out what path it can't write to due to permissions. I'm at a loss and losing a lot of valuable clientele. My host has told me to do a fresh install of the OS/CP on my server, which is so far beyond unacceptable I had to tear the representative a new one.

I need help, please.

As an addendum;

This is only happening for one domain (geeklan.com). Removing the domain and recreating it temporarily fixes the problem. All other mail delivery functions on all other domains work fine.

I can't extract an answer from Parallels or their dead forums.

[GeekApp]

I have rebuilt the entire mailqueue, but that was obviously not the issue from the beginning (it's failing to deliver to a specific user only).

I've switched back to Postfix, but the same problem exists with some further caveats;

Postfix shows nothing in the log.
No bounce comes either. The message just disappears into the ether.

Oh, to wish for the day that Parallels would actually support the bugs they crap out on us. This only happened after 9.2.2 was installed, so there's the start of it.

[scott]

Interesting one there, recreating the domain would zap /var/qmail/mailnames/geeklan.com/. The Maildir's under that do contain a tmp/ sub directory that at least for qmail should be owned by popuser. Have you checked the permissions on those directories?

The other thing to look at would be /usr/local/psa/handlers/before-local/ to see if there are any apps living in there. I can see a condition where a script that is broken in that directory (drweb for example) could cause a failure because it cant write/read from something unrelated to the mail queue.

Last one to check would be the individual mailboxes .qmail files. My gut tells me that these arent the problem though, because youd get a much more specific log message (from qmail anyway) on that.

[GeekApp]

Hate to bump this one up again..

But I've still go nothing. As said above, even flipping over to Postfix doesn't help. Different error, same overall issue.

Recreating the domain is still only a temporary fix and there is no telling how long it will last before it blows up again (Generally <1 day).

I've googled the hell out of the error and still have nothing. Naturally, Parallels is quiet on the subject.

[mikeshinn]

Hmmmm, if recreating the mail domain solves the problem that sounds like its a permission problem in the Maildir itself. Does this domain use any maildrop rules or anything that might drop mail in certain folders, or might change permissions on the mailname domains tree? Also, do you use selinux on this box and have you checked those logs to see if its a policy issue?

[GeekApp]

SELinux is off.

I have made no changes to how qMail/Postfix function by default in the PLESK Environment. Permissions of all directories within the Maildir are consistent with a clean install of PLESK.