[wplug] Interesting shell problem (possibly OT?)

[Mark S.]

Love it!

Can think of more On Topic....


On Fri, Jan 7, 2011 at 3:34 AM, Vance Kochenderfer <vkochend at nyx.net> wrote:

> Absolutely not off-topic, IMO.
>
> Nathan Embery wrote:
> > Apparently, find or it's subshell is exiting early for some reason, or
> > at least not returning all the files correctly. The directory contents
> > are virtually guaranteed to change while this is processing, but I
> > wouldn't think that would be the issue since this is fairly common (esp
> > things like `find . -type f -print | xargs -n 1 rm` etc)
>
> Hmm, I'm not too experienced with using process substitution to
> feed a while loop as you're doing, but in theory it ought to work.
> Bash should handle it by creating a named pipe or a file in
> /dev/fd, so you shouldn't run into a limit on the amount of data
> that can be transferred to the loop.
>
> I would suggest instrumenting the script to provide additional
> details as it's going through the directories.  To start, I'd add
> this after the while loop (make it the line before "else"):
>  echo Removed $ACOUNT of $MCOUNT files checked
>
> This would give you a cumulative directory-by-directory count you
> could compare to the expected values.  If this is coming up short,
> try changing the last line of the while loop to:
>  done < <( find $MAILDIR/$i/cur -type f | pv -l )
> or if you don't have pv installed,
>  done < <( find $MAILDIR/$i/cur -type f | tee >(wc -l >&2) )
> to see how many files the 'find' command is picking up.  If it is
> getting the right number, but $MCOUNT is wrong, then you've run
> into some limitation of bash, I think.  (If it is a process
> substitution limit, the second form may be less useful in picking
> it up as it uses process substitution to determine the count.)
>
> A general cleanup is probably also in order - it's good practice
> to quote variables so that filenames with spaces get properly
> handled (e.g., use "$fname" instead of $fname).  The script will
> also fail for filenames that contain a newline, but this is
> admittedly a pathological case.
>
> The fastest and most reliable way would probably be to avoid the
> shell looping construct entirely and use a 'grep -R -l -Z' command
> in /var/mail/vmail to feed a list of names into 'xargs -0 rm'.
> But that wouldn't give you the file counts, which I gather are
> important.
>
> Vance Kochenderfer        |  "Get me out of these ropes and into a
> vkochend at nyx.net          |   good belt of Scotch"    -Nick Danger
> _______________________________________________
> wplug mailing list
> wplug at wplug.org
> http://www.wplug.org/mailman/listinfo/wplug
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.wplug.org/pipermail/wplug/attachments/20110107/1883acdb/attachment.html