Anyone knows how to use includePattern and ignorePattern in flume spooldir source?


4 Answer(s)


Hi Kathir,

Even I have been trying lots of permutations and combinations for the regex used in ignore pattern.. but still no luck. Please let me also know if anyone has been able to succeed.

 

-Madhura


Hi guys,

I wrote a few lines of java code to help get what regex would apply. See it and the result below.

Code


import java.util.regex.Pattern;

public class RegexFinder {

    public static void main(String[] args) {
        String[] words = new String[]{"credit_note.txt", "prices_20140324.csv", "reports_20140324.csv", "symbols_20140324.txt"};
        Pattern pattern = Pattern.compile("prices(.+)");
        for (String word : words) {
            System.out.printf("%s %s %s\n", word, pattern.matcher(word).matches() ? "matches" : "does not match", pattern.pattern());
        }
    }
}

 

Result is

credit_note.txt does not match prices(.+)
prices_20140324.csv matches prices(.+)
reports_20140324.csv does not match prices(.+)
symbols_20140324.txt does not match prices(.+)

I hope this guides you in application.

Regards


Hi Kathiresan,

includePattern ^.*$ Regular expression specifying which files to include. It can used together with ignorePattern. If a file matches both ignorePattern and includePattern regex, the file is ignored.
ignorePattern ^$ Regular expression specifying which files to ignore (skip). It can used together with includePattern. If a file matches both ignorePattern and includePattern regex, the file is ignored.

For example:

ignorePattern=\.*txt$

Hope this helps.

Thanks.


Thanks Michael, it worked:)