Anyone knows how to use includePattern and ignorePattern in flume spooldir source?



0

Hi,

     Anyone knows how to use includePattern and ignorePattern in flume spooldir source? to include specifc file types and ignore '.txt' files? Thanks in advance

 

-Kathir

     


4 Answer(s)


0

Hi Kathir,

Even I have been trying lots of permutations and combinations for the regex used in ignore pattern.. but still no luck. Please let me also know if anyone has been able to succeed.

 

-Madhura


0

Hi guys,

I wrote a few lines of java code to help get what regex would apply. See it and the result below.

Code


import java.util.regex.Pattern;

public class RegexFinder {

    public static void main(String[] args) {
        String[] words = new String[]{"credit_note.txt", "prices_20140324.csv", "reports_20140324.csv", "symbols_20140324.txt"};
        Pattern pattern = Pattern.compile("prices(.+)");
        for (String word : words) {
            System.out.printf("%s %s %s\n", word, pattern.matcher(word).matches() ? "matches" : "does not match", pattern.pattern());
        }
    }
}

 

Result is

credit_note.txt does not match prices(.+)
prices_20140324.csv matches prices(.+)
reports_20140324.csv does not match prices(.+)
symbols_20140324.txt does not match prices(.+)

I hope this guides you in application.

Regards


0

Hi Kathiresan,

includePattern ^.*$ Regular expression specifying which files to include. It can used together with ignorePattern. If a file matches both ignorePattern and includePattern regex, the file is ignored.
ignorePattern ^$ Regular expression specifying which files to ignore (skip). It can used together with includePattern. If a file matches both ignorePattern and includePattern regex, the file is ignored.

For example:

ignorePattern=\.*txt$

Hope this helps.

Thanks.


0

Thanks Michael, it worked:)

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password