Wiki example, mixer config, include and exclude

[Nevermetyou65]

Hi, I am trying to follow this

https://github.com/allenai/dolma/blob/main/docs/getting-started.md

I am confused with the logic of include and exclude in mixer config
The doc state that

Optional content-based filtering. Default = keep everything. Documents are retained if they match any of the include patterns (or if no include patterns are specified) AND if they match none of the exclude patterns. Pattern syntax is jsonpath.

And for exclude

Optional content-based filtering. Default = keep everything. Documents are retained if they match any of the include patterns (or if no include patterns are specified) AND if they match none of the exclude patterns. Pattern syntax is jsonpath.

Here is a row from the result of running this mixer config

{
    "streams": [
      {
        "name": "getting-started",
        "documents": [
          "v0/documents/*.gz"
        ],
        "output": {
          "path": "wikipedia/example1/documents",
          "max_size_in_bytes": 1000000000
        },
        "attributes": [
          "exp",                           
          "bff_duplicate_paragraph_spans"  
        ],
        "filter": {
          "include": [
            "$.attributes[?(@.exp__whitespace_tokenizer_with_paragraphs_v1__document[0][2] < 100000)]"
          ],
          "exclude": [
            "$.attributes[?(@.exp__whitespace_tokenizer_with_paragraphs_v1__document[0][2] < 50)]",
            "$.attributes[?(@.exp__ft_lang_id_en_paragraph_with_doc_score_v2__doc_en[0][2] <= 0.5)]",
            "$.attributes[?(@.bff_duplicate_paragraph_spans && @.bff_duplicate_paragraph_spans[0] && @.bff_duplicate_paragraph_spans[0][2] >= 1.0)]"
          ]
        },
      }
    ],
    "processes": 16
  }

{'bff_duplicate_paragraph_spans': [[397, 408, 1]],
 'exp__char_length_with_paragraphs_v1__document': [[0, 2488, 2488.0]],
 'exp__char_length_with_paragraphs_v1__paragraph': [[0, 7, 7.0],
  [8, 185, 177.0],
  [185, 397, 212.0],
  [397, 408, 11.0],
  [408, 932, 524.0],
  [932, 1155, 223.0],
  [1155, 1339, 184.0],
  [1339, 1609, 270.0],
  [1609, 2040, 431.0],
  [2040, 2433, 393.0],
  [2433, 2445, 12.0],
  [2445, 2488, 43.0]],
 'exp__ft_lang_id_en_paragraph_with_doc_score_v2__doc_en': [[0,
   2488,
   0.9643100000000001]],
 'exp__ft_lang_id_en_paragraph_with_doc_score_v2__doc_not_en': [[0,
   2488,
   0.03569]],
 'exp__ft_lang_id_en_paragraph_with_doc_score_v2__en': [[0,
   7,
   0.8609300000000001],
  [8, 185, 0.9408700000000001],
  [185, 397, 0.9891300000000001],
  [397, 408, 0.9542700000000001],
  [408, 932, 0.9900300000000001],
  [932, 1155, 0.9732400000000001],
  [1155, 1339, 0.95964],
  [1339, 1609, 0.9844],
  [1609, 2040, 0.99004],
  [2040, 2433, 0.9910200000000001],
  [2433, 2445, 0.7232900000000001],
  [2445, 2488, 0.07972000000000001]],
 'exp__ft_lang_id_en_paragraph_with_doc_score_v2__not_en': [[0, 7, 0.13907],
  [8, 185, 0.05913],
  [185, 397, 0.010870000000000001],
  [397, 408, 0.04573000000000001],
  [408, 932, 0.009970000000000001],
  [932, 1155, 0.026760000000000003],
  [1155, 1339, 0.04036],
  [1339, 1609, 0.015600000000000001],
  [1609, 2040, 0.00996],
  [2040, 2433, 0.00898],
  [2433, 2445, 0.27671],
  [2445, 2488, 0.9202800000000001]],
 'exp__random_number_v1__random': [[0, 2488, 0.88631]],
 'exp__whitespace_tokenizer_with_paragraphs_v1__document': [[0, 2488, 530.0]],
 'exp__whitespace_tokenizer_with_paragraphs_v1__paragraph': [[0, 7, 2.0],
  [8, 185, 38.0],
  [185, 397, 48.0],
  [397, 408, 4.0],
  [408, 932, 109.0],
  [932, 1155, 44.0],
  [1155, 1339, 37.0],
  [1339, 1609, 64.0],
  [1609, 2040, 92.0],
  [2040, 2433, 77.0],
  [2433, 2445, 3.0],
  [2445, 2488, 12.0]]}

I don't understand why this document is retained. The last element of bff_duplicate_paragraph_spans is 1, I thought this row should be excluded.