Request to placeholder leads to wrong behaviour

[tadjik1]

Hi there,
we've noticed a strange behaviour that seems like a bug.

In some cases user receives documents from placeholder service even when there are layers and categories parameters in request and they are not matched with placeholder response.

Example: https://search.mapzen.com/v1/search?text=myrtle%20ave,%20New%20York&categories=transport&layers=venue note: please use your own api_key

In this query you can see layers=venue and categories=transport, so let's assume that we want to find public transport stop. However,

1. query string parsed as

{
  controller:libpostal: {
    parsed_text: {
      street: "myrtle ave",
      city: "new york"
    }
  }
},

2. and then condition for placeholder request with placeholderIdsLookupShouldExecute matches. because it doesn't check if there are categories and layers parameters.

// execute placeholder if libpostal identified address parts but ids need to
//  be looked up for admin parts
const placeholderIdsLookupShouldExecute = all(
  not(hasResponseDataOrRequestErrors),
  isPlaceholderServiceEnabled,
  // check clean.parsed_text for several conditions that must all be true
  all(
    // run placeholder if clean.parsed_text has 'street'
    hasParsedTextProperties.any('street'),
    // don't run placeholder if there's a query or category
    not(hasParsedTextProperties.any('query', 'category')),
    // run placeholder if there are any adminareas identified
    hasParsedTextProperties.any('neighbourhood', 'borough', 'city', 'county', 'state', 'country')
  )
);

3. so finally api returns placeholder results that do not match requested categories:

{
   type:"Feature",
   geometry:{
      type:"Point",
      coordinates:[
         -73.391666,
         43.025018
      ]
   },
   properties:{
      id:"polyline:9125029",
      gid:"openstreetmap:street:polyline:9125029",
      layer:"street", // layer street - but we whitelisted before only venue layer
      source:"openstreetmap",
      source_id:"polyline:9125029",
      name:"Myrtle Avenue",
      locality_gid:"whosonfirst:locality:85978637",
      localadmin:"White Creek",
      localadmin_gid:"whosonfirst:localadmin:404523327",
      street:"Myrtle Avenue",
      country_gid:"whosonfirst:country:85633793",
      locality:"Cambridge",
      label:"Myrtle Avenue, Cambridge, NY, USA",
      match_type:"exact",
      region_a:"NY",
      country_a:"USA",
      county:"Washington County",
      accuracy:"centroid",
      confidence:1,
      continent:"North America",
      region:"New York",
      region_gid:"whosonfirst:region:85688543",
      country:"United States",
      county_gid:"whosonfirst:county:102083153",
      continent_gid:"whosonfirst:continent:102191575"
      // no categories at all, but we whitelisted only transport
   },
   bbox:[
      -73.392243,
      43.022705,
      -73.390992,
      43.028144
   ]
}

As a possible solution I would add one more subcondition to placeholderIdsLookupShouldExecute condition (copy that from placeholderGeodisambiguationShouldExecute condition), so it will be something like:

// execute placeholder if libpostal identified address parts but ids need to
//  be looked up for admin parts
const placeholderIdsLookupShouldExecute = all(
  not(hasResponseDataOrRequestErrors),
  isPlaceholderServiceEnabled,
  not(
    // don't look up for admin parts if categories were requested
    hasRequestCategories
  ),
  // check clean.parsed_text for several conditions that must all be true
  all(
    // run placeholder if clean.parsed_text has 'street'
    hasParsedTextProperties.any('street'),
    // don't run placeholder if there's a query or category
    not(hasParsedTextProperties.any('query', 'category')),
    // run placeholder if there are any adminareas identified
    hasParsedTextProperties.any('neighbourhood', 'borough', 'city', 'county', 'state', 'country')
  )
);

[missinglink]

hi @tadjik1, I was going through the issues today and saw this, could you please open a PR?

[orangejulius]

This is also related to pelias/pelias#670. I think a fix for one is a fix for the other!

[orangejulius]

Update: pelias/pelias#670 has been fixed, but it is not the same issue. Here, this looks like the search with IDs query does not respect layers.
http://pelias.github.io/compare/#/v1/search%3Fdebug=true&layers=venue&text=myrtle%20ave,%20new%20york

(all the results are in the street layer)

[orangejulius]

This looks to be fixed now (likely by #1307, which fixed an issue where the address_search_with_ids query type would return results from layers that did not match what were allowed by the layers param.

The query from my last comment now returns only venues. 🎉

Let us know if there are any more cases like this, we really want to fix any problems with sources or layers parameter handling.