Recast: FindDistanceToWall can return incorrect results on shared edges with Excluded Nav Area

Context

ARecastNavMesh::FindDistanceToWall can be used to find the closest 'wall' on a nav mesh, given a queried position. A wall can be:

• any boundary towards the end of the nav mesh (outer rectangle),
• walkable and non-walkable area
• walkable area and an Excluded Nav Area (Nav Area marked with a Nav Modifier, then specified in the query via Filter to excluded that Nav Area)

Problem

User reported this and provided a 100% minimal repro project. In some geometric situations a polygon edge can be shared by a walkable polygon and an excluded NavArea's polygon. The user has provided a repro project where FindDistanceToWall will then return incorrect points on an edge that is in the middle of walkable area. The bug seems to occur when a part of the edge is on excluded nav area. This makes FindDistanceToWall not always reliable.

Repro project, screenshots and a video are attached internally.

As per reporter:

• Setup an empty level with a navmesh
  • On the RecastNavMesh actor, check the 'Draw Triangle Edges' setting so that the triangles are rendered in the viewport when the nav mesh is drawn (toggle with P)
• Create a new NavArea blueprint 'NA_MyArea'
• Create a NavigationQueryFilter blueprint 'NQF_MyFilter' that is set to exclude NA_MyArea
• Place a NavigationTestingActor to the level.
  • Set its FilterClass to NQF_MyFilter
  • Check its 'Draw Distance to Wall' setting
• Drag the NavigationTestingActor around in the level editor viewport
• A line should be drawn from the actor to the closest wall
• Observe: In some cases the line is drawn towards an edge that is on the middle of the nav mesh, so not a wall. See attached screenshots and video.
• Expected: In those cases, FindDistanceToWall returns an incorrect result. It should always find a point on the nav mesh's outer edge or on the boundary between excluded and included area.