Remove specific statistics

You can use the APIs to remove statistical data and prevent adding personally identifiable information (PII) to the search index. For example, you can prevent data that contains an at sign (@) from being indexed.

This feature involves the following APIs.

• StatisticsGetGDPR – Search GDPR data from statistics. Use to get statistics items that match a query pattern. An extension method of IStatisticsClient.
  public static GDPRQueryResult StatisticGetGDPR(this IStatisticsClient client, string query)
• StatisticsDeleteGDPR – Delete GDPR data from statistics. Used to delete statistics items that match the query pattern. An extension method of IStatisticsClient.
  public static GDPRDeleteResult StatisticDeleteGDPR(this IStatisticsClient client, string query)

In addition, you can use these APIs to prevent PII data from appearing in search query results.

ITrackSanitizerPatternRepository

• Add – Add patterns to storage, which, in turn, are used to prevent PII data from search queries.
  string Add(TrackSanitizerPattern pattern)
void Add(IEnumerable<TrackSanitizerPattern> patterns)
• Get – Get pattern by Pattern ID from storage.
  TrackSanitizerPattern Get(string patternId)
• GetAll – Get patterns from storage.
  IEnumerable<TrackSanitizerPattern> GetAll()
• Update – Update existing patterns.
  string Update(TrackSanitizerPattern pattern)
void Update(IEnumerable<TrackSanitizerPattern> patterns)
• Delete – Delete pattern by ID from storage.
  void Delete(string patternId)
• DeleteAll – Delete patterns from storage.
  void DeleteAll()

See also Prevent indexing of PII data.

Example: remove PII data from statistics

To retrieve and remove PII data from statistics, you can search for a range of data:

• Email including domain name, for example "[email protected]"
• Full name, for example, "John Doe"

Using a predefined pattern, you can prevent sensitive data in a custom search query from being saved to the statistics.

public class Sample {
  protected IClient _client;
  protected IStatisticsClient _statisticsClient;
  protected ITrackSanitizerPatternRepository _trackSaniziterRepository;
  public Sample(IClient client) {
    _client = client;
    _statisticsClient = client.Statistics();
    _trackSaniziterRepository = client.TrackSanitizer().TrackSaniziterRepository;
  }

  public void Test() {
    // Setting and add sanitizer patterns.
    _trackSaniziterRepository.Add(new List<TrackSanitizerPattern> {
      new TrackSanitizerPattern {
        PatternString = "admin",
          PatternType = TrackSanitizerFilterType.PlainText
      },
      new TrackSanitizerPattern {
        PatternString = "email",
          PatternType = TrackSanitizerFilterType.PlainText
      },
      new TrackSanitizerPattern {
        PatternString = "*@mail.com",
          PatternType = TrackSanitizerFilterType.Wildcard
      },
      new TrackSanitizerPattern {
        PatternString = "1#1",
          PatternType = TrackSanitizerFilterType.Wildcard
      },
      new TrackSanitizerPattern {
        PatternString = "c[a-e]ll",
          PatternType = TrackSanitizerFilterType.Wildcard
      },
      new TrackSanitizerPattern {
        PatternString = @ "\w+([-+.]\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*",
          PatternType = TrackSanitizerFilterType.Regex
      }
    });

    // Doing Tracking behavior
    var result = _client
      .UnifiedSearchFor(@ "[email protected]")
      .StatisticsTrack()
      .GetResult();

    // Try to get GDPR data by exact term matched sanitize pattern.
    var response = _statisticsClient.GetGDPR("[email protected]", x => {});
  }
};

When using a wildcard pattern, review these Microsoft wildcard samples.

Search, filter, and delete PII data

See Prevent indexing of PII data for installing a sample and verifying the deletion.