Extracting Data from Configurations via Pattern Matching

The data model exposes device configurations via the config field of the files field of Device objects. Specifically, the config field contains a list of config line records. To reflect the nesting structure of config lines in device configurations, config line records have a children field which contains the immediate nested config lines of the parent config line. See the data model for more details.

Data can be extracted from configurations by pattern-matching against the lines, using either the patternMatch or the patternMatches functions. We explain both here, though typically, using patternMatches will lead to simpler query logic.

The patternMatches function takes two arguments: the first is a list of config line records (which has type List<ConfigLine>), and the second is a pattern (which has type Pattern<T> for some type T that depends on the pattern). The function returns a list of all successful matches of the given config lines against the given pattern. Each successful match is a record that has two fields: line holds the matched config line, while data holds the matched data. For example, the following query lists all TACACS servers, by device, and includes the text of the matched config line as well:

foreach device in network.devices
foreach match in patternMatches(device.files.config, `tacacs-server host {server:string}`)
select {device: device.name, tacacsServer: match.data.server, line: match.line.text}

In the above query, the patternMatches function is applied to all top-level lines of config in each device. The second argument to patternMatches is the pattern, which is enclosed in back-ticks (``). This pattern matches lines that start with the specific keywords tacacs-server and host, and then continue with a string of arbitrary non-whitespace characters. The function returns each successful match against the top-level lines of config. Because the pattern tacacs-server host {server:string} has a pattern {server:string}, the type of the pattern is Pattern<{server:string}>. Then for each match, match.data holds a record that has a single field named server whose type is a string. Note that the name of the field is determined by the name given to the component within the pattern.

The subsequent section has more details on the syntax and semantics of patterns. Also, see the Examples for more example checks using this capability.

This patternMatch function works similar to patternMatches, but takes a single string as its first argument. Instead of returning a list of all successful matches, it returns a single match, or null if the match fails. For example, the following query lists all TACACS servers, by device:

foreach d in network.devices
foreach line in d.files.config
let match = patternMatch(line.text, `tacacs-server host {server:string}`)
where isPresent(match)
select {device: d.name, tacacsServer: match.server}

Note that the result of patternMatch(line.text, `tacacs-server host {server:string}`) is bound to variable match using a let qualifier, rather than a foreach qualifier as was used in the above example using patternMatches. A let qualifier is used because patternMatch returns a single match, rather than a list of matches. Also note the isPresent condition on the fourth line, which limits the query to just the successful matches.

In most cases, using patternMatches is recommended, since it leads to shorter, simpler query logic.

Details on Patterns

As mentioned above, a pattern is written enclosed in backticks (``). A pattern can be matched against text and also extracts data for named components within the pattern.

A pattern consists of a space-separated sequence of pattern items, where each item is either

• a literal: matches a specific keyword, such as tacacs-server or interface,
• a pattern expression (described below), enclosed in curly braces, such as named components as in the query above ({server:string}).

The sequence of pattern items matches some text if each item matches, in sequence, against the text.

A pattern expression (enclosed within curly braces in a pattern) can be any of the following:

• A string literal, which matches the exact text. For example, {"interface Vlan1"}.

• string: matches any sequence of characters up to the next whitespace.

• number: matches any integer number (e.g. 0, 2, -22, or 98231, but not 3.1415).

• float: matches any decimal number (e.g. 0.1, .5, 3., -2.2 or 98.231, but not 3).

• ipv4Address: matches any IPv4 address, such as 10.22.31.99.

• ipv6Address: matches any IPv6 address, such as 2001:0db8:85a3:0000:0000:8a2e:0370:7334.

• ipv4Subnet: matches any IPv4 subnet, such as 20.30.0.0/16.

• ipv6Subnet: matches any IPv6 subnet, such as 2001:0db8:85a3:0000:0000:8a2E:0370:7334/24.

• macAddress: matches any MAC address, such as AA:bb:33:44:55:66, 00aa.11bb.22cc or 00aa11.bb22cc.

• A literal, enclosed in double-quotes: matches any specific sequence of characters. For example, "tacacs-server", matches the literal string tacacs-server. Note that literals can already by written at the top-level of a pattern as described above. However, it also possible to write literals within pattern expressions (enclosed by curly braces).

• eof matches when there is no input left. For example, {string} {eof} matches "foo" but not "foo bar".

• empty always matches successfully and returns an empty record. This pattern is often useful in combination with | pattern builder, because it allows you to indicate an optional argument in a pattern. For example logging {"host" | empty } {ipv4Address} matches commands like "logging host 1.2.3.4" and "logging 1.2.3.4" where the second argument can be "host" or can be omitted.

• var:pattern, where var is a variable name and pattern is a pattern expression: matches the same text that pattern matches. In addition, it extracts data: the result of matching var:pattern will be a record (aka object) with a field named var whose value is the value extracted from matching pattern. For example, if name:string is matched against the string "abc.def", the match will succeed and return a record {name: "abc.def"} having a single name field whose value is the value extracted by the string pattern.

• pattern1 pattern2 ... patternN, is the sequence of several patterns: matches if each of the patterns can be matched in sequence. The return value is an object with fields for all the named components contained in the patterns.

• pattern1 | pattern2, is the "or" of the given patterns: matches if either pattern matches. The returned value is a record with two fields, named left and right. If pattern1 matches then its value will be in the left field and the right field will be null. Likewise, if pattern2 matches then its value will be in the right field and the left field will be null. For example, if number | ipv4Address were matched against text "123", then the result would be {left: 123, right: null}, whereas if this pattern were matched against text "1.2.3.4", the result would be {left: null, right: 1.2.3.4}. There is no backtracking, which means that if pattern1 matches the next word then pattern2 is never considered even if the rest of the pattern does not match the rest of the input.

• pattern*: matches the given pattern zero or more times. It always succeeds, and returns a list of the values matched by the pattern.

• pattern+: matches the given pattern one or more times. When it succeeds it returns a list of the values matched by the pattern.

• !pattern: matches whenever the given pattern does not match. It consumes no input and returns an empty record. To consume input and return a record containing the matched input, follow the negated pattern with the string pattern. For example, {!"tag" string} matches (and consumes) any word that is not "tag". This can be used to ignore elements up to an element of interest. For example, extracting the IP address from

  Name Vendor Model OS Version IP Address
  alt-app-lb01 F5 BIG-IP Virtual Edition 11.6.1 10.100.0.117
  atl-ce01 Juniper vmx 14.1R5.4 147.75.201.115

  is a bit difficult because the length of the model varies. However, this is achieved via the pattern

  {(!ipv4Address string)*} {ipAddress:ipv4Address}

Extracting Configuration Blocks from Arbitrary Strings

The parseConfigBlocks(os, text) function can be used to parse a collection of configuration blocks from a string that holds a device configuration. In other words, an expression like parseConfigBlocks(OS.NXOS, "interface eth1\n no shut\n") could be used to parse some NXOS configuration in the given string. The resulting value can then be used with various pattern-matching functions.

This function may be useful to process the response to custom commands, for example the response to command show running-config all.