<jupyter_start><jupyter_text>.. highlight:: none.. _storm-ref-pivot:Storm Reference - Pivoting==========================Pivot operations are performed on the output of a previous Storm operation such as a lift or filter. Pivot operators are used to perform pivot operations by navigating from one set of nodes to another based on a specified relationship. Most often, this relationship is two properties (primary and / or secondary) that share the same **value**, and that may also share the same :ref:`data-type`. That is, other than some specialized use cases (such as pivoting to or from tags), most pivots involve navigating between the following kinds of properties:- primary to secondary;- secondary to primary;- secondary to secondary; or- primary to primary... NOTE::    Primary to primary property pivots are a specialized use case that is commonly handled using :ref:`raw-pivot-syntax`, below.Where the source and target properties have the same value **and** the same type, Storm can leverage Synapse's :ref:`gloss-type-aware` to simplify pivot operations and identify implicit relationships. For example, type awareness is used for :ref:`implicit-pivot-syntax` and also for "wildcard" pivot operations (specialized use cases for the :ref:`pivot-out` and the :ref:`pivot-in`).The pivot operations available within Storm are:- `Pivot Out Operator`_- `Pivot In Operator`_- `Pivot With Join`_- `Traverse (Walk) Light Edges`_- `Pivot Out and Walk`_- `Pivot In and Walk`_- `Pivot to Digraph (Edge) Nodes`_- `Pivot Across Digraph (Edge) Nodes`_- `Pivot to Tags`_- `Pivot from Tags`_- `Implicit Pivot Syntax`_- `Raw Pivot Syntax`_.. NOTE::    Light edges represent a special use case within the Synapse hypergraph; navigating (traversing) light edges (:ref:`light-edge`) is included here as a "pivot-like" operation.See :ref:`storm-ref-syntax` for an explanation of the syntax format used below.See :ref:`storm-ref-type-specific` for details on special syntax or handling for specific data types. .. _pivot-out:Pivot Out Operator------------------The pivot out operator ( ``->`` ) is the primary Storm pivot operator. The pivot out operator is used for:- primary to secondary property pivots,- secondary to primary property pivots,- secondary to secondary property pivots, and- "wildcard" pivot out - pivot from any / all **secondary** properties of the inbound set of nodes to the equivalent **primary** property of any nodes, leveraging Synapse's :ref:`gloss-type-aware`.`Pivot to Digraph (Edge) Nodes`_ and `Pivot Across Digraph (Edge) Nodes`_ are covered separately below.**Syntax:**- *Primary to Secondary:*    ** **->** ** **:** **- *Secondary to Primary:*    ** **:** ** **->** **- *Secondary to Secondary:*    ** **:** ** **->** ** **:** **- *"Wildcard" Pivot Out:*    ** **-> *****Examples:***Pivot from primary property ( = ) to secondary property ( = ):*- Pivot from a set of domains to all of their subdomains regardless of depth (i.e., from a domain to all of the domains where the inbound domain is a zone):<jupyter_code>q = '[inet:fqdn=hurr.derp.woot.com inet:fqdn=um.wut.woot.com inet:fqdn=vertex.link inet:ipv4=172.16.17.32 inet:ipv4=5.6.7.8]'
podes = await core.eval(q, num=5, cmdr=True)
# Use previous temp cortex, define and print test query
q = '<inet:fqdn> '
q1 = 'inet:fqdn=woot.com '
q2 = '-> inet:fqdn:zone'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=5, cmdr=False)<jupyter_output><empty_output><jupyter_text>- Pivot from a set of domains to the DNS A records for those domains:<jupyter_code># Make some DNS A records:
q = '[inet:dns:a=(woot.com,172.16.17.32)]'
q1 = '[inet:dns:a=(woot.com,192.168.3.11)]'
q2 = '[inet:dns:a=(woot.com,8.8.8.8)]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=False)
podes = await core.eval(q1, num=1, cmdr=False)
podes = await core.eval(q2, num=1, cmdr=False)
# Define and print test query
q = '<inet:fqdn> '
q1 = 'inet:fqdn=woot.com '
q2 = '-> inet:dns:a:fqdn'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Pivot from secondary property ( = ) to primary property ( = ):*- Pivot from a set of DNS A records to the resolution IP addresses contained in those records:<jupyter_code># Define and print test query using previous data
q = '<inet:dns:a> '
q1 = 'inet:dns:a:fqdn=woot.com '
q2 = ':ipv4 -> inet:ipv4'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Pivot from secondary property ( = ) to secondary property ( = ):*- Pivot from the WHOIS records for a set of domains to the DNS A records for the same domains:<jupyter_code># Make some WHOIS records and related nodes:
q = '[inet:whois:rec=(woot.com,2018/05/22) :registrant="woot hostmaster" :registrar="markmonitor"]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=True)
# Define and print test query using existing data
q = '<inet:whois:rec> '
q1 = 'inet:whois:rec:fqdn=woot.com '
q2 = ':fqdn -> inet:dns:a:fqdn'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>*"Wildcard" pivot out - pivot from all secondary properties to the primary properties of the equivalent forms ( =  to  = ):*- Pivot from a set of WHOIS records to all nodes whose primary property equals *any* of the secondary properties of the WHOIS record:<jupyter_code># Define and print test query
q = '<inet:whois:rec> '
q1 = 'inet:whois:rec:fqdn=woot.com '
q2 = '-> *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>In the example above, the pivot would navigate from the ``:fqdn``, ``:registrar``,  and ``:registrant`` secondary properties of the ``inet:whois:rec`` nodes (for example) to the associated ``inet:fqdn``, ``inet:whois:rar``, and ``inet:whois:reg`` nodes. **Usage Notes:**- When pivoting **from** a secondary property (* = *), the secondary property **must** be specified using the relative property name only (``:baz`` vs. ``foo:bar:baz``). If you specify the full property name before the pivot, Storm interprets that as an additional lift (i.e., `` inet:dns:a:fqdn -> inet:fqdn`` would be interpreted as "take a set of DNS A records from an initial query, lift **all** DNS A records with an :fqdn property (i.e., every DNS A node in the Cortex), and then pivot to the associated FQDN nodes").- Pivoting out using the asterisk wildcard ( ``*`` ) is sometimes called a **refs out** pivot because it pivots from **all** secondary properties of the inbound nodes to **all nodes referenced by** those properties. That is, for each inbound node, the "refs out" pivot will pivot from the node's **secondary properties** to all the nodes that have a **primary property** equal to that type and value.- Pivoting using the wildcard is based on strong data typing within the Synapse data model and Synapse's **type awareness**, so will only pivot out to properties that match both ** and ** / **. This means that the following nodes will **not** be returned by a wildcard pivot out:  - Nodes with matching ** / ** but of different **. For example, if a node’s secondary property is a string (type **) that happens to contain a valid domain (type **), a wildcard pivot out from the node with the string value will **not** return the ``inet:fqdn`` node.  - Digraph (edge) nodes, whose properties are of type ** (node definition, or *,* tuples). See `Pivot to Digraph (Edge) Nodes`_ and `Pivot Across Digraph (Edge) Nodes`_ for details on pivoting to / through those forms.  - It is possible to perform an explicit pivot between properties of different types. For example: `` :name -> inet:fqdn``- See :ref:`pivot-out-and-walk` for a more comprehensive alternative to the wildcard pivot out. .. _pivot-in:Pivot In Operator-----------------The pivot in ( ````) operator. The pivot in operator pivots to the set of nodes that **reference** the current set of nodes.Logically, any pivot in operation can be expressed as an equivalent pivot out operation. For example, the following two pivots would be functionally equivalent:- Pivot from a set of domains to their associated DNS A records:  `` -> inet:dns:a:fqdn``- Use "pivot in" to navigate from a set of domains to the DNS A records that **reference** a set of domains:  `` <- inet:dns:a:fqdn``Because of this equivalence, and because "left to right" logic is generally more intuitive, **only pivot out has been fully implemented in Storm.** (The second example, above, will actually return an error.) The pivot in operator exists, but is only used for certain special case pivot operations:- "wildcard" pivot in - pivot from any / all **primary** properties of the inbound set of nodes to the equivalent **secondary** property of any nodes, leveraging Synapse's :ref:`gloss-type-aware`, and- reverse `Pivot to Digraph (Edge) Nodes`_ and reverse `Pivot Across Digraph (Edge) Nodes`_ (covered separately below).**Syntax:**- *"Wildcard" Pivot In*    ** **<- *** **Example:***Pivot from all primary properties to all nodes with an equivalent secondary property ( =  to  = ):*- Pivot from a set of domains to all nodes with a secondary property that references the domains:<jupyter_code># Define and print test query using existing data
q = '<inet:fqdn> '
q1 = 'inet:fqdn=woot.com '
q2 = '<- *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=11, cmdr=False)<jupyter_output><empty_output><jupyter_text>In this example, the pivot might return nodes with secondary properties such as ``inet:whois:email:fqdn``, ``inet:dns:ns:zone``, ``inet:dns:query:name:fqdn``, and so on. **Usage Notes:**- Pivoting in using the asterisk wildcard ( ``*`` ) is sometimes called a **refs in** pivot because it pivots from the inbound nodes to **all nodes that reference** those nodes. That is, for each inbound node, the "refs in" pivot will pivot from the **primary property** of a node to all nodes that have a **secondary property** equal to that type and value.- Pivoting in using the wildcard will return an instance of a node for **each** matching secondary property. For example, where a node may have the same ** for two different secondary properties (such as ``:domain`` and ``:zone`` on an ``inet:fqdn`` node), the pivot in will return two copies of the node. Results can be de-duplicated using the Storm :ref:`storm-uniq` command.- Pivoting using the wildcard is based on strong data typing within the Synapse data model and Synapse's **type awareness**, so will only pivot in from properties that match both ** and ** / **. This means that the following nodes will **not** be returned by a wildcard pivot in:  - Nodes with matching ** / ** but of different **. For example, if a node’s primary property (such as a domain, type **) - happens to be referenced as as a different type (such as a string, type **) as a secondary property of another node, a wildcard pivot in to the ``inet:fqdn`` node will **not** return the node with the string value.  - Digraph (edge) nodes, whose properties are of type ** (node definition, or *,* tuples). See `Pivot to Digraph (Edge) Nodes`_ and `Pivot Across Digraph (Edge) Nodes`_ for details on pivoting to / through those forms.- Other than digraph (edge) node navigation / traversal, **pivot in can only be used with the wildcard** ( ``*`` ). That is, pivot in does not support specifying a particular target form:  ``inet:fqdn=woot.com <- inet:dns:a:fqdn``  The above query will return an error. A filter operation (see :ref:`storm-ref-filter`) can be used to downselect the results of a wildcard pivot in operation to a specific set of forms:    ``inet:fqdn=woot.com <- * +inet:dns:a``- See :ref:`pivot-in-and-walk` for a more comprehensive alternative to the wildcard pivot in. .. _pivot-join:Pivot With Join---------------The pivot and join operator ( ``-+>`` ) performs the specified pivot operation but joins the results with the inbound set of nodes. That is, the inbound nodes are retained and combined with the results of the pivot.Another way to look at the difference between a pivot and a join is that a pivot operation **consumes** nodes (the inbound set is discarded and only nodes resulting from the pivot operation are returned) but a pivot and join does **not** consume the inbound nodes.The pivot and join operator is used to retain the inbound nodes in any of the following cases:- primary to secondary property pivots,- secondary to primary property pivots,- secondary to secondary propoerty pivots, and- "wildcard" pivot out - pivot from any / all **secondary** properties of the inbound set of nodes to the equivalent **primary** property of any nodes.**Syntax:**- *Primary to Secondary*    ** **-+>** ** **:** **- *Secondary to Primary*    ** **:** ** **-+>** **- *Secondary to Secondary*    ** **:** ** **-+>** ** **:** **- *"Wildcard" Pivot Out and Join*    ** **-+> *****Examples:***Pivot and join from primary property ( = ) to secondary property ( = ):*- Return a set of domains and all of their immediate subdomains:<jupyter_code># Use existing data
# Define and print test query
q = '<inet:fqdn> '
q1 = 'inet:fqdn=woot.com '
q2 = '-+> inet:fqdn:domain'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Pivot and join from secondary property ( = ) to primary property ( = ):*- Return a set of DNS A records and their associated IP addresses:<jupyter_code># Define and print test query using previous data
q = '<inet:dns:a> '
q1 = 'inet:dns:a:fqdn=woot.com '
q2 = ':ipv4 -+> inet:ipv4'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=6, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Pivot and join from secondary property ( = ) to secondary property ( = ):*- Return the WHOIS records for a set of domains and the DNS A records for the same domains:<jupyter_code># Define and print test query using existing data
q = '<inet:whois:rec> '
q1 = 'inet:whois:rec:fqdn=woot.com '
q2 = ':fqdn -+> inet:dns:a:fqdn'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>**"Wildcard" pivot out and join - pivot from all secondary properties to the primary properties of the equivalent forms ( =  to  = ):**- Return a set of WHOIS records and all nodes whose primary property equals any of the secondary properties of the WHOIS record:<jupyter_code># Define and print test query
q = '<inet:whois:rec> '
q1 = 'inet:whois:rec:fqdn=woot.com '
q2 = '-+> *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Usage Notes:**- A pivot out and and join operation follows the same caveats and constraints as the standard :ref:`pivot-out`. .. _walk-light-edge:Traverse (Walk) Light Edges---------------------------The traverse (walk) light edges operator ( ``-()>`` or ``)-`` ) is used to traverse from a set of inbound nodes to the set of nodes they are linked to by the specified light edge(s). Because a light edge is not a node, the navigation is technically a "traversal" of the light edge as opposed to a property-to-property pivot. (Contrast with :ref:`pivot-to-edge` and :ref:`pivot-across-edge` below.)Similar to an edge in a traditional directed graph, light edges have a "direction" (i.e., the  relationship represented by a light edge is "one way"). From a Storm syntax perspective, light edges can be traversed in either direction.See the Synapse data model background documents on :ref:`form-edge-reps` for a discussion of the use of light  edges vs. edge (digraph) nodes.**Syntax:**- *Walk - Single Light Edge*    ** **-(** ** **)>** ***** | **    ** *** **)-** ***** | **- *Walk - Multiple Light Edges*    ** **-( (** ** **,** **  [ **,** ** ...] **) )>** ***** | **    ** *** **,** **  [ **,** ** ...] **) )-** ***** | **- *Walk - Any Light Edge (Wildcard)*    ** **-(** ***** **)>** ***** | **    ** *****Examples:***Traverse the "refs" light edge from an article to the FQDNs "referenced" by the article:*<jupyter_code># Make some nodes and light edges
q = '[media:news=(d41d8cd98f00b204e9800998ecf8427e,) inet:fqdn=vertex.link inet:ipv4=192.168.3.11]'
q1 = 'inet:fqdn=vertex.link inet:ipv4=192.168.3.11 [ <(refs)+ { media:news=a3759709982377809f28fc0555a38193 }]'
# Run the query and test
podes = await core.eval(q, num=3, cmdr=False)
podes = await core.eval(q1, num=2, cmdr=False)
# Define and print test query
q = '<media:news> '
q1 = 'media:news=a3759709982377809f28fc0555a38193 '
q2 = '-(refs)> inet:fqdn'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Traverse the "refs" light edge from an article to all of the nodes "referenced" by the article:*<jupyter_code># Define and print test query
q = '<media:news> '
q1 = 'media:news=a3759709982377809f28fc0555a38193 '
q2 = '-(refs)> *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=2, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Traverse the "hasip" light edge from an IPv4 address to the CIDR block(s) the IP is part of:*<jupyter_code># Make some nodes and light edges
q = '[inet:cidr4=172.16.31.10/12 inet:ipv4=172.16.58.3]'
q1 = 'inet:cidr4=172.16.31.10/12 [ +(hasip)> { inet:ipv4=172.16.58.3 } ]'
# Run the query and test
podes = await core.eval(q, num=2, cmdr=False)
podes = await core.eval(q1, num=1, cmdr=False)
# Define and print test query
q = '<inet:ipv4> '
q1 = 'inet:ipv4=172.16.58.3 '
q2 = '<(hasip)- inet:cidr4'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Traverse the "hasip" and "ipwhois" light edges from an IPv4 address to any nodes linked via those light edges (i.e., typically the CIDR block(s) the IP is part of and the netblock registration record(s) for the IP):*<jupyter_code># Define and print test query
q = '<inet:ipv4> '
q1 = 'inet:ipv4=172.16.58.3 '
q2 = '<((hasip, ipwhois))- *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Traverse any / all light edges from an article to all nodes linked by any light edge:*<jupyter_code># Define and print test query
q = '<media:news> '
q1 = 'media:news=a3759709982377809f28fc0555a38193 '
q2 = '-(*)> *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=2, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Usage Notes:**- The traversal syntax allows specification of a single verb, a list of verbs, or the "wildcard" / asterisk ( ``*`` ) to reference any / all light edge verbs that may be present.- There are no light edges (i.e., specific light edge verbs) defined in a Cortex by default. Users can create and define their own according to their needs.- The Storm :ref:`storm-model`, :ref:`storm-edges`, and :ref:`storm-lift-byverb` commands can be used to work with  light edges in a Cortex. .. _pivot-out-and-walk:Pivot Out and Walk------------------The pivot out and walk (traverse) light edges operator ( ``--> *`` ) combines a wildcard pivot out ("refs out") operation ( ``-> *`` ) with a wildcard walk light edges operation ( ``-(*)>`` ).**Syntax:**** **----> *****Examples:***Pivot from an IP netblock registration record to all nodes referenced by the record's secondary properties and all nodes linked to the record by light edges:*<jupyter_code># Make some nodes and light edges
q = '[inet:whois:iprec=(d41d8cd98f00b204e9800998ecf8427e,) :id="172.16.31.10-123.127.255.255" :net4=(172.16.31.10,192.168.127.12)]'
q1 = 'inet:whois:iprec=a3759709982377809f28fc0555a38193 [ <(ipwhois)- { inet:ipv4=172.16.58.3 } ]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=False)
podes = await core.eval(q1, num=1, cmdr=False)
# Define and print test query
q = '<inet:whois:iprec> '
q1 = 'inet:whois:iprec=a3759709982377809f28fc0555a38193 '
q2 = '--> *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Usage Notes:**- The pivot out and walk operator can only be used with a wildcard ( ``*`` ); it is not possible to specify a particular form as the target of the operation. A filter operation can be used to refine the results of the pivot and walk operation if necessary.- The pivot and walk operators (pivot out and walk / pivot in and walk) are useful for "exploring" data in a Cortex as they will return all the nodes "next to" the working set of nodes (subject to :ref:`gloss-type-aware`) without requiring the user to have specific knowledge of the data model.- The Storm :ref:`storm-tee` command can be used to perform concurrent pivot in and walk / pivot out and walk operations on an inbound set of nodes:  .. parsed-literal::     | tee { --> * } { <-- * } .. _pivot-in-and-walk:Pivot In and Walk------------------The pivot in and walk (traverse) light edges operator ( ``<-- *`` ) combines a wildcard pivot in ("refs in") operation ( ``<- *`` ) with a wildcard walk light edges operation ( ``<(*)-`` ).**Syntax:**** **<---- *****Examples:***Pivot from a set of IP addresses to all nodes that reference the IPs and all nodes linked to the IPs by light edges:*<jupyter_code># Make some nodes and light edges
q = '[media:news=(d41d8cd98f00b204e9800998ecf8427e,)]'
q1 = 'media:news=a3759709982377809f28fc0555a38193 [ +(refs)> { [ inet:ipv4=172.16.58.3 ] } ]'
q2 = '[inet:dns:a=(woot.com,172.16.58.3) inet:server=tcp://172.16.31.103]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=False)
podes = await core.eval(q1, num=1, cmdr=False)
podes = await core.eval(q2, num=2, cmdr=False)
# Define and print test query
q = '<inet:ipv4> '
q1 = 'inet:ipv4=172.16.58.3 '
q2 = '<-- *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Usage Notes:**- The pivot in and walk operator can only be used with a wildcard ( ``*`` ); it is not possible to specify a particular form as the target of the operation. A filter operation can be used to refine the results of the pivot and walk operation if necessary.- The pivot and walk operators (pivot out and walk / pivot in and walk) are useful for "exploring" data in a Cortex as they will return all the nodes "next to" the working set of nodes (subject to :ref:`gloss-type-aware`) without requiring the user to have specific knowledge of the data model.- The Storm :ref:`storm-tee` command can be used to perform concurrent pivot in and walk / pivot out and walk operations on an inbound set of nodes:  .. parsed-literal::     | tee { --> * } { <-- * } .. _pivot-to-edge:Pivot to Digraph (Edge) Nodes-----------------------------Digraph (edge) nodes (:ref:`form-edge`) are of type ``edge`` or ``timeedge``. These nodes (forms) are unique in that their primary property value is a pair of **node definitions** (type :ref:`gloss-ndef`) - that is, *, * tuples. (``timeedge`` forms are comprised of two *, * tuples and an additional ** value). Each  *, * tuple from the primary property is broken out as secondary property ``:n1`` or ``:n2`` of type **. This means that pivoting to and from digraph nodes is a bit different than pivoting to and from nodes whose properties are a simple ** or **.See the Synapse data model background documents on :ref:`form-edge-reps` for a discussion of the use of light  edges vs. edge (digraph) nodes.**Syntax:**** **->** ** | ** [**:n2**]** **-+>** ** | ** [**:n2**]** *** | ****Examples:***Pivot out from a set of nodes whose ndefs (, ) are the first element (:n1) in a set of a digraph nodes:*- Pivot out from a person node to the set of digraph nodes representing things that person “has”:<jupyter_code># Make some nodes:
q = '[ps:person="26786e8fb9b9e7050d0b8b4e38e1d431" :name="<NAME>"]'
q1 = '[inet:email=<EMAIL>]'
q2 = '[mat:item="32d0bea68c5e9ec82b8f0fc867ccacda" :name="<NAME>\'s dog"]'
q3 = '[edge:has=((ps:person, 26786e8fb9b9e7050d0b8b4e38e1d431), (inet:email,<EMAIL>))]'
q4 = '[edge:has=((ps:person, 26786e8fb9b9e7050d0b8b4e38e1d431), (mat:item, 32d0bea68c5e9ec82b8f0fc867ccacda))]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=True)
podes = await core.eval(q1, num=1, cmdr=True)
podes = await core.eval(q2, num=1, cmdr=True)
podes = await core.eval(q3, num=1, cmdr=True)
podes = await core.eval(q4, num=1, cmdr=True)
# Define and print test query
q = '<ps:person> '
q1 = 'ps:person=26786e8fb9b9e7050d0b8b4e38e1d431 '
q2 = '-> edge:has'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=2, cmdr=False)<jupyter_output><empty_output><jupyter_text>- Pivot out from a person node to the set of ``timeedge`` digraph nodes representing places that person has been to (and when):<jupyter_code># Make some nodes:
q = '[geo:place=5859abb1ba6e4a418bf31dfe2fc3c08a :name="test place" :latlong=(37.4168957,-121.9218271)]'
q1 = '[edge:wentto=((ps:person, bb2a3e42ef3fc0d2b2a4e6145396cb65), (geo:place, 5859abb1ba6e4a418bf31dfe2fc3c08a), "2018/12/15 08:35:00")]'
q2 = '[edge:wentto=((ps:person, bb9d1c4270ccd9076ea30f0bb2491bbd), (geo:place, 5859abb1ba6e4a418bf31dfe2fc3c08a), "2017/06/30 14:27:43")]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=False)
podes = await core.eval(q1, num=1, cmdr=False)
podes = await core.eval(q2, num=1, cmdr=False)
# Define and print test query
q = '<ps:person> '
q1 = 'ps:person '
q2 = '-> edge:wentto'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=2, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Pivot in from a set of nodes whose ndefs (, ) are the second element (:n2) in a set of a digraph nodes:*- Pivot in from an article to the set of digraph nodes representing things that “have” the article (e.g., people or organizations who authored the article):<jupyter_code># Make some nodes:
q = '[ps:person=<KEY> :name=Alice]'
q1 = '[ps:person=<KEY> :name=Bob]'
q2 = '[edge:has=((ps:person, bb2a3e42ef3fc0d2b2a4e6145396cb65), (media:news, 2aa767aebe9d0172601cef3c5867abea))]'
q3 = '[edge:has=((ps:person, bb9d1c4270ccd9076ea30f0bb2491bbd), (media:news, 2aa767aebe9d0172601cef3c5867abea))]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=False)
podes = await core.eval(q1, num=1, cmdr=False)
podes = await core.eval(q2, num=1, cmdr=False)
podes = await core.eval(q3, num=1, cmdr=False)
# Define and print test query
q = '<media:news> '
q1 = 'media:news=2aa767aebe9d0172601cef3c5867abea '
q2 = '<- edge:has'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=2, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Usage Notes:**- To simplify working with digraph nodes and their ``ndef`` properties, Storm makes some assumptions (optimizations) when using the pivot out and pivot in operators:  - When pivoting to or from a set of nodes to a set of digraph nodes, pivot using the ``ndef`` (*,*) of the inbound nodes and not their primary property (**) alone.  - When pivoting **out** to a digraph node, the inbound nodes’ *,* ``ndef`` will be the **first** element (``:n1``) of the digraph. You must explicitly specify ``:n2`` as the target property to pivot using the second element.  - When pivoting **in** to a digraph node, the inbound nodes’ *,* ``ndef`` will be the **second** element (``:n2``) of the digraph. It is not possible to pivot in to the ``:n1`` value.- Pivoting to / from digraph nodes is one of the specialized use cases for the pivot in ( ``<-``) operator, however the primary use case of pivot in with digraph nodes is reverse edge traversal (see `Pivot Across Digraph (Edge) Nodes`_). See `Pivot In Operator`_ for general limitations of the pivot in operator. .. _pivot-across-edge:Pivot Across Digraph (Edge) Nodes---------------------------------Because digraph nodes represent generic edge relationships, analytically we are often more interested in the nodes on "either side" of the edge than in the digraph node itself. For this reason, the pivot operators have been optimized to allow a syntax for easily navigating "across" these digraphs (edges).See the Synapse data model background documents on :ref:`form-edge-reps` for a discussion of the use of light  edges vs. edge (digraph) nodes.**Syntax:**** **->** ** | ** **->** ***** | **** *** | ** *****Examples:**- Traverse a set of ``edge:has`` nodes to pivot from a person to all the things the person "has":<jupyter_code># Define and print test query using existing data
q = '<ps:person> '
q1 = 'ps:person=26786e8fb9b9e7050d0b8b4e38e1d431 '
q2 = '-> edge:has -> *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=2, cmdr=False)<jupyter_output><empty_output><jupyter_text>- Traverse a set of ``edge:wentto`` nodes to pivot from a person to the locations the person has visited:<jupyter_code># Define and print test query using existing data
q = '<ps:person> '
q1 = 'ps:person=bb2a3e42ef3fc0d2b2a4e6145396cb65 '
q2 = '-> edge:wentto -> *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Usage Notes:**- Storm makes the following assumptions to optimize the two pivots:  - For pivots out, the first pivot is to the digraph nodes’ ``:n1`` property and the second pivot is from the digraph nodes’ ``:n2`` property.  - For pivots in, the first pivot is to the digraph nodes’ ``:n2`` property and the second pivot is from the digraph nodes’ ``:n1`` property.- Pivoting "across" the digraph nodes still performs two pivot operations (i.e., to the digraph nodes and then from them). As such it is still possible to apply an optional filter to the digraph nodes themselves before the second pivot. .. _pivot-to-tags:Pivot to Tags-------------Pivot to tags syntax allows you to pivot from a set of nodes with tags to the set of ``syn:tag`` nodes representing those tags. This includes:- pivot to all leaf tag nodes,- pivot to all tag nodes,- pivot to all tag nodes matching a specified pattern, and- pivot to tag nodes matching an exact tag.See the :ref:`analytical-model-tags` document for additional discussion of tags as nodes (``syn:tag`` nodes) and tags as labels applied to other nodes.**Syntax:**** **-> ** [ ***** | ** **.*** | ** ]**Examples:***Pivot to all leaf tag nodes:*- Pivot from a set of domains to the ``syn:tag`` nodes for all **leaf** tags applied to those domains (i.e., the  longest / final tag in each tree applied to each node):<jupyter_code># Make some tagged nodes:
q = '[inet:fqdn=aoldaily.com +#aka.feye.thr.apt1 +#cno.infra.sink.hole.kleissner +#cno.infra.sink.hole.snk4 +#cno.ttp.se.masq.aol]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=True)
# Define and print test query using existing data
q = '<inet:fqdn> '
q1 = 'inet:fqdn=aoldaily.com '
q2 = '-> #'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Pivot to ALL tag nodes:*- Pivot from a set of files to the ``syn:tag`` nodes for **all** tags applied to those files (i.e., each tag in each tag tree applied to each node, from root to leaf):<jupyter_code># Make some tagged nodes:
q = '[file:bytes=sha256:ce82f2b530f028644c8c7238c065eb88e4af153598447179aa784482efba454e +#aka.feye.mal.greencat +#aka.feye.thr.apt1 +#rep.vt.armadillo +#rep.vt.pedll]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=True)
# Define and print test query using existing data
q = '<file:bytes> '
q1 = 'file:bytes=sha256:ce82f2b530f028644c8c7238c065eb88e4af153598447179aa784482efba454e '
q2 = '-> #*'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=10, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Pivot to all tag nodes matching the specified pattern:*- Pivot from a set of IP addresses to the ``syn:tag`` nodes for all tags applied to those IPs that are part of the anonymized infrastructure tag tree:<jupyter_code># Make some tagged nodes:
q = '[inet:ipv4=172.16.31.10 +#cno.infra.anon.tor +#cno.infra.anon.vpn.airvpn]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=True)
# Define and print test query using existing data
q = '<inet:ipv4> '
q1 = 'inet:ipv4=172.16.31.10 '
q2 = '-> #cno.infra.anon.*'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Pivot to tag nodes exactly matching the specified tag:*- Pivot from a set of nodes to the ``syn:tag`` node for ``foo.bar`` (if present on the inbound set of nodes):<jupyter_code># Make some tagged nodes:
q = '[inet:fqdn=woot.com +#foo.bar]'
# Run the query and test
podes = await core.eval(q, num=1, cmdr=True)
# Define and print test query using existing data
q = '<query> '
q1 = 'inet:fqdn=woot.com '
q2 = '-> #foo.bar'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Usage Notes:**- Pivot to all tags ( ``*`` ) and pivot by matching an initial pattern ( ``.*`` ) will match **all** tags in the relevant tag trees from the inbound nodes, not just the leaf tags. For example, for an inbound node with tag ``foo.bar.baz``, ``*`` will return the ``syn:tag`` nodes for ``foo``, ``foo.bar``, and ``foo.bar.baz``.- When using the asterisk / wildcard ( ``*`` ) to match a pattern, the wildcard(s) can be used anywhere within the tag name (value); they are not limited to matching elements within the tag's dotted namespace. For example, all of the following are valid (though may return different results):  - ``-> aka.thr.*``  - ``-> aka.t*``  - ``-> a*``  - ``-> *thr*``  - ``-> *.thr.*``- The pivot to tags operator does not support pivoting directly to a set of tags specified by a prefix match ( ``^`` ) or regular expression ( ``~`` ). However, these operators can be used as part of a subsequent filter operation to further refine the results of the pivot. .. _pivot-from-tags:Pivot from Tags---------------Pivot from tags syntax allows you to pivot from a set of ``syn:tag`` nodes to the set of nodes that have those tags.**Syntax:**** **->** ***** | ****Examples:**- Pivot to all domains tagged with tags from any of the inbound ``syn:tag`` nodes:<jupyter_code># Define and print test query using existing data
q = '<syn:tag> '
q1 = 'syn:tag=aka.feye.thr.apt1 '
q2 = '-> inet:fqdn'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>- Pivot to **all** nodes tagged with tags from any of the inbound ``syn:tag`` nodes:<jupyter_code># Define and print test query using existing data
q = '<syn:tag> '
q1 = 'syn:tag=aka.feye.thr.apt1 syn:tag=cno.infra.anon '
q2 = '-> *'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=3, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Usage Notes:**- In many cases, pivot from tags is functionally equivalent to :ref:`lift-tag`. That is, the following queries will both return all nodes tagged with ``aka.feye.thr.apt1``:  ``syn:tag=aka.feye.thr.apt1 -> *``    ``aka.feye.thr.apt1``    Pivoting from tags is most useful when used in conjunction with `Pivot to Tags`_ - that is, taking a set of inbound nodes, pivoting to the ``syn:tag`` nodes for any associated tags (pivot to tags), and then pivoting out again to other nodes tagged with some or all of those tags (pivot from tags). .. _implicit-pivot-syntax:Implicit Pivot Syntax---------------------Pivot operations in Storm can always be executed by **explicitly** specifying the source and target properties for the pivot. This is referred to as **explicit pivot syntax** or explicit syntax. For example:.. parsed-literal::    inet:fqdn=vertex.link -> inet:dns:a:fqdn :ipv4 -> inet:ipv4The above query:- lifts the FQDN ``vertex.link``,- explicitly pivots from the primary property of the FQDN to any ``inet:dns:a:fqdn`` secondary property with the same value, and- explicitly pivots from the ``:ipv4`` secondary property of the ``inet:dns:a`` nodes to the primary property of  any ``inet:ipv4`` nodes with the same value.Using explicit pivot syntax tells Storm **exactly** what you want to do; there is no ambiguity in the query. (Explicit syntax may also be useful when first learning Storm to reinforce exactly what navigation is being carried out when you perform a pivot operation.) However, the need to fully specify target properties (using form and property names) and specifically reference source properties (using relative property names) can add overhead ("more typing") to a Storm query that is not necessary if the query is unambiguous (i.e., based on the inbound and outbound forms).For this reason, Storm also supports **implicit pivot syntax** for certain types of pivots. Implicit pivot syntax takes advantage of Synapse's :ref:`gloss-type-aware` to "know" which properties can be pivoted to (or from), given the forms that are inbound to and outbound from the pivot operation. In these cases, the source and/or target do not need to be explicitly specified. This allows for more concise Storm syntax in cases where the source and / or target of the pivot is self-evident given the forms used.Implicit pivot syntax can be used in the following cases where the source and target properties have both the same **type** AND the same **value**:- Primary to secondary property pivots.- Secondary to primary property pivots.Implicit pivot syntax **cannot** be used for the following:- Primary to primary property pivots (see :ref:`raw-pivot-syntax`, below)- Secondary to secondary property pivots.- Pivots between primary and secondary (or secondary and primary) properties with the same value but of different **types**. **Examples:***Pivot from primary property ( = ) to secondary property ( = ) using implicit syntax:*- Pivot from a set of domains to their associated DNS A records:**Explicit syntax:**<jupyter_code># Define and print test query using existing data
q = '<inet:fqdn> '
q1 = 'inet:fqdn=woot.com '
q2 = '-> inet:dns:a:fqdn'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Implicit syntax:**<jupyter_code># Define and print test query using existing data
q = '<inet:fqdn> '
q1 = 'inet:fqdn=woot.com '
q2 = '-> inet:dns:a'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>With implicit syntax, the target property ``:fqdn`` can be omitted because it is the only logical target given ``inet:fqdn`` nodes as the source and ``inet:dns:a`` nodes as the target of the pivot... NOTE::  While the ``inet:fqdn`` form has secondary properties that are also of type ``inet:fqdn`` (e.g., both ``:domain`` and ``:zone``) implicit syntax can only be used to pivot between primary and secondary OR secondary and primary properties, but not both. That is, implicit syntax does not allow you to go from any / all properties of a given type in the source nodes to  any / all properties with the same type and value in the target nodes.  Because the target of the pivot is ``inet:dns:a`` nodes, the only logical target given the inbound nodes is the ``:fqdn`` **secondary** property, which means the only logical source is the **primary** property of the ``inet:fqdn``. *Pivot from secondary property ( = ) to primary property ( = ) using implicit syntax:*- Pivot from a set of DNS A records to their associated IP addresses:**Explicit syntax:**<jupyter_code># Define and print test query using existing data
q = '<inet:dns:a> '
q1 = 'inet:dns:a:fqdn=woot.com '
q2 = ':ipv4 -> inet:ipv4'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Implicit syntax:**<jupyter_code># Define and print test query using existing data
q = '<inet:dns:a> '
q1 = 'inet:dns:a:fqdn=woot.com '
q2 = '-> inet:ipv4'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>With implicit syntax, the source property ``:ipv4`` can be omitted because it is the only logical source given a set of ``inet:ipv4`` nodes as the target... NOTE::    Similar to the last example, while the ``inet:dns:a`` form has both ``:ipv4`` and ``:fqdn`` secondary prroperties, implicit syntax can only be used to pivot between primary and secondary OR secondary and primary properties, but not both. Because the target of the pivot is ``inet:ipv4`` nodes, the only logical source property is the ``:ipv4`` secondary property of the ``inet:dns:a`` node. *Use of multiple implicit pivots:*- Pivot from a set of domains to their DNS A records and then to the associated IP addresses:**Regular (full) syntax:**<jupyter_code># Define and print test query using existing data
q = '<inet:fqdn> '
q1 = 'inet:fqdn=woot.com '
q2 = '-> inet:dns:a:fqdn :ipv4 -> inet:ipv4'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Implicit syntax:**<jupyter_code># Define and print test query using existing data
q = '<inet:fqdn> '
q1 = 'inet:fqdn=woot.com '
q2 = '-> inet:dns:a -> inet:ipv4'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=4, cmdr=False)<jupyter_output><empty_output><jupyter_text>The above example simply combines the previous two examples to illustrate the use of multiple implicit pivot operations in a longer query. *Implicit syntax with multiple target properties:*- Pivot from a set of domains to the associated DNS MX records:**Implicit syntax:**<jupyter_code># Make some nodes
q = '[inet:dns:mx=(google.com,aspmx3.googlemail.com) inet:dns:mx=(djliker.com,google.com)]'
# Execute the query to teste
podes = await core.eval(q, num=2, cmdr=False)
# Define and print test query
q = '<inet:fqdn> '
q1 = 'inet:fqdn=google.com '
q2 = '-> inet:dns:mx'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=2, cmdr=False)<jupyter_output><empty_output><jupyter_text>In the example above, given the source and target forms, the logical pivot for implicit syntax is from the primary property of the inbound FQDN to the secondary properties of the DNS MX nodes. However, an ``inet:dns:mx`` form has **two** secondary properties of type ``inet:fqdn``: the mail exchange server (``inet:dns:mx:mx``) and the domain that uses the MX (``inet:dns:mx:fqdn``).Using implicit syntax, Storm wil pivot from the source FQDN(s) to any DNS MX records where the domain matches **either** of those secondary properties. For example, querying the FQDN ``google.com`` will return DNS MX records **for** Google (i.e., ``inet:dns:mx:fqdn=google.com``) as well as MX records that may **use** Google as their mail exchange (i.e., ``inet:dns:mx:mx=google.com``).If you want only one or the other of those types of records, you need to use explicit syntax to specify the target property, i.e.:<jupyter_code># Define and print test query
q = '<inet:fqdn> '
q1 = 'inet:fqdn=google.com '
q2 = '-> inet:dns:mx:fqdn'
q3 = '\n\n'
q4 = 'or'
q5 = '\n\n'
q6 = '-> inet:dns:mx:mx'
print(q + q2 +q3 + q4 + q5 + q + q6)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)
podes = await core.eval(q1 + q6, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>*Implicit syntax with multiple source properties:*- Pivot from a set of files to their associated SHA256 hashes:**Implicit syntax:**<jupyter_code># Make some nodes
q = '[file:bytes=fc56fac29e8c93d71f14cbe8028f826d289b8b2e7cb6e94539d980966ec05e05 :mime:pe:richhdr=fc56fac29e8c93d71f14cbe8028f826d289b8b2e7cb6e94539d980966ec05e05]'
# Execute the query to teste
podes = await core.eval(q, num=1, cmdr=False)
# Define and print test query
q = '<file:bytes> '
q1 = 'file:bytes=fc56fac29e8c93d71f14cbe8028f826d289b8b2e7cb6e94539d980966ec05e05 '
q2 = '-> hash:sha256'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=2, cmdr=False)<jupyter_output><empty_output><jupyter_text>In the example above, given the source and target forms, the logical pivot for implicit syntax is from the secondary properties of the inbound file to the primary property of the SHA256 nodes. However, a ``file:bytes`` form has **two** secondary properties of type ``hash:sha256``: the file's SHA256 hash (``file:bytes:sha256``) and the hash of the file's rich header data (if the file is a PE executable - ``file:bytes:mime:pe:richhdr``).Using implicit syntax, Storm will pivot from **both** source properties (where present) to **all** of the associated SHA256 nodes - that is, those that match either the ``:sha256`` or ``:mime:pe:richhdr`` value.If you want only one or the other of those types of records, you need to use explicit syntax to specify the source  property, i.e.:<jupyter_code># Define and print test query
q = '<file:bytes> '
q1 = 'file:bytes=fc56fac29e8c93d71f14cbe8028f826d289b8b2e7cb6e94539d980966ec05e05 '
q2 = ':sha256 -> hash:sha256'
q3 = '\n\n'
q4 = 'or'
q5 = '\n\n'
q6 = ':mime:pe:richhdr -> hash:sha256'
print(q + q2 +q3 + q4 + q5 + q + q6)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)
podes = await core.eval(q1 + q6, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>.. _raw-pivot-syntax:Raw Pivot Syntax----------------For certain edge cases, standard Storm pivot syntax (explicit or implicit) is insufficient. In these instances raw pivot syntax acts as a "get out of jail free" card to perform specialized pivot operations. These include:- primary-to-primary property pivots;- pivots where the value of the target property (primary or secondary) is computed from the input node(s);- extramodel pivots.In raw pivot syntax, the target of the pivot is specified as a Storm query enclosed in curly braces. Raw pivots often involve specifying a variable derived from the inbound node(s) and performing the raw pivot using the variable, though this is not technically required. (See :ref:`storm-adv-vars` for a discussion of using variables in Storm).For some raw pivot syntax use cases, you can compose an equivalent Storm query using lift and filter operations. For example: - lift a set of nodes; - define a variable based on those nodes; - lift a second set of nodes using the variable; - filter out the original nodes you lifted, thus leaving only the second set of lifted nodes. However, executing this type of query using raw pivot syntax is slightly more efficient; the Storm query within the raw pivot's curly braces may still be a lift operation, but performing it inside a raw pivot means you do not have to explicitly drop (filter out) your original nodes. (As with a regular pivot, the inbound nodes are consumed by the pivot operation itself, eliminating the need for the filter.)As always, these efficiencies may be trivial for smaller queries but can be significant for larger queries.**Syntax:**** **->** **{** ** **}****Examples:**- Pivot from a string (``it:dev:str``) representing an FQDN to the ``inet:fqdn`` node for that FQDN (i.e., pivot between two primary properties of different types).**Standard syntax (no raw pivot, lift / filter only):**<jupyter_code># Make some nodes
q = '[ it:dev:str=hugesoft.org inet:fqdn=hugesoft.org ]'
# Execute and test
podes = await core.eval(q, num=2, cmdr=False)
# Define and print test query using existing data
q = '<it:dev:str> '
q1 = 'it:dev:str=hugesoft.org '
q2 = '$fqdn=$node.value() inet:fqdn=$fqdn -it:dev:str'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)<jupyter_output><empty_output><jupyter_text>**Raw pivot syntax:**<jupyter_code># Define and print test query using existing data
q = '<it:dev:str> '
q1 = 'it:dev:str=hugesoft.org '
q2 = '$fqdn=$node.value() -> { inet:fqdn=$fqdn }'
print(q + q2)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q1 + q2, num=1, cmdr=False)
# Close cortex because done
_ = await core.fini()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12804767924869062, 'fraction_numerical': 0.030421608171292115, 'mean_word_length': 4.7385997236296635, 'pattern_counts': {'":': 1, '<': 225, '<?xml version=': 0, '>': 287, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 24, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '40', 'dir': 'jupyter-structured-clean-dedup', 'id': '200963', 'license': 'Apache-2.0', 'n_tokens_mistral': 16932, 'n_tokens_neox': 15901, 'n_words': 6430, 'path': 'docs/synapse/userguides/storm_ref_pivot.ipynb', 'repo_name': 'ackroute/synapse', 'size': '49926'}