Augmented-by Addition into the IETF-YANG-LibraryHuaweiTownsend Street, 4th Floor George's CourtDublinIrelandzephyre888@gmail.comHuaweibenoit.claise@huawei.comTelefonica Innovacion DigitalRonda de la Comunicacion, S/NMadrid 28050Spainignacio.dominguezmartinez@telefonica.com
OPS
NETCONF
This document augments the ietf-yang-library in [RFC8525] to
provide the augmented-by list. It facilitates the process of
obtaining the entire dependencies of YANG model, by directly
querying the server's YANG module.
Source for this draft and an issue tracker can be found at .
The YANG library specifies a YANG
module that provides the information about the YANG models and
datastores to facilitate a client application to fully utilize
and understand the YANG data modelling language. To know the
YANG dependencies, has defined and
provided the submodule list and the YANG modules deviation list.
However, the YANG modules augmentation is not provided.
According to , both augmentations
and deviations are defining contents external to the model,
but applying internally for the model. It is
important to know the augmentation and deviation as they are
dependencies of the model, but it is also difficult because
they are defined externally.
When we try to use the ietf-yang-library in
to obtain the reverse dependencies
(Augmentations and deviations), the augmentation is not defined
in it.
However, the augmentation and the deviation work similarly
as YANG modules dependency. Therefore, it is reasonable to
document them the same way in the IETF YANG library. Besides,
it will be easier to determine the reverse dependency if the
augmentation is directly available, through a GET request
into this new YANG model.
This draft augment the ietf-yang-library to
include the YANG modules augmentation information.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 when, and only
when, they appear in all capitals, as shown here.The terminology from is used in this documentTree diagrams in this document use the notation defined in
.When using one YANG model, it is important to make sure
that all its dependencies are presented. In
there are four dependencies for
one YANG mode:
Import: the "import" statement allows a module or
submodule to reference definitions defined in other modules.
Include: the "include" statement is used in a module
to identify each submodule that belongs to it.
Augmentation: the "augment" statement defines the
location in the data model hierarchy where additional
nodes are inserted
Deviation: the "deviation" statement defines a
hierarchy of a module that the server does not
implement faithfully.
The import and include are direct dependencies while the
augmentation and deviation are reverse dependencies. To
know a specific YANG model's direct dependencies, we can
parse this YANG model as the dependencies are directly
specified (import and include statements").
As for the reverse dependencies, since they are defined
externally, we cannot parse the YANG model itself to get
them. Among all the methods for getting reverse dependency,
getting the ietf-yang-library content is one of the most
convenient ways.
However, the ietf-yang-library only provides the deviation
list, but not the augmentation. It is reasonable to update
it to provide the augmentation information since both
augmentation and deviation have similar way of working (both
are applied to the original model but invisible to them). A
noticeable difference between deviations and augmentations
is that the deviations are required to understand the API
contract between the client and the server. But with some
use cases arise as the requirement of automate network
management, the augmentation becomes essential information
for understanding the network, too.
As the demand arises for
YANG-based telemetry , there is
a need for real-time knowledge of a specific YANG model's
dependency list when a specific YANG-Push message is
received. The alternative, for a YANG-push receiver, to
collect and store the entire module set for every single
server who could be streaming data, is not always practical.
See the following reasons:
For a YANG-push collector => we never know in advance which
telemetry content will be received and from whom.
Querying all the YANG modules is time consuming =>
we lose the real-time.
With similar central idea, two use cases are introduced
in this section. One target solving the dependencies problems in
a Data Mesh Telemetry System while the other aims at building a
data catalog which makes YANG model inforamtion easily accessible.
A network analytics architecture that integrates YANG-push and
Kafka is proposed in 2022 and is continuously growing and gaining
influence, refer to the draft:
An
Architecture for YANG-Push to Apache Kafka Integration.
In this open-sourced project covering
Support of Versioning in YANG Notifications Subscription,
Support of Network Observation Timestamping in YANG Notifications
, among others, the purpose is to provide adequate
information in the YANG-Push notification so that when it is
received, the model and its dependency can be parsed and found
automatically from the vantage point. The architecture relies on
the information of YANG model and their dependency to realize,
as one of its main goals is to solve the problem of missing YANG
semantics when data is received in Time Series Database in the end.
To solve the problem, a schema registry is introduced to store
YANG models and all their relationship (direct dependency and
reverse dependency).
Currently, the method used for finding model's reverse
dependency is get-all-schemas, that is to retrieve all YANG
modules from the device to the client's disk, enabling the
client to fully understand the YANG model relationship. This
process is heavy because in real situation, each device may
have few thousands or even more YANG module implemented.
Besides, pressure is also introduced due to the need of parsing
modules and find all the dependencies.
Considering the telemetry real-time aspects, this extra
delay in processing the dependencies through get-all-schemas
is not ideal.
The YANG model proposed in the draft can be used in this
architecture to release the stress of get-all-schemas and bring
some extra benefits.
By providing the augmentation information, it enables the collector
to get the YANG reverse dependencies by simply sending one
<get> query to ietf-yang-library. In this regard, the
process of acquiring full dependency
becomes real-time action. Compared with get-all-schemas, it enables
collector to fetch up-to-date data since the queries are sent
at run time, while the old approach forces collector to always
work with never updated data. On another hand, user do not bother
waiting for ten minutes every time when starting collector to either
get data updated or to obtain YANG relationship.
Finding the YANG models implemented by a network device is paramount for
configuring and monitoring the status of a network. However, since the
inception of YANG the network industry has experienced a tsunami of
YANG models developed by SDOs, open-source communities,
and network vendors. This heterogeneity of YANG models, that vary
from one network device model to another, makes the management of a
multi-vendor network a big challenge
for operators. In this regard, a data catalog provides a registry of the datasets
exposed by remote data sources for consumers to discover data
of interest. Besides the location of the dataset
(i.e., the data source), the data catalog registers
additional metadata such as the data model (or schema) followed in the
dataset or even related terms defined in a business glossary.Data catalog solutions typically implement collectors that
ingest metadata from the data sources themselves and also external
metadata sources. For example, a Kafka Schema Registry is a
metadata source that provides metadata about the data models
followed by some data stored in a Kafka topic.In this sense, a YANG-enabled network device can be considered
as another kind of data source, which the Data Catalog can
pull metadata from. For instance, the data catalog can include a
connector that fetches metadata about the YANG models implemented
by the network device. Combining these metadata with other such as
the business concept "interface", would enable data consumers to
discover which datasets related to the concept "interface"
are exposed by the network device.Network devices that implement YANG Library expose metadata
about which YANG modules are implemented, and which are only imported.
However, what a data consumer needs at the end are the YANG models
implemented by the device, hence, the combination of implemented YANG
modules with other YANG modules that might deviate or augment the formers.Coming back to the example of datasets related to the "interface"
concept, say we have a network device that implements the
ietf-interfaces module
and the ietf-ip module ,
where the latter augments the former. For a data catalog to
collect these metadata, a connector would retrieve YANG Library
data from the target device. However, the current version
of YANG Library would not satisfy the use case as it would
tell that the device implements both ietf-interfaces and ietf-ip
modules, but will miss the augment dependency between them.The current workaround to this limitation is to, in combination with the
YANG library data, additionally fetch both YANG modules and process them
to discover that there is an augment dependency. This adds extra burden
on the connector, which is forced to combine multiple metadata collection
mechanisms. This process could be softened by extending YANG Library
to also capture augment dependencies, in a similar fashion to
deviation dependencies.
This YANG module augments the ietf-yang-library module by adding the
augmented-by list in the "yang-library/module-set". The name Augmented-by
indicated the modules by which the current module is being augmented.
Note that this module only augments the ietf-yang-library defined in
. At the time of writing this document,
most router vendors support , a previous
revision of the ietf-yang-library YANG module; The module that
augments is provided in the appendix B.
The following is the YANG tree diagram for model ietf-yang-library-augmentedby. ../../yanglib:module/name
]]>
The following is the YANG tree diagram
for the ietf-yang-library with the augmentation defined in
module ietf-yang-library-augmentedby, including the RPCs and
notifications.
../../module/name
| | | +--ro yanglib-aug:augmented-by*
-> ../../yanglib:module/name
| | +--ro import-only-module* [name revision]
| | +--ro name yang:yang-identifier
| | +--ro revision union
| | +--ro namespace inet:uri
| | +--ro location* inet:uri
| | +--ro submodule* [name]
| | +--ro name yang:yang-identifier
| | +--ro revision? revision-identifier
| | +--ro location* inet:uri
| +--ro schema* [name]
| | +--ro name string
| | +--ro module-set* -> ../../module-set/name
| +--ro datastore* [name]
| | +--ro name ds:datastore-ref
| | +--ro schema -> ../../schema/name
| +--ro content-id string
x--ro modules-state
x--ro module-set-id string
x--ro module* [name revision]
x--ro name yang:yang-identifier
x--ro revision union
+--ro schema? inet:uri
x--ro namespace inet:uri
x--ro feature* yang:yang-identifier
x--ro deviation* [name revision]
| x--ro name yang:yang-identifier
| x--ro revision union
x--ro conformance-type enumeration
x--ro submodule* [name revision]
x--ro name yang:yang-identifier
x--ro revision union
+--ro schema? inet:uri
notifications:
+---n yang-library-update
| +--ro content-id -> /yang-library/content-id
x---n yang-library-change
x--ro module-set-id -> /modules-state/module-set-id
]]>The YANG module source code of ietf-yang-library-augmentedby
in which augmentation to the ietf-yang-library of
is defined. file "ietf-yang-library-augmentedby@2023-10-27.yang"
module ietf-yang-library-augmentedby {
yang-version 1.1;
namespace "urn:ietf:params:xml:ns:yang:ietf-yang-library-augmentedby";
prefix yanglib-aug;
import ietf-yang-library {
prefix yanglib;
reference
"RFC 8525: YANG library";
}
organization
"IETF NETCONF (Network Configuration) Working Group";
contact
"WG Web:
WG List:
Author: Zhuoyao Lin
Benoit Claise
IGNACIO DOMINGUEZ MARTINEZ-CASANUEVA
";
description
"This module augments the ietf-yang-library defined in
[RFC8525] to provide not only the deviation list, but also
the augmented-by list, in order to give sufficient
information about the YANG models reverse dependency. It
facilitates the process of obtaining the entire
dependencies of YANG model.
The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL',
'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED',
'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document
are to be interpreted as described in BCP 14 (RFC 2119)
(RFC 8174) when, and only when, they appear in all
capitals, as shown here.
Copyright (c) 2022 IETF Trust and the persons identified as
authors of the code. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, is permitted pursuant to, and subject
to the license terms contained in, the Revised BSD License
set forth in Section 4.c of the IETF Trust's Legal Provisions
Relating to IETF Documents
(https://trustee.ietf.org/license-info).
This version of this YANG module is part of RFC XXXX; see the
RFC itself for full legal notices. ";
revision 2023-10-27 {
description
"Added list augmented-by in yang-library/module-set/module to
make the module store the entire reverse dependency information
(augmented-by and deviation).";
reference
"RFC XXXX: Support of augmentedby in ietf-yang-library";
}
augment "/yanglib:yang-library/yanglib:module-set/yanglib:module" {
description
"Augment the augmented-by list from module info with the
module-augmented-by grouping" ;
leaf-list augmented-by {
type leafref {
path "../../yanglib:module/yanglib:name";
}
description
"Leaf-list of the augmentation used by this server to
modify the conformance of the module associated with
this entry. Note that the same module can be used for
augmented-by for multiple modules, so the same
entry MAY appear within multiple 'module' entries.
This reference MUST NOT (directly or indirectly)
refer to the module being augmented.
Robust clients may want to make sure that they handle a
situation where a module augments itself (directly or
indirectly) gracefully.";
}
}
}
]]>
Note to the RFC-Editor: Please remove this section before
publishing.
Here is the github repository for the YANG source code of this draft:
https://github.com/Zephyre777/draft-lincla-netconf-yang-library-augmentation.git.
For the demo, please refer to the demo folder in this repository. There is a netopeer2 instance running with updated version of libyang and sysrepo(links are listed in the demo instruction).
The list name has been updated from "augmentation" to
"augmented-by", in order to represent the usage clearly.
The leafref has been changed from absolute path
"/yanglib:yang-libraray/yanglib:module-set/yanglib:module/yanglib:name"
to relative path "../../yanglib:module/yanglib:name".
The YANG validation in the appendix A shows that this path
can work as expected.
Section 5 Implementation and section 6 Changes has been added.TBCThis document has no actions for IANA.An Architecture for YANG-Push to Apache Kafka IntegrationSwisscom This document describes the motivation and architecture of a native YANG-Push notifications and YANG Schema integration into Apache Kafka Message Broker and YANG Schema Registry. Toward Building a Semantic Network Inventory for Model-Driven TelemetryUniversidad Politecnica de Madrid, Telefonica I+DUniversidad Politecnica de MadridUniversidad Politecnica de MadridUniversidad Politecnica de MadridTelefonica I+DIEEE Communications MagazineThis section gives a few examples that the user can
try themselves with yanglint. This is created to prove
the syntax correctness.The valid example should pass the validation while
the invalid one will not, because the module has augmented a
module in another module-set, which is illegal according
to the YANG source code. file "example_valid.xml"
1ms1module12024-02-29urn:ietf:params:xml:ns:yang:module1module2module3module22024-02-29urn:ietf:params:xml:ns:yang:module2module32024-02-29urn:ietf:params:xml:ns:yang:module30]]> file "example_invalid.xml"
1ms1module12024-02-29urn:ietf:params:xml:ns:yang:module1module2module3module22024-02-29urn:ietf:params:xml:ns:yang:module2module32024-02-29urn:ietf:params:xml:ns:yang:module30]]>
This section defines the ietf-yang-library-rfc7895-augmentedby that augments
the ietf-yang-library defined in . The module-state/module
list of this YANG module version is also defined in the
version though deprecated.
The following is the YANG tree diagram for ietf-yang-library-rfc7895-augmentedby
augmenting RFC7895. /yanglib:modules-state/module/name
+--ro revision -> /yanglib:modules-state/module/revision
]]>The following is the full YANG tree diagram of ietf-yang-library-rfc7895-augmentedby
augmenting ietf-yang-library defined in RFC7895. /yanglib:modules-state/module/name
+--ro yanglib-aug:revision
-> /yanglib:modules-state/module/revision
notifications:
+---n yang-library-change
+--ro module-set-id -> /modules-state/module-set-id
]]>The YANG module that augments the ietf-yang-library RFC7895. file "ietf-yang-library-rfc7895-augmentedby@2023-10-27.yang"
module ietf-yang-library-rfc7895-augmentedby {
yang-version 1.1;
namespace "urn:ietf:params:xml:ns:yang:ietf-yang-library-rfc7895-augmentedby";
prefix yanglib-aug;
import ietf-yang-library {
prefix yanglib;
revision-date 2016-06-21;
reference
"RFC 7895: YANG Module Library.";
}
organization
"IETF NETCONF (Network Configuration) Working Group";
contact
"WG Web:
WG List:
Author: Zhuoyao Lin
Author: Benoit Claise
Author: IGNACIO DOMINGUEZ MARTINEZ-CASANUEVA
";
description
"This module augments the ietf-yang-library defined in [RFC7895]
to provide not only the deviation list, but also the
augmentedby list, in order to give sufficient information
about the YANG models reverse dependency. It facilitates
the process of obtaining the entire dependencies of YANG model.
The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL',
'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED',
'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document
are to be interpreted as described in BCP 14 (RFC 2119)
(RFC 8174) when, and only when, they appear in all
capitals, as shown here.
Copyright (c) 2022 IETF Trust and the persons identified as
authors of the code. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, is permitted pursuant to, and subject
to the license terms contained in, the Revised BSD License
set forth in Section 4.c of the IETF Trust's Legal Provisions
Relating to IETF Documents
(https://trustee.ietf.org/license-info).
This version of this YANG module is part of RFC XXXX; see the
RFC itself for full legal notices. ";
revision 2023-10-27 {
description
"Added list augmentedby in yang-library/modules-state/module to
make the module store the entire reverse dependency information
(augmentedby and deviation).";
reference
"RFC XXXX: Support of augmentedby in ietf-yang-library
defined in RFC7895";
}
augment "/yanglib:modules-state/yanglib:module" {
description
"Augment the augmentedby from module info with the
module-augmented-by grouping" ;
uses yanglib-aug:module-state-augmented-by;
}
/*
* Groupings
*/
grouping module-state-augmented-by {
description
"This grouping defines a list with keys being the module
name and revison. The list contains the augmented-by list.";
list augmented-by {
key "name revision";
status deprecated;
description
"List of YANG augmented-by module names and revisions
used by this server to modify the conformance of
the module associated with this entry. Note that
the same module can be used for augmented-by for
multiple modules, so the same entry MAY appear
within multiple 'module' entries.
The augment module MUST be present in the 'module'
list, with the same name and revision values.
The 'conformance-type' value will be 'implement' for
the augment module.";
leaf name {
type leafref {
path "/yanglib:modules-state/yanglib:module/yanglib:name";
}
description
"Identifies a given module in the yang library by
its name.";
}
leaf revision {
type leafref {
path "/yanglib:modules-state/yanglib:module/yanglib:revision";
}
description
"Revision of the module";
}
}
}
}
]]>The following people all contributed to creating this document:The author would like to thanks Jan Lindblad and Jean Quilbeuf
for his help during the design of the YANG module.