In-Line Network Management Prediction

This work in progress describes a mechanism that allows a distributed model, injected into a network, to predict the state of the network. The concept is illustrated in . The state to be predicted is modeled within each actual network node. Thus, a distributed model, shown in the top plane, is formed within the actual network, shown in the bottom plane. The top plane slides ahead of wallclock time, although in an asynchronous manner. This means that each simulated node MAY have its own notion of simulation time. ________________________________________________ / /---------o... / / o----o... / / /------o---o... / /_Distributed Network Model Plane_______________/ (spatially located inside the actual network below, but temporally located ahead of the actual network) -------------------------------------------------------> Wallclock ________________________________________________ / / / /---------o... / / o----o... / / /------o---o... / /_Actual Network Plane__________________________/ : The Distributed Model Inside the Network. This concept opens up a set of interoperability issues which do not appear to have been fully addressed. How can distributed model components be injected into an existing network? In-line models are injected into the network assuming the overlay environment shown in . In-line models in are designed to run as fast as possible in order to maintain a simulation time that is ahead of wallclock, communicating via virtual messages with future timestamps. What if messages are processed out-of-order because they arrive out-of-order at a node? How long do you wait (and slow your simulation down) to make sure they are not out-of-order? This specification provides a framework that allows synchronization to be handled in any manner; e.g. via a conservative (blocking) or optimistic (Time-Warp) manner within the network. Additionally, how can the models verify and maintain a reasonable amount of accuracy? A mechanism is provided in this document to allow local verification of prediction accuracy. Attempts to adjust accuracy are implementation dependent. How do independent model developers allow their models to work coherently in this framework? Model operation is implementation dependent, however, this specification attempts to make certain that model messages will at least be transported in an inter-operable manner, both across and WITHIN, intermediate network devices. How does one publish their model descriptions? How are predicted values represented and accessed? Suggestion solutions for these questions are presented in this document as well.

In-line predictive network management, which enables greater performance and fault tolerance, is based upon algorithmic information injected into a network allowing system state to be predicted and efficiently propagated throughout the network. This paradigm enables management of the network with continuous projection and refinement of future state in real time. In other words, the models injected into the network allow state to be predicted and propagated throughout the network enabling the network to operate simultaneously in real time and in the future. The state of traffic, security, mobility, health, and other network properties found in typical Simple Network Management Protocol (SNMP) Management Information Bases (MIB) is available for use by the management system. To enable predictive management of applications, new MIBs will have to be defined that hold both current values as well as values expected to exist in the future. The AgentX protocol begins to address the issue of independent SNMP agent developers dynamically and seamlessly interconnecting their agents into a single MIB under the control of a master agent. AgentX specifies the protocol between the master and sub-agents allowing the sub-agents to connect to the master agent. The AgentX specification complements this work-in-progress, namely, in-line network management prediction. The in-line network management prediction specification provides the necessary interface between agent functionality injected remotely via an Active Packet and dynamically 'linked' into a MIB. The agent code may enhance an existing MIB value by allowing it to return predicted values. Otherwise, coexistence with AgentX is SUGGESTED. The in-line network management prediction specification enables faster development of MIB modules with more dynamic algorithmic capability because Active and Programmable networks allow lower-level, secure, dynamic access to network devices. This has allowed injection of predictive capability into selected portions of existing MIBs and into selected portions of active or programmable network devices resulting in greater performance and fault tolerance.

This document proposes standards for the following aspects of in-line predictive management: SNMP Object Time Series Representation and Manipulation Common Algorithmic Description Multi-Party In-line Predictive Model Access and Control Common Framework for Injecting Models into the Network Model Interface with the Framework The high-level components of this proposed standard are shown in . The Active Network Framework is a work in progress. In-line Predictive Management is the subject of this document. The Internet Protocol and SNMP are well-known.

shows the various ways in which in-line predictive management can be used in an active network given an implementation in a particular execution environment. The in-line predictive management application runs as an active application on an active node. The framework is independent of the underlying architecture of the active network, which can take one of two forms. The protocol stack on the left shows a fully active network in which the Node Operating System runs one or more Execution Environments . Multiple active applications may execute in any Execution Environment. The protocol stack on the right shows the architecture of an active network overlay over IP. Essentially, the overlay scheme uses the Active Network Encapsulation Protocol (ANEP) as a conduit to use the underlying IP network. The predictive management application executes alongside the other active applications and interacts with any managed active applications to provide their future state. Since the predictive management application requires only the execution environment to run in, it is independent of whether the active network is implemented as an overlay or it is available as a fully active network. +-------+-------+-----------+ +--------+---------+-------------+ |Active |Active | In-line | | Active | Active | In-line | | Appl | Appl | Predictive| | Appl | Appl | Predictive | | | | Management| | | | Management | +-------+-------+-----------+ +--------+---------+-------------+ | Active Net EE | | Active Net EE | +---------------------------+ +--------------------------------+ | NodeOS | | Node OS | +---------------------------+ +------------------+-------------+ | ANEP | | ANEP | +---------------------------+ +------------------+-------------+ | Internet Protocol| SNMP | +------------------+-------------+ Active Network over IP : Relationship Among Underlying Assumptions about the Predictive Management Environment. The next section provides basic definitions. Following that, the goals of this proposed standard are laid out. The remainder of the document develops into progressively more detail defining interoperability among algorithmic in-line network management prediction components. Specifically, predictive capability requires careful handling of the time dimension. Rather than change the SNMP standard, a tabular technique is suggested. Then, in order to simplify design of predictive management objects, an extension to Case Diagrams is suggested for review and comment. This is followed by the specification of a distributed predictive framework. It is understood that multiple distributed predictive mechanisms exist, however, this framework is presented for comment and review because it contains all the necessary elements. Finally, the detailed interface between the active or programmable code and IP standard interfaces is presented.

The following acronyms and definitions are helpful in understanding the general concept of predictive network management. In-line Located within, or immediately adjacent to, the flow of network traffic. Predictive Network Management The capability of reliably predicting network events or the state of the network at a time greater than wall-clock time. Fine-Grained Models Small, light-weight, executable code modules that capture the behavior of a network or application component to enable predictive network management. Algorithmic Information Information, in the form of algorithms contained inside executable code, as opposed to static, non-executable data. Depending upon the complexity of the information to be transferred, an algorithmic form, or an optimal tradeoff between algorithmic and non-algorithmic form can be extremely flexible and efficient. Non-Algorithmic Information Information that cannot be executed. Generally requires a highly structured protocol to transfer with well-defined code pre-installed at all points in route including source and destination. Small-State Information caches that can be created at network nodes, intended for use by executable components of the same application. Global-State Information caches created at network nodes, intended to be used by executable components of different applications. Multi-Party In-line Predictive Management Model An in-line predictive management model comprised of multiple in-line algorithmic models that are developed, installed, utilized, and administered by multiple domains. The following acronyms and definitions are useful in understanding the details of the specific predictive network management framework described in this document. A (Anti-Toggle) Used to indicate an anti-message. The anti-message is initiated by rollback and is used to keep the system within a specific range of prediction accuracy. AA (Active Application) An active network protocol or service that is injected into the network in the form of active packets. The active packets are executed within the EE. Active Network A network that allows executable code to be injected into the nodes of the network and allows the code to be executed at the nodes. Active Packet The executable code that is injected into the nodes of an active network. Anti-Message An exact duplicate of a virtual message except for that the Anti-toggle bit is set. An Anti-message is used to annihilate an invalid virtual message. This is an implementation specific feature relevant to optimistic distributed simulation. DP (Driving Process) Generates virtual messages. Generally, the DP is implemented as an algorithm that samples network state and transforms the state into a prediction. The prediction is represented by a virtual message. EE (Execution Environment) The active network execution environment. The environment that resides on active network nodes that executes active packets. Lookahead The difference between Wallclock and LVT. This value is the distance into the future for which predictions are made. LP (Logical Process) An LP consists of the Physical Process and additional data structures and instructions which maintain message order and correct operation as a system executes ahead of real time. LVT (Local Virtual Time) The LP contains a notion of time local to itself known as LVT. A node's LVT may differ from other nodes' LVT and Wallclock. LVT is a local, asynchronous notion of time. M (Message) The message portion of a Virtual Message is implementation specific. This proposed standard SUGGESTS that the message contents be opaque, however, an SNMP varbind, intended to represent future state, MAY be transported. Executable code may also be transported within the message contents. NodeOS (Node Operating System) The active network Operating System. The supporting infrastructure on intermediate networks nodes that supports one or more execution environments. PP (Physical Process) A PP is an actual process. It usually refers the actual process being modeled, or whose state will be predicted. QS (Send Queue) A queue used to hold copies of messages that have been sent by an LP. The messages in the QS may be sent as anti-messages if a rollback occurs. Rollback The process of adjusting the accuracy of predictive components due to packets arriving out-of-order or out-of-tolerance. Rollback is specific to optimistic distributed simulation techniques and is thus an implementation specific feature. RT (Receive Time) The time message value is predicted to be valid. RQ (Receive Queue) A queue used in the algorithm to hold incoming messages to an LP. The messages are stored in the queue in order by receive time. SQ (State Queue) The SQ is used as a LP structure to hold saved state information for use in case of a rollback. The SQ is the cache into which pre-computed results are stored. Tolerance A user-specified limit on the amount of prediction error allowed by an LP's prediction. TR (Real Time) The current time as a time-stamp within a virtual message. TS (Send Time) The LVT that a virtual message has been sent. This value is carried within the header of the message. The TS is used for canceling the effects of false messages. VM (Virtual Message) A message, or state, expected to exist in the future. Wallclock The current time.

The goals of this document are... Simplicity This document attempts to describe the minimum necessary elements for in-line management prediction. Model developers should be able to inject models into the network allowing SNMP Object value prediction. Such models should work seamlessly with other predictive models in the network. The goal is to minimize the burden on the model developer while also insuring model interoperability. Conformance This document attempts conformance with existing standards when and where it is possible to do so. The concept is to facilitate a gradual transition to the active and programmable networking paradigm. In-line Algorithmically-Based Management This document attempts to introduce the use of in-line algorithmic management information.

SNMP, as currently defined, has a very limited notion of time associated with state information. The temporal semantics are expected to be applied to the state by the applications reading the information. On the other hand, predictive management requires generation, handling and transport of information that understands the temporal characteristics of the state, i.e. whether the information is current, future, or perhaps past information. In other words, capability for handling the time dimension of management information needs to be extended and standardized in some manner. In this section, we propose a mechanism for handling time issues in predictive management that require minimal changes from the SNMP standard. A proposed standard technique for handling the time dimension in predictive state systems is to build the SNMP Object as a Table Object indexed by time. This is shown in the following excerpt from a Load Prediction MIB...

. . ~ . . loadPrediction OBJECT IDENTIFIER ::= { loadPredMIB 1 } loadPredictionTable OBJECT-TYPE SYNTAX SEQUENCE OF LoadPredictionEntry MAX-ACCESS not-accessible STATUS current DESCRIPTION "Table of load prediction information." ::= { loadPrediction 1 } loadPredictionEntry OBJECT-TYPE SYNTAX LoadPredictionEntry MAX-ACCESS not-accessible STATUS current DESCRIPTION "Table of Atropos LP prediction information." INDEX { loadPredictionPort } ::= { loadPredictionTable 1 } LoadPredictionEntry ::= SEQUENCE { loadPredictionID DisplayString, loadPredictionPredictedLoad INTEGER, loadPredictionPredictedTime INTEGER } loadPredictionID OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "The LP identifier." ::= { loadPredictionEntry 1 } loadPredictionPredictedLoad OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the predicted load on the link." ::= { loadPredictionEntry 2 } loadPredictionPredictedCPUTime OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the predicted processor time used by a packet on this node." ::= { loadPredictionEntry 3 } loadPredictionPredictedTime OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the time at which the predicted event will be valid." ::= { loadPredictionEntry 4 } . . ~ . . : MIB Structure for Handling Object Values with Predictive Capability. In , the result of an SNMP query of the relevant predictive MIB Object is displayed. Because the identifiers are suffixed by time, the object values are sorted temporally. If a client wishes to know the next predicted event on or before a given time, the the query can be formulated as a GET-NEXT with the next predicted event time to be determined as the suffix. The GET-NEXT-RESPONSE will contain the next predicted event along with its time of occurrence. Otherwise, a value outside the table will be returned if no such predicted value yet exists.

. . ~ . . loadPredictionTable.loadPredictionEntry.loadPredictionID.1 -> OCTET STRING- (ascii):AN-1 loadPredictionTable.loadPredictionEntry.loadPredictionPort.1 -> INTEGER: 3325 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedLoad.4847 -> INTEGER: 240 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedLoad.20000 -> INTEGER: 420 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedLoad.40000 -> INTEGER: 460 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedLoad.60000 -> INTEGER: 497 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedLoad.80000 -> INTEGER: 540 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedLoad.100000 -> INTEGER: 580 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedLoad.120000 -> INTEGER: 619 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedLoad.140000 -> INTEGER: 660 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedTime.4847 -> INTEGER: 4847 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedTime.20000 -> INTEGER: 20000 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedTime.40000 -> INTEGER: 40000 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedTime.60000 -> INTEGER: 60000 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedTime.80000 -> INTEGER: 80000 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedTime.100000 -> INTEGER: 100000 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedTime.120000 -> INTEGER: 120000 loadPredictionTable.loadPredictionEntry.loadPredictionPredictedTime.140000 -> INTEGER: 140000 loadPredictionTable.loadPredictionEntry.loadPredictionCurrentLoad.1 -> INTEGER: 15949 loadPredictionTable.loadPredictionEntry.loadPredictionCurrentTime.1 -> INTEGER: 25639 . . ~ . . : Output from a Query of the MIB Structure for Handling Object Values with Predictive Capability. This allows SNMP GET-NEXT operations from a client to locate an event nearest to the requested time as well as search in temporal order for next predicted events.

SNMP, as currently defined, assumes that non-algorithmic descriptive information will be generated, handled, or transported. Prediction requires model development and execution. This proposed standard SUGGESTS that models are to be small, low-overhead, and fine-grained. Fine-grained refers to the fact that the models are locally constrained in time and space. In this section, we propose algorithmic descriptions of management models designed to encourage the understanding and use of in-line predictive management techniques. Case Diagrams provide a well-known representation for the relation of management information to information flow as shown in . The details of Case Diagrams will not be discussed here (see the previous reference for more information). The purpose of this section is to illustrate an enhancement to the diagram that allows algorithmic information to be specified, particularly for multi-party predictive model interaction.

An excerpt of an SNMP Case Diagram serves to provide a flavor of its current format. The diagram below shows packets arriving from a lower network layer. Some packets are determined to have encoding errors and are discarded. The remaining packets flow to the upper layer. ^ Upper Layer | ==+== outPackets | ~ | +==> encodingErrors | ~ | ==+== inPackets ^ | Lower Layer : An Example Case Diagram. For the purposes of in-line predictive management, models SHOULD be specified and injected into the system. These models MAY coexist with the current SNMP management model supplementing the information with predictive values. This is denoted by adding algorithmic model information to the Case Diagram. A '+' sign after the name of an Object Identifier identifies the object as one that can return future values. The model used to predict the future information is written within braces near the Object identifier and incorporates the name of the SNMP object identifiers. This document SUGGESTS using a common syntax for the notation such as that used for code blocks by the C Programming Language block constructs, Java Programming Language blocks, or the notation used by any number of other languages. Standardization of the model syntax is outside the scope of interest for this document. All functions MUST be defined. Operating system function calls MAY NOT be used. The salient point is that the algorithm must be clearly and concisely defined. The algorithm must also be a faithful representation of the actual predictive model injected into the system. As shown in , 'encodingErrors' is predictively enhanced to be 10% of 'inPackets' for future values. The predictive algorithm MUST run on the network node and MUST be immediately available as input for other predictively enhanced objects. The predicted value MUST be available as a response to SNMP queries for future state information, or for transfer to other nodes via virtual messages, explained later in this document. SNMP Objects that are enhanced with predictive capability are assumed to always have the actual monitored value at Wallclock time.

^ Upper Layer | ==+== outPackets | ~ | +==> encodingErrors+ { 0.1 * inPackets } | ~ | ==+== inPackets ^ | Lower Layer : A Sample Algorithmic Description. If this were a wireless network, a more realistic algorithmic model would likely incorporate channel quality SNMP Objects into the 'encodingErrors' prediction algorithm. In many cases, the algorithmic portion of the Case Diagram will involve SNMP objects from other nodes. Syntax should include the ability to identify general topological information in the description of external objects. For example, 'inPackets[adj]' or 'inPackets[edge]' should indicate immediately adjacent nodes or nodes at the topological edge of the network. In the example shown in , a 'packetsForwarded' object has predictive capability denoted by the '+' symbol. The predictive capability comes from an algorithmic model specified within the braces next to the object name. In this case, the prediction will be the value of the 'driverForwarded' object from the node closest to the edge of the network.

^ Upper Layer | ==+== outPackets | ~ | +==> packetsForwarded+ { driverForwarded[edge] } | ~ | ==+== inPackets ^ | Lower Layer : An Algorithmic Description Using State Generated from Another Node Described in . In the following figure, which is an SNMP diagram of the edge node, the 'driverForwarded' object is predicted by executing the algorithm in braces. This algorithm predicts 'driverForwarded' packets to be a linear approximation of a sample of 'appPackets'. The sample is 'epsilon' time units apart and the prediction is 'delta' time units into the future.

^ Upper Layer | ==+== driverPackets | ~ | +==> driverForwarded+ | { delta * (appPackets(t-epsilon) - appPackets(t))/ epsilon } ~ | ==+== inPackets ^ | Lower Layer : A Node Generating State Information Used by the Node in .

Multiple developers and administrators of in-line predictive algorithmic models will require mechanisms to ensure correct understanding and operation of each others' models and intentions.

It may be necessary to register predictive models. Registration is often an IANA function . Algorithmic model registration needs to be handled more dynamically than AgentX models. Algorithmic models, while not necessary doing so, have the capability to install/de-install at rapid rates. The in-line model installation and de-installation proposed standard is described in .

Multiple models residing on a node need to inter-operate with one another. This document proposes to use SNMP Object Identifiers as much as possible for communication of state information among models. In addition, multiple Active Application models may choose to communicate with one another via global state.

Querying an IP addressable node for SNMP objects that are predictively enhanced should appear transparent to the person polling the node. Multiple ports, etc.. should not be required. A program injected into a node that serves to extend an SNMP MIB MAY do so using global state. A global state cache holds the SNMP object values and responds via an internal port to connect with a master SNMP agent for the node.

This section specifies an algorithmic predictive management framework. The framework allows details of distributed simulation, such as time management, state saving, and model development to be implementation dependent while ensuring in-line inter-operability both with, and within, the network. The general predictive network management architecture MUST contain at least one Driving Processes (DP), MAY contain Logical Processes (LP), and MUST use Virtual Messages (VM).

illustrates network nodes containing DPs and LPs. The annotation under nodes AH-1 and AN-1 are an SNMP Object Identifier. SNMP Object Identifier 'oid_1' represents state of node AH-1. The predictively enhanced SNMP Object Identifier, 'oid+' on node AN-1 is a function of 'oid_1'. Note that 'f()' is shown as an arbitrary function in the figure, but MUST be well-defined in practice. +------+ +-----+ | LP |-->... | VM | |(node)| |(msg)| /+------+ +------+ +-----+ +------+/ | DP |------------------->| LP | |(node)| |(node)|---->... | AH-1 | | AN-1 | +------+ +------+ oid_1 oid+ {f(oid_1)} \ \ +------+ | LP |-->... |(node)| +------+ : Framework Entity Types. The framework makes a distinction between a Physical Process and a Logical Process. A Physical Process is nothing more than an executable task defined by program code i.e. it is the implementation of a particular model or a hardware component or a direct connection to a hardware component representing a device. An example of a Physical Process is the packet forwarding process on a router. Each Physical Process MUST be encapsulated within a Logical Process, labeled LP in . A Logical Process consists of a Physical Process, or a model of the Physical Process and additional implementation specific data structures and instructions to maintain message order and correct operation as the system executes ahead of current (or Wallclock) time as illustrated in greater detail in . The details of the DP and LP structure and operation are implementation specific, while the inter-operation of the DP/LP system must be specified. The LP architecture is abstracted in . The flow of messages through the LP is shown by the arrows entering from the left side of the figure. The in-line predictive framework components are shown in , where AH-1 and AN-1 are Active Host 1 and Active Node 1 respectively. In this context, active hosts are nodes that can inject new packets into the network while active nodes are nodes that behave as intermediate hops in a network. The Logical Process MUST handle time management for the model. The Logical Process and the model that it implements MAY be implemented in any manner, however, they must be capable of inter-operating. The framework MUST be capable of supporting both conservative and optimistic time management within the network. Conservative time management REQUIRES that the model block when messages MAY be received out-of-order while optimistic time management MAY allow model processing to continue, even when messages are received out-of-order. However, additional implementation specific mechanisms MAY be used to account for out-of-order messages. Such mechanisms MAY be embedded within the Logical Process and this specification does not attempt to standardize them.

Virtual input messages directed to a Logical Process MUST be received by the Logical Process, passed to the model, and processed. Virtual output messages MAY be generated as a result. +-------------------------------------------------------------+ | Active Application | | | | +-----------------------------+ | | | Logical Process | | | | (Time Management) | | | | +-------------+ | | | | | Model | | | | Virtual Input Msgs | | Virtual Output Msgs | ========================>| |==========================> | | | | | | | | +------/\-----+ | | | | || State | | | | +--------\/-------+ | | | | | State Queue | | | | +------| Predicted Values|----+ | | | (Small-state) | | | +--------/\-------+ | +-----------------------------||------------------------------+ \/ SNMP : A High-Level View of the Logical Process Framework Component within an Active Application.

Virtual messages contain the following fields: Send Time (TS) which MUST contain the LVT (local simulation time) at which the message was sent Receive Time (TR) which MUST denote the time the message is expected to exist in the future MAY contain an (optional) Anti-toggle (A) bit for out-of-order message handling purposes such as message cancellation and rollback MUST contain the message content itself (M) which is model specific Thus, a Virtual Message (VM) MUST have the following structure... 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Send-Time (TS) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Receive-Time (RT) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Real-Time (TR) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |A| . | +-+ . | | . | ~ Message (M) ~ | . | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : An In-line Management Prediction Virtual Message. These in-line predictive messages, or virtual messages, that contain invalid fields because the transmitting Logical Processes used an incompatible time management technique MUST be dropped. However, it is SUGGESTED that a count of such packets be maintained in a general in-line predictive management framework MIB. The Receive Time field MUST be filled with the time that this message is predicted to be valid at the destination Logical Process. The Send Time field MUST be filled with the time that this message was sent by the originating Logical Process. The Anti-Toggle (A) field MUST be used for creating an anti-message to remove the effects of false messages as described later. A message MUST also contain a field for the current Real Time (RT). If a message arrives at a Logical Process out-of-order or with invalid information, that is, out of a pre-specified tolerance for prediction accuracy, it is called a false message. The method for handling false messages is implementation specific. The Receive Queue, shown in , maintains newly arriving messages in order by Receive Time (TR). The implementation of the Receive Queue is implementation specific.

The Driving and Logical Processes MUST communicate via virtual messages as shown in . The Driving Process MAY generate predictions based upon SNMP queries of other layers on the local node. The Logical Process MAY check its prediction accuracy via SNMP queries of other layers on its local node. +------------------+/--+ +------------------+/--+ | DP |\-|| SNMP | LP |\-|| SNMP +------------------+ || +------------------+ || | Virtual Messages | || | Virtual Messages | || +------------------+ || +------------------+ || | ANEP |__|| | ANEP |__|| +------------------+--|| +------------------+--|| | IP |__|| | IP |__|| +------------------+---+ +------------------+---+ Driving Process Logical Process : Facility for Checking Accuracy with Actual Network SNMP Objects in the In-line Predictive Management Framework.

The in-line predictive framework MAY allow for prediction refinement and correction by communicating with the actual component whose state is to be predicted via an SNMP query. The asynchronous prediction mechanism has the following architecture for Logical Process... +-------------------------------------------------------+ | Logical Process | | | | State Queue (MIB) | | +-+ | | | | | | +-+ | | | | | Virtual Message Route +-+ | ========> ]O =============>| |=========> ]O ===============> | Receive Queue +-+ Send Queue | | Model | +--------------------------/\---------------------------+ || SNMP Object Id (oid) || +-------------------------------------------------------+ | Actual Component Whose State is to be Predicted | +-------------------------------------------------------+ : A Logical Process Implementation and Interface. All of the Logical Process queues and caches MAY reside in an active node's Small-State. Small-State is a persistent memory cache left behind by an active packet that is available to trailing active packets that have the proper access rights. Typically, any type of information can be stored in Small-State. The Receive Queue MAY maintain active virtual message ordering and scheduling. All active packets MUST be encapsulated inside Active Packets following the Active Network Encapsulation Protocol format. Once a virtual message leaves the Receive Queue, the virtual time of the Logical Process, known as Local Virtual Time, MUST be updated to the value of the Receive Time from the departing virtual message. Virtual messages MUST originate from Driving Processes, shown in that predict future events and inject them into the system as virtual messages. The development of a Driving Process and Logical Process are dependent upon the model used to enhance the desired state of the system with predictive capability. Logical Processes MUST only operate upon the the arrival of virtual input messages and MUST NEVER spontaneously generate virtual messages. Following the arrows across , virtual messages enter either the Physical Process. The state of the Logical Process is periodically saved in the State Queue (SQ) shown as the State Cache in . State Queue values are used to restore the Logical Process to a known safe state when false messages are received. State values are continuously compared with actual values from the Physical Process to check for prediction accuracy, which in the case of load prediction is the number and arrival times of predicted and actual packets received. If the prediction error exceeds a specified tolerance, a rollback MAY occur. An important part of the architecture for network management is the fact that the State Queue within the in-line management prediction architecture is the node's Management Information Base. The State Queue values are the SNMP Management Information Base Object values; but unlike legacy SNMP values, these values are expected to occur in the future. The State Queue operation is implementation dependent, however, it holds the predicted SNMP Objects, is SUGGESTED to be implemented in small-state, and MUST use the interface specified in to respond to SNMP queries. The current version of SNMP has no mechanism to indicate that a managed object is reporting its future state; currently all results are reported with a timestamp that contains the current time. In working on predictive active network management prediction there is a need for managed entities to report their state information at times in the future. These times are unknown to the requester. A simple means to request and respond with future time information is to append the future time to all Management Information Base Object Identifiers that are predicted. This requires making these objects members of a Management Information Base table indexed by predicted time as discussed in . This can be seen in the loadPredictionTable shown in . Thus a Simple Network Management Protocol client, who does not know the exact time of the next predicted value, can issue a get-next command appending the current time to the known object identifier. The managed object responds with the requested object valid at the closest future time. The figure illustrates an SNMP request and the corresponding response. Future times are the LVT of the Logical Process running on a particular node. As Wallclock approaches a particular future time, predicted values MAY be adjusted, allowing the prediction to become more accurate. The table of future values MAY be maintained within a sliding Lookahead window, so that old values are removed and the prediction does exceed a given future time. Continuing along the arrows in , any virtual messages that are generated as a result of the Physical Process or model computation proceed to the Send Queue (QS). The Send Queue is implementation dependent, however, it MAY maintain copies of virtual messages to be transmitted in order of their send times. The Send Queue is required for the generation of anti-messages during rollback. Anti-Messages annihilate corresponding virtual messages when they meet to correct for previously sent false messages. Annihilation is simply the removal of both the actual and the anti-message. Where the annihilation occurs is implementation specific and left to the implementor. After leaving the Send Queue, virtual messages travel to their destination Logical Process. Further details on the optimistic synchronization mechanism are implementation dependent and outside the scope of this work in progress.

An in-line management prediction model developer MUST implement at least one Driving Processing and MAY implement a Logical Process using the same time management technique. The model developer MAY include an SNMP client within the model in order to query the modeled component in order to improve prediction accuracy. The model developer's Driving Process MUST generate virtual messages. The Logical Process MUST receive and process those messages. The Logical Process MAY respond to virtual messages by generating virtual message(s). The Logical Process MAY use active network node Small-state to hold a time series of the SNMP Object Id whose value is being continuously predicted. The interface to the SNMP MIB small-state is specified in the following section.

The general active network architectural framework, without any specific network management paradigm implementation, is shown in . Active Applications +----+ +----+ +----+ +----+ |AA 1| |AA 2| |AA 3| |AA 4| +----+ +----+ +----+ +----+ EE-specific ____________ ____________ Programming i/f's +----------+ +----------+ Execution | | | | Environments | EE 1 | | EE 2 | | | | | +----------+ +----------+ NodeOS i/f ========================== Low-level channels, threads, Abstractions state storage, ... : The Active Network Framework. In-line network management prediction requires a general active network framework that supports active applications to be injected into the proper execution environments. The in-line management prediction framework enforces certain minimal requirements on the execution environment, which are listed below.

The execution environment MUST provide an information cache called 'Small State' as defined in to enable information exchange between active packets, defined in . The execution environment MAY also provide an information cache called 'Global State', defined in , to enable the in-line management prediction framework to communicate with a predictively managed active application to query its current state. The EE MUST provide an API to be able to store and query both 'Small State' and also to 'Global State', if it is implemented. The EE SHOULD provide appropriate access control mechanisms to both 'Small State' and also to 'Global State', if it is implemented.

The execution environment MUST provide an interface that enables both the in-line management prediction values and the values of the actual component being managed to publish their state to an SNMP MIB. This enables the in-line management prediction framework to store the predicted state in a well-known format and also enables legacy SNMP tools to query the predicted state using SNMP operations. Additionally, the managed application is also able to update its current state using SNMP, which the Logical Process will be able to query. In a particular implementation of such an interface, a generic SNMP agent coded as an active application MAY be injected into the active nodes. The agent creates a 'Global State' on the active node with a well-known name. The agent reads information coded in a known format that has been written to the 'Global State' and publishes it to the MIB. Any active application that wishes to advertise its state uses an interface that enables it to store its information in the well-known 'Global State' in the given format.

The format of the messages that are posted between the SNMP agent and an active application are shown below, +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Message Type | Object ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | . | ~ Value ~ | . | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : Message Packet.

The SNMP Agent and the active application MAY use special interfaces to implement messaging between them. A Message Packet, whose format is shown in , is the basic unit of inter-application communication. Each message consists of a message type. The type SHOULD assume one of the following values: MSG_ADDINT: to add a new MIB Object of type SNMP INTEGER MSG_UPDATEINT: to update the value of an MIB Object of type SNMP INTEGER MSG_GETINT: to get the value of an MIB Object of type SNMP INTEGER MSG_ADDLONG: to add a new MIB Object of type SNMP LONG MSG_UPDATELONG: to update the value of an MIB Object of type SNMP LONG MSG_GETLONG: to get the value of an MIB Object of type SNMP LONG MSG_ADDSTRING: to add a new MIB Object of type SNMP STRING MSG_UPDATESTRING: to update the value of an MIB Object of type SNMP STRING MSG_GETSTRING: to get the value of an MIB Object of type SNMP STRING The active application SHOULD send a message of the valid message type to the SNMP agent to perform the required operation. On receipt of a message, the SNMP agent SHOULD attempt to perform the requested operation. It MUST then respond with an acknowledgment message in a format shown in .

The acknowledgment message has the following format. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Status Code | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | . | ~ Status Message ~ | . | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : Acknowledgment Message Packet. The status code MUST have one of the following values: OK: to indicate successful operation ERR_DUPENTRY: if for a MSG_ADD operation, an Object identifier of given name already exists ERR_NOSUCHID: if for a MSG_UPDATE operation, an Object identifier of given name does not exist. The Status message MAY be any descriptive string explaining the nature of the failure or SHOULD be "Success" for a successful operation.

Models injected into the network allow network state to be predicted and efficiently propagated throughout the active network enabling the network to operate simultaneously in real time as well as project the future state of the network. Network state information, such as load, capacity, security, mobility, faults, and other state information with supporting models, is automatically available for use by the management system with current values and with values expected to exist in the future. In the current version, sample load and processor usage prediction applications have been experimentally validated using the Atropos Toolkit. The toolkit's distributed simulation infrastructure takes advantage of parallel processing within the network, because computation occurs concurrently at all participating active nodes. The network being emulated can be queried in real time to verify the prediction accuracy. Measures such as rollbacks are taken to keep the simulation in line with actual performance.

Further details on the in-line network management prediction concept can be found in Active Networks and Active Network Management. The SNMP MIB for the in-line predictive management system described in this proposed standard follows in the next section.

ATROPOS-MIB DEFINITIONS ::= BEGIN IMPORTS MODULE-IDENTITY, OBJECT-TYPE, experimental, Counter32, TimeTicks FROM SNMPv2-SMI DisplayString FROM SNMPv2-TC; atroposMIB MODULE-IDENTITY LAST-UPDATED "9801010000Z" ORGANIZATION "GE CRD" CONTACT-INFO "Stephen F. Bush bushsf@crd.ge.com" DESCRIPTION "Experimental MIB modules for the Active Virtual Network Management Prediction (Atropos) system." ::= { experimental active(75) 4 } -- -- Logical Process Table -- lP OBJECT IDENTIFIER ::= { atroposMIB 1 } lPTable OBJECT-TYPE SYNTAX SEQUENCE OF LPEntry MAX-ACCESS not-accessible STATUS current DESCRIPTION "Table of Atropos LP information." ::= { lP 1 } lPEntry OBJECT-TYPE SYNTAX LPEntry MAX-ACCESS not-accessible STATUS current DESCRIPTION "Table of Atropos LP information." INDEX { lPIndex } ::= { lPTable 1 } LPEntry ::= SEQUENCE { lPIndex INTEGER, lPID DisplayString, lPLVT INTEGER, lPQRSize INTEGER, lPQSSize INTEGER, lPCausalityRollbacks INTEGER, lPToleranceRollbacks INTEGER, lPSQSize INTEGER, lPTolerance INTEGER, lPGVT INTEGER, lPLookAhead INTEGER, lPGvtUpdate INTEGER, lPStepSize INTEGER, lPReal INTEGER, lPVirtual INTEGER, lPNumPkts INTEGER, lPNumAnti INTEGER, lPPredAcc DisplayString, lPPropX DisplayString, lPPropY DisplayString, lPETask DisplayString, lPETrb DisplayString, lPVmRate DisplayString, lPReRate DisplayString, lPSpeedup DisplayString, lPLookahead DisplayString, lPNumNoState INTEGER, lPStatePred DisplayString, lPPktPred DisplayString, lPTdiff DisplayString, lPStateError DisplayString, lPUptime TimeTicks } lPIndex OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS not-accessible STATUS current DESCRIPTION "The LP table index." ::= { lPEntry 1 } lPID OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "The LP identifier." ::= { lPEntry 2 } lPLVT OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the LP Local Virtual Time." ::= { lPEntry 3 } lPQRSize OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the LP Receive Queue Size." ::= { lPEntry 4 } lPQSSize OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the LP send queue size." ::= { lPEntry 5 } lPCausalityRollbacks OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the number of rollbacks this LP has suffered." ::= { lPEntry 6 } lPToleranceRollbacks OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the number of rollbacks this LP has suffered." ::= { lPEntry 7 } lPSQSize OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the LP state queue size." ::= { lPEntry 8 } lPTolerance OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the allowable deviation between process's predicted state and the actual state." ::= { lPEntry 9 } lPGVT OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is this system's notion of Global Virtual Time." ::= { lPEntry 10 } lPLookAhead OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is this system's maximum time into which it can predict." ::= { lPEntry 11 } lPGvtUpdate OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the GVT update rate." ::= { lPEntry 12 } lPStepSize OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the lookahead (Delta) in milliseconds for each virtual message as generated from the driving process." ::= { lPEntry 13 } lPReal OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the total number of real messages received." ::= { lPEntry 14 } lPVirtual OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the total number of virtual messages received." ::= { lPEntry 15 } lPNumPkts OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the total number of all Atropos packets received." ::= { lPEntry 16 } lPNumAnti OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the total number of Anti-Messages transmitted by this Logical Process." ::= { lPEntry 17 } lPPredAcc OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the prediction accuracy based upon time weighted average of the difference between predicted and real values." ::= { lPEntry 18 } lPPropX OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the proportion of out-of-order messages received at this Logical Process." ::= { lPEntry 19 } lPPropY OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the proportion of out-of-tolerance messages received at this Logical Process." ::= { lPEntry 20 } lPETask OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the expected task execution wallclock time for this Logical Process." ::= { lPEntry 21 } lPETrb OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the expected wallclock time spent performing a rollback for this Logical Process." ::= { lPEntry 22 } lPVmRate OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the rate at which virtual messages were processed by this Logical Process." ::= { lPEntry 23 } lPReRate OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the time until next virtual message." ::= { lPEntry 24 } lPSpeedup OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the speedup, ratio of virtual time to wallclock time, of this logical process." ::= { lPEntry 25 } lPLookahead OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the expected lookahead in milliseconds of this Logical Process." ::= { lPEntry 26 } lPNumNoState OBJECT-TYPE SYNTAX INTEGER (0..2147483647) MAX-ACCESS read-only STATUS current DESCRIPTION "This is the number of times there was no valid state to restore when needed by a rollback or when required to check prediction accuracy." ::= { lPEntry 27 } lPStatePred OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the cached value of the state at the nearest time to the current time." ::= { lPEntry 28 } lPPktPred OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the predicted value in a virtual message." ::= { lPEntry 29 } lPTdiff OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the time difference between a predicted and an actual value." ::= { lPEntry 30 } lPStateError OBJECT-TYPE SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the difference between the contents of an application value and the state value as seen within the virtual message." ::= { lPEntry 31 } lPUptime OBJECT-TYPE SYNTAX INTEGER (0..2147483647) --SYNTAX DisplayString MAX-ACCESS read-only STATUS current DESCRIPTION "This is the time in milliseconds that Atropos has been running on this node." ::= { lPEntry 32 } END : The Atropos MIB.

Clearly, the power and flexibility to increase performance via the ability to inject algorithmic information also has security implications. Fundamental active network framework security implications will be discussed in .