US006108782A United States Patent   Patent Number: Fletcher et al.   DISTRIBUTED REMOTE MONITORING (DRMON) FOR NETWORKS  Inventors: Rick Fletcher, San Jose; Prakash Banthia, Santa Clara, both of Calif, Date of Patent: 6,108,782 Aug. 22, 2000 J ander, “Midlevel Managers Ease SNMP Information Over load,” Data Communications, vol. 22, No. 17, Nov. 1993, pp. 53, 54, 56, 58. Jander, “Lightening the Load on Management Stations,” Data Communications, vol. 23, No. 9, Jun. 1994, pp. 45, 46. Johnson, “A three—Layered Solution for Managing the Enterprise,” Data Communications, vol. 24, No. 8, Jun.  Assignee; 3COM Corporation, Santa Clara, Calif.  Continuation-in-part of application No. 08/766,274, Dec. 13, 1996, abandoned. Provisional application No. 60/040,876, Mar. 21, 1997, abandoned- 1995, pp. 41, 42. Larsen, “Mastering Distributed Domains via the Web,” Data Communications, vol. 25, No. 7, May 1996, pp. 36, 38. Lee, “A Distributed Network Management System,” Pro ceedings of the Global Telecommunications Conference, San Francisco, CA, Nov. 28—Dec. 2, 2994, vol. 1, Nov. 1994, Institute of Electronics Engineers, pp. 548—552. Roberts, “RMON Adapters Shed Light on LAN’s,” Data Communications, V01- 25, NO~ 6, May, 1996, PP~ 43, 44~ Schwager, “Remote Network Monitoring MIB,” Annual Review of Communications, National Engineering Consor tium, Chicago, IL, vol. 46, Jan. 1992, pp. 752—754.   Int. c1.7 ............................. .. H04L 9/00- G06F 11/30 US. Cl. ........................ .. 713/153; 713/201; 713/202; Sta11ing$> “Patching the Cracks in SNMP? BYE VOL 21> No 8’ Aug' 1996’ PP' 5556'  Field of Search ................................... .. 709/224, 235,  Appl' NO” 08/882’207  Filed; Jun, 24, 1997 Related US, Application Data  709/224; 709/235; 370/245; 370/252 709/227, 230, 248; 370/252, 241, 245; 713/201, 202, 151, 153, 160, 162  References Cited Primary Examiner—Gilberto Barron, Jr. Attorney) Agent» Or Firm—[email protected] Murabito & Hao LLP  ABSTRACT Distributed remote monitoring (dRMON) of network traffic and performance uses distributed nodes to collect traffic statistics at distributed points in the network. These statistics U_S_ PATENT DOCUMENTS are forwarded to collectors which compile the statistics to create combined views of network performance. A collector 4,817,080 3/1989 Soha ...................................... .. 370/252 may mimic a prior art, nomdistributed, network probe and 572517152 10/1993 Notess 709/224 may interact with network management software as though 5’45O’6O1 709/224 it were a stand alone network probe thereby simplifying a 9/1995 Okudf‘ """ " Desai Ct 8.1. ............................. .. 5,961,596 10/1999 Takubo et al. ........................ .. userzs interaction 709/224 network management protocols including SNMP, RMON, FOREIGN PATENT DOCUMENTS 0 573 248 A1 12/1993 European Pat. Off. . 0 726 664 A2 8/1996 European Pat Off- WO 96/38955 12/1996 WIPO - and RMON2 but is not limited to those environments. The invention has applications in a variety of communication system environments including local area networks, cable television distribution systems, ATM systems, and advanced OTHER PUBLICATIONS Green?eld, “Network Management Filters Down to the Desktop,” Data Communications, vol. 20, No. 13, Sep. 1991, pp. 39, 40, 42. telephony systems. A speci?c embodiment of the invention solves is particularly optimized to work in LAN environ ments with end systems running under Windows-compatible network operating systems. 26 Claims, 6 Drawing Sheets INPUT PASSWORD FOR USE IN COLLECTOR AND AGENT ENCRYPT PASSWORD AT COLLECTOR AND EMBED ENCRYPTED PASSWORD IN COLLECTOR EXECUTABLE FILE SEPARATELY ENCRYPT PASSWORD AT AGENT USING DIFFERENT ALGORITHM FROM COLLECTOR AND EMBED DIFFERENT ENCRVF'TED PASSWORD IN AGENT EXECUTABLE AT RUN'TIME, DECRYPT EMBEDDED PAS$WORD AT AGENT AND AT COLLECTOR AND PRODUCE IDENTICAL ORIGINAL PASSWORD AT BOTH, ORIGINAL PASSWORD ONLY STORED IN RUN-TIME MEMORYv AT COLLECTOR] PLACE PASSWORD IN AUTHENTICATION FIELD OF POLL PACKET AND SUBSEQUENT PACKETS. ENCRYPT PACKETS To BE TRANSMIT'I'ED USING PRIOR ART NETWORK (MD5) ENCRYPTION ALGORITHM AT AGENT, DECRYPT PACKET AND COMPARE PASSWORD IN AUTHENTICATION FIELD OF POLL PACKET TO PASSWORD. DO PASSWORDS MATCH’? RESPOND TO POLL PACKET the distributed System‘ Tihe invention is designed to work in accordance with a variety of standard IGNORE POLL PACKET U.S. Patent Aug. 22, 2000 Sheet 1 0f 6 6,108,782 52a E Agent 62 52b 72a E / , I FEE-2E Agent .0 52c IS / dRMON Collector 529 52f Agent Agent 72d Agent 526 52d Agent Agent 518 El 50b /70e _ Non-Agent 50c IS / dRMON Agent Collector w/proxy 50d Q" Non-Agent 54 Management Console U COLLECTOR 42 73b 4 / dRMON DOMAIN SERVER/ROUTER FIG. 1 COLLECTOR U.S. Patent Aug. 22, 2000 Sheet 2 0f 6 ETHERNET HEADER AMP HEADER ETHERNET AMP ADDR(48) ID DATA 6,108,782 ETHERNET TRAILER (24 BITS) FIG. 2 HIGH LAYER NAME (NUMBER) QILSLCK DATA PROTOCOLS HIGHER LAYER PROTOCOLS APPL'CAT'ON (7) Protocol Manager APP Packets/MlBs PRESENTATION (6) SESSION (5) SESSIONS PACKET TRANSPORT (4) NETWORK (3) LOW DATA LINK (2) PHYSICAL (0,1) STREAM ROUTING DTA DRIVER PACKETS MC PACKETS ADAPTOR BITS (DTA DRIVER) FIG. 3 SNMP, FTP, HTTP Data Encryption/ Compression FULL/HALF DUPLEX TCP, UDP IP ETHERNET ETHERNET U.S. Patent Aug. 22, 2000 6,108,782 Sheet 3 0f 6 dRMON Agent Windows 95 or NT User Mode DTA DLL DTA TDI Driver Kernel Mode NIC Driver FIG. 4 RMON RMON Engine Data 112 Structures 110 114 dRMON Data Structures DLM Mgr. 1L 115 dRMON interface Module SNMP 118 / dRMON Protocol 116 System / FIG. 5 Windows 95 or NT dRMON Collector User Mode SNMP Extensions Kernel Mode MS SNMP Agent DTA TDl Driver NIC Driver FIG.6 DTA DLL U.S. Patent Aug. 22,2000 Sheet 4 0f6 6,108,782 146 SSN61 dMRE MM45 2PCNaM DMLgMr. P ————— / m ts R81dDl OeNMdeO RDaSGLu NdrOm OdDSRNa.“Mmmm ,n o w wNm m mmmw a. M0wmN l CS M w8O / b m m n .Wmo mNlo SNMP Management Stations 80/ 81\ Ddmmmmnm C 80 Work Group Collectors > x of 9 dRMON U.S. Patent Aug. 22,2000 Sheet 5 0f6 6,108,782 INPUT PASSWORD FOR USE IN COLLECTOR AND AGENT I ENCRYPT PASSWORD AT COLLECTOR AND EMBED ENCRYPTED PASSWORD IN COLLECTOR EXECUTABLE FILE I SEPARATELY ENCRYPT PASSWORD AT AGENT USING DIFFERENT ALGORITHM FROM COLLECTOR AND EMBED DIFFERENT ENCRYPTED PASSWORD IN AGENT EXECUTABLE FILE I AT RUN-TIME, DECRYPT EMBEDDED PASSWORD AT AGENT AND AT COLLECTOR AND PRODUCE IDENTICAL ORIGINAL PASSWORD AT BOTH, ORIGINAL PASSWORD ONLY STORED IN RUN-TIME MEMORY. I AT COLLECTOR, PLACE PASSWORD IN AUTHENTICATION FIELD OF POLL PACKET AND SUBSEQUENT PACKETS. ENCRYPT PACKETS TO BE TRANSMITTED USING PRIOR ART NETWORK (MD5) ENCRYPTION ALGORITHM I AT AGENT, DECRYPT PACKET AND COMPARE PASSWORD IN AUTHENTICATION FIELD OF POLL PACKET TO PASSWORD. DO PASSWORDS MATCH? RESPOND TO POLL PACKET IGNORE POLL PACKET FIG. 9 6,108,782 1 2 DISTRIBUTED REMOTE MONITORING computers, Workstations, and printers and additionally may (DRMON) FOR NETWORKS be digital devices such as digital telephones or real-time video displays. Different types of ESs can operate together This application claims priority from provisional patent on the same LAN. In one type of LAN, LAN intermediate systems (IS) 60—63 are referred to as bridges or sWitches or hubs and WAN ISs 64 and 66 are referred to as routers, application 60/040,876, ?led Mar. 21, 1997 noW expired abandoned. This application is a continuation-in-part of Ser. No. 08/766,274LE A1EU ( )757 Pk2-D15 306-3101, ?led Dec. 13, 1996 noW abandoned. MICROFICHE APPENDIX 10 This application has been ?led With a micro?che appendix containing a user manual relating to one speci?c embodi ment of a system incorporating aspects of the invention. A portion of the disclosure of this patent document contains material Which is subject to copyright protection. The copy containing a header having at least a destination address specifying an ultimate destination and generally also having 15 a source address and other transmission information such as transmission priority. Packets are generally formatted according to a particular protocol and contain a protocol identi?er of that protocol. Packets may be encased in other packets. FIG. 2 illustrates a packet. right oWner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Of?ce patent ?le or records, but otherWise reserves all copyright rights What 20 soever. layers, data is vieWed and organiZed differently, different 25 protocols are folloWed, different packets are de?ned and different physical devices and softWare modules handle the data traffic. FIG. 3 illustrates one example of a layered netWork standard having a number of layers, Which We Will refer to herein as: the Physical Layer, the Data Link Layer, 30 the Routing Layer, the Transport Layer, the Session Layer, the Presentation Layer and the Application Layer. These for monitoring and analysis of netWork traf?c using a distributed remote traf?c monitoring (DRMON) technology. Related technology is discussed in co-assigned co-pending US. patent applications Ser. Nos. 08/506,533, Layers Modern communication standards, such as the TCP/IP Suite and the IEEE 802 standards, organiZe the tasks nec essary for data communication into layers. At different BACKGROUND OF THE INVENTION This invention relates to transmission of information betWeen multiple digital devices on a netWork. More particularly, this invention relates to a method and apparatus hoWever many different LAN con?gurations are possible, and the invention is not limited in application to the netWork shoWn in FIG. 1. Packets In a LAN such as 40, data is generally transmitted betWeen ESs as independent packets, With each packet entitled METHOD AND APPARATUS FOR ASYNCHRO NOUS PPP AND SYNCHRONOUS PPP CONVERSION, layers correspond roughly to the layers as de?ned Within the TCP/IP Suite. (The 802 standard and other standards have different organiZational structures for the layers.) ?led Jul. 25, 1995 now US. Pat. No. 5,666,362; and 08/542,157, entitled METHOD AND APPARATUS FOR TRANSPARENT INTERMEDIATE SYSTEM BASED FILTERING ONA LAN OF MULTICAST PACKETS, ?led Oct. 12, 1995 now US. Pat. No. 5,818,838 and incorporated herein by reference to the extent necessary to understand the invention. 35 Generally, When an ES is communicating over a netWork Networking Devices Standards This speci?cation presumes familiarity With the general concepts, protocols, and devices currently used in LAN 40 using a layered protocol, a different softWare module may be running on the ES at each of the different layers in order to handle netWork functions at that layer. Examples of softWare modules existing Within an ES at different layers are shoWn in FIG. 3. Drivers and Adapters Each of the ISs and ESs in FIG. 1 includes one or more netWorking applications and in WAN internetWorking appli cations. These standards are publicly available and dis cussed in more detail in the above referenced and other adapters and a set of drivers. An adaptor generally includes circuitry and connectors for communication over a segment 45 co-assigned patent applications. This speci?cation also presumes some familiarity With the speci?c netWork and operating system components dis cussed brie?y in the folloWing paragraphs, such as the simple netWork management protocol (SNMP) for manage and translates data from the digital form used by the com puter circuitry in the IS or ES into a form that may be transmitted over the segment, Which may be electrical signals, optical signals, radio Waves, etc. A driver is a set of instructions resident on a device that alloWs the device to ment of LAN and WAN netWorks, and the RMON MIBs accomplish various tasks as de?ned by different netWork protocols. Drivers are generally softWare programs stored on de?ned for remote netWork monitoring and management. the ISs or ESs in a manner that alloWs the drivers to be General NetWork Topology modi?ed Without modifying the IS or ES hardWare. NIC Driver FIG. 1 illustrates a local area netWork 50 40 of a type that might be used today in a moderate siZed enterprise as an example of a netWork in Which the present invention may be deployed. LANs are arrangements of various hardWare and softWare elements that operate together to alloW a number of digital devices to exchange data Within the LAN and also 55 may include internet connections to external Wide area 60 The loWest layer adaptor softWare operating in one type of netWork ES is generally referred to as a NIC (NetWork Interface Card) driver. A NIC driver is layer 2 softWare designed to be tightly coupled to and integrated With the adaptor hardWare at the adaptor interface (layer 1) and is netWorks (WANs) such as WANs 42 and 44. Typical modern also designed to provide a standardiZed interface betWeen layer 2 and 3. Ideally, NIC drivers are small and are designed LANs such as 40 are comprised of one to many LAN so that even in an ES With a large amount of installed intermediate systems such as 60—63 that are responsible for data transmission throughout the LAN and a number of end netWork softWare, neW adaptor hardWare can be substituted systems (ESs) such as ESs 50a—d, 51a—c, and 52a—g, that 65 With a neW NIC driver, and all other ES softWare can continue to access the netWork Without modi?cation. represent the end user equipment. The ESs may be familiar NIC drivers communicate through one of several avail end-user data processing equipment such as personal able NIC driver interfaces to higher layer netWork protocols. 6,108,782 4 3 Examples of NIC driver interface speci?cations are NDIS Prior Art RMON Overview (Network Driver Interface Speci?cation developed by Prior art Remote Monitoring (RMON) technology is a set of software and hardware speci?cations designed to facili tate the monitoring and reporting of data traf?c statistics in Microsoft and 3Com) and ODI (Open Data-Link Interface developed by Apple Computer and Novell). Generally, when an ES is booting up and begins building its stack of network protocol software, the NIC driver loads RMON a local area was network originally(LAN) de?ned or by wide thearea IETFnetwork (Internet Engi neering Task Force) in 1991. RMON de?ned an independent ?rst and tends to be more robust than other network software modules because of its limited functions and because it is tightly designed to work with a particular hardware adaptor. Management and Monitoring of Individual ESs in a Net work Environment A network such as that shown in FIG. 1 is generally managed and monitored within an enterprise by a central 10 machines provided the various functions described by the de?ning IETF RFC documents, RFC-1271, RFC-1513 and RFC-1757. According to the original standards, a special application Information Services department (ISD), which is respon sible for handling all the interconnections and devices shown. The same ISD is generally responsible for managing the applications and system components on each of the network probe, which was generally implemented as a separate CPU-based system residing on the monitored net work. Software running on the probe and associated program, sometimes referred to as an RMON Manager, 15 individual ESs in the network. Many prior art systems have been proposed to allow an IS staff person to manage and partially monitor network infra structure remotely over a network. Such systems include IBM’s NetView, HP’s OpenView or Novell’s Network Man controlled the operation of the probe and collected the statistics and data captured by the probe. In order to track network traf?c and perform commands issued to it by the RMON Manager, a prior art probe operated in a promiscu ous mode, where it read every packet transmitted on network segments to which it was connected. The probe performed analyses or stored packets as requested by the RMON Manager. agement System (NMS). However, these systems generally Prior art RMON builds upon the earlier Simple Network rely on a full network protocol stack to be correctly running effectively on the remote ES in order to accomplish any remote ?le management operations. Management Protocol (SNMP) technology while offering 25 four advantages over SNMP agent-based solutions: (1) RMON provides autonomous Network Management/ Monitoring, unlike SNMP which required periodic polling Simple Network Management Protocol (SNMP) A common protocol used for managing network infra structure over the network is the Simple Network Manage ment Protocol (SNMP). SNMP is a layer 7 network and cation when a user wishes to access information kept at the system management protocol that handles network and probe. of ESs. RMON stand-alone probes are constantly on duty and only require communication with a management appli system management functions and can be implemented as a (2) RMON’s alarm capability and user-programmable driver (or SNMP agent) interfacing through UDP or some other layer 4 protocol. Prior art SNMP installations largely event triggers furnish a user with asynchronous noti?cation of network events without polling ESs. This reduces the network bandwidth used and allows across-WAN links without concern for performance costs. were not placed in ESs because SNMP did not handle ES 35 management or monitoring functions and because SNMP (3) RMON automatically tracks network traf?c volume agents are processor and memory intensive. SNMP is designed to provide a simple but powerful cross and errors for each ES MAC address seen on a segment and platform protocol for communicating complex data struc maintains a Host Matrix table of MAC address pairs that have exchanged packets and the traf?c volume and errors associated with those address pairs. tures important to network infrastructure management. However, its power and platform-independent design makes it computationally intensive to implement, and for that (4) RMON permits the collection and maintenance of reason it has limited applications in end system management or monitoring. It is primarily used in network infrastructure management, such as management of network routers and 45 bridges. capture capabilities which allowed a user to collect impor SNMP is designed to support the exchange of Manage tant network packet exchanges and analyZe them at the management console. The new capabilities of RMON were quickly appreciated and RMON probes soon became the preferred choice for remote monitoring. It has become common place for ISs, particularly hubs and switch/bridges to embed RMON probe ment Information Base (MIB) objects through use of two simple verbs, get and set. MIB objects can be control structures, such as a retry counter in an adaptor. Get can get the current value of the MIB and set can change it. While the SNMP protocol is simple, the MIB de?nitions can be dif?cult to implement because MIB ids use complex data structures which create cross-platform complexities. SNMP has to translate these complex MIB de?nitions into ASN.1 which is a cross-platform language. historical network performance metrics thereby facilitating trend analysis and proactive performance monitoring. (5) RMON includes fairly sophisticated packet ?lter and 55 functions. RMON2 Shortly after adoption of RMON, users wanted more which will often be the case when the network connection is management information than the layer 2 statistics RMON provided. In particular, network managers wanted to track higher layer protocols and the sessions based upon those protocols to learn which applications were using which protocols at what expense in available network bandwidth. failing. When working, SNMP provides a protocol interface Therefore, a new version of RMON, RMONZ was devel Even if installed in an ES, an SNMP agent cannot be used to manage or diagnose an ES or update system components where the UDP protocol stack is not working properly, for higher layer prior art management applications. oped to provide more advanced capabilities. RMON2 pro vides network header layer (layer 3) through application SNMP is described in detail in a number of standard reference works. The wide adoption of SNMP throughout the networking industry has made compatibility with SNMP an important aspect of new management and monitoring tools. 65 layer (layer 7) monitoring for a number of commonly used protocols and applications, including the Internet protocol suite (IP and UDP) and Internet applications (FTP, Telnet, TCP and SNMP). 6,108,782 5 6 Limitations of IS-Based (Hub-Based/SWitch-Based RMON (telephone, television) on any such communication system A traditional stand-alone RMON probe, connected to a sWitch like any other host device, only sees network traf?c ?owing on the segments to Which it is connected, greatly limiting its usefulness in modern, more complicated netWork or to encompass distributed points in the netWork interme diate of an end systems. It is also intended that the Word “packet” as used in the speci?cation and claims be read to cover any unit of transmitted data, Whether an ethernet topologies. One solution is to place the RMON probe Within the sWitch itself and have it monitor all ports simultaneously. unless the context requires otherWise. packet, a cell, or any other data unit transmitted on a netWork HoWever, this requires considerable processing capability in order to handle the large bandWidth made possible by modern sWitching architectures. SUMMARY OF THE INVENTION 10 In a conventional 10 Mb Ethernet or 4/16 Mb Token Ring environment, a stand-alone RMON probe on a single net Work segment could usually be implemented on a 486-class processor. HoWever, Where multiple netWork interfaces must be monitored or Where netWork bandWidths are higher, (such as With 100Base-T LANs or sWitching hubs/ATM), it is considerably more costly to build a probe With suf?cient processing poWer to capture all, or even most, of the netWork components, are placed Within each (or a subset) of the ESs 15 RMON functional groups but only capture and analyZe embodiments captures packets that the ES communicates that RMON products claiming to keep up With higher bandWidth netWork traf?c generally cannot, in fact, keep up With an ES that does not have an dRMON agents installed; as a result, the processing requirements of the dRMON agents are kept Well Within the range of the ES (or host) CPU’s capabilities and generally do not result in a notice 25 some of the previously referenced applications because it can act as a proxy for, or mimic, the behavior of a prior art dRMON probe), existing someWhere on the WAN/LAN. The collector combines received agent data thereby creating called a copy port) Where all packets are repeated to the 35 at the collector the vieW that a prior-art stand-alone RMON probe Would have if all the ESs Were on the same LAN segment With the probe. According to the invention, the collector may be a stand-alone device connected to the LAN, such as 61b or 65a, or may be implemented Within a sWitch In general, What is needed is an ef?cient and Workable mechanism for the distributed collection of performance statistics in a communication system. Within the speci?c environment just described, What is needed is an RMON in the LAN such as 62 or Within a server, such as 64. According to one embodiment of the invention, a dRMON collector can mimic the SNMP responses of a prior technology Whereby RMON functionality can be imple mented in a LAN/WAN Without unduly harming netWork performance and not requiring additional expensive netWork hardWare to support. Ideally, this technology Would be compatible With standard RMON and RMON2 technology able loss of performance. According to the invention, on a periodic basis, initiated by a polling packet from the collector in one embodiment, the dRMON agents forWard their statistics and/or captured packets to a dRMON collector (referred to as a proxy in sWitch vendor has provided a monitor port (sometimes external RMON probe. HoWever, this approach decreases data traffic performance in the sWitch, and does nothing to reduce the processing overhead required of the probe. such as 50a—c, 51a—c, and 521-g, connected to the LAN or Within server machines. These agents implement prior art packets that their native ES sends or receives, or in some packets being exchanged. Independent laboratory tests shoW With all data How during peak netWork rates. The situation Worsens considerably When attempting to do RMON2 analy sis of netWork packets in high bandWidth environments. Processing poWer required can be easily ?ve times greater than needed to simply capture packets, and data storage requirements can easily increase ten fold. Use of ?ltering sWitches and hubs (discussed in the above referenced patent applications) in netWorks further limits the usefulness of probes because, unlike repeaters, not all the packets appear at every output port of the sWitch. This makes the use of external stand-alone probes infeasible unless the The present invention is a method and apparatus for distributed remote netWork monitor (dRMON) in a LAN. According to an embodiment of the invention, dRMON agents, Which are softWare or softWare plus hardWare 45 so it could operate effectively With existing netWork man agement softWare. For purposes of clarity, the present discussion refers to netWork devices and concepts in terms of speci?c examples. HoWever, the method and apparatus of the present invention art non-distributed RMON probe so that existing netWork management or monitoring softWare can interact With the collector as though the collector Were a prior art probe. Therefore prior art netWork management softWare need not be aWare of the existence of the dRMON agents. According to a further embodiment, multicast domains are handled specially. In a default mode, ESs in the same multicast domain are treated by a collector as though they are on one LAN segment. This approach alloWs other vendor’s RMON netWork management applications to inter may operate With a Wide variety of types of netWork devices act With the collector as though it Were a prior art probe; including netWorks and communication systems dramati cally different from the speci?c examples illustrated in FIG. hoWever, When used With enhanced dRMON Managers, a user is provided the ability to combine ports and hosts in order to create Virtual LAN (VLAN) de?nitions Which Would cause the monitoring function to behave as though all 1 and described beloW. It should be understood that While the invention is described in terms of a computer netWork, the invention has applications in a variety of communication systems, such as advanced cable television systems, advanced telephone netWorks, ATM, or any other commu 55 selected hosts Were on the same LAN segment being served by the same RMON probe. A dRMON collector in this embodiment could create and maintain several such vieWs nication system that Would bene?t from distributed perfor mance monitoring and centraliZed collection and compila With each appearing as one interface to conventional RMON tion. It is therefore not intended that invention be limited, except as indicated by the appended claims. It is intended According to a further embodiment, agent proxies are provided to be placed in IS systems such as bridges to handle the dRMON agent functions for ESs that do not have agents. Management applications. that the Word “netWor ” as used in the speci?cation and claims be read to cover any communication system unless 65 These proxies can be used in environments Where some ESs the context requires otherWise and likeWise “end system” and “node” be read to encompass any suitable end system are running operating systems for Which dRMON agents are not yet available. According to the invention, using a proxy 6,108,782 7 8 agent in an IS for just some of the ESs can allow that IS to FIG. 5 is a more detailed diagram of a particular embodi ment of an agent according to the invention and other collect just those statistics needed for agent-less ESs and therefore does not overburden the IS processing capabilities. There are several key advantages to various embodiments of the invention When compared to other solutions. among these advantages are scalability, affordability, true end-to components upon Which it depends. FIG. 6 is a block diagram of an embodiment of a dRMON Collector according to the invention. FIG. 7 is a more detailed internal vieW a of an embodi end response time monitoring, redundancy, visibility into client node, distributed architecture, and Web support. ment of a dRMON Collector according to the invention. FIG. 8 is a diagram illustrating hierarchical collectors Because each agent is analyZing only its oWn directed according to an embodiment of the invention. FIG. 9 is a How chart illustrating a security mechanism according to an embodiment of the invention. FIG. 10 shoWs a particular embodiment of a simpli?ed ?le that may be used to communicate netWork statistics data to traf?c, or possibly its oWn traf?c and the traf?c of a limited number of other ESs, dRMON can handle extremely high bandWidth environments With relative ease. Compared to stand-alone probes, dRMON is more affordable as a remote monitoring tool, particularly in sWitched environments. Very inexpensive PC technology can be used to host the Collector softWare resulting in loW equipment costs. 15 a remote terminal. FIG. 11 is a diagram of a computer system as an example of a system used to deploy the invention. RMON2, for all its poWer, still does not afford the netWork manager one of the most asked for features, that being continual response time monitoring. RMON2 appli DESCRIPTION OF SPECIFIC EMBODIMENTS cations can only do this if packet capture is used to forWard the protocol streams to the management station, at a price in netWork utiliZation and performance. dRMON Agents rou FIG. 1 is a block diagram illustrating the deployment of the invention in an example netWork according to a speci?c embodiment of the invention. The invention includes tWo tinely perform this analysis and forWard the results (not the types of primary components, the agents that reside in ESs entire packets) to the Collector. The fact that dRMON agents in the ESs themselves are collecting the data additionally creates a more precise vieW of the LAN since any LAN ’s characteristics vary based upon Where in the Wire a node is connected; furthermore, because of their cost, probes are often located close to the backbone 25 dRMON Agent In one embodiment, the dRMON agent is implemented in the C programming language. The agent executable code is Where feWer probes can see more of the traf?c. This approach prevents the netWork manager from spotting infra structure problems and delays occurring betWeen the probe’s location and the desktop. Only dRMON can perform true, accurate, end-to-end response time analysis. Since data collection is done by the managed nodes and and the collector or collectors that collect and compile the netWork statistics and interacts With netWork management applications (such as an application running on console 54) to provide a management/monitoring picture to the netWork. launched each time an ES is started or rebooted and the agent may be tightly bound to ES adaptor driver softWare. Because the dRMON agent has no visible ES user interface, the ES user is unaWare of the agent’s presence, and can do 35 RMON Collectors can substitute for each other, there is no single point-of-failure and dRMON therefore inherently nothing With regards to recon?guring the ES that Would inadvertently disable the agent. FIG. 4 shoWs one particular embodiment of an agent and provides monitoring redundancy. In the case of monolithic probes or management add-in cards, unless multiple probes other components upon Which it depends. An NDIS Desk Top Agent type module (DTA) is used to bind to the netWork adapter driver, thus establishing a source of directed packets are deployed on each LAN segment, a probe’s failure can be disastrous When attempting remote monitoring. to analyZe as Well as a means to communicate With the Because the dRMON agent softWare of the invention resides in ESs, it can capitaliZe upon native operating system interface mechanisms (for example OS APIs such as Microsoft’s WIN32) to gather information about the ES that could never be ascertained from the Wire via packet capture dRMON collector via the netWork. Multiple NIC bindings may be supported by the agent and may alloW the agent to monitoring traf?c on different segments having different 45 Among the important functions that can be performed by and analysis. Examples of the kinds of information avail able: (1) NetWork protocol stack con?gurations and NIC agents according to various embodiments of the invention are: (1) receiving and responding to messages from the collector and con?guring its operation to conform to col con?gurations including problematic situations; (2) Appli cation information such as What protocols an application is lector instructions; (2) performing RMON analysis and compiling statistics regarding netWork traf?c for forWarding to the collector; (3) performing packet capture at the agent for forWarding packet streams to the collector, and (4) bound to, the application’s manufacturer, version, ?le date and time, DLLs used and their versions, etc.; (3) ES system information such as memory, CPU, disk space, current resource utiliZations, etc.; and (4) System performance met rics. The invention Will be further understood upon revieW of providing a mechanism for receiving and executing doWn 55 the folloWing detailed description in conjunction With the BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram of a local area netWork of one type in tures and tables are built and maintained Within the section Which the invention may be effectively employed. FIG. 2 is a diagram of a packet. FIG. 3 is a diagram shoWing a layered netWork protocol. FIG. 4 is a diagram of a particular embodiment of an agent Which it depends. loadable modules. FIG. 5 provides an exploded vieW of the dRMON Agent’s internal components. Central to the agent’s functionality is RMON Engine 110. This module takes the packet stream received from the netWork via the DTA and subjects it to RMON analyses as con?gured via the collector. Data struc draWings. according to the invention and other components upon layer 1 protocols. 65 labeled RMON Data Structures 112 in order to accomplish and store the results of this RMON analysis. The agent compares packets to ?lters in effect that have been set by the collector and, upon a match, an event is generated for analysis and/or the packet is retained and added to a capture channel. The invention may include support for DoWn 6,108,782 10 Loadable-Modules (DLMs) through DLM manager 111. protocol traf?c With its Agents. The DTA is also used as a packet interface to alloW the Collector to monitor its oWn This allows the user to download executables such as diagnostics from the RMON management console that can directed traf?c as Well as the broadcast and multicast traf?c perform system analysis, network analysis or both. dRMON ?oWing Within its sphere of management. To prevent dupli data structures 114 are used to store information necessary cation of statistics, only the Collector maintains RMON for agent-to-collector functioning according to the information on broadcast and multicast traffic. Since, in one embodiment, the Collector must communi invention, such as, in one embodiment, a list at the agent of the layer 2 (MAC) addresses of all other ES that include cate With RMON Manager applications using SNMP, a full functioning dRMON agents. set of SNMP interfaces and services 142 exists in the Collector Which is not found in the dRMON Agent. In the The dRMON Interface Module 115 is intended to isolate the Agent core from ES platform and netWork protocol dependencies to maximiZe the portability of the agent executable code and therefore to facilitate the porting of the WindoWs95(TM) and WindoWsNT(TM) environments, Microsoft(TM) offers an extensible SNMP agent. This agent provides the UDP/IP protocol stack, PDU parser and basic agent softWare to other operating system (OS) platforms. BeloW dRMON Interface Module 115 are the loWer layer components used to communicate With the dRMON 15 MIB-II support, but alloWs a user-provided extension to register MIB objects that are to be maintained by the user-provided extension. When the extensible agent receives collector, the DTA and the operating system. dRMON protocol box 116 is Where the dRMON protocol and DTA registered objects, it passes the request to a user-provided interfaces are realiZed. While dRMON protocol is used for callback function for processing. In one embodiment, a communication With the Collector, many requests coming collector according to the invention registers the full RMON MIB With the Extensible Agent. In embedded applications an SNMP PDU referencing one or more of the user from the Collector, such as requests to set ?ltering or packet capture parameters, are essentially SNMP protocol data units (i.e. PDUs or packets) encapsulated in dRMON pro tocol; hence, the presence of the SNMP interface and decoder module 118, Which decodes the necessary subset of (e.g., sWitches), the Microsoft Extensible Agent may be replaced With customiZed SNMP services. FIG. 7 gives a more detailed internal vieW of the Collector 25 SNMP. In an alternate embodiment, the invention could use a different (possibly routable) protocol instead of dRMON protocol for Agent-to-Collector exchanges. The dRMON Interface Module provides for this by isolating the protocol dRMON Mapper 144 performs the task of mapping betWeen details from the Agent’s core. dRMON Collector The dRMON Collector receives RMON analysis and capture data from the agents and sorts, collates, and aggre gates that information into a cohesive database that recreates the vieW a prior art RMON probe Would have if the ESs Were all on the same LAN segment With the prior art probe. The executable. Again, the architecture is very similar to that of the dRMON Agent and may use a third-party RMON2 engine as RMON2 engine 140. The SNMP Services com ponent 142 provides the RMON extensions that are regis tered With the Microsoft Extensible SNMP Agent. The 35 RMON MIB objects and their internal representations con tained Within the module labeled RMON Data Structures 146. The Integrator 148 merges RMON statistics, tables and capture streams coming from the remote dRMON agents With the equivalent output from the Collector’s analysis of its oWn directed traf?c combined With the broadcast and multicast traffic present at its interface. The ?nal result is an collector can then makes this information available to man agement applications, either using SNMP and the MIB-II and RMON MIBs or optionally, to WEB broWsers via HTTP or other Web interface language. Different instances of the Collector, like the Agent, can be developed to support a integrated vieW of all of the monitored traf?c just like one Would get from a conventional RMON probe. The other loWer-layer components such as the dRMON Interface Module 150 provide the same platform isolation number of different operating systems. function that they do for the dRMON Agent thus permitting Any SNMP operation on the netWork Which Would affect the con?guration or operation of a stand-alone RMON probe is captured by the collector and forWarded, as appropriate, to the agents so that the agents can modify their behavior the other modules to be implemented in a Way Which 45 maximiZes their portability. Protocol for Communications BetWeen Adaptor and Collec tor accordingly. An example Would be an SNMP packet setting According to the invention, a protocol is de?ned for ?lter de?nitions for Which packets ?oWing on the netWork are captured for later analysis. Such a packet Would be received by the collector and then passed along to dRMON agents Which Would each individually compare received packets to the ?lter de?nitions. communications betWeen a collector and its agents. The speci?c details of the protocol are not necessary for an understanding of the invention, and the protocol may be a prior art netWork management protocol, such as SNMP or a subset of standards-based SNMP. HoWever, the invention is also able to Work With a simple While the invention may be most easily described as a netWork having a single collector, because the actual data 55 gathering and monitoring is being performed at the managed and more ef?cient protocol for speci?cally communicating certain kinds of netWork management information and this ESs, it is possible to have another collector on the LAN/ represents a preferred embodiment. A preferred protocol WAN assume the data collection duties of a defective or tors on a LAN, in Which case in this embodiment an Would encompass both an application level protocol that handles MIB objects and a netWorking protocol that is poWerful for the particular purposes for Which it Was identi?er is used so that an agent communicates With only designed. A preferred protocol does not require and is not one collector. In one embodiment, this identi?er also acts as susceptible to con?guration by an ES user, so that it is not as easily inadvertently disabled by a user as many other off-line collector. It is also possible to have multiple collec a security passWord as described beloW. netWork protocols are. A preferred protocol Would bind FIG. 6 is a block diagram of an embodiment of a dRMON Collector according to the invention. Like the Agent, the Collector loads automatically When the system starts and depends upon the same DTA services to exchange dRMON 65 more directly to a NIC driver so that the protocol Will load and be functional even if other netWork protocol stacks do not load or are not operating properly. A preferred protocol 6,108,782 11 12 Will generally require no acknowledgement by default, but discovery request is broadcast. In one embodiment, this delay is set by each agent based on a random number. In other embodiments, as described beloW, response delay is Will include the ability to establish acknowledgements for reliability and also to include encryption features for secu rity. based on some characteristic attached to each speci?c ES, such as MAC address. Discovery requests are repeated periodically to detect nodes Which have been added or Apreferred protocol may be restricted to communication betWeen intermediate system collectors and end system poWered-up since the last discovery operation. Time Synchronization and Polling To facilitate proper time-based ordering of captured pack agents, an area Where users do not otherWise need to interface. The collector, in one embodiment, is designed to interface With other netWork management softWare through a standards based protocol, like SNMP, to facilitate interop erability With netWork management softWare. A preferred protocol Will result in loWer netWork traf?c, 10 be very reliable and require a small installation on the agent maintains a clock, derived from its system clock. To end-systems. A preferred protocol Will be designed With an aWareness of the reliability of modern netWork infrastruc be meaningful, the clocks in each Agent must be kept fairly 15 tures realiZing that many prior art protocols are designed they are transmitted. In modern netWorks, in fact, packets rarely get dropped once they are sent by the transmitter. A sends out a time synchroniZation message periodically. These messages may also be used to trigger the return of statistics from the Agents. As elseWhere described herein, each Agent sets a random delay interval before sending its data to prevent ?ooding the collector. preferred protocol, therefore, eliminates much of the acknoWledgement and redundant traffic generated by other netWork protocols that unnecessary for reliable netWork operation. 25 In a speci?c embodiment, agents and Collectors keep time in 100-nanosecond increments, each individual agent and collector ultimately deriving its count from the CPU clock of its oWn host. The Collector includes in each poll, sent out every 5 seconds, its current uptime counter Which is the number of 100-nanosecond increments that have occurred since the collector Was started. Agents compare this value With their oWn count, computed from their oWn system clock, and compute any corrections that need to be made to account for variations in system hardWare at each node. agents communicate over the netWork as the dRMON pro tocol. Unless the context otherWise requires, the dRMON protocol should be understood to represent any possible protocol betWeen collectors and agents for the eXchange of management/monitoring information, generally in the form of MIBs, including prior art SNMP-type protocols or includ ing a preferred specialized protocol as just described. Collector and Agent Functions From the perspective of the user, the primary functions of the agents and the collector are to collectively implement the close to those of its peers and their Collector, although precise alignment is generally not possible and is not required by the invention. In order to keep agent time-stamps aligned, the Collector With the assumption that netWork traf?c Will be very unre liable and that packets Will often get dropped or lost after For the purposes of this description of the invention, We Will refer to the protocol by Which dRMON collectors and ets at the Collector and to ensure that statistics are placed into the proper time period buckets, statistics and packets coming from the Agents to the collector are time-stamped by the agents. In order to accomplish this time-stamp, each Agents use their oWn corrected counters to provide a relative 35 time stamp on the statistics and captured packets that they return. In a speci?c embodiment, the agent and collector monitoring, management, and packet capture capabilities counters are each roll-over counters. de?ned from RMON2, SNMP, and related netWorking stan dards With enhancements resulting from the distributed In one embodiment, average latencies in the path betWeen the agent and the collector are ignored, because in most real-World local area netWorks, the transmission delay Will be effectively Zero. Other embodiments are possible Where nature of the invention as herein described. As these primary functions are described in publicly available standards and documents and are Well-knoW to practitioners in the art, the agents compute average latencies and adjust their time stamps accordingly. details of the netWork statistics gathering, packet capture, or standards-based con?guration of those function are not described here. What folloWs is a description of the func tions according to the invention that alloWs the invention to During packet capture, the collector time-sorts captured 45 ordered correctly in the capture channels. The timestamps added by the agents Will normally be sufficient to do this, but perform netWork monitoring, in a distributed fashion. Some collector functions Will noW be described. In addi tion to performing RMON2 analysis on its oWn directed traf?c as Well as all multicast and broadcast traffic the at times, because of corrections made at the agents, some captured packets may get returned With nearly identical time-stamps. In that case, the collector uses some protocol Collector performs several other functions pertain to the management or con?guration of its remote agents. dRMON embodiments may be designed to interoperate With a variety of RMON Management applications and all major SNMP Management Platforms (e.g., HP OpenVieW) that support packets returned to it to ensure that protocol eXchanges are interpretation (such as sequence numbers, or request/ response indications) to correctly order the captured packets. Agent Management by Collector 55 Agent Management can be roughly divided into tWo areas: agent con?guration and RMON con?guration. the original RMON MIBs. Doing so requires only that the collector be programmed to communicate With the particular Agent con?guration refers to such issues as hoW much memory/storage for the agent to reserve for RMON data management application and that ?ltering functions required by the management application be translatable by the col space, version management, etc. These actions are not lector to directives to the agent. dRMON protocol management frames. RMON con?guration consists of ?lter settings, historical de?ned in prior art RMON and are accomplished using Agent Discovery by Collector The collector is responsible for automatically discovering all of the dRMON Agents Within its management sphere. According to one speci?c embodiment, a special multicast discovery frame is used to solicit identifying responses from the agents. Each agent sets a different response delay so as not to ?ood the Collector With discovery responses When a sampling intervals and other RMON MIB-de?ned user settable options as Well as the neWly accepted Aspen MIB for standards-based probe con?guration. dRMON protocol 65 frames are used to carry these exchanges but Within them are SNMP-formatted PDUs carrying the actual management information. 6,108,782 13 14 Optimization of Network Traf?c by Agents and Collectors is used by the Collector, as needed, to supplement the use of the timestamps in recreating correct time order of the According to the invention, network traf?c betWeen agents and Collector is designed to be “?nite”, i.e., in as captured packets. many cases as possible, agents and collectors communicate In one embodiment, captured packets Within a poll using a minimum number of packets. The folloWing steps are taken by the invention to help optimiZe and minimiZe interval are grouped and sent to the Collector on the sub netWork traf?c betWeen the collector and the agent: resources at the agent for more packet-captures ahead. sequent multicast request. This frees up memory and system 1. For discovery, reporting of statistics, and time synchronization, the collector generates a multicast poll to Which each of the agents replies. If a multicast poll is dropped at any agent, no retransmission or acknoWledge ment is attempted. This is possible because according to the In one embodiment, the invention does not protect against loss of captured packets once those packets are transmitted 10 at a node, the captured packets it contains Will be lost. HoWever, other elements of the invention as described herein, reduce the dangers that a collector Will not receive a packet once it has been transmitted by a node. In an invention, traf?c information reported by the agents to the collector is in the form of cumulative counters; if a report packet from an agent is dropped or is missed by the from an ES. If, for some reason, a packet cannot be received 15 collector, a subsequent report packet from the agent Will correct the statistics at the Collector. alternative embodiment, an acknoWledgement based proto col connection is established When captured packets are to be transmitted. Coverage of End-Systems Without dRMON Agents and 2. Conversation traf?c information sent by the agents is time-?ltered, i.e., only the conversation entries that Were updated since the last retrieval by this collector are sent by the agent. 3. Traffic information sent by the agents to the Collector gather statistics or capture packets pertaining to other ES in the response is complete Within one packet; no response depends on the availability or arrival of a second packet from the agent, so responses can be processed immediately. Which do not have active dRMON agents. In this Way, the invention may be effectively employed in a netWork even Where all ES are not equipped With dRMON agents. In this Duplicate Data Filtering According to one embodiment of the invention, provi sions are made for ESs With dRMON agents installed to 25 Even if certain response packets get lost, impact to overall embodiment, the collector and agents Work together to eliminate duplicate statistics information reported to the accuracy of collector statistics is minimal. 4. Agents generate a statistics response packet only in collector by various agents and to reduce unnecessary net Work traf?c overhead. To avoid both of these problems, according to this embodiment, the collector maintains a list response to a request by a collector. In general, there is no other traffic generated by agents unless speci?cally requested by the collector in a multicast packet. of identi?ers of ESs With active dRMON agents. In one Distribution of Packet-Capture Among dRMON Agents embodiment, this list consists of the MAC (layer 2) According to the invention, the agent and collector can also perform capture of speci?c packet streams, as de?ned by RMON for stand alone RMON probes. To accomplish packet-capture, an RMON-management application sets up addresses of the ESs With agents. This list is communicated to every dRMON agent controlled by the collector piece by-piece, With a certain number (N) of ES indications noti?ed to all agents in each multicast request. Agents 35 the proper ?lters, channel and buffer control parameters at capture and use this information to reduce unnecessary traffic as described herein. The information may be con the Collector, as described in standard RMON MIBs, and as Would be done in a standard RMON probe. All neW ?lter tained Within the agent ES in any type of table structure and de?nitions, channel de?nitions, and buffer control de?ni in one embodiment the information is stored in a binary tree tions are then forWarded by the collector to all dRMON Agents using multicast packets, as soon as these de?nitions are set up at the Collector. In addition, the collector may received ES addresses to determine Whether or not this agent communicate existing de?nitions periodically to all dRMON agents. Based on these de?nitions, dRMON agents capture pack table in order to facilitate quick look-up at the agent of 45 Will capture that received traffic. Agents and the collector folloW certain rules to reduce netWork traf?c overhead. In general, agents report statistics captured packets only in non-multicast conversations in regarding only conversations that are (1) directed (i.e. not multicast), and are (2) to them (i.e. received (RX) traffic). For transmitted traf?c, the agent reports statistics for directed Which it is an active member. If the conversation is With a traffic only When the receiving ES does not have an active non-agent ES, then the agent node is responsible for the capture. If the conversation is With another dRMON agent, dRMON agent according to the reporting agent’s list. Other rules are possible that eliminate duplicate reporting. ets and forWard them to the collector. Each dRMON agent then in one embodiment to maintain the time order of In cases Where for some reason an agent incorrectly captured packets (i.e., the response is after the request, etc.), reports transmitted traffic to another active agent, the col lector can eliminate duplicate reports by giving higher only one of the tWo agents in a conversation captures the packets and is responsible for sending these packets to the In general, if only one side of a conversation has an active priority to reports from the agent at Which the traffic Was received. Therefore, if agents A and B both report traffic betWeen them, a collector Will use part of the traf?c information from A in Which traf?c is directed to A and part of the traf?c information from B in Which traf?c is directed to B. Another eXample, in Which A is an agent ES and Z is not, conver agent, that side captures packets for both sides. sation betWeen them Will be reported by A only, and there is 55 Collector. In one embodiment, if both sides of a conversa tion contain an active agent, a simple comparison of MAC Addresses is made and a MAC Address Which is lexico graphically bigger becomes responsible for capture. Other rules for determining priority for packet capture are possible. no duplication to be avoided. In some embodiments, in some situations, both sides of a conversation Will be reporting captured packets. Where necessary, the periodic synchroniZing timebase messages from the Collector are used to keep the dRMON Agent’s According to speci?c embodiments of the invention, a number of other strategies may be used to prevent transmit ting duplicate data to the collector or, When duplicate data is packet timestamps in close alignment and protocol analysis transmitted, to prevent that duplicated data from being 65 6,108,782 15 16 counted twice at the collector. These strategies can vary based on Whether the data is captured packet data streams forWarded to the collector or is RMON statistics only sent from the agent to the collector. the interoperability design issues to the module (agent or collector) necessary for interface With that system. Speci?c Adapter/NetWork Operating System (NOS) Support by Agent The ?rst release of one speci?c embodiment of the invention includes support for NDIS 3.X Which encom Furthermore, to prevent duplication of multicast and broadcast statistics, in one embodiment only the Collector itself tracks multicast and broadcast packets and ES agent passes WindoWs for Workgroups 3.11, WindoWs 95 and tracking is disabled for those packets. Agents do not report WindoWs NT 3.51 or later. Novell’s Client 32 Will be supported in these same environments via the NDIS 3 any traffic statistics based on broadcasts. Currently, multi cast traf?c is also handled by Collector only. In some alternative embodiments, it may be desirable to have agents Wrapper Which Will still be present. Any vendor’s NIC Which offers an NDIS 3.X compliant driver can and Will be participate in reporting of multicast traffic. Preventing Flooding of Collector According to the invention, the collector sends out a multicast request to all its agents every polling interval. If all the agents respond to this request immediately, or With a similar delay, there is a chance of ?ooding the Collector With supported, although NIC drivers designed for use With the invention may be enhanced to provide additional features. All Microsoft-de?ned Physical Media Management OIDs 15 more packets than it can receive and process. The collector in turn may have to drop some packets depending on the buffer-resources available. To avoid that, each agent uses a A special Transmit Callback from the dRMON Agent is supported in drivers designed for use With the invention. This transmit callback alloWs outbound traf?c from the host to be monitored by dRMON Without the performance pen delay algorithm, Which calculates amount of time to Wait before sending the response. Ideally, this delay algorithm is such to spread responses from the agents as evenly as possible over a poll response period so that the collector can more easily handle all response packets. In one embodiment, an agent derives a delay value from its unique MAC address of the ES to distribute response packets across the desired alty resulting from putting the adapter in promiscuous mode, 25 systems there is no Way for a higher layer protocol (such as the dRMON agent) to signal to the driver that it Wants to see copies of data that is being transmitted on the netWork. random number generator, seeded With the unique MAC According to the invention, the dRMON agent performs a set operation against the NIC driver using the transmit callback OID, indicating a 32-bit pointer to the dRMON agent’s call-back routine. If that operation succeeds, then the 35 agent responses, including deterministic algorithms based on the number of agents responding to a given collector. Aging out of Agents and Collector Agents age-out collectors When the agent no longer receives any multicast requests for a prolonged period. 45 order to check for the presence of a neW or reaWakened collector on the netWork. The collector also times-out an agent if it does not receive a response to a series of multicast polls for a prolonged This alternative aspect of the invention is necessary because in Microsoft’s original NDIS architecture, an adap tor NIC driver communicating through the NDIS Wrapper does not have the ability to pass transmitted packets back up to a different higher layer protocol than the protocol that originated the packets. Therefore, in a prior art NDIS NIC, the agent cannot get access to packets transmitted by other higher layer protocol. period. In addition to freeing up resources in the collector that are no longer needed, this information (i.e., that this particular ES is no longer a dRMON agent) is communi cated to other agents. Other agents can then start reporting for this neW non-agent, as explained elseWhere in the application. dRMON agent knoWs that the NIC driver includes code to support the transmit callback. The agent then can instruct the NIC driver, using set operations, to set NIC driver ?lters to monitor directed transmit traf?c. If the callback set operation fails, then the agent sets the adaptor ?lters to promiscuous mode, in Which case the adaptor reads all packets that appear on the Wire, including packets it transmits, and those packets are available to higher layer protocols. When an agent ages out all collectors, the agent can free up the ES resources no longer needed, and also it no longer needs to process every single packet because there is no one to Whom it can send packet statistics. Only the dRMON protocol packets need to be processed by dormant agents in as is currently required in many prior art drivers in order to see transmit traf?c. In some current netWork operating response time. In another embodiment, an agent uses a address of the ES, to distribute response packets across the desired response time. In other embodiments, agents seed a random number generator With tWo numbers, one based on a changing value such as a system clock. This redistributes responses from ESs during each response time. Other response distribution algorithms are possible that distribute (object ID.) will be implemented including those catego riZed as optional. This alloWs dRMON agents to detect all media-based error events When running on adapters and drivers designed for use With dRMON. Transmit Callback The NDIS Wrapper does, hoWever, give a driver the ability to hold a packet in buffer memory until the driver has determined from the adaptor card that the packet has been copied to the card and sent. According to this aspect of the 55 According to one embodiment, there is very little memory invention, a driver takes advantage of this mechanism to communicate directly With an dRMON agent DTA TDI that a transmitted packet in the buffer and can Wait until the TDI a collector packet, at Which time the RMON engine is has read and analyZed the packet before signalling to the NDIS Wrapper that processing on the packet is complete. initialiZed and a number of buffers are allocated. Security requirement (less than 10K bytes) for the agent until it sees Compatibility and Interoperability Communication betWeen a dRMON agent and Collector According to the present invention, collectors and agents may be designed to operate effectively With a number of different netWork interface cards (NICs) and NOS architec tures and a number of different management applications. The separability of the agent and collectors alloW the management system according to the invention to be adapted to different operating environments While localiZing is secure. Before either an agent or a Collector is installed, the user sets a passWord that one collector and all agents With Which it is communicating use to encrypt all messages betWeen them. The passWord is separately encrypted by each 65 agent and by the collector and an embedded key is stored in each image (executable) ?le of the dRMON Agents and the Collector. According to the invention, the agent and the 6,108,782 17 18 collector each use a slightly different algorithm that produce The present invention in one embodiment reduces this traffic by having a collector continuously update one or a group of simple ?les at the collector that contained data different embedded keys from the password, though the tWo algorithms are guaranteed to alWays be able to reproduce the representing the compiled statistics of netWork operation. same passWord at When they are run. This mechanism is employed so that a “hacker” can not simply do a comparison of a dRMON collector and agent executable ?les in order to derive the embedded key. The invention protects against a hacker from simply dif?ng the eXecutable ?les to locate the passWord and then inserting that in a “rogue” Collector executable. These ?le may be stored as simple teXt ?le. A management station or a display terminal enabled to receive and display this data can then make one request for a compiled ?le then and use the data in the ?le to display a representation of netWork operations. A dRMON collector, according to an 10 In some types of netWorks, those con?gured to be one done is prior art interfaces. One application for this embodi large LAN, several collectors may be deployed to handle all the agents in the LAN and each collector Will collect ment Would be to make the data available over an internet type netWork, and displayable by a Web broWser. statistics from one group of ESs on the LAN. Communica FIG. 10 shoWs a representation of an eXample of one tion betWeen the ES and its collector is controlled by the shared passWord because agents Will not correctly be able to decode and Will simply ignore poll packets that do not use the passWord set for themselves and their collector. In one embodiment, the dRMON agents have tWo passWords, one a dRMON-and-auto-update passWord, the other an auto-update-only passWord (also referred to as a back door key). Both of these are stored Within the dRMON agent in an intermediate, encrypted form. At run time, the dRMON agent and the Collector, using 15 slightly different algorithms, calculate a ?nal passWord key 25 simpli?ed data ?le that may be used to report statistics according to the invention. The ?rst line, “ipcount”, identi ?es Whether the data has changed. “Pktdist,” “pktrate,” “stats,” are keyWords that preceed data lines for a particular class of data. In this example, data in quotes is treated as labels and ?oating point numbers are treated as values. Hierarchical Collectors Multiple alternative deployments of dRMON collectors from their stored intermediate passWord. This derived value Will be the same at both ends (both the collector and the agent) and Will be kept in run-time memory only and never stored anyWhere the user might hack. This Collector’s calculated key is carried in the Authentication ?eld (also 16 are possible according to the invention, With different embodiments including different sets of the features described herein. In addition to distributing the data collection process, the data archiving and retrieval process may also be distributed. Today’s management systems traditionally have focused on a centraliZed management console model Where all the data ultimately lives at one management station such as 54 after having been retrieved from its remote sources. The obvious bytes long) of the dRMON protocol Common Header. Once and signi?cant disadvantage to this is that the information is unavailable to a netWork manager Who is physically located elseWhere. the key is placed on the netWork, some type of netWork encrypting, such as MDS, is used to protect the security of the packets on the netWork. If the Collector’s ?nal calculated key does not match embodiment of the invention, may also include an SNMP interface alloWing it to report individual counter values as is 35 either of the dRMON agents’ keys (normal or backdoor keys), the dRMON agents Will reject its request. If this key matches the back-door-key, then auto-update Will be alloWed. If this key matches With dRMON agent’s key, then Most larger netWorks already have various information sources already deployed at some locations such as RMON probes, embedded RMON implementations (often partial of a security feature according to the invention. group support) or embedded SNMP Agents. It is advanta geous to incorporate their input into the dRMON vieW, supplementing it When possible With more complete man agement data. An enhanced collector provides sophisticated manage In one embodiment, once an agent has validated a col lector it stores an indication for an address of the collector include in the standard collector, especially When the Col auto-update as Well as other dRMON information is pro vided to it. FIG. 9 provides a How chart of one embodiment and does not have to validate subsequent packets received from the collector. Other embodiments are possible that use security features ment capabilities that are too dif?cult or costly to routinely 45 provided by the netWork operation system and that therefore do not require a user to set a passWord. In such embodiments, a different, but possibly related mechanism may be used to alloW multiple collectors to be heard by only a subset of agents. Thus, the invention provides a number of alternative security measures that together provide secure communica tion betWeen agents and collectors. lector is embedded in a hub, sWitch, or router. Such enhanced capabilities might include WEB support With 55 JAVA server capability, the ability to feed management data into standard databases or intelligent analysis of manage ment data to spot problems before those problems become critical. FIG. 8 illustrates hoW this concept may be implemented according to an embodiment of the invention and hoW it may be distributed Within the netWorking environment. TWo classes of Collectors are depicted: Workgroup Collectors 81 and Domain Collectors 80. All collectors are addressable by Ef?cient Reporting of dRMON Data over a NetWork Management stations 84, but often only Domain Collectors Prior art RMON probes typically communicate informa tion about the netWork’s operation With a management are in fact addressed by a management application. Workgroup Collectors oversee smaller regions of the station using RMON de?ned MIBs and ?lters that are netWork such as a single ?oor in a multilevel building. individually reported to the management station upon Because their sphere of management is smaller, a Workgroup request of individual MIB data. Prior art RMON de?nes a number of different counters, each of Which an RMON probe can report to a management station upon query by that etc.) are also smaller; as a result, they can often be embedded in sWitch or hub. In smaller netWorks, these Collectors station through SNMP or another generic netWork manage ment protocol. This can potentially lead to a large amount of traf?c ?oWing betWeen a prior art probe and a management station in order to display an overall picture of the network. collectors’ physical requirements (CPU poWer, memory, Would probably be adequate for their management needs and 65 a second tier of Domain Collectors Would not be required. Domain Collectors (DCs) are used in larger netWorks to collect and archive management data from Workgroup Col 6,108,782 19 20 lectors Within their sphere of management. DCs typically represent larger regions Within the enterprise network such Domain Collectors as a remote of?ce or a Whole building on a large campus. on the front-end (i.e. at the ES level), it is Domain Collectors 80 Which distribute it on the back-end (i.e. at the manage ment terminal level). DCs are generally implemented on While dRMON Agents distribute RMON’s functionality Each one can support multiple management stations 84, thus permitting any manager to monitor that domain from any Where in the enterprise. Because of their greater scope of poWerful hardWare, possibly based upon Pentium/Pentium responsibility and the need to provide considerable long Pro systems running WindoWs NT. DCs are concentrators for large amounts of netWork management data. In one term and nonvolatile data storage, DCs are generally much more poWerful devices than Workgroup Collectors and as such, are generally implemented as stand alone stackable embodiment, DCs alloW capturing more netWork monitoring data Without overly burdening distributed collectors by periodically off-loading statistics from the 15s, freeing up devices generally located With the sWitches and hubs they oversee. those IS resources to continue to capture neW data. This data A more detailed description of these Collector types and various alternative embodiments folloW. is gathered from a variety of possible sources, such as: dRMON Workgroup Collectors, Embedded RMON (full or Workqroup Collectors AWorkgroup class dRMON Collector is located in a prior 15 and organiZes this various information to create a seemingly cated device. There are advantages and disadvantages to each of these hardWare implementations as discussed beloW. (1) Probe Based. RMON probes often have more resources available than do management cards embedded in sWitches and hubs and are often strategically located throughout the netWork in a Way that makes them prime candidates for collection points for dRMON. Combined homogenous vieW of its management domain. The manage ment domain may include different LANs that communicate across routers and domain collectors generally are able to communicate via a routed netWork protocol, such as IP. The merged vieW is then made accessible in any variety of possibly Ways, including to compliant SNMP-based man With a desire to monitor devices Which do not have a dRMON agent installed, locating a Collector in the probe has further advantages. For example, a dual-interface agement applications, published using WEB protocols, via 25 RMON probe could be connected to tWo sWitch ports Which are shared With a number of older PCs, Mackintoshes and station anyWhere in the enterprise. Other features that may be included in alternative embodi ments of DCs or in higher performance collectors include: the other sWitch ports. Ideally, the probe Would be con?g urable to provide a choice of vieWs such that the user could Data sourcing for popular database products. ODBC in select to have the probe combine the Collector’s data With its this embodiment are used to cull important management data from the domain vieW and feed it to databases created oWn to create one interface vieW or to present them as separate interfaces. 35 in manageable versions including management functions, so it is a natural option to place a dRMON Collector Within them. The primary disadvantages to this approach are that management cards are often resource constrained both in available CPU poWer as Well as in RAM capacity, With the and maintained by the user. This capability alloWs users to use the database query and reporting tools they use every day to also access and analyZe their netWork management data. WEB-based device management. The Domain Collector may provide a WEB front-end to the SNMP device man RAM limitations often enough to preclude doing much in agement thus making any broWser-equipped station a device management station. Expert Analysis. One of RMON’s greatest strengths is its the Way of packet capture and store, and that to one degree or another, the inclusion of RMON analysis in the sWitch usually negatively affects overall sWitch performance. Nevertheless, many users may prefer this approach and it dial-up, etc. Because of the large and extensible storage capabilities that may be included With DCs, considerable historical data and many large captured packet streams could be maintained and archived and offered to any management UNIX Workstations Which do not have dRMON Agents. All other dRMON-equipped nodes Would be distributed across (2) Hub/Switch Based. Most Hubs or SWitches are offered partial) in sWitches/hubs, RMON probes and/or Embedded SNMP Management Agents in sWitches/hubs. A DC merges art type RMON probe, a hub/sWitch, or a stackable dedi 45 ?lter and capture capabilities. HoWever, unless the user is a protocol expert, most of the poWer of this feature is lost to dedicated dRMON Collector Whose packaging may be iden them. Expert systems tools, like those noW appearing for WindoWs NT, may be used in this embodiment to provide ongoing analysis of the management data and alert the user to problems before they become critical and can suggest tical to that of the stackable hubs Which it Would manage. It may be based upon proprietary hardWare or possibly a PC possible resolutions. Systems Management Integration. At present, manage Without monitor or keyboard. This Collector has a more ment tools vendors have lined up on opposite sides of the fence: there are those Who focus on systems management tools and those Who have concentrated efforts on netWork enables an RMON solution for products that do not have the resources to support full embedded RMON. (3) Stackable/Stand alone. The Stackable Collector is a poWerful CPU than most embedded management cards and is capable of holding considerable RAM and, optionally, hard disk storage; as a result, it can hold much more RMON 55 management. Unfortunately, many of the real World prob lems users face are not cleanly isolated to one side or the data such as large amounts of historical data or captured packets. It may also provide additional services such as other. There are numerous systems management tools such WEB-based RMON management and even WEB-based device management of the rest of the stack. The inclusion of as LANDesk and Microsoft’s SMS Which could be coupled into a DC via interfacing softWare. In combination With many of these enhanced capabilities into this Collector’s speci?cations are facilitated by basing it upon the PC expert analysis, DCs could then provide problem detection and resolution of many common problems regardless of Whether they Were system problems, netWork problems or a combination of the tWo. architecture and using an OS such as WindoWs NT to support various add-ons. The development tools for the PC platform are also far ahead of those for embedded processors, thus shortening substantially the time-to-market and maximiZing the availability of experienced program mers. 65 The invention may be embodied in a set of executable computer program code Which may be stored into a ?xed computer medium such as a disk, diskette, volatile memory or non-volatile memory, or any other medium for storing 6,108,782 21 22 5. The method according to claim 1 further comprising: capturing netWork data streams at said nodes; and forWarding said captured data streams to said collector. computer code. In such a case When such instructions are loaded and executed in an appropriately con?gured netWork intermediate system, the intermediate system Will perform as described herein. A representation of such a system 700 in shoWn in FIG. 11, containing CPU 707, optional input 6. The method according to claim 4 Wherein said com piled statistics are as de?ned by published RMON or devices 709 and 711, disk drives 715 and optional monitor RMON2 monitoring protocols. 705. Fixed media 717 may be used to program such a system and could represent a disk-type optical or magnetic media or a memory. A system such as 700 may be used in conjunction 7. The method according to claim 1 Wherein a plurality of said nodes are end systems that provide netWork commu With the invention as embodied on a ?xed media to generate executable ?les that can be distributed throughout a netWork to various netWork components as described herein. The invention has noW been explained With reference to nications to a user. 10 speci?c embodiments. Other embodiments Will be apparent to those of skill in the art. In particular, method steps have been grouped and labelled as being part of various sub methods in order to increase clarity of the disclosure, protocol, said ?rst protocol being a higher layer protocol 15 de?ned for the monitoring and management of netWorks and Wherein said node communicates With said collector using a second protocol, said second protocol being a loWer layer protocol that in unacknoWledged and is speci?cally designed hoWever, these steps could be differently grouped Without changing the essential operation of the invention. for loWer layer netWork management communication. Furthermore, it should be understood that While the inven tion has been described in terms of a computer netWork, the invention has applications in a variety of communication 10. The method according to claim 1 Wherein said col lector and said nodes communicate via a protocol in Which: statistics data from nodes to the collector is generated only in response to a poll packet received from a systems, such as advanced cable television or telephone netWorks, or any other communication system including collector; system performance monitoring at distributed points in the system and reported back to a centraliZed collector. It is therefore not intended that this invention be limited, except as indicated by the appended claims. It is also intended that 8. The method according to claim 1 Wherein a plurality of said nodes communicate using an ethernet protocol. 9. The method according to claim 1 Wherein said collector communicates With said netWork manager using a ?rst 25 poll and response packets are not acknoWledged or retransmitted; nodes report all netWork statistics in terms of cumulative the Word “netWor ” as used in the speci?cation and claims be read to cover any communication system unless the counters so that any failure of any poll or response packet does not result in erroneous data at the collector context requires otherWise and likeWise “end system” be read to encompass any suitable end system (telephone, but merely results in a delay in the collector receiving the data. 11. The method according to claim 10 Wherein said television) on any such communication system or to encom pass distributed points in the netWork intermediate of an end systems. It is also intended that the Word “packet” as used in the speci?cation and claims be read to cover any unit of protocol further provides that node responses to a poll from a collector are complete in one data unit so that a received 35 transmitted data, Whether an ethernet packet, a cell, or any other data unit transmitted on a netWork unless the context tiple nodes each respond to said multicast poll data unit from said collector and ?ooding of the collector is prevented by having each node delay its response by said random value, requires otherWise. What is claimed is: 1. Amethod for distributed collecting of netWork statistics Wherein said random value determined at each node and derived from an address of said node. 13. The method according to claim 1 Wherein said mul comprising: gathering netWork statistics at a plurality of nodes dis tributed in a netWork; transmitting data containing said statistics to a collector; tiple nodes each repeatedly respond to repeated multicast poll data units from said collector and ?ooding of the combining said statistics from said plurality of nodes into group netWork statistics to form complied statistics; collector is prevented by having each node delay its response by said random value, Wherein said random value reporting netWork performance data based on said com piled statistics, from said collector, to a netWork man ager; and determined at each node and derived from an address of said node and a changing value such that responses to a multicast poll data unit are redistributed With each poll. 14. The method according to claim 1 Wherein a node and Wherein multiple nodes each respond to a multicast poll data unit from a collector and ?ooding of the collector a collector each have embedded Within them an identical is prevented by having each node delay its response by passWord that is separately encrypted by different reversible a random value. 2. The method according to claim 1 further comprising: setting values at said collector to con?gure said collecting of netWork statistics; and forWarding con?guration data by said collector to said nodes to con?gure said gathering by said nodes. 3. The method according to claim 1 further comprising: launching an agent in nodes participating in said distrib uting collecting, said agent being an executable module for gathering netWork statistics and communicating With said collector. 4. The method according to claim 1 Wherein said com piled statistics are as de?ned in a standard de?ned for the gathering of netWork-Wide performance statistics. response from a node can be processed Without depending that any other data unit be received. 12. The method according to claim 1 Wherein said mul algorithms and Wherein said collector and said node unen 55 crypt their identical passWords at run-time only said Wherein said collector places said identical passWord in an initial poll data unit and Wherein said node responds to that collector only if a passWord in a poll data unit matches its passWord. 15. The method according to claim 9 Wherein said ?rst protocol is a standard-based SNMP protocol alloWing said collector to communicate With standard netWork manage ment applications and said second protocol is a non-routed layer 2 protocol optimiZed for unacknoWledged communi cation betWeen a collector and a node. 65 16. The method according to claim 1 Wherein said col lector is a set of functions embedded Within a netWork intermediate system. 6,108,782 24 23 17. The method according to claim 3 wherein said agent at said node, establishing capture channels resident on is a set of functions incorporated in other driver or system softWare installed in a node. said node for storing said captured data units prior to encapsulating and transmitting said units to said col 18. The method according to claim 1 further comprising: lector. 21. The method according to claim 19 further comprising: launching an agent in nodes participating in said distrib uted capture, said agent being an executable module for transmitting data containing compiled statistics from said collector to a domain collector; compiling statistics from a plurality of collectors at said domain collector; and providing reports based on said compiled statistics, from establishing capture channels, capturing packets, and 10 said domain collector, to a netWork manager. 19. A method for distributed capture of data unit streams 22. The method according to claim 19 Wherein a plurality of said nodes are end systems that provide netWork com comprising: munications to a user. capturing data units at a plurality of nodes distributed in a netWork; 15 encapsulating said captured units and transmitting said de?ned for the monitoring and management of netWorks and combining said captured units from said nodes into group Wherein said node communicates With said collector using a capture channels; second protocol, said second protocol being a loWer layer protocol that is ?exibly either unacknowledged or acknowledged, has loW overhead, and is speci?cally designed for loWer layer netWork management communica reporting said group capture channels, from said collector, to a netWork manager; and collector; using said synchroniZation data at said nodes to maintain a time at said nodes that is in synchroniZation With the time at said collector; and time-stamping captured data at said nodes When said data is transmitted from said nodes; said time-stamp representing an elapsed time at said node from When said data is received at said and When said encapsulated data is transmitted to said collector; examining said time-stamp at said collector to determine and order said captured data units. 20. The method according to claim 19 further comprising: setting values at said collector to con?gure capture chan nels for said data; forWarding con?guration data by said collector to said nodes to establish capture channels and ?ltering de? nitions; and 23. The method according to claim 19 Wherein said collector communicates With said netWork manager using a ?rst protocol, said ?rst protocol being a higher layer protocol encapsulated data to a collector; transmitting at periodic intervals from said collector to said nodes a synchronization data unit, said synchro niZation data unit representing an elapsed time at said communicating With said collector. 25 tion. 24. The method according to claim 19 Wherein said collector furthers examines a time stamp for captured data units and, Where necessary, examines other protocol infor mation in said data units to determine a correct order for said data units. 25. The method according to claim 19 Wherein a node records the identity of all other nodes capable of performing distributed capture and only captures data if: 35 the data traf?c is directed data traf?c either to or from that one node; and if the other node is either not capable of performing data capture or if the address of the other node indicates that said node is designated to perform packet capture. 26. The method according to claim 19 Wherein a node transmits encapsulated data only in response to a poll signal from a collector.