Configuring a QFX3000-M QFabric System

Network Configuration Example

Configuring a QFX3000-M QFabric System

Modified: 2016-12-16

Copyright © 2017, Juniper Networks, Inc.

Juniper Networks, Inc.

1133 Innovation Way

Sunnyvale, California 94089

USA

408-745-2000 www.juniper.net

Copyright © 2016, Juniper Networks, Inc. All rights reserved.

Juniper Networks, Junos, Steel-Belted Radius, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United

States and other countries. The Juniper Networks Logo, the Junos logo, and JunosE are trademarks of Juniper Networks, Inc. All other trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.

Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice.

Network Configuration Example Configuring a QFX3000-M QFabric System


All rights reserved.

The information in this document is current as of the date on the title page.

YEAR 2000 NOTICE

Juniper Networks hardware and software products are Year 2000 compliant. Junos OS has no known time-related limitations through the year 2038. However, the NTP application is known to have some difficulty in the year 2036.

END USER LICENSE AGREEMENT

The Juniper Networks product that is the subject of this technical documentation consists of (or is intended for use with) Juniper Networks software. Use of such software is subject to the terms and conditions of the End User License Agreement (“EULA”) posted at http://www.juniper.net/support/eula.html

. By downloading, installing or using such software, you agree to the terms and conditions of that EULA.

ii Copyright © 2017, Juniper Networks, Inc.

Table of Contents

Chapter 1

Chapter 2

Chapter 3

Understanding the QFX3000-M QFabric System . . . . . . . . . . . . . . . . . . . . . . . 5

QFabric System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Legacy Data Center Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

QFX Series QFabric System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Understanding QFabric System Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Understanding Interfaces on the QFabric System . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Four-Level Interface Naming Convention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

QSFP+ Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Link Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Understanding the QFabric System Hardware Architecture . . . . . . . . . . . . . . . . . . 18

QFabric System Hardware Architecture Overview . . . . . . . . . . . . . . . . . . . . . . 18

QFX3000-G QFabric System Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

QFX3000-M QFabric System Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Initial Setup for the QFX3000-M QFabric System . . . . . . . . . . . . . . . . . . . . . 23

QFabric System Initial and Default Configuration Information . . . . . . . . . . . . . . . 23

Converting the Device Mode for a QFabric System Component . . . . . . . . . . . . . . 25

Example: Configuring EX4200 Switches for the QFX3000-M QFabric System

Control Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Importing a QFX3000-M QFabric System Control Plane EX4200 Switch

Configuration with a USB Flash Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Generating the MAC Address Range for a QFabric System . . . . . . . . . . . . . . . . . . 56

Performing the QFabric System Initial Setup on a QFX3100 Director Group . . . . . 57

Performing an Initial Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Restoring a Backup Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

QFabric System Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Understanding QFabric System Administration Tasks and Utilities . . . . . . . . . . . 63

Gaining Access to the QFabric System Through the Default Partition . . . . . . . . . . 67

Example: Configuring QFabric System Login Classes . . . . . . . . . . . . . . . . . . . . . . 68

Configuring Node Groups for the QFabric System . . . . . . . . . . . . . . . . . . . . . . . . . 76

Configuring the Port Type on QFX3600 Node Devices . . . . . . . . . . . . . . . . . . . . . . 81

Configuring the QSFP+ Port Type on QFX5100 Devices . . . . . . . . . . . . . . . . . . . . 85

Example: Configuring SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Example: Configuring System Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Configuring Graceful Restart for QFabric Systems . . . . . . . . . . . . . . . . . . . . . . . . . 92

Enabling Graceful Restart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Configuring Graceful Restart Options for BGP . . . . . . . . . . . . . . . . . . . . . . . . 93

Configuring Graceful Restart Options for OSPF and OSPFv3 . . . . . . . . . . . . 94

Tracking Graceful Restart Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Optimizing the Number of Multicast Flows on QFabric Systems . . . . . . . . . . . . . 96


iii


Chapter 4

Chapter 5

QFabric System Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Generating the License Keys for a QFabric System . . . . . . . . . . . . . . . . . . . . . . . . 97

Adding New Licenses (CLI Procedure) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Installing a License Using a Configuration Statement . . . . . . . . . . . . . . . . . . 99

Installing Licenses Using the CLI Directly . . . . . . . . . . . . . . . . . . . . . . . . 100

Installing Licenses Using a Configuration File . . . . . . . . . . . . . . . . . . . . . 101

Installing a License Using an Operational Command . . . . . . . . . . . . . . . . . . 102

Adding a License to a Device with a Single Routing Engine . . . . . . . . . . 102

Adding a License to a Device with Dual Routing Engines . . . . . . . . . . . . 103

Deleting a License (CLI Procedure) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Using the Operational Command to Delete Licenses . . . . . . . . . . . . . . . . . . 104

Using a Configuration Command to Delete Licenses . . . . . . . . . . . . . . . . . . 104

Saving License Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Verifying Junos OS License Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Displaying Installed Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Displaying License Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

QFabric System Backup and Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Performing System Backup and Recovery for a QFabric System . . . . . . . . . . . . . 109

Performing a QFabric System Recovery Installation on the Director Group . . . . . 110

(Optional) Creating an Emergency Boot Device Using a Juniper Networks

External Blank USB Flash Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Performing a Recovery Installation Using a Juniper Networks External USB

Flash Drive with Preloaded Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Performing a Recovery Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Creating an Emergency Boot Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

iv Copyright © 2017, Juniper Networks, Inc.

CHAPTER 1

Understanding the QFX3000-M QFabric

System

•

QFabric System Overview on page 5

•

Understanding QFabric System Terminology on page 9

•

Understanding Interfaces on the QFabric System on page 14

•

Understanding the QFabric System Hardware Architecture on page 18

QFabric System Overview

The architecture of legacy data centers contrasts significantly with the revolutionary

Juniper Networks data center solution.

This topic covers:

•

Legacy Data Center Architecture on page 5

•

QFX Series QFabric System Architecture on page 7

Legacy Data Center Architecture

Service providers and companies that support data centers are familiar with legacy multi-tiered architectures, as seen in

Figure 1 on page 6

.


5


Figure 1: Legacy Data Center Architecture

6

The access layer connects servers and other devices to a Layer 2 switch and provides an entry point into the data center. Several access switches are in turn connected to intermediate Layer 2 switches at the aggregation layer (sometimes referred to as the

distribution layer) to consolidate traffic. A core layer interconnects the aggregation layer switches. Finally, the core switches are connected to Layer 3 routers in the routing layer to send the aggregated data center traffic to other data centers or a wide area network

(WAN), receive external traffic destined for the data center, and interconnect different

Layer 2 broadcast domains within the data center.

The problems that exist with the multi-tiered data center architecture include:

• Limited scalability—The demands for electrical power, cooling, cabling, rack space, and port density increase exponentially as the traditional data center expands, which prohibits growth after minimal thresholds are met.

• Inefficient resource usage—Up to 50 percent of switch ports in a legacy data center are used to interconnect different tiers rather than support server and storage connections. In addition, traffic that ideally should move horizontally between servers within a data center often must also be sent vertically up through the tiers to reach a router and down through the tiers to reach the required destination server.

• Increased latency—By requiring the devices at each tier level to perform multiple iterations of packet and frame processing, the data plane traffic takes significantly longer to reach its destination than if the sending and receiving devices were directly connected. This processing overhead results in potentially poor performance for time-sensitive applications, such as voice, video, or financial transactions.


Chapter 1: Understanding the QFX3000-M QFabric System

QFX Series QFabric System Architecture

In contrast to legacy multi-tiered data center architectures, the Juniper Networks QFX

Series QFabric System architecture provides a simplified networking environment that solves the most challenging issues faced by data center operators. A fabric is a set of devices that act in concert to behave as a single switch. It is a highly scalable, distributed,

Layer 2 and Layer 3 networking architecture that provides a high-performance, low-latency, and unified interconnect solution for next-generation data centers as seen in

Figure 2 on page 7 .

Figure 2: QFX Series QFabric System Architecture

Director devices

Interconnect devices

Node devices

Virtual Chassis control plane

A QFabric system collapses the traditional multi-tiered data center model into a single tier where all access layer devices (known in the QFabric system model as Node devices) are essentially directly connected to all other access layer devices across a very large scale fabric backplane (known in the QFabric system model as the Interconnect device).

Such an architecture enables the consolidation of data center endpoints (such as servers, storage devices, memory, appliances, and routers) and provides better scaling and network virtualization capabilities than traditional data centers.


7


Essentially, a QFabric system can be viewed as a single, nonblocking, low-latency switch that supports thousands of 10-Gigabit Ethernet ports or 2-Gbps, 4-Gbps, or 8-Gbps Fibre

Channel ports to interconnect servers, storage, and the Internet across a high-speed, high-performance fabric. The entire QFabric system is managed as a single entity through a Director group, containing redundant hardware and software components that can be expanded and scaled as the QFabric system grows in size. In addition, the Director group automatically senses when devices are added or removed from the QFabric system and dynamically adjusts the amount of processing resources required to support the system.

Such intelligence helps the QFabric system use the minimum amount of power to run the system efficiently, but not waste energy on unused components.

As a result of the QFabric system architecture, data center operators are now realizing the benefits of this next-generation architecture, including:

•

Low latency—Because of its inherent advantages in this area, the QFabric system provides an excellent foundation for mission-critical applications such as financial transactions and stock trades, as well as time-sensitive applications such as voice and video.

•

Enhanced scalability—The QFabric system can be managed as a single entity and provides support for thousands of data center devices. As Internet traffic continues to grow exponentially with the increase in high-quality video transmissions and rise in the number of mobile devices used worldwide, the QFabric system can keep pace with the demands for bandwidth, applications, and services offered by the data center.

• Virtualization-enabled—The QFabric system was designed to work seamlessly with virtual servers, virtual appliances, and other virtual devices, allowing for even greater scalability, expandability, and rapid deployment of new services than ever before.

Migrating to virtual devices also results in significant costs savings, fueled by reduced space requirements, decreased needs for power and cooling, and increased processing capabilities.

•

Simplicity—Although the QFabric system can scale to hundreds of devices and thousands of ports, you can still manage the QFabric system as a single system.

• Flexibility—You can deploy the QFabric system as an entire system or in stages.

• Convergence—Because the congestion-free fabric is lossless, all traffic in a QFabric system can be converged onto a single network. As a result, the QFabric system supports Ethernet, Fibre Channel over Ethernet, and native Fibre Channel packets and frames.

Flat, nonblocking, and lossless, the network fabric offered by the QFabric system has the scale and flexibility to meet the needs of small, medium, and large-sized data centers for years to come.

Related

Documentation

•

Understanding QFabric System Terminology

•

Understanding the QFabric System Hardware Architecture

•

Understanding the QFabric System Software Architecture

8 Copyright © 2017, Juniper Networks, Inc.


Understanding QFabric System Terminology

To understand the QFabric system environment and its components, you should become familiar with the terms defined in

Table 1 on page 9 .

Table 1: QFabric System Terms

Term Definition

Clos network fabric

Director device

Director group

Director software fabric control Routing Engine fabric manager Routing Engine

Three-stage switching network in which switch elements in the middle stages are connected to all switch elements in the ingress and egress stages. In the case of QFabric system components, the three stages are represented by an ingress chipset, a midplane chipset, and an egress chipset in an Interconnect device (such as a QFX3008-I Interconnect device). In Clos networks, which are well known for their nonblocking properties, a connection can be made from any idle input port to any idle output port, regardless of the traffic load in the rest of the system.

Hardware component that processes fundamental QFabric system applications and services, such as startup, maintenance, and inter-QFabric system device communication. A set of Director devices with hard drives can be joined to form a Director group, which provides redundancy and high availability by way of additional memory and processing power. (See also Director group.)

Set of Director devices that host and load-balance internal processes for the

QFabric system. The Director group handles tasks such as QFabric system network topology discovery, Node and Interconnect device configuration, startup, and DNS, DHCP, and NFS services. Operating a Director group is a minimum requirement to manage a QFabric system.

The Director group runs the Director software for management applications and runs dual processes in active/standby mode for maximum redundancy and high availability. (See also Director software and Director device.)

Software that handles QFabric system administration tasks, such as fabric management and configuration. The Junos OS-based Director software runs on the Director group, provides a single, consolidated view of the QFabric system, and enables the main QFabric system administrator to configure, manage, monitor, and troubleshoot QFabric system components from a centralized location. To access the Director software, log in to the default partition. (See also Director device and Director group.)

Virtual Junos OS Routing Engine instance used to control the exchange of routes and flow of data between QFabric system hardware components within a partition. The fabric control Routing Engine runs on the Director group.

Virtual Junos OS Routing Engine instance used to control the initialization and maintenance of QFabric system hardware components belonging to the default partition. The fabric manager Routing Engine runs on the Director group.

infrastructure QFabric system services processed by the virtual Junos Routing Engines operating within the Director group. These services, such as fabric management and fabric control, support QFabric system functionality and high availability.


9


Table 1: QFabric System Terms (continued)

Term Definition

Interconnect device QFabric system component that acts as the primary fabric for data plane traffic traversing the QFabric system between Node devices. Examples of Interconnect devices include the QFX3008-I Interconnect device in a QFX3000-G QFabric system, the QFX5100-24Q configured as an Interconnect device, and the

QFX3600-I Interconnect device in a QFX3000-M QFabric system. (See also

Node device.)

Junos Space Carrier-class network management system for provisioning, monitoring, and diagnosing Juniper Networks routing, switching, security, and data center platforms.

Set of one to eight Node devices that connects to an external network.

network Node group network Node group Routing Engine Virtual Junos OS Routing Engine instance that handles routing processes for a network Node group. The network Node group Routing Engine runs on the

Director group.

Node device partition

QFabric system

Routing and switching device that connects to endpoints (such as servers or storage devices) or external network peers, and is connected to the QFabric system through an Interconnect device. You can deploy Node devices similarly to the way a top-of-rack switch is implemented. Examples of Node devices include the QFX3500 Node device, QFX3600 Node device, and QFX5100 Node device. (See also Interconnect device and network Node group.)

Collection of physical or logical QFabric system hardware components (such as Node devices) that provides fault isolation, separation, and security.

In their initial state, all QFabric system components belong to a default partition.

Highly scalable, distributed, Layer 2 and Layer 3 networking architecture that provides a high-performance, low-latency, and unified interconnect solution for next-generation data centers. A QFabric system collapses the traditional multi-tier data center model, enables the consolidation of data center endpoints (such as servers, storage devices, memory, appliances, and routers), and provides better scaling and network virtualization capabilities than traditional data centers.

Essentially, a QFabric system can be viewed as a single, nonblocking, low-latency switch that supports thousands of 10-Gigabit Ethernet ports or

2-Gbps, 4-Gbps or 8-Gbps Fibre Channel ports to interconnect servers, storage, and the Internet across a high-speed, high-performance fabric. The QFabric system must have sufficient resources and devices allocated to handle the

Director group, Node device, and Interconnect device functions and capabilities.




Term Definition

QFabric system control plane Internal network connection that carries control traffic between QFabric system components. The QFabric system control plane includes management connections between the following QFabric system hardware and software components:

•

•

•

•

Node devices, such as the QFX3500 Node device.

Interconnect devices, such as the QFX3008-I Interconnect device.

Director group processes, such as management applications, provisioning, and topology discovery.

Control plane Ethernet switches to provide interconnections to all QFabric system devices and processes. For example, you can use EX Series EX4200 switches running in Virtual Chassis mode for this purpose.

To maintain high availability, the QFabric system control plane uses a different network than the QFabric system data plane, and uses a fabric provisioning protocol and a fabric management protocol to establish and maintain the

QFabric system.

QFabric system data plane

QFabric system endpoint

QFabric system fabric

QFX3500 Node device

Redundant, high-performance, and scalable data plane that carries QFabric system data traffic. The QFabric system data plane includes the following high-speed data connections:

•

•

•

10-Gigabit Ethernet connections between QFabric system endpoints (such as servers or storage devices) and Node devices.

40-Gbps quad small form-factor pluggable plus (QSFP+) connections between Node devices and Interconnect devices.

10-Gigabit Ethernet connections between external networks and a Node device acting as a network Node group.

To maintain high availability, the QFabric system data plane is separate from the QFabric system control plane.

Device connected to a Node device port, such as a server, a storage device, memory, an appliance, a switch, or a router.

Distributed, multistage network that consists of a queuing and scheduling system that is implemented in the Node device, and a distributed cross-connect system that is implemented in Interconnect devices. The QFabric system fabric is part of the QFabric system data plane.

Node device that connects to either endpoint systems (such as servers and storage devices) or external networks in a QFabric system. It is packaged in an industry-standard 1U, 19-inch rack-mounted enclosure.

The QFX3500 Node device provides up to 48 10-Gigabit Ethernet interfaces to connect to the endpoints. Twelve of these 48 interfaces can be configured to support 2-Gbps, 4-Gbps or 8-Gbps Fibre Channel, and 36 of the interfaces can be configured to support Gigabit Ethernet. Also, there are four uplink connections to connect to Interconnect devices in a QFabric system. These uplinks use 40-Gbps quad small form-factor pluggable plus (QSFP+) interfaces. (See also QFX3500 switch.)


11



Term Definition

QFX3500 switch Standalone data center switch with 10-Gigabit Ethernet access ports and

40-Gbps quad, small form-factor pluggable plus (QSFP+) uplink interfaces.

You can (optionally) configure some of the access ports as 2-Gbps, 4-Gbps, or 8-Gbps Fibre Channel ports or Gigabit Ethernet ports.

The QFX3500 switch can be converted to a QFabric system Node device as part of a complete QFabric system. The switch is packaged in an industry-standard 1U, 19-inch rack-mounted enclosure. (See also QFX3500

Node device.)

QFX3600 Node device

QFX3600 switch

Node device that connects to either endpoint systems (such as servers and storage devices) or external networks in a QFabric system. It is packaged in an industry-standard 1U, 19-inch rack-mounted enclosure.

The QFX3600 Node device provides 16 40-Gbps QSFP+ ports. By default, 4 ports (labeled

Q0 through

Q3

) are configured for 40-Gbps uplink connections between your Node device and your Interconnect device, and 12 ports (labeled

Q4 through

Q15

) use QSFP+ direct-attach copper (DAC) breakout cables or

QSFP+ transceivers with fiber breakout cables to support 48 10-Gigabit

Ethernet interfaces for connections to either endpoint systems (such as servers and storage devices) or external networks. Optionally, you can choose to configure the first eight ports ( Q0 through Q7 ) for uplink connections between your Node device and your Interconnect device, and ports Q2 through Q15 for

10-Gigabit Ethernet connections to either endpoint systems or external networks. (See also QFX3600 switch.)

Standalone data center switch with 16 40-Gbps quad, small form-factor pluggable plus (QSFP+) interfaces. By default, all the 16 ports operate as

40-Gigabit Ethernet ports. Optionally, you can choose to configure the 40-Gbps ports to operate as four 10-Gigabit Ethernet ports. You can use QSFP+ to four

SFP+ breakout cables to connect the 10-Gigabit Ethernet ports to other servers, storage, and switches.

The QFX3600 switch can be converted to a QFabric system Node device as part of a complete QFabric system. The switch is packaged in an industry-standard 1U, 19-inch rack-mounted enclosure. (See also QFX3600

Node device.)




Term Definition

QFX5100 Node device redundant server Node group rolling upgrade

Routing Engine

QFabric system Node device that connects to either endpoint systems (such as servers and storage devices) or external networks. All three supported models are packaged in an industry-standard 1U, 19-inch rack-mounted enclosure. A QFX5100 Node device can be any of these models:

•

•

•

QFX5100-48S

By default, the QFX5100-48S Node device provides 48 10-Gigabit Ethernet interfaces to connect to the endpoints. There are also six 40-Gbps quad small form-factor pluggable plus (QSFP+) interfaces, of which four are uplinks (FTE).

QFX5100-48T

By default, the QFX5100-48T Node device provides 48 10GBASE-T interfaces to connect to endpoints. There are also six 40-Gbps QSFP+ interfaces, of which four are uplinks (FTE)

QFX5100-24Q

By default, the QFX5100-24Q Node device provides 24 40-Gigabit Ethernet

QSFP+ interfaces to connect to the endpoints. The QFX5100-24Q has two expansion bays. The number of additional interfaces available depends on the expansion module and the System mode configured for the Node device.

By default, on the QFX5100-48S Node device and QFX5100-48T Node device, the first 4 ports (labeled fte-0/1/0 through fte-0/1/3

) are configured for

40-Gbps uplink connections between your Node device and your Interconnect devices, and 2 ports (labeled xle-0/1/4 and xle-0/1/5 ) use QSFP+ direct-attach copper (DAC) breakout cables or QSFP+ transceivers with fiber breakout cables to support 8 10-Gigabit Ethernet interfaces for connections to either endpoint systems (such as servers and storage devices) or external networks.

Optionally, you can choose to configure the middle 2 ports ( xle-0/1/2 and xle-0/1/3

) for additional connections to either endpoint systems or external networks.

(See also QFX3500 Node device and QFX3600 Node device.)

Set of two Node devices that connect to servers or storage devices. Link aggregation group (LAG) interfaces can span the Node devices within a redundant server Node group.

Method used in the QFabric system to upgrade the software for components in a systematic, low-impact way. A rolling upgrade begins with the Director group, proceeds to the fabric (Interconnect devices), and finishes with the

Node groups.

Juniper Networks-proprietary processing entity that implements QFabric system control plane functions, routing protocols, system management, and user access. Routing Engines can be either physical or virtual entities.

The Routing Engine functions in a QFabric system are sometimes handled by

Node devices (when connected to endpoints), but mostly implemented by the Director group (to provide support for QFabric system establishment, maintenance, and other tasks).


13



Term Definition routing instance Private collection of routing tables, interfaces, and routing protocol parameters unique to a specific customer. The set of interfaces is contained in the routing tables, and the routing protocol parameters control the information in the routing tables.

(See also virtual private network.) server Node group virtual LAN (VLAN) virtual private network (VPN) flow group

Set of one or more Node devices that connect to servers or storage devices.

Unique Layer 2 broadcast domain for a set of ports selected from the components available in a partition. VLANs allow manual segmentation of larger Layer 2 networks and help to restrict access to network resources. To interconnect VLANs, Layer 3 routing is required.

Layer 3 routing domain within a partition. VPNs maintain privacy with a tunneling protocol, encryption, and security procedures. In a QFabric system, a Layer 3 VPN is configured as a routing instance.

Force redundant multicast streams to flow through different interconnect devices to prevent a single interconnect device from potentially dropping both streams of multicast traffic during a failure.

Related

Documentation

•

QFabric System Overview

•

Understanding the QFabric System Hardware Architecture

•

Understanding the QFabric System Software Architecture

•

Understanding Fibre Channel Terminology

• Understanding QFabric Multicast Flow Control

Understanding Interfaces on the QFabric System

This topic describes:

•

Four-Level Interface Naming Convention on page 14

•

QSFP+ Interfaces on page 15

•

Link Aggregation on page 18

Four-Level Interface Naming Convention

When you configure an interface on the QFabric system, the interface name needs to follow a four-level naming convention that enables you to identify an interface as part of either a Node device or a Node group. Include the name of the network or server Node group at the beginning of the interface name.

The four-level interface naming convention is:

device-name:type-fpc/pic/port


Chapter 1: Understanding the QFX3000-M QFabric System where device-name is the name of the Node device or Node group. The remainder of the naming convention elements are the same as those in the QFX3500 switch interface naming convention.

An example of a four-level interface name is:

node2:xe-0/0/2

QSFP+ Interfaces

The QFX3500 Node device provides four 40-Gbps QSFP+ (quad small form-factor pluggable plus) interfaces (labeled Q0 through Q3) for uplink connections between your

Node device and your Interconnect devices.

The QFX3600 Node device provides 16 40-Gbps QSFP+ interfaces. By default, 4 interfaces

(labeled Q0 through Q3) are configured for 40-Gbps uplink connections between your

Node device and your Interconnect devices, and 12 interfaces (labeled Q4 through Q15) use QSFP+ direct-attach copper (DAC) breakout cables or QSFP+ transceivers with fiber breakout cables to support 48 10-Gigabit Ethernet interfaces for connections to either endpoint systems (such as servers and storage devices) or external networks. Optionally, you can choose to configure the first eight interfaces (Q0 through Q7) for uplink connections between your Node device and your Interconnect devices, and interfaces

Q2 through Q15 for 10-Gigabit Ethernet or 40-Gigabit Ethernet connections to either endpoint systems or external networks (see Configuring the Port Type on QFX3600 Node

Devices).

Table 2 on page 15

shows the port mappings for QFX3600 Node devices.

Table 2: QFX3600 Node Device Port Mappings

Port Number

10-Gigabit Ethernet Interfaces

(On PIC 0)

40-Gigabit Ethernet

Interfaces (On PIC 1)

40-Gigabit Data Plane

Uplink Interfaces (On PIC 1)

Q0

Q1

Q2

Not supported on this port

Not supported on this port xe-0/0/8 xe-0/0/9 xe-0/0/10 xe-0/0/11 xle-0/1/0 xle-0/1/1 xle-0/1/2 fte-0/1/0 fte-0/1/1 fte-0/1/2

Q3 xe-0/0/12 xe-0/0/13 xe-0/0/14 xe-0/0/15 xle-0/1/3 fte-0/1/3


15


Table 2: QFX3600 Node Device Port Mappings (continued)

Port Number


(On PIC 0)

40-Gigabit Ethernet


Q4 xle-0/1/4

Q5

Q6

Q7

Q8

Q9

Q10 xe-0/0/32 xe-0/0/33 xe-0/0/34 xe-0/0/35 xe-0/0/36 xe-0/0/37 xe-0/0/38 xe-0/0/39 xe-0/0/40 xe-0/0/41 xe-0/0/42 xe-0/0/43 xe-0/0/24 xe-0/0/25 xe-0/0/26 xe-0/0/27 xe-0/0/28 xe-0/0/29 xe-0/0/30 xe-0/0/31 xe-0/0/16 xe-0/0/17 xe-0/0/18 xe-0/0/19 xe-0/0/20 xe-0/0/21 xe-0/0/22 xe-0/0/23 xle-0/1/5 xle-0/1/6 xle-0/1/7 xle-0/1/8 xle-0/1/9 xle-0/1/10


Uplink Interfaces (On PIC 1) fte-0/1/4 fte-0/1/5 fte-0/1/6 fte-0/1/7






Table 2: QFX3600 Node Device Port Mappings (continued)

Port Number


(On PIC 0)

40-Gigabit Ethernet


Q11 xle-0/1/11

Q12

Q13

Q14

Q15 xe-0/0/52 xe-0/0/53 xe-0/0/54 xe-0/0/55 xe-0/0/56 xe-0/0/57 xe-0/0/58 xe-0/0/59 xe-0/0/44 xe-0/0/45 xe-0/0/46 xe-0/0/47 xe-0/0/48 xe-0/0/49 xe-0/0/50 xe-0/0/51 xe-0/0/60 xe-0/0/61 xe-0/0/62 xe-0/0/63 xle-0/1/12 xle-0/1/13 xle-0/1/14 xle-0/1/15


Uplink Interfaces (On PIC 1)






The QFX5100-48S Node device provides 48 10-Gigabit Ethernet interfaces and 6 40-Gbps

QSFP+ interfaces. By default, 4 interfaces (labeled 48 through 51) are configured for

40-Gbps uplink connections between your Node device and your Interconnect devices, and 2 interfaces (labeled 52 and 53) support 40-Gigabit Ethernet connections to either endpoint systems (such as servers and storage devices) or external networks. Optionally, you can choose to configure the middle two interfaces (50 and 51) for 40-Gigabit Ethernet connections to either endpoint systems or external networks, and you can choose to configure the last two interfaces (52 and 53) for uplink connections between your Node device and your Interconnect devices (see Configuring the QSFP+ Port Type on QFX5100

Devices).

Table 3 on page 18

shows the port mappings for QFX5100-48S Node devices.


17


49

50

51

Table 3: QFX5100-48S Node Device Port Mappings

Port Number

40-Gigabit Ethernet Interfaces (On PIC

1)

40-Gigabit Data Plane Uplink Interfaces

(On PIC 1)

48 Not supported on this PIC

Not supported on this PIC xle-0/1/2 fte-0/1/0 fte-0/1/1 fte-0/1/2 xle-0/1/3 xle-0/1/4 fte-0/1/3 fte-0/1/4

52

53 xle-0/1/5 fte-0/1/5

Link Aggregation

Link aggregation enables you to create link aggregation groups across Node devices within a network Node group or redundant server Node group. You can include up to eight

Ethernet interfaces in a LAG. You can have up to 48 LAGs within a redundant server Node group, and 128 LAGs in a network Node group. To configure a LAG, include the aggregated-devices statement at the [edit chassis node-group node-group-name] hierarchy level and the device-count statement at the [edit chassis node-group node-group-name aggregated-devices ethernet] hierarchy level. Additionally, include any aggregated Ethernet options (minimum-links and link-speed) at the [edit interfaces interface-name aggregated-ether-options] hierarchy level and the 802.3ad statement at the [edit interfaces interface-name ether-options] hierarchy level. To configure the Link Aggregation

Control Protocol (LACP), include the lacp statement at the [edit interfaces aggregated-ether-options] hierarchy level.

Related

Documentation

•

Configuring the Port Type on QFX3600 Node Devices

•

Configuring the QSFP+ Port Type on QFX5100 Devices

Understanding the QFabric System Hardware Architecture

•

QFabric System Hardware Architecture Overview on page 18

•

QFX3000-G QFabric System Features on page 21

•

QFX3000-M QFabric System Features on page 21

QFabric System Hardware Architecture Overview

The QFabric system is a single-layer networking tier that connects servers and storage devices to one another across a high-speed, unified core fabric. You can view the QFabric system as a single, extremely large, nonblocking, high-performance Layer 2 and Layer 3 switching system. The reason you can consider the QFabric system as a single system is that the Director software running on the Director group allows the main QFabric system administrator to access and configure every device and port in the QFabric system from


Chapter 1: Understanding the QFX3000-M QFabric System a single location. Although you configure the system as a single entity, the fabric contains four major hardware components. The hardware components can be chassis-based, group-based, or a hybrid of the two. As a result, it is important to understand the four types of generic QFabric system components and their functions, regardless of which hardware environment you decide to implement. A representation of these components is shown in

Figure 3 on page 19

.

Figure 3: QFabric System Hardware Architecture

Director devices

Node devices

Interconnect devices Virtual Chassis

(control plane)

The four major QFabric system components include the following:

• Director group—The Director group is a management platform that establishes, monitors, and maintains all components in the QFabric system. It is a set of Director devices that run the Junos operating system (Junos OS) on top of a CentOS foundation.

The Director group handles tasks such as QFabric system network topology discovery,

Node and Interconnect device configuration and startup, and Domain Name System

(DNS), Dynamic Host Configuration Protocol (DHCP), and Network File System (NFS) services. The Director group also runs the software for management applications, hosts and load-balances internal processes for the QFabric system, and starts additional

QFabric system processes as requested.

• Node devices—A Node device is a hardware system located on the ingress of the

QFabric system that connects to endpoints (such as servers or storage devices) or external networks, and is connected to the heart of the QFabric system through an

Interconnect device. A Node device can be used in a manner similar to how a top-of-rack switch is implemented. By default, Node devices connect to servers or storage devices.

However, when you group Node devices together to connect to a network that is external to the QFabric system, the formation is known as a network Node group.

• Interconnect devices—An Interconnect device acts as the primary fabric for data plane traffic traversing the QFabric system between Node devices. To reduce latency to a


19

Configuring a QFX3000-M QFabric System minimum, the Interconnect device implements multistage Clos switching to provide nonblocking interconnections between any of the Node devices in the system.

• Control plane network—The control plane network is an out-of-band Gigabit Ethernet management network that connects all QFabric system components. For example, you can use a group of EX4200 Ethernet switches configured as a Virtual Chassis to enable the control plane network. The control plane network connects the Director group to the management ports of the Node and Interconnect devices. By keeping the control plane network separate from the data plane, the QFabric system can scale to support thousands of servers and storage devices.

The four major QFabric system components can be assembled from a variety of hardware options. Currently supported hardware configurations are shown in

Table 4 on page 20 .

Table 4: Supported QFabric System Hardware Configurations

QFabric

System

Configuration Director Group Node Device

Interconnect

Device Control Plane Device

QFX3000-G

QFabric system

QFX3100 Director group

QFX3500, QFX3600, and

QFX5100-48S, QFX5100-48T, and

QFX5100-24Q Node devices

NOTE: There can be a maximum of 128 Node devices in the

QFX3000-G QFabric system.

QFX3008-I

Interconnect device

NOTE: There can be a maximum of four Interconnect devices in the

QFX3000-G

QFabric system.

Two Virtual Chassis composed of either four

EX4200-48T switches each (for a copper-based control plane) or eight

EX4200-24F switches each (for a fiber-based control plane)

QFX3000-M

QFabric system

QFX3100 Director group

NOTE: For a copper-based

QFX3000-M

QFabric system control plane network, use

QFX3100 Director devices with RJ-45 network modules installed. For a fiber-based control plane network, use

QFX3100 Director devices with SFP network modules installed.

QFX3500, QFX3600, and

QFX5100-48S, QFX5100-48T, and

QFX5100-24Q Node devices

QFX5100-24Q or

QFX3600-I

Interconnect devices

NOTE:

• There can be a maximum of 16

Node devices in the QFX3000-M

QFabric system using QFX3600-I as Interconnect devices and 32

Node devices using the

QFX5100-24Q as Interconnec devices.

NOTE: QFX5100-24Q

Interconnect devices and

QFX3600-I Interconnect devices cannot be mixed on the same

QFabric system.

NOTE: There can be a maximum of four Interconnect devices in the

QFX3000-M

QFabric system.

• For a copper-based QFX3000-M

QFabric system control plane network, use QFX3500 Node devices with a 1000BASE-T management board installed. For a fiber-based control plane network, use QFX3500 Node devices with an SFP management board installed.

Two EX4200 Ethernet switches

NOTE: For a copper-based

QFX3000-M QFabric system control plane network, use

EX4200-24T switches with an SFP+ uplink module installed. For a fiber-based control plane network, use

EX4200-24F switches with an SFP+ uplink module installed.



To complete the system, external Routing Engines (such as the fabric manager Routing

Engines, network Node group Routing Engines, and fabric control Routing Engines) run on the Director group and implement QFabric system control plane functions. The control plane network provides the control plane connections between the Node devices, the

Interconnect devices, and the Routing Engines running on the Director group.

QFX3000-G QFabric System Features

A QFX3000-G QFabric system provides the following key features:

•

Support for up to 128 Node devices and 4 Interconnect devices, which provides a maximum of 6144 10-Gigabit Ethernet ports.

• Low port-to-port latencies that scale as the system size grows from 48 to 6144

10-Gigabit Ethernet ports.

•

Support for up to 384,000 total ingress queues at each Node device to the QFabric system Interconnect backplane.

• Support for Converged Enhanced Ethernet (CEE) traffic.

QFX3000-M QFabric System Features

A QFX3000-M QFabric system provides the following key features:

• Support for up to 32 Node devices and 4 QFX5100-24Q Interconnect devices or 16

Node device and 4 QFX3600-I Interconnect devices.

NOTE: You may not mix QFX5100-24Q Interconnect devices with

QFX3600-I Interconnect devices on the same QFX3000-M QFabric system.

•

Low port-to-port latencies that scale as the system size grows from 48 to 768

10-Gigabit Ethernet ports.

Related

Documentation

•

Understanding QFabric System Terminology

•

Understanding the QFabric System Software Architecture

•

Understanding the Director Group

•

Understanding Routing Engines in the QFabric System

•

Understanding Interconnect Devices

•

Understanding Node Devices

•

Understanding Node Groups

•

Understanding Partitions


21



CHAPTER 2

Initial Setup for the QFX3000-M QFabric

System

•

QFabric System Initial and Default Configuration Information on page 23

•

Converting the Device Mode for a QFabric System Component on page 25

•

Example: Configuring EX4200 Switches for the QFX3000-M QFabric System Control

Plane on page 30

•

Importing a QFX3000-M QFabric System Control Plane EX4200 Switch Configuration with a USB Flash Drive on page 55

•

Generating the MAC Address Range for a QFabric System on page 56

•

Performing the QFabric System Initial Setup on a QFX3100 Director Group on page 57

QFabric System Initial and Default Configuration Information

Once you install the hardware for the QFabric system, you can configure the Junos operating system (Junos OS) to begin using the system. This topic discusses which setup activities you need to perform and which activities are handled automatically by the

QFabric system.

The fabric manager Routing Engine in the Director group automatically handles some of the initial setup activities, including:

• Assignment of IP addresses and unique identifiers to each QFabric system component by way of the management control plane

•

Inclusion of all QFabric system devices within the default partition

• Establishment of interdevice communication and connectivity through the use of a fabric provisioning protocol and a fabric management protocol

The initial configuration tasks you need to perform to bring up the QFabric system and make it operational include:

•

Converting any standalone devices, such as QFX3500 and QFX3600 devices, to Node device mode (see Converting the Device Mode for a QFabric System Component)

• Setting up the QFabric system control plane cabling, topology, and configuration


23


•

To set up the control plane cabling, topology, and configuration for the QFX3000-G

QFabric system, see Example: Configuring the Virtual Chassis for a Copper-Based

QFX3000-G QFabric System Control Plane.

•

To set up a copper or fiber-based control plane cabling, topology, and configuration for the QFX3000-M QFabric system, see

“Example: Configuring EX4200 Switches for the QFX3000-M QFabric System Control Plane” on page 30 .

• Accessing the Director group through a console connection, turning on the devices, and running through the initial setup script which prompts you to:

•

Set IP addresses for the Director devices in the Director group.

•

Set an IP address for the default partition.

• Add the software serial number for your QFabric system. (Review the e-mail containing the software serial number that you received from Juniper Networks when you purchased your QFabric system.)

•

Set the starting MAC address and the range of MAC addresses for the QFabric system.

for this information.)

• Set a root password for the Director devices.

•

Set a root password for the QFabric system components, such as Node devices,

Interconnect devices, and infrastructure.

•

Logging into the default partition by using the IP address you configured when you ran the Director group initial setup script

• Configuring basic system settings for the default partition, such as time, location, and default gateways

NOTE: Unlike other Juniper Networks devices that run Junos OS, a QFabric system does not have a default factory configuration (containing the basic configuration settings for system logging, interfaces, protocols, and so on) that is loaded when you first install and power on the Director devices.

Therefore, you must configure all the settings required for your QFabric system through the default partition CLI.

•

Configuring aliases for Node devices

•

Configuring VLANs and interfaces for the QFabric system devices

• Configuring redundant server Node groups to provide resiliency for server and storage connections

•

Configuring a network Node group to connect the QFabric system to external networks

•

Configuring the port type on QFX3600 Node devices

• Configuring routing protocols to run on the network Node group interfaces and reach external networks


Chapter 2: Initial Setup for the QFX3000-M QFabric System

NOTE: When you configure routing protocols on the QFabric system, you must use interfaces from the Node devices assigned to the network Node group. If you try to configure routing protocols on interfaces from the Node devices assigned to server Node groups, the configuration commit operation fails.

• Generating and adding the license keys for the QFabric system

Converting the Device Mode for a QFabric System Component

You can configure some devices to act as a standalone switch or participate in a QFabric system in a particular role. To change the role of your device, you must set the device mode.

Table 5 on page 25

shows the device modes available for various devices.

Table 5: Support for device mode options

Device mode QFX3500 QFX3600 QFX5100

Interconnect device

Node device

Standalone

N/A

Supported

Supported

Supported

Supported

Supported

Supported for QFX3000-M

Supported

N/A

To convert a device to a different mode, issue the request chassis device-mode command and specify the desired device mode. You verify the current and future device mode with the show chassis device-mode command.

When you convert a device from standalone mode to either Node device or Interconnect device mode, the software prepares the device to be configured automatically by the

QFabric system. However, changing the device mode erases all configuration data on the device.

NOTE: The QFX3600 switch requires Jloader Release 1.1.8 before you can convert the switch to Interconnect device mode. For more information, see:

Jloader 1.1.8 Release for QFX-Series Platforms

.

CAUTION: We recommend that you back up your device configuration to an external location before converting a device to a different device mode.

The following procedures illustrate the conversion options available when you modify a device mode:

• Convert from standalone switch mode to Node device mode

•

Convert from Node device mode to Interconnect device mode


25


•

Convert from Interconnect device mode to Node device mode

• Convert from Node device mode or Interconnect device mode to standalone switch mode

Standalone Switch to

Node Device

To convert your device from standalone mode to Node device mode, follow these steps:

1.

Connect to your standalone device through the console port and log in as the root user.

2.

Back up your device configuration to an external location.

3.

root@switch# save configuration-name external-path

Upgrade the software on your device to a QFabric system Node and Interconnect device software package that matches the QFabric system complete software package used by your QFabric system. If the complete software package for your

QFabric system is named jinstall-qfabric-13.2X52-D10.2.rpm, you need to install the jinstall-qfabric-5-13.2X52-D10.2-domestic-signed.tgz

package on your QFX5100 device and the jinstall-qfx-13.2X52-D10.2-domestic-signed.tgz package on your QFX3500 or

QFX3600 device. Matching the two software packages ensures a smooth and successful addition of the device to the QFabric system inventory.

root@switch# request system software add software-package-name reboot

NOTE: After you install the correct software, the QFX5100 device is placed into Node device mode by default and cannot be converted to any other mode in Junos OS Release 13.2X52-D10.

4.

Check the current device mode by issuing the show chassis device-mode command.

root@switch> show chassis device-mode

Current device-mode : Standalone

Future device-mode after reboot : Standalone

5.

Issue the request chassis device-mode command and select the desired device mode.

root@switch> request chassis device-mode node-device

Device mode set to 'node-device' mode.

Please reboot the system to complete the process.

6.

Verify the future device mode by issuing the show chassis device-mode command.


Current device-mode : Standalone

Future device-mode after reboot : Node-device

7.

Reboot the device.

root@switch> request system reboot

Reboot the system ? [yes,no] (no) yes

Shutdown NOW!

[pid 34992] root@switch>



*** FINAL System shutdown message from root@switch ***

System going down IMMEDIATELY

8.

Verify that the new device mode has been enabled by issuing the show chassis device-mode command.


Current device-mode : Node-device


9.

To enable a converted device to participate in the QFabric system, locate the applicable network cables for your device and connect the device ports to the control plane and data plane.

10.

(Optional) If you change the device back from Node device mode to standalone mode, restore the saved backup configuration from your external location.

root@switch# load configuration-name external-path

Node Device to

Interconnect Device

To convert your device from Node device mode to Interconnect device mode, follow these steps:

1.

From the default partition CLI prompt, back up your QFabric system configuration to an external location.

2.

user@qfabric# save configuration-name external-path

Connect to your device through the console port and log in as the root user.

3.





4.


root@switch> request chassis device-mode interconnect-device

Device mode set to 'interconnect-device' mode.


5.




Future device-mode after reboot : Interconnect-device

6.

Reboot the device.



Shutdown NOW!





27


7.



Current device-mode : Interconnect-device


8.

To enable a converted device to participate in the QFabric system in its new role, move the device to a different rack (as needed), locate the applicable network cables for your device, connect the device ports to the control plane and data plane per the design for your specific QFabric system, and reconfigure any aliases for the device at the QFabric default partition CLI prompt.

Interconnect Device to

Node Device

To convert your device from Interconnect device mode to Node device mode, follow these steps:

1.


2.


Connect to your device through the console port and log in as the root user.

3.





4.


root@switch> request chassis device-mode node-device

Device mode set to 'node-device' mode.


5.





6.

Reboot the device.



Shutdown NOW!




7.







8.

To enable a converted device to participate in the QFabric system in its new role, move the device to a different rack (as needed), locate the applicable network cables for your device, connect the device ports to the control plane and data plane per the design for your specific QFabric system, and reconfigure any aliases for the device at the QFabric default partition CLI prompt.

QFabric Component

(Interconnect or Node

Device) to Standalone

Switch

To convert your QFabric component from either Interconnect device mode or Node device mode to standalone switch mode, follow these steps:

1.


2.


Connect to the desired QFabric component through the console port of the device and log in as the root user.

3.


root@node1> show chassis device-mode



4.

Issue the request chassis device-mode standalone command to convert the component to standalone switch mode, while the component is still connected to the QFabric system.

root@node1> request chassis device-mode standalone

Device mode set to 'standalone' mode.


NOTE: Always convert the device mode to standalone before you remove the component from the QFabric system. If you remove the component from the QFabric system before converting the device mode to standalone, the switch might not operate properly. For example, the output of the show chassis hardware command might display no FPCs or interfaces for the switch.

5.


root@node1> show chassis device-mode


Future device-mode after reboot : Standalone

6.

Reboot the component to complete the conversion process.

root@node1> request system reboot


Shutdown NOW!


29


[pid 34992] root@node1>

*** FINAL System shutdown message from root@node1 ***


7.

Disconnect and remove the component from the QFabric system. You may now operate the device as a standalone switch.


Plane

This example shows you how to connect QFabric system components and configure the

EX4200 switches used for the QFX3000-M QFabric system control plane network. Proper wiring of Director devices, Interconnect devices, and Node devices to the EX4200 switches, combined with a standard configuration, enables you to bring up the internal QFabric system management network and prepare your QFabric system for full operation.

NOTE: The EX4200 switch configuration is the same for both the copper-based and fiber-based QFX3000-M QFabric system control plane networks. Hence, a separate example for configuring EX4200 switches for the fiber-based control plane network is not provided.

However, because you cannot mix and match fiber and copper in the same control plane network, you must select only one type of control plane for each QFX3000-M QFabric system you install. The primary focus of this example is a copper-based control plane network. Before you use this example to configure a fiber-based control plane network, ensure that you have installed and wired the QFabric system hardware and EX4200 switches as required for a fiber-based control plane network (see QFX3000-M QFabric

System Installation Overview).

•

Requirements on page 30

•

Overview on page 31

•

Configuration on page 39

•

Verification on page 50

Requirements

This example uses the following hardware and software components:

•

One QFX3000-M QFabric system containing:

•

Two QFX3100 Director devices with 1000BASE-T network modules installed

• Two QFX3600-I Interconnect devices

• Eight QFX3500 Node devices with a 1000BASE-T management board installed

• Two EX4200-24T switches with SFP+ uplink module installed



Overview

•

Junos OS Release 13.2X52-D10 for the QFabric system components

• Junos OS Release 12.3R6.6 for the EX Series switches

Before you begin:

• Rack, mount, and install your QFabric system hardware (Director group, Interconnect devices, and Node devices). For more information, see Installing and Connecting a

QFX3100 Director Device, Installing and Connecting a QFX3600 or QFX3600-I Device, and Installing and Connecting a QFX3500 Device.

• Rack, mount, and install your EX4200 switches. For more information, see Installing

and Connecting an EX4200 Switch.

The QFX3000-M QFabric system control plane network connects the Director group,

Interconnect devices, and Node devices in a QFabric system across a pair of redundant

EX4200 switches. By separating the management control plane from the data plane, the QFabric system can scale efficiently. The copper-based control plane network uses

Gigabit Ethernet cabling and connections between components, and two 1-Gigabit

Ethernet connections configured in a link aggregation group (LAG) between the redundant

EX4200 switches.

Specific ports have been reserved on the EX4200 switches to connect to each of the

QFabric system device types. Such design simplifies installation and facilitates timely deployment of a QFabric system. It also permits the use of a standard EX4200 switch configuration included as part of this example. The standard configuration can scale from the 8 Node devices shown in this example to a maximum of 16 Node devices.

Topology

Figure 4 on page 31

shows the general port ranges where QFabric system devices must be connected to the EX4200 switches. For each EX4200 switch, connect ports 0 through

15 to Node devices, ports 16 through 19 to Interconnect devices, ports 20 through 23 to

Director devices, and uplink ports 0 and 1 to the other control plane switch as an inter-switch LAG.

Table 6 on page 32

shows the details of the QFabric system component-to-EX4200 switch port mappings.

Figure 4: QFX3000-M QFabric System Control Plane—EX4200 Switch

Port Ranges

EX Series

CAUTION:

• The control plane network within a QFabric system is a critical component of the system that should not be shared with other network traffic. In order


31

Configuring a QFX3000-M QFabric System to scale efficiently, the control plane network must be reserved for the

QFabric system and its components. As a result, the ports of the QFabric system control plane must never be used for any purpose other than to transport QFabric system control plane traffic, and we neither recommend nor support the connection of other devices to the QFabric system control plane network.

•

Do not install Junos Space and AI-Scripts (AIS) on the control plane network

EX4200 switches in a QFX3000-M QFabric system.

Table 6 on page 32

shows the specific mappings of QFabric system control plane network ports from the QFabric system components to the EX4200 switches.

NOTE: The uplink ports 2 and 3 on the EX4200 switches are reserved for future use.

Table 6: QFX3000-M QFabric System Copper-Based Control Plane—QFabric

Component-to-EX4200 Switch Port Mappings

EX4200 Switch 1 (EX0) EX4200 Switch 2 (EX1)

QFabric System

Component

Node device 0 Node0, management port

C0 to port

0

(ge-0/0/0)

Node1, management port C0 to port 1

(ge-0/0/1)

Node2, management port

C0 to port

2

(ge-0/0/2)


C1 to port

0

(ge-0/0/0)


(ge-0/0/1)


C1 to port

2

(ge-0/0/2)

Node device 1

Node device 2

Node device 3 Node3, management port C0 to port 3

(ge-0/0/3)


C0 to port

4

(ge-0/0/4)


C0 to port

5

(ge-0/0/5)


(ge-0/0/6)


C0 to port

7

(ge-0/0/7)


(ge-0/0/8)


(ge-0/0/3)


C1 to port

4

(ge-0/0/4)


C1 to port

5

(ge-0/0/5)


(ge-0/0/6)


C1 to port

7

(ge-0/0/7)


(ge-0/0/8)

Node device 4

Node device 5

Node device 6

Node device 7

Node device 8




Component-to-EX4200 Switch Port Mappings (continued)


QFabric System

Component

Node device 9 Node9, management port C0 to port 9

(ge-0/0/9)


C0 to port

10

(ge-0/0/10)


C0 to port

11

(ge-0/0/11)


(ge-0/0/12)


C0 to port

13

(ge-0/0/13)


(ge-0/0/14)


C0 to port

15

(ge-0/0/15)


(ge-0/0/9)


C1 to port

10

(ge-0/0/10)


C1 to port

11

(ge-0/0/11)


(ge-0/0/12)


C1 to port

13

(ge-0/0/13)


(ge-0/0/14)


C1 to port

15

(ge-0/0/15)

Node device 10

Node device 11

Node device 12

Node device 13

Node device 14

Node device 15

Interconnect device 0 IC0, management port C0 to port 16

(ge-0/0/16)

IC1, management port

C0 to port

17

(ge-0/0/17)


C0 to port

18

(ge-0/0/18)

IC3, management port C0 to port 19

(ge-0/0/19)

DG0 module 0, port

0 to port

20

(ge-0/0/20)

DG0 module 0, port 1 to port 21

(ge-0/0/21)

DG1 module 0, port

0 to port

22

(ge-0/0/22)

DG1 module 0, port

1 to port

23

(ge-0/0/23)

EX0, uplink port 0 to EX1, uplink port 0

(ge-0/1/0)


(ge-0/0/16)


C1 to port

17

(ge-0/0/17)


C1 to port

18

(ge-0/0/18)


(ge-0/0/19)

DG0 module 1, port

0 to port

20

(ge-0/0/20)

DG0 module 1, port 1 to port 21

(ge-0/0/21)

DG1 module 1, port

0 to port

22

(ge-0/0/22)

DG1 module 1, port

1 to port

23

(ge-0/0/23)


(ge-0/1/0)

Interconnect device 1



Director device 0

Director device 0

Director device 1

Director device 1

Inter-EX4200 switch LAG


33



Component-to-EX4200 Switch Port Mappings (continued)


QFabric System

Component

Inter-EX4200 switch LAG EX0, uplink port 1 to EX1, uplink port 1

(ge-0/1/1)

Reserved

Uplink port 2 (ge-0/1/2)

Reserved

Uplink port

3

(ge-0/1/3)


(ge-0/1/1)

Reserved

Uplink port 2 (ge-0/1/2)

Reserved

Uplink port

3

(ge-0/1/3)

Future use

Future use

Next, connect the Director devices to the EX4200 switches. In general, you want to accomplish the following:

• Connect two ports from one network module in a Director device to the first EX4200 switch, and two ports from the second network module to the second EX4200 switch.

•

Connect the Director devices to each other and create a Director group. You can use either straight-through RJ-45 patch cables or crossover cables, because the Director devices contain autosensing modules. Connect one port from each network module on the first Director device to one port in each network module on the second Director device.

Figure 5 on page 34

shows the specific ports on the Director group that you must connect to the EX4200 switches and interconnect between the Director devices.

Figure 5: QFX3000-M QFabric System Control Plane—Director Group to

EX4200 Switch Connections

34

EX Series EX Series

In this specific example, connect ports 0 and 1 from module 0 on Director device DG0 to ports 20 and 21 on EX4200 switch EX0 (ge-0/0/20 and ge-0/0/21), and connect ports

0 and 1 from module 1 to ports 20 and 21 on the second EX4200 switch EX1 (ge-0/0/20 and ge-0/0/21).



For Director device DG1, connect ports 0 and 1 from module 0 to ports 22 and 23 on

EX4200 switch EX0 (ge-0/0/22 and ge-0/0/23), and connect ports 0 and 1 from module 1 to ports 22 and 23 on the second EX4200 switch EX1 (ge-0/0/22 and ge-0/0/23).

To form the Director group, connect port 3 on module 0 on Director device DG0 to port

3 on module 0 on Director device DG1. Similarly, connect port 3 on module 1 on Director device DG0 to port 3 on module 1 on Director device DG1.

Table 7 on page 35

shows the port mappings for the Director group in this example.

Table 7: Director Group Port Mappings

Director

Device EX4200 Switch EX0 EX4200 Switch EX1

DG0

DG1

•

•

•

DG0 module 0, port

0 to port

20 on EX0

(ge-0/0/20)

DG0 module 0, port

1 to port

21 on EX0

(ge-0/0/21)

DG0 module 0, port 3 to module 0, port 3 on DG1

•

•

•

DG1 module 0, port

0 to port

22 on EX0

(ge-0/0/22)

DG1 module 0, port 1 to port 23 on EX0

(ge-0/0/23)


•

•

•

DG0 module 1, port

0 to port

20 on EX1

(ge-0/0/20)

DG0 module 1, port

1 to port

21 on EX1

(ge-0/0/21)


•

•

•

DG1 module 1, port

0 to port

22 on EX1

(ge-0/0/22)

DG1 module 1, port 1 to port 23 on EX1

(ge-0/0/23)


In the software, the ports of each network module on a Director device are reversed, numbered from right to left, and incremented sequentially across modules. If you issue interface operational commands directly on the Director device, note the following port mappings as shown in

Table 8 on page 35 :

Table 8: Hardware to Software Port Mappings for Director Device Network Modules

Network Module Port 0 Port 1 Port 2 Port 3

Module 0

Module 1 eth5 eth9 eth4 eth8 eth3 eth7 eth2 eth6

Figure 6 on page 36

shows the specific ports on the QFX3600-I Interconnect devices that you must connect to the EX4200 switches. In general, connect the first management port in an Interconnect device to the first EX4200 switch, and the second management port to the second EX4200 switch.


35


Figure 6: QFX3000-M QFabric System Control Plane—Interconnect Device to EX4200 Switch Connections

EX Series EX Series

In this specific example, for both Interconnect devices IC0 and IC1, connect management port C0 to EX4200 switches EX0 and EX1 and management port C1 to EX4200 switches

EX0 and EX1. Connect the management port C0 cables to port 16 on EX4200 switches

EX0 and EX1 (ge-0/0/16), and connect the management port C1 cables to port 17 on

EX4200 switches EX0 and EX1 (ge-0/0/17).

Table 9 on page 36

shows the port mappings for the Node devices in this example.

Table 9: Interconnect Device Port Mappings

Interconnect Device EX4200 Switch EX0 EX4200 Switch EX1

IC0 IC0, management port

C0 to port

16

(ge-0/0/16)


C1 to port

16

(ge-0/0/16)

IC1 IC1, management port C0 to port 17

(ge-0/0/17)


(ge-0/0/17)

Figure 7 on page 36

,

Figure 8 on page 36 , and

Figure 9 on page 37

show the specific ports on the Node devices that you must connect to the EX4200 switches when using a copper-based control plane. In general, connect the first management port from a Node device to the first EX4200 switch, and the second management port to the second

EX4200 switch.

Figure 7: QFX3000-M QFabric System Control Plane—QFX3500 Node

Device to EX4200 Switch Connections

EX Series EX Series



EX Series EX Series





EX Series EX Series

When implementing a fiber-based control plane, refer to

Figure 10 on page 37 ,

Figure 11 on page 37 , and Figure 12 on page 37

for the proper control plane connections.

Figure 10: QFX3000-M QFabric System Fiber-Based Control

Plane—QFX3500 Node Device to EX4200 Switch Connections

EX Series EX Series



EX Series EX Series



EX Series EX Series

In this specific example, for Node device Node0, connect management port C0 (also known as me5) to EX4200 switch EX0 port 0 (ge-0/0/0), and connect management port C1 (also known as me6) to the second EX4200 switch EX1 port 0 (ge-0/0/0).

For the remaining seven Node devices, connect management port C0 to the ge-0/0/X port on EX4200 switch EX0 that matches the Node device number. Similarly, connect management port C1 to the port on the second EX4200 switch EX1 that matches the

Node device number. For example, you would connect Node device Node5 to port 5

(ge-0/0/5).

Table 10 on page 38

shows the full set of port mappings for the Node devices in this example.


37


Table 10: Node Device to EX4200 Switch Port Mappings

Node Device EX4200 Switch EX0 EX4200 Switch EX1

Node0

Node1

Node2

Node3


(ge-0/0/0)


C0 to port

1

(ge-0/0/1)


(ge-0/0/2)


C0 to port

3

(ge-0/0/3)


(ge-0/0/0)


C1 to port

1

(ge-0/0/1)


(ge-0/0/2)


C1 to port

3

(ge-0/0/3)

Node4

Node5

Node6

Node7


(ge-0/0/4)


C0 to port

5

(ge-0/0/5)


C0 to port

6

(ge-0/0/6)


(ge-0/0/7)


(ge-0/0/4)


C1 to port

5

(ge-0/0/5)


C1 to port

6

(ge-0/0/6)


(ge-0/0/7)

Figure 13 on page 38

shows the specific uplink ports on the first EX4200 switch that you must connect to the second EX4200 switch. These connections create a link aggregation group (LAG) that provides redundancy and resiliency for the EX4200 switch portion of the control plane. In general, connect each 1-Gigabit Ethernet uplink port from the first

EX4200 switch to the corresponding 1-Gigabit Ethernet uplink port on the second EX4200 switch.

Figure 13: QFX3000-M QFabric System Control Plane—Inter-EX4200

Switch LAG Connections

EX Series EX Series

In this specific example, for EX4200 switch EX0, connect uplink port 0 (ge-0/1/0) to

EX4200 switch EX1 uplink port 0 (ge-0/1/0). Then connect uplink port 1 (ge-0/1/1) on

EX4200 switch EX0 to uplink port 1 (ge-0/1/1) on EX4200 switch EX1.

Table 11 on page 39

shows the port mappings for the EX4200 switch LAG connections in this example.



Table 11: EX4200 Switch LAG Port Mappings

EX0 and EX1 EX0

Uplink port 0

Uplink port

1 ge-0/1/0 to ge-0/1/0 ge-0/1/1 to ge-0/1/1

EX1 ge-0/1/0 to ge-0/1/0 ge-0/1/1 to ge-0/1/1

Configuration

CLI Quick

Configuration

• [xref target has no title]

To configure the QFX3000-M QFabric system control plane EX4200 switches quickly, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network, and then copy and paste the commands into the EX4200 switch CLI at the [edit] hierarchy level.

set groups qfabric chassis aggregated-devices ethernet device-count 3 set groups qfabric chassis alarm management-ethernet link-down ignore set groups qfabric chassis lcd-menu fpc 0 menu-item maintenance-menu disable set groups qfabric protocols rstp interface ae2.0 mode point-to-point set groups qfabric protocols rstp interface all edge set groups qfabric protocols rstp interface all no-root-port set groups qfabric protocols rstp bpdu-block-on-edge set groups qfabric protocols lldp interface all set groups qfabric ethernet-switching-options storm-control interface all bandwidth

10000 set groups qfabric vlans qfabric vlan-id 100 set groups qfabric vlans qfabric dot1q-tunneling set groups qfabric-int interfaces <*> mtu 9216 set groups qfabric-int interfaces <*> unit 0 family ethernet-switching port-mode access set groups qfabric-int interfaces <*> unit 0 family ethernet-switching vlan members qfabric set groups qfabric-ae interfaces <*> aggregated-ether-options link-speed 1g set groups qfabric-ae interfaces <*> aggregated-ether-options lacp active set apply-groups qfabric set chassis fpc 0 pic 1 sfpplus pic-mode 1g set interfaces interface-range Node_Device_Interfaces member "ge-0/0/[0-15]" set interfaces interface-range Node_Device_Interfaces description "QFabric Node Device" set interfaces interface-range Node_Device_Interfaces mtu 9216 set interfaces interface-range Node_Device_Interfaces unit 0 family ethernet-switching port-mode access set interfaces interface-range Node_Device_Interfaces unit 0 family ethernet-switching vlan members qfabric set interfaces interface-range Interconnect_Device_Interfaces member "ge-0/0/[16-17]" set interfaces interface-range Interconnect_Device_Interfaces description "QFabric

Interconnect Device" set interfaces interface-range Interconnect_Device_Interfaces mtu 9216 set interfaces interface-range Interconnect_Device_Interfaces unit 0 family ethernet-switching port-mode access set interfaces interface-range Interconnect_Device_Interfaces unit 0 family ethernet-switching vlan members qfabric set interfaces interface-range Director_Device_DG0_LAG_Interfaces member

"ge-0/0/[20-21]"


39


40 set interfaces interface-range Director_Device_DG0_LAG_Interfaces description "QFabric

Director Device - DG0" set interfaces interface-range Director_Device_DG0_LAG_Interfaces ether-options speed

1g set interfaces interface-range Director_Device_DG0_LAG_Interfaces ether-options 802.3ad

ae0 set interfaces interface-range Director_Device_DG1_LAG_Interfaces member

"ge-0/0/[22-23]" set interfaces interface-range Director_Device_DG1_LAG_Interfaces description "QFabric

Director Device - DG1" set interfaces interface-range Director_Device_DG1_LAG_Interfaces ether-options speed

1g set interfaces interface-range Director_Device_DG1_LAG_Interfaces ether-options 802.3ad

ae1 set interfaces interface-range Control_Plane_Inter_LAG_Interfaces member "ge-0/1/[0-1]" set interfaces interface-range Control_Plane_Inter_LAG_Interfaces description "QFabric

Control Plane (Inter - Switch LAG)" set interfaces interface-range Control_Plane_Inter_LAG_Interfaces ether-options 802.3ad

ae2 set interfaces ae0 apply-groups qfabric-int set interfaces ae0 apply-groups qfabric-ae set interfaces ae0 description "QFabric Director Device - DG0" set interfaces ae1 apply-groups qfabric-int set interfaces ae1 apply-groups qfabric-ae set interfaces ae1 description "QFabric Director Device - DG1" set interfaces ae2 description "QFabric Control Plane (Inter-Switch LAG)" set interfaces ae2 mtu 9216 set interfaces ae2 aggregated-ether-options link-speed 1g set interfaces ae2 aggregated-ether-options lacp active set interfaces ae2 unit 0 family ethernet-switching vlan members qfabric set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_3 loss-priority low code-points 110 set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_3 loss-priority low code-points 111 set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_2 loss-priority low code-points 100 set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_2 loss-priority high code-points 101 set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_0 loss-priority low code-points 010 set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_0 loss-priority high code-points 001 set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_3 loss-priority low code-points 110 set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_3 loss-priority low code-points 111 set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_2 loss-priority low code-points 100 set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_2 loss-priority high code-points 101 set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_0 loss-priority low code-points 010 set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_0 loss-priority high code-points 001 set class-of-service forwarding-classes class class_3 queue-num 7 set class-of-service forwarding-classes class class_2 queue-num 2


Chapter 2: Initial Setup for the QFX3000-M QFabric System set class-of-service forwarding-classes class class_0 queue-num 0 set class-of-service interfaces ge-*/0/* scheduler-map cpe_network_smap set class-of-service interfaces ge-*/0/* unit 0 classifiers ieee-802.1 onep_qfabric_classifier set class-of-service interfaces ge-*/0/* unit 0 classifiers inet-precedence

IP_qfabric_classifier set class-of-service interfaces ae* scheduler-map cpe_network_smap set class-of-service interfaces ae* unit 0 classifiers ieee-802.1 onep_qfabric_classifier set class-of-service interfaces ae* unit 0 classifiers inet-precedence IP_qfabric_classifier set class-of-service scheduler-maps cpe_network_smap forwarding-class class_3 scheduler scheduler_3 set class-of-service scheduler-maps cpe_network_smap forwarding-class class_2 scheduler scheduler_2 set class-of-service scheduler-maps cpe_network_smap forwarding-class class_0 scheduler scheduler_0 set class-of-service schedulers scheduler_3 buffer-size percent 30 set class-of-service schedulers scheduler_3 priority strict-high set class-of-service schedulers scheduler_2 transmit-rate percent 75 set class-of-service schedulers scheduler_2 buffer-size percent 30 set class-of-service schedulers scheduler_2 priority low set class-of-service schedulers scheduler_0 transmit-rate percent 25 set class-of-service schedulers scheduler_0 buffer-size percent 40 set class-of-service schedulers scheduler_0 priority low set system host-name qfabric-control-plane set system services ssh set system services telnet set system services web-management http set system syslog user * any emergency set system syslog file messages any notice set system syslog file messages authorization info set system syslog file messages archive world-readable set system syslog file messages explicit-priority set system syslog file interactive-commands interactive-commands any set system syslog file secure authorization info set system syslog file default-log-messages any any set system syslog file default-log-messages structured-data set system syslog file console any error set system syslog time-format millisecond set interfaces me0 unit 0 family inet address 192.168.157.26/24 set routing-options static route 0.0.0.0/0 next-hop 192.168.157.1

Step-by-Step

Procedure

The following example requires that you navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration

Mode in the CLI User Guide.

To configure a EX4200 switch for the QFX3000-M QFabric system control plane network:

1.

Create a configuration group called qfabric to define global QFabric system control plane properties. Set up the number of aggregated Ethernet devices, configure alarm and LCD management, activate loop prevention and storm control, specify a global VLAN (VLAN 100) and 802.1q tunneling, define options for aggregated

Ethernet interfaces, and apply the qfabric group settings to the configuration.

[edit] user@switch# set groups qfabric chassis aggregated-devices ethernet device-count

3


41


42

2.

user@switch# set groups qfabric chassis alarm management-ethernet link-down ignore user@switch# set groups qfabric chassis lcd-menu fpc 0 menu-item maintenance-menu disable user@switch# set groups qfabric protocols rstp interface ae2.0 mode point-to-point user@switch# set groups qfabric protocols rstp interface all edge user@switch# set groups qfabric protocols rstp interface all no-root-port user@switch# set groups qfabric protocols rstp bpdu-block-on-edge user@switch# set groups qfabric protocols lldp interface all user@switch# set groups qfabric ethernet-switching-options storm-control interface all bandwidth 10000 user@switch# set groups qfabric vlans qfabric vlan-id 100 user@switch# set groups qfabric vlans qfabric dot1q-tunneling user@switch# set groups qfabric-int interfaces <*> mtu 9216 user@switch# set groups qfabric-int interfaces <*> unit 0 family ethernet-switching port-mode access user@switch# set groups qfabric-int interfaces <*> unit 0 family ethernet-switching vlan members qfabric user@switch# set groups qfabric-ae interfaces <*> aggregated-ether-options link-speed 1g user@switch# set groups qfabric-ae interfaces <*> aggregated-ether-options lacp active user@switch# set apply-groups qfabric

Configure interfaces for the QFabric system control plane network. Enable the

EX4200 switch SFP+ uplink module for 1-Gigabit Ethernet operation. Set the interface ranges where Node devices (0 through 15), Interconnect devices (16 and

17), and Director devices (20 through 23) connect to the control plane network through the EX4200 switches. Configure the inter-EX4200 switch LAG connections for the ae2 interface and apply the qfabric-int and qfabric-ae configuration groups to the aggregated Ethernet interfaces (ae0 and ae1) for the Director devices.

[edit] user@switch# set chassis fpc 0 pic 1 sfpplus pic-mode 1g user@switch# set interfaces interface-range Node_Device_Interfaces member

"ge-0/0/[0-15]" user@switch# set interfaces interface-range Node_Device_Interfaces description

"QFabric Node Device" user@switch# set interfaces interface-range Node_Device_Interfaces mtu 9216 user@switch# set interfaces interface-range Node_Device_Interfaces unit 0 family ethernet-switching port-mode access user@switch# set interfaces interface-range Node_Device_Interfaces unit 0 family ethernet-switching vlan members qfabric user@switch# set interfaces interface-range Interconnect_Device_Interfaces member

"ge-0/0/[16-17]" user@switch# set interfaces interface-range Interconnect_Device_Interfaces description "QFabric Interconnect Device" user@switch# set interfaces interface-range Interconnect_Device_Interfaces mtu

9216 user@switch# set interfaces interface-range Interconnect_Device_Interfaces unit

0 family ethernet-switching port-mode access user@switch# set interfaces interface-range Interconnect_Device_Interfaces unit

0 family ethernet-switching vlan members qfabric user@switch# set interfaces interface-range Director_Device_DG0_LAG_Interfaces member "ge-0/0/[20-21]"



3.

user@switch# set interfaces interface-range Director_Device_DG0_LAG_Interfaces description "QFabric Director Device - DG0" user@switch# set interfaces interface-range Director_Device_DG0_LAG_Interfaces ether-options speed 1g user@switch# set interfaces interface-range Director_Device_DG0_LAG_Interfaces ether-options 802.3ad ae0 user@switch# set interfaces interface-range Director_Device_DG1_LAG_Interfaces member "ge-0/0/[22-23]" user@switch# set interfaces interface-range Director_Device_DG1_LAG_Interfaces description "QFabric Director Device - DG1" user@switch# set interfaces interface-range Director_Device_DG1_LAG_Interfaces ether-options speed 1g user@switch# set interfaces interface-range Director_Device_DG1_LAG_Interfaces ether-options 802.3ad ae1 user@switch# set interfaces interface-range Control_Plane_Inter_LAG_Interfaces member "ge-0/1/[0-1]" user@switch# set interfaces interface-range Control_Plane_Inter_LAG_Interfaces description "QFabric Control Plane (Inter - Switch LAG)" user@switch# set interfaces interface-range Control_Plane_Inter_LAG_Interfaces ether-options 802.3ad ae2 user@switch# set interfaces ae0 apply-groups qfabric-int user@switch# set interfaces ae0 apply-groups qfabric-ae user@switch# set interfaces ae0 description "QFabric Director Device - DG0" user@switch# set interfaces ae1 apply-groups qfabric-int user@switch# set interfaces ae1 apply-groups qfabric-ae user@switch# set interfaces ae1 description "QFabric Director Device - DG1" user@switch# set interfaces ae2 description "QFabric Control Plane (Inter-Switch

LAG)" user@switch# set interfaces ae2 mtu 9216 user@switch# set interfaces ae2 aggregated-ether-options link-speed 1g user@switch# set interfaces ae2 aggregated-ether-options lacp active user@switch# set interfaces ae2 unit 0 family ethernet-switching vlan members qfabric

Enable class of service (CoS) for the QFabric system control plane network. Establish forwarding classes, priorities, scheduler maps, classifiers, and queues for three types of traffic: control traffic, interdevice traffic, and best-effort traffic.

[edit] user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_3 loss-priority low code-points 110 user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_3 loss-priority low code-points 111 user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_2 loss-priority low code-points 100 user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_2 loss-priority high code-points 101 user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_0 loss-priority low code-points 010 user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_0 loss-priority high code-points 001 user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_3 loss-priority low code-points 110 user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_3 loss-priority low code-points 111


43


44

4.

user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_2 loss-priority low code-points 100 user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_2 loss-priority high code-points 101 user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_0 loss-priority low code-points 010 user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_0 loss-priority high code-points 001 user@switch# set class-of-service forwarding-classes class class_3 queue-num 7 user@switch# set class-of-service forwarding-classes class class_2 queue-num 2 user@switch# set class-of-service forwarding-classes class class_0 queue-num 0 user@switch# set class-of-service interfaces ge-*/0/* scheduler-map cpe_network_smap user@switch# set class-of-service interfaces ge-*/0/* unit 0 classifiers ieee-802.1

onep_qfabric_classifier user@switch# set class-of-service interfaces ge-*/0/* unit 0 classifiers inet-precedence IP_qfabric_classifier user@switch# set class-of-service interfaces ae* scheduler-map cpe_network_smap user@switch# set class-of-service interfaces ae* unit 0 classifiers ieee-802.1

onep_qfabric_classifier user@switch# set class-of-service interfaces ae* unit 0 classifiers inet-precedence

IP_qfabric_classifier user@switch# set class-of-service scheduler-maps cpe_network_smap forwarding-class class_3 scheduler scheduler_3 user@switch# set class-of-service scheduler-maps cpe_network_smap forwarding-class class_2 scheduler scheduler_2 user@switch# set class-of-service scheduler-maps cpe_network_smap forwarding-class class_0 scheduler scheduler_0 user@switch# set class-of-service schedulers scheduler_3 buffer-size percent 30 user@switch# set class-of-service schedulers scheduler_3 priority strict-high user@switch# set class-of-service schedulers scheduler_2 transmit-rate percent

75 user@switch# set class-of-service schedulers scheduler_2 buffer-size percent 30 user@switch# set class-of-service schedulers scheduler_2 priority low user@switch# set class-of-service schedulers scheduler_0 transmit-rate percent

25 user@switch# set class-of-service schedulers scheduler_0 buffer-size percent 40 user@switch# set class-of-service schedulers scheduler_0 priority low

Configure settings to enable the EX4200 switches to interoperate with your management network. Set a hostname, system services (such as Telnet), system log thresholds, management interface parameters, default routes, and any additional preferences you might have.

[edit] user@switch# set system host-name qfabric-control-plane user@switch# set system services ssh user@switch# set system services telnet user@switch# set system services web-management http user@switch# set system syslog user * any emergency user@switch# set system syslog file messages any notice user@switch# set system syslog file messages authorization info user@switch# set system syslog file messages archive world-readable user@switch# set system syslog file messages explicit-priority user@switch# set system syslog file interactive-commands interactive-commands any


Chapter 2: Initial Setup for the QFX3000-M QFabric System user@switch# set system syslog file secure authorization info user@switch# set system syslog file default-log-messages any any user@switch# set system syslog file default-log-messages structured-data user@switch# set system syslog file console any error user@switch# set system syslog time-format millisecond user@switch# set interfaces me0 unit 0 family inet address 192.168.157.26/24 user@switch# set routing-options static route 0.0.0.0/0 next-hop 192.168.157.1

Results To view the configuration, issue the show command in configuration mode or the show configuration command in operational mode. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.

The following configuration is the standard configuration that applies universally to both

EX4200 switches in your QFabric system control plane network.

[edit] groups { qfabric { chassis { aggregated-devices { ethernet { device-count 3;

}

} alarm { management-ethernet { link-down ignore;

}

} lcd-menu { fpc 0 { maintenance-menu disable;

}

}

} protocols { rstp { interface ae2.0 { mode point-to-point;

} interface all { edge; no-root-port;

} bpdu-block-on-edge;

} lldp { interface all;

}

} ethernet-switching-options { storm-control { interface all { bandwidth 10000;

}


45


}

} vlans { qfabric { vlan-id 100; dot1q-tunneling;

}

}

} qfabric-int { interfaces {

<*> { mtu 9216; unit 0 { family ethernet-switching { port-mode access; vlan { members qfabric;

}

}

}

}

}

} qfabric-ae { interfaces {

<*> { aggregated-ether-options { link-speed 1g; lacp { active;

}

}

}

}

}

} apply-groups [qfabric]; chassis { fpc 0 { pic 1 { sfpplus { pic-mode 10g;

}

}

}

} interfaces { interface-range Node_Device_Interfaces { member "ge-0/0/[0-15]"; description "QFabric Node Device"; mtu 9216; unit 0 { family ethernet-switching { port-mode access; vlan {


Chapter 2: Initial Setup for the QFX3000-M QFabric System members qfabric;

}

}

}

} interface-range Interconnect_Device_Interfaces { member "ge-0/0/[16-17]"; description "QFabric Interconnect Device"; mtu 9216; unit 0 { family ethernet-switching { port-mode access; vlan { members qfabric;

}

}

}

} interface-range Director_Device_DG0_LAG_Interfaces { member "ge-0/0/[20-21]"; description "QFabric Director Device - DG0"; ether-options { speed {

1g;

}

802.3ad ae0;

}

} interface-range Director_Device_DG1_LAG_Interfaces { member "ge-0/0/[22-23]"; description "QFabric Director Device - DG1"; ether-options { speed {

1g;

}

802.3ad ae1;

}

} interface-range Control_Plane_Inter_LAG_Interfaces {

} ae0 { apply-groups [ qfabric-int qfabric-ae ]; description "QFabric Director Device - DG0";

} member "ge-0/1/[0-1]"; description "QFabric Control Plane (Inter-Switch LAG)"; ether-options {

802.3ad ae2;

} ae1 { apply-groups [ qfabric-int qfabric-ae ]; description "QFabric Director Device - DG1";

} ae2 { description "QFabric Control Plane (Inter-Switch LAG)"; mtu 9216;


47

Configuring a QFX3000-M QFabric System aggregated-ether-options { link-speed 1g; lacp { active;

}

} unit 0 { family ethernet-switching { vlan { members qfabric;

}

}

}

}

} class-of-service { classifiers { ieee-802.1 onep_qfabric_classifier { forwarding-class class_3 { loss-priority low code-points [ 110 111 ];

} forwarding-class class_2 { loss-priority low code-points 100; loss-priority high code-points 101;


}

} inet-precedence IP_qfabric_classifier { forwarding-class class_3 { loss-priority low code-points [ 110 111 ];



}

}

} forwarding-classes { class class_3 queue-num 7; class class_2 queue-num 2; class class_0 queue-num 0;

} interfaces { ge-*/0/* { scheduler-map cpe_network_smap; unit 0 { classifiers { ieee-802.1 onep_qfabric_classifier; inet-precedence IP_qfabric_classifier;



}

}

} scheduler-maps { cpe_network_smap { forwarding-class class_3 scheduler scheduler_3; forwarding-class class_2 scheduler scheduler_2; forwarding-class class_0 scheduler scheduler_0;

}

} schedulers { scheduler_3 { buffer-size percent 30; priority strict-high;

} scheduler_2 { transmit-rate percent 75; buffer-size percent 30; priority low;

} scheduler_0 { transmit-rate percent 25; buffer-size percent 40; priority low;

}

}

}

}

} ae* { scheduler-map cpe_network_smap; unit 0 { classifiers { ieee-802.1 onep_qfabric_classifier; inet-precedence IP_qfabric_classifier;

}

}

The following portion of the configuration applies to the specific requirements of your management network. Modify this section to meet the needs of your network.

[edit] system { host-name qfabric-control-plane; services { ssh; telnet; web-management { http;

}

} syslog { user * { any emergency;

} file messages {


49

Configuring a QFX3000-M QFabric System any notice; authorization info; archive world-readable; explicit-priority;

} file interactive-commands { interactive-commands any;

} file secure { authorization info;

} file default-log-messages { any any; structured-data;

} file console { any error;

} time-format millisecond;

}

} interfaces { me0 { unit 0 { family inet { address 192.168.157.26/24;

}

}

}

} routing-options { static { route 0.0.0.0/0 next-hop 192.168.157.1;

}

}

To verify the syntax of your configuration prior to committing it, enter commit check from configuration mode. If you are done configuring the device, enter commit from configuration mode.

Verification

Confirm that the EX4200 switch configuration is working properly.

•

Verifying the QFX3000-M QFabric System Control Plane—EX4200 Switch

EX0 on page 50

•

Verifying the QFX3000-M QFabric System Control Plane—EX4200 Switch

EX1 on page 52

Verifying the QFX3000-M QFabric System Control Plane—EX4200 Switch EX0

Purpose Verify that the control plane is properly connected on your first EX4200 switch.



Action Connect to the Junos OS CLI of EX4200 switch EX0, either from your management network or from the console port of the switch. In operational mode, enter the show interfaces terse command.

Sample Output

user@ex0> show interfaces terse

Interface Admin Link Proto Local Remote ge-0/0/0 up up ge-0/0/0.0 up up eth-switch ge-0/0/1 up up ge-0/0/1.0 up up eth-switch ge-0/0/2 up up ge-0/0/2.0 up up eth-switch ge-0/0/3 up up ge-0/0/3.0 up up eth-switch ge-0/0/4 up up ge-0/0/4.0 up up eth-switch ge-0/0/5 up up ge-0/0/5.0 up up eth-switch ge-0/0/6 up up ge-0/0/6.0 up up eth-switch ge-0/0/7 up up ge-0/0/7.0 up up eth-switch ge-0/0/8 up down ge-0/0/8.0 up down eth-switch ge-0/0/9 up down ge-0/0/9.0 up down eth-switch ge-0/0/10 up down ge-0/0/10.0 up down eth-switch ge-0/0/11 up down ge-0/0/11.0 up down eth-switch ge-0/0/12 up down ge-0/0/12.0 up down eth-switch ge-0/0/13 up down ge-0/0/13.0 up down eth-switch ge-0/0/14 up down ge-0/0/14.0 up down eth-switch ge-0/0/15 up down ge-0/0/15.0 up down eth-switch ge-0/0/16 up up ge-0/0/16.0 up up eth-switch ge-0/0/17 up up ge-0/0/17.0 up up eth-switch ge-0/0/18 up down ge-0/0/18.0 up down eth-switch ge-0/0/19 up down ge-0/0/19.0 up down eth-switch ge-0/0/20 up up ge-0/0/20.0 up up aenet --> ae0.0

ge-0/0/21 up up ge-0/0/21.0 up up aenet --> ae0.0





vcp-0 up down


51

Configuring a QFX3000-M QFabric System vcp-0.32768 up down vcp-1 up down vcp-1.32768 up down ae0 up up ae0.0 up up eth-switch ae1 up up ae1.0 up up eth-switch ae2 up up ae2.0 up up eth-switch bme0 up up bme0.32768 up up inet 128.0.0.1/2

128.0.0.16/2

128.0.0.32/2

tnp 0x10 dsc up up gre up up ipip up up lo0 up up lo0.0 up up inet 127.0.0.1 --> 0/0 lsi up up me0 up up me0.0 up up inet 192.168.157.26/24 mtun up up pimd up up pime up up tap up up vlan up up vme up down

Meaning In the output of the show interfaces terse command, if all interfaces that connect to the

QFabric system devices are listed as up (such as ge-0/0/16 and ge-0/0/17 for the

Interconnect devices; ge-0/0/20 through ge-0/0/23 for the Director devices; ge-0/0/0 through ge-0/0/7 for the Node devices; and ge-0/1/0 and ge-0/1/1 for the inter-EX4200 switch connections), the control plane is properly connected.

Verifying the QFX3000-M QFabric System Control Plane—EX4200 Switch EX1

Purpose Verify that the control plane is properly connected on your second EX4200 switch.

Action Connect to the Junos OS CLI of EX4200 switch EX1, either from your management network or from the console port of the switch. In operational mode, enter the show interfaces terse command.

Sample Output

user@ex1> show interfaces terse

Interface Admin Link Proto Local Remote ge-0/0/0 up up ge-0/0/0.0 up up eth-switch ge-0/0/1 up up ge-0/0/1.0 up up eth-switch ge-0/0/2 up up ge-0/0/2.0 up up eth-switch ge-0/0/3 up up ge-0/0/3.0 up up eth-switch ge-0/0/4 up up ge-0/0/4.0 up up eth-switch ge-0/0/5 up up


Chapter 2: Initial Setup for the QFX3000-M QFabric System ge-0/0/5.0 up up eth-switch ge-0/0/6 up up ge-0/0/6.0 up up eth-switch ge-0/0/7 up up ge-0/0/7.0 up up eth-switch ge-0/0/8 up down ge-0/0/8.0 up down eth-switch ge-0/0/9 up down ge-0/0/9.0 up down eth-switch ge-0/0/10 up down ge-0/0/10.0 up down eth-switch ge-0/0/11 up down ge-0/0/11.0 up down eth-switch ge-0/0/12 up down ge-0/0/12.0 up down eth-switch ge-0/0/13 up down ge-0/0/13.0 up down eth-switch ge-0/0/14 up down ge-0/0/14.0 up down eth-switch ge-0/0/15 up down ge-0/0/15.0 up down eth-switch ge-0/0/16 up up ge-0/0/16.0 up up eth-switch ge-0/0/17 up up ge-0/0/17.0 up up eth-switch ge-0/0/18 up down ge-0/0/18.0 up down eth-switch ge-0/0/19 up down ge-0/0/19.0 up down eth-switch ge-0/0/20 up up ge-0/0/20.0 up up aenet --> ae0.0






vcp-0 up down vcp-0.32768 up down vcp-1 up down vcp-1.32768 up down ae0 up down ae0.0 up down eth-switch ae1 up down ae1.0 up down eth-switch ae2 up up ae2.0 up up eth-switch bme0 up up bme0.32768 up up inet 128.0.0.1/2

128.0.0.16/2

128.0.0.32/2

tnp 0x10 dsc up up gre up up ipip up up lo0 up up lo0.0 up up inet 127.0.0.1 --> 0/0


53

Configuring a QFX3000-M QFabric System lsi up up me0 up up me0.0 up up inet 192.168.157.26/24 mtun up up pimd up up pime up up tap up up vlan up up vme up down

Meaning In the output of the show interfaces terse command, if all interfaces that connect to the

QFabric system devices are listed as up (such as ge-0/0/16 and ge-0/0/17 for the

Interconnect devices; ge-0/0/20 through ge-0/0/23 for the Director devices; ge-0/0/0 through ge-0/0/7 for the Node devices; and ge-0/1/0 and ge-0/1/1 for the inter-EX4200 switch connections), the control plane is properly connected.

Related

Documentation

•

QFX3000-M QFabric System Installation Overview

•

Installing and Connecting a QFX3100 Director Device

•

Installing and Connecting a QFX3600 or QFX3600-I Device

•

Installing and Connecting a QFX3500 Device

•

Installing and Connecting an EX4200 Switch

•

Understanding the QFabric System Control Plane



Importing a QFX3000-M QFabric System Control Plane EX4200 Switch Configuration with a USB Flash Drive

There are two methods of importing the configuration file to the QFX3000-M QFabric system control plane EX4200 switches.

•

Download the configuration file onto a USB flash drive from the Juniper Networks software download site before inserting the USB flash drive into the EX4200 switch

USB port

•

Copy and paste the configuration from

“Example: Configuring EX4200 Switches for the QFX3000-M QFabric System Control Plane” on page 30

.

NOTE: The EX4200 switch configuration is the same for both the copper-based and fiber-based QFX3000-M QFabric system control plane networks.

Before you begin:

•

Rack, mount, and install your QFabric system hardware (Director group, Interconnect devices, and Node devices). For more information, see Installing and Connecting a

QFX3100 Director Device, Installing and Connecting a QFX3600 or QFX3600-I Device, and Installing and Connecting a QFX3500 Device.

•

Rack, mount, and install your EX4200 switches for the QFabric system control plane.

For more information, see Installing and Connecting an EX4200 Switch.

• Select a USB flash drive that meets the EX4200 switch USB port specifications. See

USB Port Specifications for an EX Series Switch.

•

Use a computer or other device to download the configuration file from the Internet and copy it to the USB flash drive.

To import the control plane EX4200 switch configuration file onto a USB flash drive:

1.

In a browser, go to http://www.juniper.net/support/downloads/junos.html

.

The Junos Platforms - Download Software page appears.

NOTE: To access the download site, you must have a service contract with Juniper Networks and an access account. If you need help obtaining an account, complete the registration form at the Juniper Networks website https://www.juniper.net/registration/Register.jsp

.

2.

In the QFX Series box, select QFX3000-M QFabric System.

The QFX3000-M QFabric System - Download Software page appears.

3.

Click the Software tab and select the software release number from the Release list that appears to the right of the Software tab.


55


A login screen appears.

4.

In the QFabric System Control Plane Network section, select QFX3000-M Control

Plane Network Configuration

.


5.

Enter your user ID and password and click Login.

6.

Read the End User License Agreement, select the I agree option button, and then click

Proceed .

7.

Save the configuration file onto the USB flash drive using your computer or other device.

8.

Insert the USB flash drive into the USB port on the EX4200 switch.

9.

Save the file to the /var/home/username directory on the EX4200 switch.

10.

Load the configuration file into the switch.

user@switch# load override filename

11.

Commit the configuration.

user@switch# commit

Load complete

12.

Remove the USB flash drive from the switch.

Related

Documentation

•


Plane on page 30

Generating the MAC Address Range for a QFabric System

Each QFabric system requires a range of reserved MAC addresses that is assigned by

Juniper Networks. You must specify the MAC address range when you perform the initial setup of the QFX3100 Director group. Additionally, refer to Activate Your QFabric System for more information.

When you purchase a QFabric system, you receive an e-mail containing a software serial number from Juniper Networks. You can use the software serial number to generate the

MAC address range for your QFabric system.



To generate the MAC address range for a QFabric system:

1.

In a browser, log in to the Juniper Networks License Management System at https://www.juniper.net/lcrs/license.do

.

The Manage Product Licenses page appears.

NOTE: To access the licensing site, you must have a service contract with

Juniper Networks and an access account. If you need help obtaining an account, complete the registration form at the Juniper Networks website https://www.juniper.net/registration/Register.jsp

.

2.

On the Generate Licenses tab, select QFX Series Product from the drop-down list, and click Go.

The Generate Licenses - QFX Series Product page appears.

3.

Select the QFX Series Product Fabric option button, and click Continue.

The Generate Licenses - QFX Series Product Fabrics page appears.

4.

In the Software Serial No field, enter the software serial number for your QFabric system, and press the Tab key.

The starting MAC address and number of MAC addresses for your QFabric system are displayed.

5.

(Optional) Click Download/Email MAC Address to download or e-mail the MAC address range.

The Download/Email MAC Address page appears.

To download the MAC address range:

•

Select the Download to this computer option button, and click OK.

To e-mail the MAC address range:

• Select the Send e-mail to e-mail ID option button, and click OK.

Related

Documentation

•

Performing the QFabric System Initial Setup on a QFX3100 Director Group

Performing the QFabric System Initial Setup on a QFX3100 Director Group

You must perform the initial setup of the QFX3100 Director group through the console port. (Before configuring the QFX3100 Director group, see Installing and Connecting a

QFX3100 Director Device.)

Before you begin connecting and configuring a QFX3100 Director group, set the following parameter values on the console server or PC:


57


•

Baud Rate—9600

• Flow Control—None

• Data—8

•

Parity—None

•

Stop Bits—1

• DCD State—Disregard

NOTE: When you use the SecureCRT client to connect to a Director device for the initial setup of a QFabric system, the backspace key does not work.

As a workaround, use the Shift+Delete key combination in SecureCRT as a backspace key equivalent or use a different UNIX client to support the backspace key natively.

The initial setup requires that you specify certain values for your QFabric system. These include:

•

Software serial number for your QFabric system (found in the e-mail containing the software serial number that you received from Juniper Networks when you purchased your QFabric system)

•

IP addresses and a default gateway IP address for your QFabric system default partition

•

IP addresses for your Director group device management ports

• Range of reserved MAC addresses for your QFabric system (see Generating the MAC

Address Range for a QFabric System or Activate Your QFabric System for this information)

• Root password for your Director group

•

Root password for the QFabric system components such as the Node devices,

Interconnect devices, and infrastructure

•

•

Performing an Initial Setup on page 58

Restoring a Backup Configuration on page 61

Performing an Initial Setup

The initial setup can be performed either manually or by using a previously saved backup configuration.

To connect and configure the QFX3100 Director group manually from the console:

1.

Connect the console port of one of the Director devices to a laptop or PC using an

RJ-45 to DB-9 rollover cable. An RJ-45 to DB-9 rollover cable is supplied with each

QFX3100 Director device. The console (CONSOLE) port is located on the front panel of the device.

2.

Log in as root. If the software booted before you connected to the console port, you might need to press the Enter key for the prompt to appear.


Chapter 2: Initial Setup for the QFX3000-M QFabric System dg0 login: root

NOTE: The prompt is either dg0 login or dg1 login depending on the Director device to which you connected your cable.

3.

For manual configuration or for initial installation, enter no when prompted to specify the backup file. The current Director device configuration is displayed.

4.

Initial Configuration

Before you can access the QFabric system, you must complete the initial setup of the Director group by using the steps that follow. If the initial setup procedure does not complete successfully, log out of the Director device and then log back in to restart this setup menu.

Continue? [y/n]: y

You may enter the configuration manually or restore from a backup.

Specify a backup file? [y/n]: n

Existing local configuration:

Enter the IP addresses and prefixes for both Director devices.

NOTE: The Director group devices and QFabric system default partition

IP addresses must be on the same subnet as your management network.

5.

Please enter the Director Group 0 IP address and prefix:

ip address/prefix

Please enter the Director Group 1 IP address and prefix:

ip address/prefix

Please enter the Director Group Subnet Mask:

subnet mask

Enter the gateway IP address for the Director group.

6.

Please enter the Director Group gateway IP address:

gateway ip address

Enter the default partition IP address. (You will use this address to log in to the QFabric system on subsequent connections.)

7.

Please enter the QFabric default partition IP address:

ip address

(Optional) Enter the IPv6 addresses for both Director devices and the gateway IPv6 address for the Director group.

8.

Would you like to input IPv6 addresses for Director Group nodes? (y/n): y

Please enter the Director Group 0 IPv6 address or 'y' to use /0:

IPv6 address

Please enter the Director Group 1 IPv6 address or 'y' to use /0:

IPv6 address

Please enter the Director Group gateway IPv6 address or 'y' to use /0 :

IPv6 address

Enter the MAC address information.

Please enter the starting MAC address:

mac address

Please enter the number of MAC addresses:

number of mac addresses

NOTE: The minimum number of MAC addresses accepted is 4000.

9.

Enter the QFabric system software serial number.

Please enter the QFabric serial ID:

serial id

10.

Create the Director device root password.

Please enter a Director device root password:

director-device-password

Please re-enter password:

director-device password


59


60

11.

Create a password for the QFabric system components.

NOTE: If you need to change the component password after the QFabric system is operational, issue the device-authentication statement at the

[edit system] hierarchy level in the QFabric default partition CLI.

Please enter a password for QFabric components (Node devices, Interconnect devices, and infrastructure):

component-password

Please re-enter password:

component-password

Note: please record your passwords for recovery purposes.

CAUTION: Carefully save your passwords for future reference, because some cannot be recovered on a QFabric system.

12.

Enter the QFabric system platform type.

Supported platform types:

1. QFX3000-G

2. QFX3000-M

Please select product type:

number corresponding to platform type

13.

Confirm the initial configuration. Ensure that the information is accurate before proceeding.

Does the following configuration appear correct?

Director Group 0 IPv4/Prefix [10.94.200.9/24]

Director Group 1 IPv4/Prefix [10.94.200.10/24]

Director Group IPv4 Gateway [10.94.200.250]

Director Group 0 IPv6/Prefix [2000:1:2:3::a5e:c809/64]

Director Group 1 IPv6/Prefix [2000:1:2:3::a5e:c80a/64]

Director Group IPv6 Gateway

[2000:0001:0002:0003:0226:88ff:fe7b:e880]

QFabric Default Partition (IPv6 address) [2000:1:2:3::0a5e:c802/64]

QFabric Serial ID [qfsn-0123456789]

Director Device Password [********]

NOTE: Only addresses of the IP version(s) you entered will appear in the configuration.

14.

Confirm the initial setup.

[y/n]: y

CAUTION: Resetting this initial configuration requires assistance from

Juniper Networks customer support or Performing a QFabric System

Recovery Installation on the Director Group. As a result, make sure you are certain the values you entered are correct before you enter yes.

15.

The director device displays the configuration.

Saving temporary configuration...

Configuring peer...



Configuring local interfaces...

Configuring interface eth0 with [10.49.214.74/24:10.49.214.254]

Configured interface eth0 with [10.49.214.74/24:10.49.214.254]

Configuring QFabric software with an initial pool of 4000 MAC addresses

[00:11:00:00:00:00 - 00:11:00:00:0f:3b]

Configuring QFabric address [10.49.214.150]

Reconfiguring QFabric software static configuration

Applying the new Director device password

Applying the QFabric component password

First install initial configuration, generating and sharing SSH keys.

First install initial configuration, generating SSH keys.

Configuration complete. Director Group services will auto start within 30 seconds.

Restoring a Backup Configuration

Before you restore a backup configuration for the Director group:

•

You must have a backup configuration file. You create the backup file with the request system software configuration-backup command and save it on an external USB flash drive.

•

If you need to reinstall the system software, perform that operation first (see Performing

a QFabric System Recovery Installation on the Director Group).

To connect and configure the Director group with a backup configuration:

1.

Log in as root. If the software booted before you connected to the console port, you might need to press the Enter key for the prompt to appear.

dg0 login: root

NOTE: The prompt is either dg0 login or dg1 login depending on the Director device to which you connected your cable.

2.

To use a previously saved backup configuration, enter yes when prompted to specify the backup file and then enter the path and filename of the backup configuration.

3.

Specify a back up file? [y/n]: y

Please specify the full path of the configuration backup file:

path/filename

Confirm the restoration of the configuration from the backup. Ensure that the information is accurate before proceeding.

Does the following configuration appear correct?

Director Group 0 IP/Prefix [10.49.214.74/24]

Director Group 1 IP/Prefix [10.49.214.75/24]

Director Group Gateway [10.49.214.254]

Starting MAC address [00:11:00:00:00:00]

Number of MAC addresses [4000]

QFabric Default Partition IP [10.49.214.150]

QFabric serial ID [qfsn-123456789]

Director Device Password [********]

QFabric component Password [********]

Product Type: [QFX3000-G]

4.

Confirm the backup restoration.

[y/n]: y


61


The Director device displays the configuration.


Configuring peer...




Configuring QFabric software with an initial pool of 4000 MAC addresses

[00:11:00:00:00:00 - 00:11:00:00:0f:3b]



Applying the new Director device password



Related

Documentation

•

Generating the MAC Address Range for a QFabric System

•

Gaining Access to the QFabric System Through the Default Partition

•

QFabric System Initial and Default Configuration Information

•

Installing and Connecting a QFX3100 Director Device

•

Performing a QFabric System Recovery Installation on the Director Group

•

request system software configuration-backup

•

device-authentication


CHAPTER 3

QFabric System Configuration

•

Understanding QFabric System Administration Tasks and Utilities on page 63

•

Gaining Access to the QFabric System Through the Default Partition on page 67

•

Example: Configuring QFabric System Login Classes on page 68

•

Configuring Node Groups for the QFabric System on page 76

•

Configuring the Port Type on QFX3600 Node Devices on page 81

•

Configuring the QSFP+ Port Type on QFX5100 Devices on page 85

•

Example: Configuring SNMP on page 87

•

Example: Configuring System Log Messages on page 90

•

Configuring Graceful Restart for QFabric Systems on page 92

•

Optimizing the Number of Multicast Flows on QFabric Systems on page 96

Understanding QFabric System Administration Tasks and Utilities

The following items describe QFabric system components, common administration tasks that you perform on the QFabric system, or utilities that help you to manage the QFabric system and its components.


63


•

Converting the device mode (QFX3500 and QFX3600 devices)—Enables you to convert a QFX3500, QFX3600, or QFX5100 device into a Node device so it can be deployed within a QFabric system. By default, QFX3500, QFX3600, and QFX5100 devices operate in standalone mode. Before the devices can participate within a QFabric system environment, you must change the device mode for the switch to node-device mode. To convert a QFX3500, QFX3600, or QFX5100 device from standalone mode to Node device mode, connect to the console port of the device, issue the request chassis device-mode node-device command, verify the future device mode with the show chassis device-mode command, connect the management port of the device to the QFabric system control plane, and reboot the device.

•

NOTE:

Before you convert the device mode, you must upgrade the software on your standalone device to a QFabric system Node and Interconnect device software package that matches the QFabric system complete software package used by your QFabric system. For example, if the complete software package for your QFabric system is named jinstall-qfabric-11.3X30.6.rpm

, you need to install the jinstall-qfx-11.3X30.6-domestic-signed.tgz

package on your standalone device. Matching the two software packages ensures a smooth and successful addition of the device to the QFabric system inventory.

• Converting the device mode erases the switch configuration. We recommend that you save your configuration to an external server or

USB flash drive before executing the device mode conversion commands and rebooting the switch.

• QFabric system control plane Ethernet network (EX4200 switches to support the

QFabric system)—Provides a separate control plane network within the QFabric system to handle management traffic. This design enables the data plane network to focus on efficient, low-latency delivery of data, voice, and video traffic.

• The QFX3000-G QFabric system control plane uses two sets of four EX4200 switches each, configured as a pair of Virtual Chassis to connect all components within the

QFabric system. The dual Virtual Chassis architecture provides redundancy and high availability to ensure reliable QFabric system operation for the Director group, the

Interconnect devices, and the Node devices.

•

The QFX3000-M QFabric system control plane uses two EX4200 switches to connect all components within the QFabric system. The two EX4200 switches provide redundancy and high availability to ensure reliable QFabric system operation for the

Director group, the Interconnect devices, and the Node devices.

Because the level of detail necessary to fully understand the control plane connections, cabling, topology, and configuration is beyond the scope of this topic, see:

•

Example: Configuring the Virtual Chassis for a Copper-Based QFX3000-G QFabric

System Control Plane for information about a QFX3000-G QFabric system with a copper-based control plane


Chapter 3: QFabric System Configuration

•

Example: Configuring EX Series Switches for the QFX3000-M QFabric System Control

Plane for information about a QFX3000-M QFabric system with a copper or fiber-based control plane

• QFabric system data plane network—Provides a separate network to handle rapid delivery of data plane traffic. The data plane uses QSFP+ interfaces and fiber-optic cabling to connect QFabric system components at speeds of 40 Gbps. By creating a redundant set of connections between the Node devices and the backplane-like

Interconnect devices, the data plane enables the Node devices to appear as if they are directly connected to one another in a single tier. To view the connection status of the

QFabric system data plane, issue the show chassis fabric connectivity command.

•

Director group (QFX3100 Director devices within a QFabric system)—Provides a redundant, resilient platform that manages the QFabric system components. Two

QFX3100 Director devices work together to ensure high availability of the system and load-balance system processes, such as the command-line interface (CLI) and shared storage. To configure the Director group for operation, install and cable two Director devices as a Director group, connect to the console port of one of the Director devices, and perform the initial setup. The setup script starts automatically the first time you power on the Director device. For more information, see Performing the QFabric System

Initial Setup on a QFX3100 Director Group. To monitor the status of the Director group, log in to the QFabric system default partition and issue the show fabric administration inventory director-group status command.

• Automatic detection and configuration of QFabric system components—Enables

QFabric system components to join the QFabric system automatically. When you install the QFabric system, activate the control plane and Director group, and power on the Node and Interconnect devices, the Director group recognizes these devices, sends each device its own portion of the Junos OS configuration, and adds them to the QFabric system inventory. By default, each individual Node device is placed into a unique server Node group that contains only that single Node device. No configuration is required for the default assignments. The default settings can be overridden when you add Node devices into a redundant server Node group (containing a pair of Node devices) or a network Node group (that can contain up to eight Node devices, run routing protocols, and connect to external networks).

•

QFabric system Routing Engines—Support the QFabric system by providing virtual, redundant instances of Junos OS that run on the Director group. The Routing Engines perform fabric management tasks, maintain control of the fabric, and host the operation of routing protocols for network Node groups. Because they are generated in pairs, the

Routing Engines provide additional high availability for the QFabric system. No configuration is required. To view the status of the QFabric system Routing Engines, issue the show fabric administration inventory infrastructure command.

• QFabric system command-line interface—Enables you to configure all components of the QFabric system from a single location by using the Junos OS CLI. To access this central location, you need to log in to the QFabric system default partition (an IP address you specify during the initial setup of the Director group). For more information, see Performing the QFabric System Initial Setup on a QFX3100 Director Group.

Most existing Junos OS configuration statements and operational mode commands are supported (for example, interfaces, VLANs, protocols, and firewall filters).


65


66

To view QFabric system components and check connectivity of the system, issue the show fabric administration inventory commands.

• Alias configuration for Director devices, Interconnect devices, and Node devices—Enables you to set user-defined aliases for QFabric system Director devices,

Interconnect devices, and Node devices to facilitate usability of the QFabric system as it scales. Aliased names appear in the output of many QFabric system operational commands, such as show fabric administration inventory. To map the hardware serial number of a Director device, Interconnect device or Node device to a user-defined name, see Configuring Aliases for the QFabric System.

•

Node group configuration—Enables you to cluster several Node devices together to provide redundancy, resiliency, and high availability at the ingress and egress points of the QFabric system. There are two types of Node groups you can configure:

• Redundant server Node group—Enables the grouped Node devices to connect the

QFabric system to local servers and storage devices. A redundant server Node group can contain a maximum of two Node devices and supports LAG connections that can span both devices.

NOTE: The Node devices in a redundant server Node group must be of the same type, such as a QFX3500 Node, a QFX3600 Node, or a QFX5100

Node. For example, you cannot add a QFX3500 and a QFX3600 Node device to the same redundant server Node group.

•

Network Node group—Enables the grouped Node devices to connect the QFabric system to external networks and run routing protocols such as BGP and OSPF. A network Node group can contain up to eight Node devices and supports LAG connections.

•

NOTE:

The name of the network Node group in the default partition, NW-NG-0, is preset. You must use this name when adding Node devices to the network Node group. You cannot specify a different name.

•

When you configure routing protocols on the QFabric system, you must use interfaces from the Node devices assigned to the network Node group. If you try to configure routing protocols on interfaces from the

Node devices assigned to server Node groups, the configuration commit operation fails.

To configure a redundant server Node group, include two Node devices with the node-device node-device-name statement at the [edit fabric resources node-group

node-group-name] hierarchy level.

To configure a network Node group, include the network-domain statement at the [edit fabric resources node-group NW-NG-0] hierarchy level. In addition, include between two and eight Node devices with the node-device node-device-name statement at the

[edit fabric resources node-group NW-NG-0] hierarchy level.



Related

Documentation

•

Converting the Device Mode for a QFabric System Component

•

Example: Configuring the Virtual Chassis for a Copper-Based QFX3000-G QFabric System

Control Plane

•

Example: Configuring EX Series Switches for the QFX3000-M QFabric System Control

Plane

•

show chassis fabric connectivity

•


•

show fabric administration inventory director-group status

•

show fabric administration inventory infrastructure

•

show fabric administration inventory

•

Configuring Aliases for the QFabric System

•

Configuring Node Groups for the QFabric System

Gaining Access to the QFabric System Through the Default Partition

This topic explains how to log in to the QFabric system default partition so you can access the Junos OS command-line interface (CLI) and configure the system.

Before you access the QFabric system default partition:

• Install the QFabric system hardware components, including connecting the network and power cables.

•

Convert any QFX3500 and QFX3600 standalone devices to node-device mode.

• Connect all components to the control plane Ethernet network.

• Turn on the Director group and run the initial setup script. Remember to write down the IP address of the default partition, which must be on the same subnetwork as your management network.

To access the default partition:

1.

Open an SSH connection to the QFabric default partition. Use the IP address you set for the default partition as part of the QFabric initial setup procedure. In your network, you can simplify access to the QFabric system by mapping the default partition IP address to a name.

[root@customer ~]# ssh [email protected]

Last login: Fri Sep 2 21:34:54 2011 from customer

Juniper QFabric Director 11.3.5043 2011-08-26 18:05:21 UTC

RUNNING ON DIRECTOR DEVICE : dg1 root@qfabric>


67


NOTE: The QFabric system is load balanced, so the CLI session might be hosted on either Director device DG0 or DG1.

2.

Enter configuration mode (the default mode in the QFabric system is configure private), configure a root password and hostname for the default partition, and assign QFabric administrator privileges to the root user.

root@qfabric> configure warning: Using private edit on QF/Director warning: uncommitted changes will be discarded on exit

Entering configuration mode

[edit] root@qfabric# set system root-authentication plain-text-password

New password: My-Password

Retype new password: My-Password root@qfabric# set system root-authentication remote-debug-permission qfabric-admin root@qfabric# set system host-name my-qfabric

[edit] root@qfabric# commit commit complete

[edit] root@my-qfabric#

3.

Configure your QFabric system as needed. You can configure routing protocols, interfaces, VLANs, and other features as needed. Keep in mind that interfaces require the four-level interface naming convention (device-name:fpc/pic/port).

Related

Documentation

•


•

QFabric System Initial and Default Configuration Information

•

•

Understanding Interfaces on the QFabric System

Example: Configuring QFabric System Login Classes

This example shows you how to assign the correct login class to users so they can access components within a QFabric system.

•


•

Overview on page 69

•


•

Verification on page 72



Requirements

Overview


•

One QFX3000-G QFabric system containing:

•

Two QFX3100 Director devices

• Two QFX3008-I Interconnect devices

• Eight QFX3500 Node devices

•

Junos OS Release 12.2 for these QFX Series components

•

Eight EX4200 switches, used to make two redundant Virtual Chassis with four members apiece

• Junos OS Release 12.1R1.9 for the EX Series switches used in the Virtual Chassis

Before you begin:

• Perform the initial setup of the QFabric system on the Director group, which includes the creation of a username and password for the QFabric system components. See

Performing the QFabric System Initial Setup on a QFX3100 Director Group.

The QFabric system offers three special preset login classes that provide different levels of access to individual components within a QFabric system (such as Node devices and

Interconnect devices). The qfabric-admin class provides the ability to log in to individual

QFabric system components and manage them. The qfabric-operator class enables the user to log in to individual components and view component-level operations and configurations. The qfabric-user class prevents access to individual QFabric system components.

You include these classes in your configuration at the [edit system login user username authentication remote-debug-permission] hierarchy level. The key task is to decide which class you should apply to users based on their need to access QFabric system components.

NOTE: To set QFabric system login classes for a root user, include the remote-debug-permission statement at the [edit system root-authentication] hierarchy level and specify the qfabric-admin class.

If you assign the qfabric-admin or the qfabric-operator class to a user, the QFabric system maps the user to a list of authorized users who are permitted to access components. To facilitate ease of use, the QFabric system uses the component password you specified during the initial setup of the Director group. When users assigned the qfabric-admin or the qfabric-operator class log in to a component by issuing the request component login operational mode command, the QFabric system verifies the class and sends the


69

Configuring a QFX3000-M QFabric System username and password to the component. The component accepts these credentials and permits access.

NOTE:

• The three QFabric system login classes give access to the components only. To provide access to the QFabric system as a whole through the default partition command-line interface (CLI), you must configure the usual Junos OS login classes or permissions (such as the super-user class).

For more information about login classes, see Junos OS Login Classes

Overview.

•

If you have completed the QFabric system initial setup and the system is operational, you can change the component password by issuing the device-authentication statement at the [edit system] hierarchy level in the

QFabric default partition CLI.

Topology

This example defines three users: Adam, Oscar, and Ulf. Adam needs to manage QFabric system components, Oscar needs limited access, and Ulf should not have any access to the components. As a result, assign the qfabric-admin class to Adam, the qfabric-operator class to Oscar, and the qfabric-user class to Ulf. However, all three users should have all permissions to access the QFabric system CLI.

Configuration

CLI Quick

Configuration

•

[xref target has no title]

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level.

set system login class all-qfabric permissions all set system login user Adam class all-qfabric set system login user Adam authentication encrypted-password

"$1$aoYSFkvE$G/dYqsTV5iSvVW2sND69U." set system login user Adam authentication remote-debug-permission qfabric-admin set system login user Oscar class all-qfabric set system login user Oscar authentication encrypted-password

"$1$3e.3wJQ8$31SrzV0.efdRbk.ZJncKm0" set system login user Oscar authentication remote-debug-permission qfabric-operator set system login user Ulf class all-qfabric set system login user Ulf authentication encrypted-password

"$1$qt9Ncm0o$okNYSN8O4fVITE/SHBdYj0" set system login user Ulf authentication remote-debug-permission qfabric-user



Step-by-Step

Procedure



To provide the same access to the QFabric system CLI for all users, but different QFabric system component-level access to different users:

1.

2.

3.

4.

Define and provide all-qfabric access and passwords to all three users. This administrator-defined class provides full permissions, enabling the users to log in to the QFabric system default partition and use the CLI. Alternatively, you can assign the super-user class to these users to accomplish the same goal.

[edit] user@qfabric# set system login class all-qfabric permissions all user@qfabric# set system login user Adam class all-qfabric user@qfabric# set system login user Adam authentication encrypted-password

"$1$aoYSFkvE$G/dYqsTV5iSvVW2sND69U." user@qfabric# set system login user Oscar class all-qfabric user@qfabric# set system login user Oscar authentication encrypted-password

"$1$3e.3wJQ8$31SrzV0.efdRbk.ZJncKm0" user@qfabric# set system login user Ulf class all-qfabric user@qfabric# set system login user Ulf authentication encrypted-password

"$1$qt9Ncm0o$okNYSN8O4fVITE/SHBdYj0"

Provide qfabric-admin component access to Adam so he can manage QFabric system components.

[edit] user@qfabric# set system login user Adam authentication remote-debug-permission qfabric-admin

Provide qfabric-operator component access to Oscar so he can view the CLI at the

QFabric system components.

[edit] user@qfabric# set system login user Oscar authentication remote-debug-permission qfabric-operator

Assign qfabric-user component restrictions to Ulf to prevent him from accessing the QFabric system components.

[edit] user@qfabric# set system login user Ulf authentication remote-debug-permission qfabric-user

Results From configuration mode, confirm your configuration by entering the show command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.

For brevity, this show command output includes only the configuration that is relevant to this example.

[edit] system { login { class all-qfabric {


71


}

} permissions all;

} user Adam { class all-qfabric; authentication { encrypted-password "$1$aoYSFkvE$G/dYqsTV5iSvVW2sND69U."; ##

SECRET-DATA remote-debug-permission qfabric-admin;

}

} user Oscar { class all-qfabric; authentication { encrypted-password "$1$3e.3wJQ8$31SrzV0.efdRbk.ZJncKm0"; ## SECRET-DATA remote-debug-permission qfabric-operator;

}

} user Ulf { class all-qfabric; authentication { encrypted-password "$1$qt9Ncm0o$okNYSN8O4fVITE/SHBdYj0"; ##

SECRET-DATA remote-debug-permission qfabric-user;

}

}

If you are done configuring the device, enter commit from configuration mode.

Verification

Confirm that the QFabric system and component-level access configuration is working properly for all three users. Adam, Oscar, and Ulf should have equivalent, full-permission access to the QFabric system CLI. Adam should have management-level access to components. Oscar should have read-only access to components. Ulf should have no component-level access.

•

Verifying qfabric-admin Access on page 72

•

Verifying qfabric-operator Access on page 74

•

Verifying qfabric-user Access on page 75

Verifying qfabric-admin Access

Purpose Verify that Adam can access the QFabric system CLI at the default partition and manage

QFabric system components.

Action From a management station on your network, issue the ssh user@qfabric command and enter the password to open an SSH session for Adam to the QFabric system. Issue the

?

command to view the CLI operational mode commands that Adam has permission to use on the QFabric system default partition.

> ssh [email protected]

Warning: Permanently added 'qfabric.network.net' (RSA) to the list of known hosts.



[email protected]'s password:

Last login: Sun Nov 20 14:12:29 2011 from 192.168.28.19


RUNNING ON DIRECTOR DEVICE : dg0

Adam@qfabric>

Adam@qfabric> ?

Possible completions:

clear Clear information in the system

configure Manipulate software configuration information

file Perform file operations

help Provide help information

load Load information from file

op Invoke an operation script

ping Ping remote target

quit Exit the management session

request Make system-level requests

restart Restart software process

save Save information to file

set Set CLI properties, date/time, craft interface message

show Show system information

telnet Telnet to another host

test Perform diagnostic debugging

traceroute Trace route to remote host

Issue the request component login ? command to view the components that Adam can access. Next, issue the request component login component-name command to log in to a Node device without being prompted for a username or password.

Adam@qfabric> request component login ?


<[Enter]> Execute this command

<node-name> Inventory name for the remote node

BBAK0372 Node device

BBAK0394 Node device

DRE-0 Diagnostic routing engine

EE3093 Node device

FC-0 Fabric control

FC-1 Fabric control

FM-0 Fabric manager

NW-NG-0 Node group

WS001/RE0 Interconnect device control board

WS001/RE1 Interconnect device control board

| Pipe through a command

Adam@qfabric> request component login EE3093

Warning: Permanently added 'qfnode-ee3093,169.254.128.14' (RSA) to the list of known hosts.

--- JUNOS 11.3I built 2011-11-04 12:46:16 UTC

{master}

Finally, issue the ? command to view the CLI operational mode commands that Adam has the permission to use on the Node device. Notice that the CLI prompt now indicates

Adam’s component access level (qfabric-admin) as the username and the Node device identifier (EE3093) as the host.

qfabric-admin@EE3093> ?




73


74




monitor Show real-time debugging information

mtrace Trace multicast path from source to receiver









ssh Start secure shell on another host

start Start shell




Meaning The output shows that Adam has received the proper permissions to access the QFabric system CLI and log in to individual components with management-level access.

Verifying qfabric-operator Access

Purpose Verify that Oscar can access the QFabric system CLI at the default partition and view the CLI on the QFabric system components.

Action From a management station on your network, issue the ssh user@qfabric command and enter the password to open an SSH session for Oscar to the QFabric system. Issue the

?

command to view the CLI operational mode commands that Oscar has permission to use on the QFabric system default partition. Notice that these permissions are the same as those given to Adam.







Oscar@qfabric>

Oscar@qfabric> ?




















Issue the request component login component-name command to log in to a Node device without being prompted for a username or password.

Oscar@qfabric> request component login EE3093

Warning: Permanently added 'qfnode-ee3093,169.254.128.14' (RSA) to the list of known hosts.

--- JUNOS 11.3I built 2011-11-04 12:46:16 UTC

{master}

Finally, issue the ? command to view the CLI operational mode commands that Oscar has permission to use on the Node device. Notice that the CLI prompt now indicates

Oscar’s component access level (qfabric-operator) as the username and the Node device identifier (EE3093) as the host. Additionally, Oscar has fewer CLI commands available than Adam because of Oscar’s read-only qfabric-operator login class.

qfabric-operator@EE3093> ?











start Start shell


Meaning The output shows that Oscar has full permissions to access the QFabric system CLI, but only read-only access when he logs in to individual components. Oscar’s permissions on the QFabric system are the same as Adam’s, but Oscar has fewer permissions than Adam on the Node device.

Verifying qfabric-user Access

Purpose Verify that Ulf has full access to the QFabric system CLI at the default partition but cannot access the QFabric system components.

Action From a management station on your network, issue the ssh user@qfabric command and enter the password to open an SSH session for Ulf to the QFabric system. Issue the ?

command to view the CLI operational mode commands that Ulf has permission to use on the QFabric system default partition. Notice that these permissions are the same as those given to Adam and Oscar.








75


Ulf@qfabric>

Ulf@qfabric> ?


















When Ulf issues the request component login component-name command, the Node device denies his access attempt.

Ulf@qfabric> request component login EE3093 error: User Ulf does not have sufficient permissions to login to device EE3093

Meaning The output shows that Ulf has full permissions to access the QFabric system CLI in the same way as Adam and Oscar. However, unlike Adam and Oscar, Ulf cannot access individual components because of the qfabric-user login class assigned to him.

Related

Documentation

•

Understanding QFabric System Login Classes

•

remote-debug-permission

•

request component login

•


•

Junos OS Login Classes Overview

Configuring Node Groups for the QFabric System

This topic explains how to configure Node groups for Node devices within the QFabric system. Node groups provide redundancy for Node devices and make your QFabric system more resilient.

There are three types of Node groups in a QFabric system:

• Automatically generated server Node groups—By default, every Node device that joins the QFabric system is placed within an automatically generated server Node group that contains one Node device (the device itself). Server Node groups connect to servers and storage devices.



•

Network Node groups—You can assign up to eight Node devices to a network Node group. When grouped together, the Node devices within a network Node group connect to other routers running routing protocols such as OSPF and BGP.

•

Redundant server Node groups—You can assign two Node devices to a redundant server Node group. When grouped together, you can create link aggregation groups

(LAGs) that span the interfaces on both Node devices to provide resiliency and redundancy.

Before you create Node groups in a QFabric system:

• Make sure your QFabric system is operational.

•

Issue the show fabric administration inventory node-devices command to display the

Node devices that are available to add to a Node group.

•

Issue the show fabric administration inventory node-groups command to display the existing Node groups.

NOTE: The following rules apply to QFabric Node group naming:

• Node group names must use alphabetic (A through Z and a through z), numeric (0 through 9), or dash (-) characters.

•

The maximum length of a Node group name is 30 characters.

•

Node group names are case sensitive. For example, MY-NG-1 and my-ng-1 refer to different components.

• You cannot use the reserved names all, fabric, or director-group as a Node group name.

NOTE: If you attempt to commit all configuration settings for a new Node group (such as the Node group itself, aliasing, and other features) at the same time, the commit operation might appear to succeed when it actually has failed. For this reason, we recommend configuring and verifying Node groups and aliases first, followed by configuring and verifying other features.

Establishing the Node groups and aliases first enables the QFabric system to reject any potentially unsupported configuration. The resulting commit errors indicate where the configuration problem lies. To verify the establishment of Node groups and aliases before configuring other features, issue the show fabric administration inventory command.

To display an automatically generated server Node group:

• Issue the show fabric administration inventory node-groups command and look for

Node groups containing a single Node device that has the same name or serial number as the server Node group.

root@qfabric> show fabric administration inventory node-groups


77


Item Identifier Connection Configuration

Node group

BBAK8281 Connected Configured

BBAK8281 Connected

BBAK8835 Connected Configured

BBAK8835 Connected

NW-NG-0 Connected Configured

Node0 BBAK8309 Connected


S1 Connected Configured



To create a network Node group:

1.

Specify the Node devices you wish to add to the network Node group by including the node-device statement at the [edit fabric resources node-group NW-NG-0] hierarchy level.

•

NOTE:

The network Node group must use the predefined name NW-NG-0. You must use this name when adding Node devices to the network Node group. You cannot specify a different name. Also, you can configure only one network Node group per partition.

•

When you configure routing protocols on the QFabric system, you must use interfaces from the Node devices assigned to the network Node group. If you try to configure routing protocols on interfaces from the

Node devices assigned to server Node groups, the configuration commit operation fails.

[edit]

2.

root@qfabric# set fabric resources node-group NW-NG-0 node-device Node0 root@qfabric# set fabric resources node-group NW-NG-0 node-device Node1

To designate the Node group as a network Node group, include the network-domain statement at the [edit fabric resources node-group NW-NG-0] hierarchy level.

[edit]

3.

root@qfabric# set fabric resources node-group NW-NG-0 network-domain

Review your configuration and issue the commit command.

[edit] root@qfabric# show fabric resources {

node-group NW-NG-0 {

network-domain;

node-device Node0;

node-device Node1;

}

}




NOTE: When you add or delete Node devices from a Node group configuration, the corresponding Node devices reboot when you commit the configuration change.

4.

To determine if your network Node group is operational, issue the show fabric administration inventory node-groups command in operational mode.

root@qfabric> show fabric administration inventory node-groups NW-NG-0


Node group

NW-NG-0 Connected Configured



To create a redundant server Node group:

1.

Specify the two Node devices you wish to add to the redundant server Node group by including the node-device statement at the [edit fabric resources node-group

node-group-name] hierarchy level.

NOTE: Ensure that the two Node devices are of the same type, either two

QFX3500 Node devices, two QFX3600 Node devices, or two QFX5100

Node devices. You cannot add different Node device types to the same redundant server Node group.

[edit]

2.

root@qfabric# set fabric resources node-group S1 node-device Node2 root@qfabric# set fabric resources node-group S1 node-device Node3


[edit] root@qfabric# show fabric resources {

node-group S1 {

node-device Node2;

node-device Node3;

}

}


NOTE: When you add or delete Node devices from a Node group configuration, the corresponding Node devices reboot when you commit the configuration change.


79


3.

To determine if your redundant server Node groups are operational, issue the show fabric administration inventory node-groups redundant-server-node-group-name command in operational mode.

root@qfabric> show fabric administration inventory node-groups S1


Node group

S1 Connected Configured



Related

Documentation

•

show fabric administration inventory node-groups

•

show fabric administration inventory node-devices

•

Understanding Node Groups

•

node-group (Resources)



Configuring the Port Type on QFX3600 Node Devices

The QFX3600 Node device provides 16 40-Gbps QSFP+ ports. By default, four ports

(labeled Q0 through Q3) operate as 40-gigabit data plane (fte) uplink ports for uplink connections between your Node device and your Interconnect devices. Twelve ports

(labeled Q4 through Q15) operate as 10-Gigabit Ethernet (xe) ports to support 48

10-Gigabit Ethernet interfaces for connections to either endpoint systems or external networks. Optionally, you can choose to configure ports Q0 through Q7 to operate as

40-gigabit data plane uplink ports, and ports Q2 through Q15 to operate as 10-Gigabit

Ethernet or 40-Gigabit Ethernet (xle) ports.

NOTE: You can use QSFP+ to four SFP+ breakout cables or QSFP+ transceivers with fiber breakout cables to connect the 10-Gigabit Ethernet ports to other devices.

NOTE: When you delete the port type configuration for an individual port or a block of ports, the ports return to operating in their default port type. For example, when you delete the 40-Gigabit Ethernet (xle) port configuration for port Q4, the port returns to operating as a 10-Gigabit Ethernet (xe) port.

NOTE: When the 40-Gigabit Ethernet (xle) ports of a QFX3600 Node device carry traffic at the full line rate, loss of untagged Layer 2 or Layer 3 traffic going across the fabric might occur, as well as increased latency on the Node device. Such effects result from the addition of a 4-byte header to packets traversing the uplink ports on the Node device. The percentage of traffic loss depends on the size of the packets: the greater the packet size, the lower the traffic loss and vice versa. This problem does not affect tagged traffic.

This topic explains how to configure the port type on QFX3600 Node devices.

Before you configure the port type on QFX3600 Node devices:

• Make sure your QFabric system is operational.

•

Issue the show fabric administration inventory node-groups command to display the existing Node groups and the Node devices in each Node group.

NOTE:

•

Only ports Q0 through Q7 can be configured to operate as 40-gigabit data plane (fte) uplink ports.

• Only ports Q2 through Q15 can be configured to operate as 10-Gigabit

Ethernet (xe) or 40-Gigabit Ethernet (xle) ports.


81


82

CAUTION: The Packet Forwarding Engine on the QFX3600 Node device is restarted when you commit the port type configuration changes. As a result, you might experience packet loss on the Node device.

The following message may be displayed in the system log file when the

Packet Forwarding Engine is restarted. You can ignore this message.

Pipe write error: Broken pipe flush operation failed

The following steps describe how to configure either a block of ports or an individual port to operate as 40-gigabit data plane uplink (fte) ports, as well as how to delete a

40-gigabit data plane uplink (fte) port configuration.

1.

To configure a block of ports to operate as 40-gigabit data plane uplink (fte) ports, specify a port range:

[edit chassis node-group name node-device name pic 1] root@qfabric# set fte port-range port-range-low port-range-high

For example, to configure ports Q4 through Q7 to operate as 40-gigabit data plane uplink ports:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 1] root@qfabric# set fte port-range 4 7

2.

To configure an individual port to operate as a 40-gigabit data plane uplink (fte) port, specify a port number:

[edit chassis node-group name node-device name pic 1] root@qfabric# set fte port port-number

For example, to configure port Q4 to operate as a 40-gigabit data plane uplink port:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 1] root@qfabric# set fte port 4

3.



4.

To delete the 40-gigabit data plane uplink (fte) port configuration for a block of ports, specify a port range:

[edit chassis node-group name node-device name pic 1] root@qfabric# delete fte port-range port-range-low port-range-high

For example, to delete the 40-gigabit data plane uplink port configuration for ports

Q4 through Q7:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 1] root@qfabric# delete fte port-range 4 7



5.

To delete the 40-gigabit data plane uplink (fte) port configuration for an individual port, specify a port number:

[edit chassis node-group name node-device name pic 1] root@qfabric# delete fte port port-number

For example, to delete the 40-gigabit data plane uplink port configuration for port

Q4:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 1] root@qfabric# delete fte port 4

The following steps describe how to configure either a block of ports or an individual port to operate as 10-Gigabit Ethernet (xe) ports, as well as how to delete a 10-Gigabit Ethernet

(xe) port configuration.

1.

To configure a block of ports to operate as 10-Gigabit Ethernet (xe) ports, specify a port range:

[edit chassis node-group name node-device name pic 0] root@qfabric# set xe port-range port-range-low port-range-high

For example, to configure ports Q4 through Q7 to operate as 10-Gigabit Ethernet ports:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 0] root@qfabric# set xe port-range 4 7

2.

To configure an individual port to operate as a 10-Gigabit Ethernet port, specify a port number:

[edit chassis node-group name node-device name pic 0] root@qfabric# set xe port port-number

For example, to configure port Q4 to operate as a 10-Gigabit Ethernet port:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 0] root@qfabric# set xe port 4

3.



4.

To delete the 10-Gigabit Ethernet (xe) port configuration for a block of ports, specify a port range:

[edit chassis node-group name node-device name pic 0] root@qfabric# delete xe port-range port-range-low port-range-high

For example, to delete the 10-Gigabit Ethernet port configuration for ports Q4 through

Q7:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 0] root@qfabric# delete xe port-range 4 7

5.

To delete the 10-Gigabit Ethernet (xe) port configuration for an individual port, specify a port number:

[edit chassis node-group name node-device name pic 0] root@qfabric# delete xe port port-number


83


For example, to delete the 10-Gigabit Ethernet port configuration for port Q4:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 0] root@qfabric# delete xe port 4

The following steps describe how to configure either a block of ports or an individual port to operate as 40-Gigabit Ethernet (xle) ports, as well as how to delete a 40-Gigabit

Ethernet (xle) port configuration.

1.

To configure a block of ports to operate as 40-Gigabit Ethernet (xle) ports, specify a port range:

[edit chassis node-group name node-device name pic 1] root@qfabric# set xle port-range port-range-low port-range-high

For example, to configure ports Q4 through Q7 to operate as 40-Gigabit Ethernet ports:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 1] root@qfabric# set xle port-range 4 7

2.

To configure an individual port to operate as a 40-Gigabit Ethernet (xle) port, specify a port number:

[edit chassis node-group name node-device name pic 1] root@qfabric# set xle port port-number

For example, to configure port Q4 to operate as a 40-Gigabit Ethernet port:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 1] root@qfabric# set xle port 4

3.



4.

To delete the 40-Gigabit Ethernet (xle) port configuration for block of ports, specify a port range:

[edit chassis node-group name node-device name pic 1] root@qfabric# delete xle port-range port-range-low port-range-high

For example, to delete the 40-Gigabit Ethernet port configuration for ports Q4 through

Q7:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 1] root@qfabric# delete xle port-range 4 7

5.

To delete the 40-Gigabit Ethernet (xle) port configuration for an individual port, specify a port number:

[edit chassis node-group name node-device name pic 1] root@qfabric# delete xle port port-number

For example, to delete the 40-Gigabit Ethernet port configuration for port Q4:

[edit chassis node-group BBAK8281 node-device BBAK8309 pic 1] root@qfabric# delete xle port 4



Related

Documentation

•

Understanding Node Devices

•

Understanding Interfaces on the QFabric System

•

pic

Configuring the QSFP+ Port Type on QFX5100 Devices

You can convert default 40-Gigabit Ethernet data plane uplink interfaces (fte) to

40-Gigabit Ethernet access interfaces (xle) ports, and default 40-Gigabit Ethernet interfaces (xle) to 40-Gigabit Ethernet data plane uplink interfaces (fte). Ports Q0 and

Q1 are fixed fte ports and cannot be changed. Ports Q2 and Q3 are fte ports by default but can be changed to xle ports. Ports Q4 and Q5 are xle ports by default but can be changed to fte ports.

NOTE: On QFX5100-24Q switches, ports Q1 through Q7 are fixed FTE ports and cannot be changed.

NOTE: You must configure xle ports in pairs, not individually, otherwise functionality is not guaranteed.

CAUTION: The Packet Forwarding Engine on a QFX5100 switch is restarted when you commit port type configuration changes (for example, configuring or deleting an fte or xle port). As a result, you might experience packet loss on the device.

The following steps describe how to configure either a block of ports or an individual port, as well as how to delete these configurations.

1.

To configure a block of ports to operate as 40-Gigabit Ethernet interfaces (xle) , specify a port range:

[edit chassis node-group name node-device name pic 1] user@switch# set xle port-range port–range-low port-range-high

For example, to configure ports Q4 through Q5 to operate as 40-Gigabit Ethernet interfaces (xle):

[edit chassis node-group name node-device name pic 1] user@switch# set xle port-range 4 5

2.

To configure a block of ports to operate as 40-Gigabit Ethernet data plane uplink interfaces (fte), specify a port range:

[edit chassis node-group name node-device name pic 1] user@switch# set fte port-range port–range-low port-range-high


85


86

For example, to configure ports Q4 through Q5 to operate as 40-Gigabit Ethernet data plane uplink interfaces (fte):

[edit chassis node-group name node-device name pic 1] user@switch# set fte port-range 4 5

3.

To configure an individual port to operate as a 40-Gigabit Ethernet data plane uplink interfaces (fte), specify a port number:

[edit chassis node-group name node-device name pic 1] user@switch# set fte port port-number

For example, to configure port Q4 to operate as a 40-Gigabit Ethernet data plane uplink interfaces (fte):

[edit chassis node-group name node-device name pic 1] user@switch# set fte port 4

4.


[edit] user@switch# commit commit complete

5.

To delete a block of ports configured as 40-Gigabit Ethernet (xle) ports, specify a port range:

[edit chassis node-group name node-device name pic 1] user@switch# delete xle port-range port-range-low port-range-high

For example, to delete the 40-Gigabit Ethernet access interface (xle) port configuration for ports Q2 through Q3:

[edit chassis node-group name node-device name pic 1] user@switch# delete xle port-range 2 3

6.

To delete an individual port configured as a 40-Gigabit Ethernet (xle) interface:

[edit chassis node-group name node-device name pic 1] user@switch# delete xle port port-number

For example, to delete the 40-Gigabit Ethernet interface (xle) for port Q2:

[edit chassis node-group name node-device name pic 1] user@switch# delete xle port 2

7.

To delete a block of ports configured as 40-Gigabit Ethernet data plane uplink interfaces (fte), specify a port range:

[edit chassis node-group name node-device name pic 1] user@switch# delete fte port-range port-range-low port-range-high

For example, to delete the block of ports configured as 40-Gigabit Ethernet data plane uplink interfaces (fte) for ports Q4 through Q5:

[edit chassis node-group name node-device name pic 1] user@switch# delete fte port-range 4 5

8.

To delete an individual port configured as a 40-Gigabit Ethernet data plane uplink interfaces (fte):

[edit chassis node-group name node-device name pic 1]


Chapter 3: QFabric System Configuration user@switch# delete fte port port-number

For example, to delete the 40-Gigabit Ethernet data plane uplink interfaces (fte) for port Q4:

[edit chassis node-group name node-device name pic 1] user@switch# delete fte port 4

9.


[edit] user@switch# commit commit complete

Related

Documentation

•

Understanding Interface Naming Conventions

•

Understanding Port Ranges and System Modes

•

pic

Example: Configuring SNMP

By default, SNMP is disabled on devices running Junos OS. This example describes the steps for configuring SNMP on the QFabric system.

•


•

Overview on page 87

•


Requirements


• Junos OS Release 12.2

•

Network management system (NMS) (running the SNMP manager)

•

QFabric system (running the SNMP agent) with multiple Node devices

Overview

Because SNMP is disabled by default on devices running Junos OS, you must enable

SNMP on your device by including configuration statements at the [edit snmp] hierarchy level. At a minimum, you must configure the community public statement. The community defined as public grants read-only access to MIB data to any client.

If no clients statement is configured, all clients are allowed. We recommend that you always include the restrict option to limit SNMP client access to the switch.

The network topology in this example includes an NMS, a QFabric system with four Node devices, and external SNMP servers that are configured for receiving traps.


87


Configuration

CLI Quick

Configuration


set snmp name “snmp qfabric” description “qfabric0 switch” set snmp location “Lab 4 Row 11” contact “qfabric-admin@qfabric0” set snmp community public authorization read-only set snmp client-list list0 192.168.0.0/24 set snmp community public client-list-name list0 set snmp community public clients 192.170.0.0/24 restrict set snmp trap-group “qf-traps” destination-port 155 targets 192.168.0.100

Step-by-Step

Procedure



To configure SNMP on the QFabric system:

NOTE: If the name, description, location, contact, or community name contains spaces, enclose the text in quotation marks (" ").

1.

2.

3.

4.

5.

Configure the SNMP system name:

[edit snmp] user@switch# set name “snmp qfabric”

Specify a description.

[edit snmp] user@switch# set description “qfabric0 system”

This string is placed into the MIB II sysDescription object.

Specify the physical location of the QFabric system.

[edit snmp] user@switch# set location “Lab 4 Row 11”

This string is placed into the MIB II sysLocation object.

Specify an administrative contact for the SNMP system.

[edit snmp] user@switch# set contact “qfabric-admin@qfabric0”

This name is placed into the MIB II sysContact object.

Specify a unique SNMP community name and the read-only authorization level.

NOTE: The read-write option is not supported on the QFabric system.



6.

7.

8.

[edit snmp] user@switch# set community public authorization read-only

Create a client list with a set of IP addresses that can use the SNMP community.

[edit snmp] user@switch# set client-list list0 192.168.0.0/24 user@switch# set community public client-list-name list0

Specify IP addresses of clients that are restricted from using the community.

[edit snmp] user@switch# set community public clients 192.170.0.0/24 restrict

Configure a trap group, destination port, and a target to receive the SNMP traps in the trap group.

[edit snmp] user@switch# set trap-group “qf-traps” destination-port 155 targets 192.168.0.100

NOTE: You do not need to include the destination-port statement if you use the default port 162.

The trap group qf-traps is configured to send traps to 192.168.0.100.

Results From configuration mode, confirm your configuration by entering the show command. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

[edit] user@switch# show snmp { name "snmp qfabric"; description "qfabric0 system"; location "Lab 4 Row 11"; contact "qfabric-admin@qfabric0"; client-list list0 {

192.168.0.0/24;

} community public { authorization read-only; clients {

197.170.0.0/24 restrict;

}

} trap-group qf-traps { destination-port 155; targets {

192.168.0.100;

}

}

}



89


Related

Documentation

•

Understanding the Implementation of SNMP on the QFabric System

•

snmp

Example: Configuring System Log Messages

The QFabric system monitors events that occur on its component devices and distributes system log messages about those events to all external system log message servers

(hosts) that are configured. Component devices may include Node devices, Interconnect devices, Director devices, and the Virtual Chassis. Messages are stored for viewing only in the QFabric system database. To view the messages, issue the show log command.

This example describes how to configure system log messages on the QFabric system.

•


•

Overview on page 90

•


Requirements


•

Junos OS Release 12.2

• QFabric system

• External servers that can be configured as system log message hosts

Overview

Component devices that generate system log message events may include Node devices,

Interconnect devices, Director devices, and the control plane switches. The following configuration example includes these components in the QFabric system:

•

Director software running on the Director group

• Control plane switches

• Interconnect device

•

Multiple Node devices

Configuration

CLI Quick

Configuration


set system syslog host 10.1.1.12 any error set system syslog file qflogs set system syslog file qflogs structured-data brief set system syslog file qflogs archive size 1g



Step-by-Step

Procedure



To configure system messages from the QFabric Director device:

1.

Specify a host, any facility, and the error severity level.

[edit system syslog] user@switch# set host 10.1.1.12 any error

NOTE: You can configure more than one system log message server

(host). The QFabric system sends the messages to each server configured.

2.

(Optional) Specify a filename to capture log messages.

NOTE: On the QFabric system, a syslog file named messages is configured implicitly with facility and severity levels of any any and a file size of 100 MBs. Therefore, you cannot specify the filename messages in your configuration, and automatic command completion does not work for that filename.

3.

[edit system syslog] user@switch# set file qflogs structured-data brief user@switch# set file qflogs

(Optional) Configure the maximum size of your system log message archive file.

This example specifies an archive size of 1 GB.

[edit system syslog] user@switch# set file qflogs archive size 1g

Results From configuration mode, confirm your configuration by entering the show system command. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

[edit] user@switch# show system syslog { file qflogs {

} host 10.1.1.12 { any error;

}

}



91


Related

Documentation

•

Understanding the Implementation of System Log Messages on the QFabric System

•

syslog (QFabric System)

•

show log

Configuring Graceful Restart for QFabric Systems

When you configure graceful restart in the QFabric CLI, the QFabric system applies the configuration to the network Node group to participate in graceful restart operations with devices external to the QFabric system. Such configuration preserves routing table state and helps neighboring routing devices to resume routing operations more quickly after a system restart. This also enables the network Node group to resume routing operations rapidly if there is a restart in the QFabric system (such as a software upgrade).

As a result, we recommend enabling graceful restart for routing protocols in the QFabric

CLI.

NOTE: The QFabric system also uses graceful restart internally within the fabric to facilitate interfabric resiliency and recovery. This internal feature is enabled by default with no configuration required.

•

Enabling Graceful Restart on page 92

•

Configuring Graceful Restart Options for BGP on page 93

•

Configuring Graceful Restart Options for OSPF and OSPFv3 on page 94

•

Tracking Graceful Restart Events on page 95

Enabling Graceful Restart

By default, graceful restart is disabled. To enable graceful restart, include the graceful-restart statement at the [edit routing-instance instance-name routing-options] or [edit routing-options] hierarchy level.

For example: routing-options { graceful-restart;

}

To configure the duration of the graceful restart period, include the restart-duration at the [edit routing-options graceful-restart] hierarchy level.

NOTE: Helper mode (the ability to assist a neighboring router attempting a graceful restart) is enabled by default when you start the routing platform, even if graceful restart is not enabled. You can disable helper mode on a per-protocol basis.

[edit] routing-options {



} graceful-restart { disable; restart-duration seconds;

}

To disable graceful restart globally, include the disable statement at the [edit routing-options graceful-restart] hierarchy level.

When graceful restart is enabled for all routing protocols at the [edit routing-options graceful-restart] hierarchy level, you can disable graceful restart on a per-protocol basis.

NOTE: If you configure graceful restart after a BGP or LDP session has been established, the BGP or LDP session restarts and the peers negotiate graceful restart capabilities. Also, the BGP peer routing statistics are reset to zero.

Configuring Graceful Restart Options for BGP

To configure the duration of the BGP graceful restart period, include the restart-time statement at the [edit protocols bgp graceful-restart] hierarchy level. To set the length of time the router waits to receive messages from restarting neighbors before declaring them down, include the stale-routes-time statement at the [edit protocols bgp graceful-restart] hierarchy level.

[edit] protocols { bgp { graceful-restart { disable; restart-time seconds; stale-routes-time seconds;

}

}

} routing-options { graceful-restart;

}

To disable BGP graceful restart capability for all BGP sessions, include the disable statement at the [edit protocols bgp graceful-restart] hierarchy level.

NOTE: To set BGP graceful restart properties or disable them for a group, include the desired statements at the [edit protocols bgp group group-name graceful-restart] hierarchy level.

To set BGP graceful restart properties or disable them for a specific neighbor in a group, include the desired statements at the [edit protocols bgp group

group-name neighbor ip-address graceful-restart] hierarchy level.


93


NOTE: Configuring graceful restart for BGP resets the BGP peer routing statistics to zero. Also, existing BGP sessions restart, and the peers negotiate graceful restart capabilities.

Configuring Graceful Restart Options for OSPF and OSPFv3

To configure the duration of the OSPF/OSPFv3 graceful restart period, include the restart-duration statement at the [edit protocols (ospf | ospf3) graceful-restart] hierarchy level. To specify the length of time for which the router notifies helper routers that it has completed graceful restart, include the notify-duration at the [edit protocols (ospf | ospf3) graceful-restart] hierarchy level. Strict OSPF link-state advertisement (LSA) checking results in the termination of graceful restart by a helping router. To disable strict LSA checking, include the no-strict-lsa-checking statement at the [edit protocols (ospf | ospf3) graceful-restart] hierarchy level.

[edit] protocols { ospf | ospfv3{ graceful-restart { disable; helper-disable no-strict-lsa-checking; notify-duration seconds; restart-duration seconds;

}

}

} routing-options { graceful-restart;

}

To disable OSPF/OSPFv3 graceful restart, include the disable statement at the [edit protocols (ospf | ospf3) graceful-restart] hierarchy level.

Starting with Release 11.3, the Junos OS supports both the standard (based on RFC 3623,

Graceful OSPF Restart) and the restart signaling-based (as specified in RFC 4811, RFC

4812, and RFC 4813) helper modes for OSPF version 2 graceful restart configurations.

Both the standard and restart signaling-based helper modes are enabled by default. To disable the helper mode for OSPF version 2 graceful restart configurations, include the helper-disable <both | restart-signaling | standard> statement at the [edit protocols ospf graceful-restart] hierarchy level. Note that the last committed statement always takes precedence over the previous one.

[edit protocols ospf] graceful-restart { helper-disable <both | restart-signaling | standard>

}

To reenable the helper mode, delete the helper-disable statement from the configuration by using the delete protocols ospf graceful-restarthelper-disable <restart-signaling | standard | both> command. In this case also, the last executed command takes precedence over the previous ones.



NOTE:

Restart signaling-based helper mode is not supported for OSPFv3 configurations. To disable helper mode for OSPFv3 configurations, include the helper-disable statement at the [edit protocols ospfv3 graceful-restart] hierarchy level.

TIP: You can also track graceful restart events with the traceoptions statement at the [edit protocols (ospf | ospf3)] hierarchy level. For more information, see

“Tracking Graceful Restart Events” on page 95

.

NOTE: If you configure BFD and graceful restart for OSPF, graceful restart might not work as expected.

Tracking Graceful Restart Events

To track the progress of a graceful restart event, you can configure graceful restart trace options flags for IS-IS and OSPF/OSPFv3. To configure graceful restart trace options, include the graceful-restart statement at the [edit protocols protocol traceoptions flag] hierarchy level:

[edit protocols] isis { traceoptions { flag graceful-restart;

}

}

(ospf | ospf3) { traceoptions { flag graceful-restart;

}

}

Related

Documentation

•

Graceful Restart Concepts

•

Verifying Graceful Restart Operation


95


Optimizing the Number of Multicast Flows on QFabric Systems

Related

Documentation

•

Because of the distributed nature of QFabric systems, the default configuration does not allow the maximum number of supported Layer 3 multicast flows to be created. To allow a QFabric system to create the maximum number of supported flows, configure the following statement: set fabric routing-options multicast fabric-optimized-distribution

After configuring this statement, you must reboot the QFabric Director group to make the change take effect.


CHAPTER 4

QFabric System Licensing

•

Generating the License Keys for a QFabric System on page 97

•

Adding New Licenses (CLI Procedure) on page 99

•

Deleting a License (CLI Procedure) on page 103

•

Saving License Keys on page 106

•

Verifying Junos OS License Installation on page 106

Generating the License Keys for a QFabric System

When you purchase a Junos OS software feature license for a QFabric system, you receive an e-mail containing an authorization code for the feature license from Juniper Networks.

You can use the authorization code to generate a unique license key (a combination of the authorization code and the QFabric system ID ) for the QFabric system, and then add the license key on the QFabric system.

Before generating the license keys for a QFabric system:

•

Purchase the required licenses for the QFabric system. See Software Features That

Require Licenses on the QFX Series.

• Note down the authorization code in the e-mail you received from Juniper Networks when you purchased the license.

•

Perform the initial setup of the QFabric system on the Director group. See Performing

the QFabric System Initial Setup on a QFX3100 Director Group.

• Log in to the QFabric system, issue the show version command, and note down the software serial number and QFabric system ID for the QFabric system.

user@qfabric> show version

Hostname: qfabric

Model: qfx3000-g

Serial Number: qfsn-0123456789

QFabric System ID: f158527a-f99e-11e0-9fbd-00e081c57cda

JUNOS Base Version [12.2I20111018_0215_dc-builder]


97


To generate the license keys for a QFabric system:

1.

In a browser, log in to the Juniper Networks License Management System at https://www.juniper.net/lcrs/license.do

.

The Manage Product Licenses page appears.

NOTE: To access the licensing site, you must have a service contract with

Juniper Networks and an access account. If you need help obtaining an account, complete the registration form at the Juniper Networks website https://www.juniper.net/registration/Register.jsp

.

2.

On the Generate Licenses tab, select QFX Series Product from the drop-down list, and click Go.

The Generate Licenses - QFX Series Product page appears.

3.

Select the QFX Series Product Fabric option button, and then click Continue.

The Generate Licenses - QFX Series Product Fabrics page appears.

4.

In the Software Serial No field, enter the software serial number for the QFabric system.

5.

In the QFabric System ID field, enter the QFabric system ID for the QFabric system.

6.

In the Authorization Code field, enter the authorization code in the e-mail you received from Juniper Networks when you purchased the license.

7.

(Optional) If you want to enter another authorization code for the same device, click

Enter More Authorization Codes to display a new authorization code field. Enter the authorization code in this field.

8.

Click Confirm.

The Confirm License Information page appears, displaying a summary of the information you submitted to the License Management System.

9.

Review the information to ensure everything is correct and then click Generate License.

The Generate Licenses - QFX Series Product Fabrics page appears, displaying a summary of your license keys, including a link that displays the details of your new license keys.

10.

Select the file format in which you want to obtain your new license keys.

11.

Select the delivery method you want to use to obtain your new license keys.


Chapter 4: QFabric System Licensing

To download the license keys:

•

Select the Download to this computer option button, and click OK.

To e-mail the license keys:

• Select the Send e-mail to e-mail ID option button, and click OK.

Related

Documentation

•

Software Features That Require Licenses on the QFX Series

•


•


•

show version

Adding New Licenses (CLI Procedure)

Before adding new licenses, complete the following tasks:

•

Purchase the required licenses.

• Establish basic network connectivity with the router or switch. For instructions on establishing basic connectivity, see the Getting Started Guide or Quick Start Guide for your device.

There are two ways to add licenses using the Junos OS CLI:

•

The system license keys key configuration statement enables you to configure and delete license keys in a Junos OS CLI configuration file.

•

The request system license add operational command installs a license immediately.

NOTE: On QFabric systems, install your licenses in the default partition of the QFabric system and not on the individual components (Node devices and Interconnect devices).

To add licenses, complete one of the following procedures:

•

Installing a License Using a Configuration Statement on page 99

•

Installing a License Using an Operational Command on page 102

Installing a License Using a Configuration Statement

Starting with Junos OS Release 15.1, you can configure and delete license keys in a Junos

OS CLI configuration file. The system license keys key statement at the [edit] hierarchy level installs a license by using a configuration statement.


99


100

NOTE: The system license keys key configuration statement is not required to install a license. The operational command request system license add installs a license immediately. But because the set system license keys key command is a configuration statement, you can use it to install a license as part of a configuration commit, either directly or by configuration file.

The license keys are validated and installed after a successful commit of the configuration file. If a license key is invalid, the commit fails and issues an error message. You can configure individual license keys or multiple license keys by issuing Junos OS CLI commands or by loading the license key configuration contained in a file. All installed license keys are stored in the /config/license/ directory.

Select a procedure to install a license using configuration:

•

Installing Licenses Using the CLI Directly on page 100

•

Installing Licenses Using a Configuration File on page 101

Installing Licenses Using the CLI Directly

To install an individual license key using the Junos OS CLI:

1.

Issue the set system license keys key name statement.

The name parameter includes the license ID and the license key. For example:

[edit] user@device# set system license keys key "JUNOS_TEST_LIC_FEAT xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx"

To install multiple license keys in the Junos OS CLI, issue the set system license keys key name statement for each license key to install. For example:

[edit] user@device# set system license keys key "key_1" set system license keys key "key_2" set system license keys key "key_2" set system license keys key "key_4"

2.

Issue the commit command.

[edit] user@device# commit commit complete

3.

Verify that the license key was installed.

For example: user@device# run show system license

License usage:

Licenses Licenses Licenses Expiry

Feature name used installed needed

sdk-test-feat1 0 1 0 permanent

Licenses installed:



License identifier: JUNOS_TEST_LIC_FEAT

License version: 2

Features:

sdk-test-feat1 - JUNOS SDK Test Feature 1

permanent

Alternatively, you can issue the show system license command from operational mode.

Installing Licenses Using a Configuration File

Before you begin, prepare the configuration file. In this example, use the Unix shell cat command to write the license.conf file:

1.

Go to the shell.

[edit] user@device# exit user@device> exit

%

2.

Open the new license.conf file.

% cat > license.conf

3.

Type the configuration information for the license key or keys:

•

For a single license, for example, type the following content: system {

license {

keys {

key "JUNOS_TEST_LIC_FEAT xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx";

}

}

}

•

For multiple license keys, for example, type something like this: system {

license {

keys {

key "key_1"

key "key_2"

key "key_3"

...

key "key_n"

}

}

}

4.

Press Ctrl+d to save the file.

To install a license key configuration in a file:

1.

Go to the CLI configuration mode.

% cli user@device> configure

[edit]


101

Configuring a QFX3000-M QFabric System user@device#

2.

Load and merge the license configuration file.

For example: user@device# load merge license.conf

load complete

3.

Issue the show | compare command to see the configuration.

For example:

[edit] user@device# show | compare

[edit system]

+ license {

+ keys {

+ key "JUNOS_TEST_LIC_FEAT xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx";

+ }

+ }

4.

Issue the commit command.

[edit] user@device# commit

5.

To verify that the license key was installed, issue the show system license command.

For example: root@switch> show system license

License usage:



sdk-test-feat1 0 1 0 permanent

Licenses installed:

License identifier: JUNOS_TEST_LIC_FEAT

License version: 2

Features:

sdk-test-feat1 - JUNOS SDK Test Feature 1

permanent

Installing a License Using an Operational Command

Complete the procedure that relates to your system:

•

Adding a License to a Device with a Single Routing Engine on page 102

•

Adding a License to a Device with Dual Routing Engines on page 103

Adding a License to a Device with a Single Routing Engine

To add a new license key to the device using an operational command:

1.

From the CLI operational mode, enter one of the following CLI commands:

•

To add a license key from a file or URL, enter the following command, specifying the filename or the URL where the key is located:


Chapter 4: QFabric System Licensing user@host> request system license add filename | url

• To add a license key from the terminal, enter the following command: user@host> request system license add terminal

2.

When prompted, enter the license key, separating multiple license keys with a blank line.

If the license key you enter is invalid, an error appears in the CLI output when you press

Ctrl+d to exit license entry mode.

3.

Go on to

“Verifying Junos OS License Installation” on page 106

.

Adding a License to a Device with Dual Routing Engines

On routers that have graceful Routing Engine switchover (GRES) enabled, after successfully adding the new license on the master Routing Engine, the license keys are automatically synchronized on the backup Routing Engine as well. However, in case GRES is not enabled, the new license is added on each Routing Engine separately. This ensures that the license key is enabled on the backup Routing Engine during changeover of mastership between the Routing Engines.

To add a new license key to a router with dual Routing Engines without GRES:

1.

After adding the new license key on the master Routing Engine, use the request chassis routing-engine master switch command to have the backup Routing Engine become the master Routing Engine.

2.

Log in to the active Routing Engine and add the new license key, repeat the same step.

NOTE: Adding a license key to the router or switch might be delayed if a kernel resynchronization operation is in progress at that time. The following message is displayed on the CLI when the license-adding operation is about to be delayed:

A kernel re-sync operation is in progress. License update may take several minutes to complete.

Related

Documentation

•


•

Junos OS Feature Licenses

•


•

request system license add

Deleting a License (CLI Procedure)

Before deleting a license, establish basic network connectivity with the router or switch.

For instructions on establishing basic connectivity, see the Getting Started Guide or Quick

Start Guide for your router or switch.


103


You can use the operational command request system license delete or the configuration command delete or deactivate to delete a license or licenses:

•

Using the Operational Command to Delete Licenses on page 104

•

Using a Configuration Command to Delete Licenses on page 104

Using the Operational Command to Delete Licenses

To delete licenses using the request system license delete command:

1.

Display the licenses available to be deleted.

user@host> request system license delete license-identifier-list ?


E00468XXX4 License key identifier

JUNOS10XXX1 License key identifier




[ Open a set of values

2.

To delete a license key or keys from a device using the CLI operational mode, select one of the following methods:

•

Delete a single license by specifying the license ID. Using this option, you can delete only one license at a time.

user@host> request system license delete license-identifier

•

Delete all license keys from the current device.

user@host> request system license delete all

• Delete multiple license keys from the current device. Specify the license identifier for each key and enclose the list of identifiers in brackets.

user@host> request system license delete license-identifier-list [JUNOS10XXX1

JUNOS10XXX3 JUNOS10XXX4 ...]

Delete license(s) ?

[yes,no] (no) yes

3.

Go on to


.

Using a Configuration Command to Delete Licenses

Starting in Junos OS Release 16.1, to remove licenses from the configuration, you can use either the configuration command delete or deactivate. The delete command deletes a statement or identifier, and all subordinate statements and identifiers contained within the specified statement path are deleted with it. The deactivate command adds the inactive: tag to a statement, effectively commenting out the statement or identifier from the configuration. Statements or identifiers marked as inactive do not take effect when you issue the commit command. To remove the inactive: tag from a statement, issue the activate command. Statements or identifiers that have been activated take effect when you next issue the commit command.

The following procedure uses the delete command, but you could use the deactivate command as well.



To delete one or all licenses using the delete command:

NOTE: You can use the deactivate command instead of the delete command in this procedure.

1.

Display the licenses available to be deleted.

Issue the run request system license delete license-identifier-list ? command from the configuration mode of the CLI.

[edit] user@host# run request system license delete license-identifier-list ?

A list of licenses on the device is displayed:


E00468XXX4 License key identifier





[ Open a set of values

2.

Delete the license or licenses you want.

•

To delete a single license, for example:

[edit] user@host# delete system license keys key “E00468XXX4”

• To delete all licenses, for example:

[edit] user@host# delete system license keys

3.

Commit the configuration.

4.

Verify the configuration.

Release History Table

Release

16.1

Description

Starting in Junos OS Release 16.1, to remove licenses from the configuration, you can use either the configuration command delete or deactivate

.

Related

Documentation

•


•



105


Saving License Keys

Before saving a license, establish basic network connectivity with the router or switch.

For instructions on establishing basic connectivity, see the Getting Started Guide or Quick

Start Guide for your router or switch.

To save the licenses installed on a device to a file using the CLI:

1.

From the CLI operational mode, enter one of the following CLI commands:

•

To save the installed license keys to a file or URL, enter the following command: user@host> request system license save filename | url

For example, the following command saves the installed license keys to a file named license.config

:

•

To save a license key from the terminal, enter the following command: user@host> request system license save ftp://user@host/license.config

2.

Go on to


.

Related

Documentation

•


•


•

Junos OS Feature Licenses

•


Verifying Junos OS License Installation

To verify Junos OS license management, perform the following tasks:

•

Displaying Installed Licenses on page 106

•

Displaying License Usage on page 107

Displaying Installed Licenses

Purpose Verify that the expected licenses are installed and active on the router or switch.

Action From the CLI, enter the show system license command.

Sample Output

user@host> show system license

License usage:



subscriber-acct 0 1 0 permanent

subscriber-auth 0 1 0 permanent

subscriber-addr 0 1 0 permanent

subscriber-vlan 0 1 0 permanent

subscriber-ip 0 1 0 permanent

scale-subscriber 0 1000 0 permanent



scale-l2tp 0 1000 0 permanent

scale-mobile-ip 0 1000 0 permanent

Licenses installed:

License identifier: E000185416

License version: 2

Features:

subscriber-acct - Per Subscriber Radius Accounting

permanent

subscriber-auth - Per Subscriber Radius Authentication

permanent

subscriber-addr - Address Pool Assignment

permanent

subscriber-vlan - Dynamic Auto-sensed Vlan

permanent

subscriber-ip - Dynamic and Static IP

permanent

Meaning The output shows a list of the license usage and a list of the licenses installed on the router or switch. Verify the following information:

•

Each license is present. Licenses are listed in ascending alphanumeric order by license

ID.

•

The state of each license is permanent.

NOTE: A state of invalid indicates that the license key is not a valid license key. Either it was entered incorrectly or it is not valid for the specific device.

•

The feature for each license is the expected feature. The features enabled are listed by license. An all-inclusive license has all features listed.

•

All configured features have the required licenses installed. The Licenses needed column must show that no licenses are required.

Displaying License Usage

Purpose Verify that the licenses fully cover the feature configuration on the router or switch.

Action From the CLI, enter the show system license usage command.

Sample Output

user@host> show system license usage



subscriber-addr 1 0 1 29 days

scale-subscriber 0 1000 0 permanent

scale-l2tp 0 1000 0 permanent

scale-mobile-ip 0 1000 0 permanent

Meaning The output shows any licenses installed on the router or switch and how they are used.

Verify the following information:


107


•

Any configured licenses appear in the output. The output lists features in ascending alphabetical order by license name. The number of licenses appears in the third column.

Verify that you have installed the appropriate number of licenses.

•

The number of licenses used matches the number of configured features. If a licensed feature is configured, the feature is considered used. The sample output shows that the subscriber address pooling feature is configured.

•

A license is installed on the router or switch for each configured feature. For every feature configured that does not have a license, one license is needed.

For example, the sample output shows that the subscriber address feature is configured but that the license for the feature has not yet been installed. The license must be installed within the remaining grace period to be in compliance.


CHAPTER 5

QFabric System Backup and Recovery

•

Performing System Backup and Recovery for a QFabric System on page 109

•

Performing a QFabric System Recovery Installation on the Director Group on page 110

•

Performing a Recovery Installation on page 118

•

Creating an Emergency Boot Device on page 120

Performing System Backup and Recovery for a QFabric System

Many routers and switches require an administrator to recover the software package and the configuration file for the device separately. In the case of a device failure, this means the administrator might need to perform two separate tasks (if neither the software package nor the configuration file can be recovered).

In contrast, the QFabric system uses a unique mechanism that saves the backup and recovery files for both the Junos OS software and the system configuration into a single collection. The following QFabric system backup and recovery mechanism simplifies and streamlines the recovery process so you can return to normal operations as quickly as possible.

To backup and recover your QFabric system:

1.

(First time only) Implement the following one-time procedure to prepare your QFabric system to use the system backup and recovery feature:

• Insert a Juniper Networks software installation USB flash drive into the master

Director device. (This drive was provided to you as one of the components of your

QFabric system shipment.)

• Issue the request system software format-qfabric-backup command. The contents and format of the USB flash drive are copied to the Director group shared directory and are used as the basis for all future backup and recovery operations.

user@qfabric> request system software format-qfabric-backup

Copying QFabric USB template image from /dev/sdb(Unigen,PQS4000,4009 MB)......

• Remove the Juniper Networks software installation USB drive from the master

Director device.

2.

Issue the request system software system-backup command to backup the software package and configuration file. This command saves the current files necessary to


109

Configuring a QFX3000-M QFabric System recover the QFabric system. The files are saved to a shared memory directory in the

Director group.

NOTE: As you upgrade your system with new software and change the system configuration over time, remember to reissue this command periodically to save the newest files for recovery purposes.

user@qfabric> request system software system-backup user@qfabric>

3.

Insert a 4 GB or larger USB flash drive into the master Director device for your Director group, and issue the request system software system-backup usb-create command.

This command copies the recovery files that have been backed up in the Director group and transfers them to the USB flash drive to create a recovery USB drive.

NOTE: Issuing this command overwrites the contents of the USB flash drive with the QFabric system recovery files.

user@qfabric> request system software system-backup usb-create /dev/sdb

Issuing this command will overwrite the contents of the USB drive.

Continue? [yes,no] (no) yes

This operation will access the USB drive on 0281042010000013.

Are you sure you want to continue? [yes,no] (no) yes

Copying QFabric recovery media to /dev/sdb...

Successfully copied QFabric recovery media to /dev/sdb

4.

Remove the recovery USB drive from the Director device, and store it securely in a known location that you will remember when you need to use the recovery USB drive.

5.

If the QFabric system fails, power off the Director group, insert the recovery USB drive into the master Director device of your Director group, turn on power to the Director device, and follow the prompts to recover your system. This step restores the software package and the configuration file for your QFabric system.

Related

Documentation

•

request system software format-qfabric-backup

•

request system software system-backup

Performing a QFabric System Recovery Installation on the Director Group

If the software on your QFabric system is damaged in some way that prevents the software from loading correctly, or you need to upgrade the software on your QFabric system, you may need to perform a recovery installation on the Director group.

If possible, perform the following steps before you perform the recovery installation:

1.

Ensure that you have an emergency boot device (for example, an external USB flash drive) for each of your Director devices to use during the recovery installation.


Chapter 5: QFabric System Backup and Recovery

You can either use the external USB flash drive containing the software supplied by

Juniper Networks, or you can use an external USB flash drive supplied by Juniper

Networks on which you install the QFabric system install media.

2.

Because the recovery installation process completely overwrites the entire contents of the Director device, make sure you back up any configuration files and initial setup information on a different external USB flash drive before you begin a recovery installation. You will need to restore this information as part of recovery process.

Use the request system software configuration-backup command to back up your configuration files and initial setup information: user@switch> request system software configuration-backup path

NOTE: To recover the Director group, you must upgrade both Director devices in parallel. If you are recovering only one Director device in a Director group, and the software version will remain the same between the two Director devices, make sure that the other Director device is powered on and operational. If the software version of the Director device you are recovering will be different, make sure that the other Director device is powered off and is not operational.

•

(Optional) Creating an Emergency Boot Device Using a Juniper Networks External

Blank USB Flash Drive on page 111

•

Performing a Recovery Installation Using a Juniper Networks External USB Flash Drive with Preloaded Software on page 113

(Optional) Creating an Emergency Boot Device Using a Juniper Networks External Blank USB

Flash Drive

If you do not have an external USB flash drive preloaded with the software from Juniper

Networks to use as an emergency boot device, you can create your own, using a blank external USB flash drive provided by Juniper Networks. Download the install media from the Juniper Networks Support website onto your UNIX workstation, uncompress and untar the software, and then burn the software image onto your Juniper Networks external

USB (4-gigabyte) flash drive. Make sure you create two emergency boot devices, one for each Director device, so you can perform a recovery installation in parallel.

1.

Using a Web browser, navigate to the http://www.juniper.net/support

.

2.

Click Download Software.

3.

In the Switchingbox, click Junos OS Platforms.

4.

In the QFX Series section, click the name of the platform for which you want to download software.

5.

Click the Software tab and select the release number from the Release drop-down list.

6.

Select the complete install media you want to download in the QFabric System Install

Media section.


111



7.

Enter your name and password and press Enter.

8.

Read the End User License Agreement, click the I agree radio button, and then click

Proceed .

9.

Log in and save the install media file to your UNIX workstation.

10.

Use FTP to access the UNIX workstation where the install media resides.

11.

ftp ftp://hostname/pathname install-media-qfabric-<version>.img.tgz

When prompted, enter your username and password.

12.

Make sure you are in binary mode by entering binary at the prompt.

binary

13.

Use the get command to transfer the installation package from the FTP host to your

UNIX workstation.

get install-media-qfabric-<version>.img.tgz

14.

Close the FTP session: bye

15.

Untar the install-media-qfabric-<version> .img.tgz file on your UNIX workstation.

16.

tar -xvzf install-media-qfabric-11.3X30.6.img.tgz

Insert a blank external USB (4-gigabyte) flash drive supplied by Juniper Networks into your UNIX workstation.

17.

Burn the software image you just downloaded to your UNIX workstation onto your external USB flash drive using the dd command: dd if=install-media-qfabric-11.3X30.6.img of=/dev/sdb bs=16k

250880+0 records in

250880+0 records out

4110417920 bytes (4.1 GB) copied, 5.10768 seconds, 805 MB/s

18.

Perform the steps in

“Performing a Recovery Installation Using a Juniper Networks

External USB Flash Drive with Preloaded Software” on page 113

to continue with the recovery installation.



Performing a Recovery Installation Using a Juniper Networks External USB Flash Drive with

Preloaded Software

This procedure describes how to perform a recovery installation using an external USB flash drive that contains Junos OS software.

NOTE: Since the recovery installation process completely overwrites the entire contents of the Director device, you will need to restore the required configuration files and initial setup information. The following procedure assumes you previously saved these backup files with the request system software configuration-backup command. Ensure that you have these backup files available on an external USB flash drive before you perform the following steps.

1.

Insert the external USB flash drive into the Director device.

2.

Perform one of the following tasks:

• If you have access to the default partition, reboot the Director device by issuing the request system reboot director-group command.

•

If you do not have access to the default partition, power cycle the Director device.

The following menu appears on the Director device console when the Director device boots up:

Juniper Networks QFabric Director Install/Recovery Media

- To boot from the local disk, wait 10 seconds or press the Enter key.

- To reinstall the QFabric software on this Director device, type: install

3.

Type install and then press Enter to install the software on the Director device.

Once the installation process is complete, the Director device reboots, and the following menu appears on the Director device console:

Juniper Networks QFabric Director Install/Recovery Media

- To boot from the local disk, wait 10 seconds or press the Enter key.

- To reinstall the QFabric software on this Director device, type: install

4.

Press Enter.

The Director device reboots from the local disk on which the software was just installed.

5.

Log in as root on the Director device.

The following menu appears on the Director device console:

Before you can access the QFabric system, you must complete the initial setup

of the Director group by using the steps that follow.

If the initial setup procedure does not complete successfully, log out of the

Director device and then log back in to restart

this setup menu.

Continue?[y/n]


113


114

6.

Enter n to bypass the initial setup script and enter the Director device root directory, where you can mount the external USB flash drive containing the configuration files and initial setup information.

7.

Issue the ls /mnt command to list the mount directory.

8.

root@dg0 ~]# ls /mnt

Issue the mkdir command to create a directory within the mount directory.

9.

root@dg0 ~]# mkdir /mnt/myusb

Issue the mount /dev/sdb2 /mnt/myusb/ command to mount the external USB flash drive to the local drive of the Director device.

root@dg0 ~]# mount /dev/sdb2 /mnt/myusb/

10.

Issue the ls -la /mnt/myusb/ command to verify the contents of your mounted external USB flashdrive.

root@dg0 ~]# ls -la /mnt/myusb/ total 1770884 drwxr-xr-x 2 root root 4096 Sep 7 05:16 .

drwxr-xr-x 3 root root 4096 Sep 7 10:15 ..

-rw-r--r-- 1 root root 4249 Sep 7 03:52 mybackup-20110907

11.

Exit the Director device and log back in as root on the Director device.

The following menu appears:

Before you can access the QFabric system, you must complete the initial setup

of the Director group by using the steps that follow.

If the initial setup procedure does not complete successfully, log out of the

Director device and then log back in to restart

this setup menu.

Continue?[y/n] y

Initial Configuration

You may enter the configuration manually or restore from a backup.

Specify a backup file? [y/n] : y

Please specify the full path of the configuration backup file. :

/mnt/myusb/mybackup-20110907

12.

Enter y to continue.

13.

Enter y and specify the path to the backup configuration file located on the external

USB flash drive.

/mnt/myusb/mybackup-20110907

The following messages appear:


Configuring peer...

connect error for 1.1.1.2:9001




Configuring QFabric software with initial pool of 4000 MAC addresses

[00:10:00:00:00:00 - 00:10:00:00:0f:3b]



Applying the new Director Device password




First install initial configuration, generating and sharing SSH keys.

First install initial configuration, generating SSH keys.

connect error for 1.1.1.2:9001

Shared SSH keys.


The Director device reboots from the local disk on which the software was just installed.

Exit the Director device session and log in to the QFabric default partition CLI.

14.

Issue the request system software configuration-restore command and specify the path to the backup configuration file located on the external USB flash drive to load the previously saved QFabric system configuration.

15.

From the default partition, issue the request system reboot node-group all command to reboot all of the Node groups in the QFabric system to ensure that all Node devices are running the same version of software as the Director-group.

16.

user@switch> request system reboot node-group all

From the default partition, issue the request system reboot fabric command to reboot the Interconnect devices and the other components in the fabric in the QFabric system to ensure that Interconnect devices are running the same version of software as the

Director group.

17.

user@switch> request system reboot fabric

Log in to the default partition and issue the show version component all command to verify that all components are running the same version of software.

user@switch> show version component all dg1:

-

Hostname: qfabric

Model: qfx3100

JUNOS Base Version [11.3X30.6] dg0:

-

Hostname: qfabric

Model: qfx3100

JUNOS Base Version [11.3X30.6]

NW-NG-0:

-

Hostname: qfabric

Model: qfx-jvre

JUNOS Base OS boot [11.3X30.6]

JUNOS Base OS Software Suite [11.3X30.6]

JUNOS Kernel Software Suite [11.3X30.6]

JUNOS Crypto Software Suite [11.3X30.6]

JUNOS Online Documentation [11.3X30.6]

JUNOS Enterprise Software Suite [11.3X30.6]

JUNOS Packet Forwarding Engine Support (QFX RE) [11.3X30.6]

JUNOS Routing Software Suite [11.3X30.6]

FC-0:

-

Hostname: qfabric

Model: qfx-jvre




115


116







FC-1:

Hostname: qfabric

Model: qfx-jvre









DRE-0:

-

Hostname: dre-0

Model: qfx-jvre









FM-0:

-

Hostname: qfabric

Model: qfx-jvre








JUNOS Routing Software Suite [11.3X30.6] nodedevice1:

-

Hostname: qfabric

Model: QFX3500








JUNOS Routing Software Suite [11.3X30.6] interconnectdevice1:

-

Hostname: qfabric



Model: QFX3108








JUNOS Routing Software Suite [11.3X30.6] warning: from interconnectdevice0: Disconnected

Related

Documentation

•


•

Upgrading Software on a QFabric System

•

request system software configuration-backup

•

request system software configuration-restore


117


Performing a Recovery Installation

If Junos OS on your device is damaged in some way that prevents the software from loading correctly, you may need to perform a recovery installation using an emergency boot device (for example, a USB flash drive) to restore the default factory installation.

Once you have recovered the software, you need to restore the device configuration. You can either create a new configuration as you did when the device was shipped from the factory, or if you saved the previous configuration, you can simply restore that file to the device.

Starting in Junos OS Release 14.1, you can also use a system snapshot as a bootup option when your Junos OS or configuration is damaged. The system snapshot feature takes a

“snapshot” of the files currently used to run the device—the complete contents of the

/config directories, which include the running Juniper Networks Junos OS, the active configuration, and the rescue configuration, as well as the host OS—and copies all of these files into an external USB flash drive. See Understanding System Snapshot.

NOTE: System snapshot is not supported on QFX10002 switches.

If at all possible, you should try to perform the following steps before you perform the recovery installation:

1.

Ensure that you have an emergency boot device to use during the installation. See

“Creating an Emergency Boot Device” on page 120

for information on how to create an emergency boot device.

2.

Copy the existing configuration in the file /config/juniper.conf.gz from the device to a remote system, such as a server, or to an emergency boot device. For extra safety, you can also copy the backup configurations (the files named /config/juniper.conf.n, where n is a number from 0 through 9) to a remote system or to an emergency boot device.

WARNING: The recovery installation process completely overwrites the entire contents of the internal flash storage.

3.

Copy any other stored files to a remote system as desired.

To reinstall Junos OS:

1.

Insert the emergency boot device into the QFX Series device.

2.

Reboot the QFX Series device.

NOTE: Do not power off the device if it is already on.

[edit system] user@device> request system reboot



If you do not have access to the CLI, power cycle the QFX Series device.

The emergency boot device (external USB install media) is detected. At this time, you can load the Junos OS from the emergency boot device onto the internal flash storage.

3.

The software prompts you with the following options:

External USB install media detected.

You can load Junos from this media onto an internal drive.

Press 'y' to proceed, 'f' to format and install, or 'n' to abort.

Do you wish to continue ([y]/f/n)? f

4.

Type f to format the internal flash storage and install the Junos OS on the emergency boot device onto the internal flash storage.

If you do no want to format the internal flash storage, type y.

The following messages are displayed:

Installing packages from external USB drive da1

Packages will be installed to da0, media size: 8G

Processing format options

Fri September 4 01:18:44 UTC 2012

-- IMPORTANT INFORMATION --

Installer has detected settings to format system boot media.

This operation will erase all data from your system.

Formatting installation disk .. this will take a while, please wait

Disabling platform watchdog - threshold 12 mins

Determining installation slice

Fri September 4 01:27:07 UTC 2012

5.

The device copies the software from the emergency boot device, occasionally displaying status messages. Copying the software can take up to 12 minutes.

When the device is finished copying the software, you are presented with the following prompt:

*** Fri September 4 01:19:00 UTC 2012***

Installation successful..

Please select one of the following options:

Reboot to installed Junos after removing install media (default) ... 1

Reboot to installed Junos by disabling install media ............... 2

Exit to installer debug shell ...................................... 3

Install Junos to alternate slice ................................... 4

Your choice: 4

NOTE: System installer will now install Junos to alternate slice

Do not power off or remove the external installer media or interrupt the installation mechanism.

6.

Select 4 to install Junos OS to the alternate slice of the partition, and then press Enter.

7.

Remove the emergency boot device when prompted and then press Enter. The device then reboots from the internal flash storage on which the software was just installed.

When the reboot is complete, the device displays the login prompt.

8.

Create a new configuration as you did when the device was shipped from the factory, or restore the previously saved configuration file to the device.


119


Release History Table

Release

14.1

Description

Starting in Junos OS Release 14.1, you can also use a system snapshot as a bootup option when your Junos OS or configuration is damaged.

Related

Documentation

•

Creating an Emergency Boot Device on page 120

Creating an Emergency Boot Device

If Junos OS on the device is damaged in some way that prevents the software from loading properly, you can use an emergency boot device to repartition the primary disk and load a fresh installation of Junos OS. Use the following procedure to create an emergency boot device.

Before you begin, you need to download the installation media image for your device and Junos OS release from http://www.juniper.net/customers/support/

.

NOTE: You can create the emergency boot device on another Juniper

Networks switch or router, or any PC or laptop that supports Linux. The steps you take to create the emergency boot device vary, depending on the device.

To create an emergency boot device:

1.

Use FTP to copy the installation media image into the /var/tmp directory on the device.

2.

Insert a USB device into the USB port.

3.

From the Junos OS command-line interface (CLI), start the shell:

4.

user@device> start shell

%

Switch to the root account using the su command:

% su

Password: password

NOTE: The password is the root password for the device. If you logged in to the device as root, you do not need to perform this step.

5.

Enter the following command on the device: root@device% dd if=/var/tmp/filename of=/dev/da1 bs=1m

The device writes the installation media image to the USB device: root@device% dd if=install-media-qfx-5e-15.1X53-D30.5-domestic.img of=/dev/da0

bs=1m

1399+0 records in



1399+0 records out

1466957824 bytes transferred in 394.081902 secs (3722469 bytes/sec)

6.

Log out of the shell: root@device% exit

% exit user@device>

Related

Documentation

•

USB Port Specifications for the QFX Series

•

Performing a Recovery Installation on page 118

•

Performing a QFabric System Recovery Installation on the Director Group

•

Performing a Recovery Installation Using an Emergency Boot Device


121



Configuring a QFX3000-M QFabric System

Network Configuration Example

Configuring a QFX3000-M QFabric System

Table of Contents

CHAPTER 1

Understanding the QFX3000-M QFabric

System

QFabric System Overview

Legacy Data Center Architecture

Figure 1: Legacy Data Center Architecture

QFX Series QFabric System Architecture

Figure 2: QFX Series QFabric System Architecture

Understanding QFabric System Terminology

Table 1: QFabric System Terms

Table 1: QFabric System Terms (continued)

Table 1: QFabric System Terms (continued)

Table 1: QFabric System Terms (continued)

Table 1: QFabric System Terms (continued)

Table 1: QFabric System Terms (continued)

Understanding Interfaces on the QFabric System

Four-Level Interface Naming Convention

QSFP+ Interfaces

Table 2: QFX3600 Node Device Port Mappings

Table 2: QFX3600 Node Device Port Mappings (continued)

Table 2: QFX3600 Node Device Port Mappings (continued)

Table 3: QFX5100-48S Node Device Port Mappings

Link Aggregation

Understanding the QFabric System Hardware Architecture

QFabric System Hardware Architecture Overview

Figure 3: QFabric System Hardware Architecture

Director devices

Node devices

Interconnect devices Virtual Chassis

(control plane)

Table 4: Supported QFabric System Hardware Configurations

QFX3000-G QFabric System Features

QFX3000-M QFabric System Features

CHAPTER 2

Initial Setup for the QFX3000-M QFabric

System

QFabric System Initial and Default Configuration Information

Converting the Device Mode for a QFabric System Component

Table 5: Support for device mode options

Example: Configuring EX4200 Switches for the QFX3000-M QFabric System Control

Plane

Requirements

Overview

Topology

Figure 4: QFX3000-M QFabric System Control Plane—EX4200 Switch

Port Ranges

Table 6: QFX3000-M QFabric System Copper-Based Control Plane—QFabric

Component-to-EX4200 Switch Port Mappings

Table 6: QFX3000-M QFabric System Copper-Based Control Plane—QFabric

Component-to-EX4200 Switch Port Mappings (continued)

Table 6: QFX3000-M QFabric System Copper-Based Control Plane—QFabric

Component-to-EX4200 Switch Port Mappings (continued)

Figure 5: QFX3000-M QFabric System Control Plane—Director Group to

EX4200 Switch Connections

Table 7: Director Group Port Mappings

Table 8: Hardware to Software Port Mappings for Director Device Network Modules

Figure 6: QFX3000-M QFabric System Control Plane—Interconnect Device to EX4200 Switch Connections

Table 9: Interconnect Device Port Mappings

Figure 7: QFX3000-M QFabric System Control Plane—QFX3500 Node

Device to EX4200 Switch Connections

Figure 8: QFX3000-M QFabric System Control Plane—QFX3600 Node

Device to EX4200 Switch Connections

Figure 9: QFX3000-M QFabric System Control Plane—QFX5100 Node

Device to EX4200 Switch Connections

Figure 10: QFX3000-M QFabric System Fiber-Based Control

Plane—QFX3500 Node Device to EX4200 Switch Connections

Figure 11: QFX3000-M QFabric System Fiber-Based Control

Plane—QFX3600 Node Device to EX4200 Switch Connections

Figure 12: QFX3000-M QFabric System Fiber-Based Control

Plane—QFX5100 Node Device to EX4200 Switch Connections

Table 10: Node Device to EX4200 Switch Port Mappings

Figure 13: QFX3000-M QFabric System Control Plane—Inter-EX4200

Switch LAG Connections

Table 11: EX4200 Switch LAG Port Mappings

Configuration

Verification