Archive for : February, 2020

NSX-T CLI Reference Guide

Here is a reference guide for some useful command line tools you can run on NSX-T.

Command line

NSX CLI Introduction

NSX CLI (nsxcli) is the command line tool for troubleshooting NSX-T.  It’s run in a non-root mode so you have to use the command structure available.  For instance there is no grep, But you can use find and pipe instead.

 

You can use the nsxcli command line tool from various elements throughout the NSX-T deployment including NSX Manager, Edges and ESXi Transport Nodes.

From Edges

SSH Access to Edges

FYI I used Edge VMs throughout this article.
Post deployment, open the console and verify the SSH service is stopped.

get service ssh

Start the SSH service.

start service ssh

Set the SSH service to autostart when the VM is powered on.

set service ssh start-on-boot

Verify that the SSH service is running and Start on boot is set to True.

get service ssh

Disable the command-line timeout.

set cli-timeout 0

Obtain routing information for the gateways.

get logical-routers

Verify that the SERVICE_ROUTER_TIER0 service gateway appears with an associated VRF ID.

 

In the command output, VRF 2 is associated with SR-T0-GW. The VRF ID in your lab might be different.

vrf

Vrf is used to access the gateway virtual routing functions.  For BGP and other Tier-0 services you access the Tier-0 service router (SR) function of the gateway.

vrf <vrf_ID>

 

verify the BGP state.

get bgp neighbor summary

Check the status is established.  A status of Active means still setting up!

 

View further information on the BGP connection.  Also shows whether the connection is established or not.

get bgp neighbor

 

View ipv4 bgp information.

get bgp ipv4

 

Exit the Tier-0 VRF service gateway mode.

  • Press q to quit out of BGP neighbor output.

Route Information from the Edges

get logical-routers
vrf #
get route

 

b > is a route learned from a BGP peer.

For DR check the forwarder for similar routing information.

get forwarding

 

In the previous example I was connected to the SR to collect that forwarding information, but you can also get it from the DR.  This is useful for seeing routing configuration for DR components throughout the environment, such as those on a Tier-1 DR.

DHCP

Runs on Edges.

get dhcp servers

get dhcp ip-pools

 

get dhcp leases

 

 

Load Balancer

get load-balancer

 

The output shows the general load balancer configuration, including UUID and Virtual Server ID.

Copy the UUID and the Virtual Server ID values and paste them.
Verify the virtual server configuration.

get load-balancer UUID virtual-server Virtual_Server_ID

 

Verify the server pool configuration.

get load-balancer UUID pools

UUID is the value that you recorded for the load balancer.

 

 

VPN Connectivity Tests on Edges

Verify that the L2VPN session is active, identify the peers, and ensure that the tunnel status is up.

get ipsecvpn session active

Verify that the sessions are up.

get ipsecvpn session status

Check whether the ipsecvpn session is up between the local and remote peers.

get ipsecvpn session summary

Get the l2vpn session, tunnel, and IPSEC session numbers, and check that the status is UP.

get l2vpn sessions

Get statistical information of the local and remote peers, whether the status is UP, count of packets received, bytes received (RX), packets transmitted (TX), and packets dropped, malformed, or loops.

get l2vpn session stats

Get the session configuration information.

get l2vpn session config

NSX Manager

Local Users

To change the password of an account run:

Set user <username> [password <password> [old-password <old-password>]]

Authentication Policy Settings for Local Users

Use the following to set:

Password length

Set auth-policy minimum-password-length <password-length>

UI and API authentication policies.
The UI and API local users have the same policy.

Set auth-policy api lockout-period <lockout-period>

Set auth-policy api lockout-reset-period <lockout-reset-period>

Set auth-policy api max-auth-failures <auth-failures>

Set CLI authentication policy

Set auth-policy cli lockout-period lockout-period <lockout-period>

View Logs

NSX CLI

get log-file policy.log

Engineering Mode

Use st en to enter engineering mode (root privileged mode)

st en

tail /var/log/policy/policy.log

Syslog – Manager and Edges

set logging-server <hostname-or-ip-address[:port]> proto <protocol> level <level>

 

Transport Nodes

Logical Switches

List all transport nodes associated with a logical switch.

get logical-switches <switch_UUID> transport-node-table

List all TEPs associated with a logical switch.

get logical-switches <switch_UUID> vtep

List MAC table associated with a logical switch.

get logical-switches <switch_UUID> mac-table

List ARP table associated with a logical switch.

get logical-switches <switch_UUID> arp-table

Deploy Manager Cluster

To deploy a manager to an existing cluster, get the cluster configuration ID, make a note of the existing Managers certificate thumbprint and use that to join the new node to the cluster.  Finally get the cluster status to confirm the new host has joined.

get cluster config

join <NSX-Manager-IP> cluster-id <cluster-id> username<NSX-Manager-username> password<NSX-Manager-password> thumbprint <NSX-Manager1's-thumbpint>

get cluster status

OR via API

POST https://<nsx-mgr>/api/v1/cluster?action=join_cluster

ESXi Configuration

Several esxcli commands can be used to aid in NSX-T configuration.

esxcli network firewall ruleset set -r syslog -e true

esxcli system syslog config set --loghost=<hostname-or-ip-address[:port]>

esxcli system syslog reload

Can also use the nsxcli command set such as:

get logical-switches

N-VDS and Tunnel Information

Show logical switch and N-VDS info
 
get logical-switch

Display the status of the overlay tunnels

Note N-VDS’s used to be called host switches, hence the get host-switch command.
 
get-host switch <N-VDS_NAME> tunnel

KVM Configuration

Syslog

Login as root

Create this file

/etc/rsyslog.d/40-vmware-remote-logging.conf

Add this line to the file

'*.*@<syslog_server_ip>:514;RFC5424fmt'

Restart syslog

Systemctl restart rsyslog

nsxcli can also be used as outlined above.  ESXi gives a bit more info as the kernel info is available in ESXi that isn’t there for KVM.

Packet Capture

Can use CLI to setup network packet capture on:

NSX Manager

start capture interface <interface-name> [file <filename>] [count <packet-count>] [expression <expression>]

NSX Edges

set capture session <session-number> interface <port-uuid> direction <direction>

Example.

set capture session 1 interface fp-eth1 direction in
set capture session 1 expression src net 172.20.10.0/24

Removed captured session information with:

del capture session 1

ESXi

Collect packets.  Can send to a file.

pktcap-uw

Can Pipe it to view captured packets on the screen.
 
pktcap-uw | tcpdump -uw

View packets.

tcpdump-uw

KVM

Tcpdump

Installation problems

get services

get service <service name>

get cluster status

get configuration

get managers

Cluster Configuration Validation

NSX Manager nodes

Get cluster status
(nsxcli)

Get services
(nsxcli)
Get log-file (nsxcli) Login as root
(Linux)
DATASTORE datastore - /var/log/corfu/corfu.9000.log
CLUSTER_BOOT_MANAGER cluster_manager - /var/log/cloudnext/nsx-ccp.log
CONTROLLER controller - /var/log/cbm/cbm.log
MANAGER manager manager.log /var/log/proton/nsxapi.log
POLICY policy policy.log /var/log/policy/policy.log
HTTP http http.log /var/log/proxy/reverse-proxy.log
- - syslog /var/log/syslog

Transport Node Preparation

Check VIBs installed

vSphere

esxcli software vib list | grep -e nsx -e vsip

This might work too.

esxcfg-module -l | grep nsx

KVM

(Ubuntu) dpkg --list | grep nsx

Redhat

rpm -qa | grep nsx

Check TEP and Hyperbus

Hyperbus is for containers.

vSphere

esxcli network ip interface ipv4 address list

Overlay (TEP and vmk10 (default vmk)).  Hyperbus (vmk50 default vmk)).

Verify TCP/IP for TEP and hyberbus.

esxcli network ip netstack list

Vxlan is GENEVE in ESXi.

KVM

Ifconfig

ovs-vsctl list Open_vSwitch

ovs-vsctl show

Ubuntu KVM hosts create two bridges.

nsx-managed
nsx-vswitch.0

Agents and Connectivity

ESXi

/etc/init.d/nsx-mpa status

esxcli network ip connection list | grep 5671

/etc/init.d/nsx-proxy status

esxcli network ip connection list | grep 1235

/etc/init.d/nsx-opsagent status

KVM

service nsx-mpa status

netstat -nap | grep 5671

service nsx-proxy status

netstat -nap | grep 1235

Alternatively, view the connection status using other CLI commands.

ESXi

type the esxcli network ip connection list | grep 1235 command.

# esxcli network ip connection list | grep 1235

KVM

Type the command netstat -anp –tcp | grep 1235.

user@host:~$ netstat -anp --tcp | grep 1235

Checking Communication from Host to Controller and Manager

ESXi

On an ESXi host using NSX-T CLI commands: 

esxi-01.corp.local> get managers
– 192.168.110.19   Connected
 
esxi-01.corp.local> get controllers
 Controller IP    Port     SSL         Status       Is Physical Master   Session State  Controller FQDN
 192.168.110.16   1235   enabled     connected             true               up               NA

KVM

On a KVM host using NSX-T CLI commands: 

kvm-01> get managers
– 192.168.110.19   Connected
 
kvm-01> get controllers
 Controller IP    Port     SSL         Status       Is Physical Master   Session State  Controller FQDN
 192.168.110.16   1235   enabled     connected             true               up               NA

View details of N-VDS

The N-VDS has its own command line tool, net-vdl2.

net-vdl2 -l

ESXi LIF MAC

View The LIF (Logical Interface) vMAC to pMAC on ESXi host.

net-vdr -C -l

02:50:56:56:44:52 is always the vMAC for the LIFs for all DRs.

DFW Validation

ESXi

nsxt-vsip is the DFW module.

Check dvFilter for firewall rules.

summarize-dvfilter | grep <VM_NAME>

KVM

Use these to validate the distributed firewall Settings.
View app firewall virtual interfaces.

ovs-appctl -t /var/run/openvswitch/nsxa-ctl dfw/vif

View firewall rules with containing addrsets.

ovs-appctl -t /var/run/openvswitch/nsxa-ctl dfw/rules <VIF_ID_NUMBER>

CLI Upgrade Confirmation

Post upgrades, use these commands to validate that NSX-T has been upgraded successfully and the correct NSX-T modules are installed.

vSphere

esxcli software vib list | grep nsx

KVM

dpkg -l | grep nsx

Red Hat

rpm -qa | egrep 'nsx|openvwitch'

nsxcli

get version