NSX-T Command Line Reference Guide

01 Feb 2020 by Simon Greaves

Here is a reference guide for some useful command line tools you can run on NSX-T.

NSX CLI Introduction

NSX CLI (nsxcli) is the command line tool for troubleshooting NSX-T. It’s run in a non-root mode so you have to use the command structure available. For instance there is no grep, But you can use find and pipe instead.

You can use the nsxcli command line tool from various elements throughout the NSX-T deployment including NSX Manager, Edges and ESXi Transport Nodes.

From Edges

SSH Access to Edges FYI I used Edge VMs throughout this article.
Post deployment, open the console and verify the SSH service is stopped.
get service ssh

Start the SSH service.
start service ssh

Set the SSH service to autostart when the VM is powered on.
set service ssh start-on-boot

Verify that the SSH service is running and Start on boot is set to True.
get service ssh

Disable the command-line timeout.
set cli-timeout 0

Obtain routing information for the gateways.
get logical-routers

Verify that the SERVICE_ROUTER_TIER0 service gateway appears with an associated VRF ID.
In the command output, VRF 2 is associated with SR-T0-GW. The VRF ID in your lab might be different.

vrf

Vrf is used to access the gateway virtual routing functions. For BGP and other Tier-0 services you access the Tier-0 service router (SR) function of the gateway.

vrf \<vrf_ID\>

verify the BGP state.
get bgp neighbor summary

Check the status is established. A status of Active means still setting up!

View further information on the BGP connection. Also shows whether the connection is established or not.
get bgp neighbor

View ipv4 bgp information.
get bgp ipv4

Exit the Tier-0 VRF service gateway mode.

• Press q to quit out of BGP neighbor output.

Route Information from the Edges

get logical-routers
vrf #
get route

b \> is a route learned from a BGP peer.

For DR check the forwarder for similar routing information.
get forwarding

In the previous example I was connected to the SR to collect that forwarding information, but you can also get it from the DR. This is useful for seeing routing configuration for DR components throughout the environment, such as those on a Tier-1 DR.

DHCP

Runs on Edges.
get dhcp servers

get dhcp ip-pools

get dhcp leases

Load Balancer

get load-balancer

The output shows the general load balancer configuration, including UUID and Virtual Server ID.

Copy the UUID and the Virtual Server ID values and paste them. Verify the virtual server configuration.
get load-balancer UUID virtual-server <Virtual_Server_ID>

Verify the server pool configuration.
get load-balancer UUID pools
UUID is the value that you recorded for the load balancer.

VPN Connectivity Tests on Edges

Verify that the L2VPN session is active, identify the peers, and ensure that the tunnel status is up.
get ipsecvpn session active

Verify that the sessions are up.
get ipsecvpn session status

Check whether the ipsecvpn session is up between the local and remote peers.
get ipsecvpn session summary

Get the l2vpn session, tunnel, and IPSEC session numbers, and check that the status is UP.
get l2vpn sessions

Get statistical information of the local and remote peers, whether the status is UP, count of packets received, bytes received (RX), packets transmitted (TX), and packets dropped, malformed, or loops.
get l2vpn session stats

Get the session configuration information.
get l2vpn session config

NSX Manager

Local Users

To change the password of an account run:
set user <username> [password <password> [old-password \<old-password>]]

Authentication Policy Settings for Local Users

Use the following to set:

Password length
set auth-policy minimum-password-length <password-length>

UI and API authentication policies. The UI and API local users have the same policy.
set auth-policy api lockout-period <lockout-period>

set auth-policy api lockout-reset-period <lockout-reset-period>

set auth-policy api max-auth-failures <auth-failures>

Set CLI authentication policy
set auth-policy cli lockout-period lockout-period <lockout-period>

View Logs

NSX CLI

get log-file policy.log

Engineering Mode

Use st en to enter engineering mode (root privileged mode)

st en  
tail /var/log/policy/policy.log  

Syslog – Manager and Edges

set logging-server <hostname-or-ip-address[:port]> proto \<protocol> level <level>

Transport Nodes

Logical Switches List all transport nodes associated with a logical switch.
get logical-switches <switch_UUID> transport-node-table

List all TEPs associated with a logical switch.
get logical-switches <switch_UUID> vtep

List MAC table associated with a logical switch.
get logical-switches <switch_UUID> mac-table

List ARP table associated with a logical switch.
get logical-switches <switch_UUID> arp-table

Deploy Manager Cluster

To deploy a manager to an existing cluster, get the cluster configuration ID, make a note of the existing Managers certificate thumbprint and use that to join the new node to the cluster. Finally get the cluster status to confirm the new host has joined.

get cluster config  
join <NSX-Manager-IP> cluster-id <cluster-id> username<NSX-Manager-username> password<NSX-Manager-password> thumbprint   <NSX-Manager1's-thumbpint>  
get cluster status  

OR via API
POST https://<nsx-mgr>/api/v1/cluster?action=join_cluster

ESXi Configuration

Several esxcli commands can be used to aid in NSX-T configuration.

esxcli network firewall ruleset set -r syslog -e true  
esxcli system syslog config set --loghost=<hostname-or-ip-address[:port]>  
esxcli system syslog reload  

Can also use the nsxcli command set such as:
get logical-switches

N-VDS and Tunnel Information

Show logical switch and N-VDS info
get logical-switch

Display the status of the overlay tunnels Note N-VDS’s used to be called host switches, hence the get host-switch command.
get-host switch <N-VDS_NAME> tunnel

KVM Configuration

Syslog

Login as root
Create this file
/etc/rsyslog.d/40-vmware-remote-logging.conf

Add this line to the file
'*.*@<syslog_server_ip>:514;RFC5424fmt'

Restart syslog
systemctl restart rsyslog

nsxcli can also be used as outlined above. ESXi gives a bit more info as the kernel info is available in ESXi that isn’t there for KVM.

Packet Capture

Can use CLI to setup network packet capture on:

NSX Manager

start capture interface <interface-name> [file <filename>] [count <packet-count>] [expression <expression>]

NSX Edges

set capture session <session-number> interface <port-uuid> direction <direction>

Example.
set capture session 1 interface fp-eth1 direction in
set capture session 1 expression src net 172.20.10.0/24

Removed captured session information with:
del capture session 1

ESXi

Collect packets. Can send to a file.
pktcap-uw

Can Pipe it to view captured packets on the screen.
pktcap-uw | tcpdump -uw

View packets.
tcpdump-uw

KVM

tcpdump

Installation problems

get services  
get service <service name>  
get cluster status  
get configuration  
get managers  

Cluster Configuration Validation

NSX Manager nodes

Transport Node Preparation

Check VIBs installed

vSphere

esxcli software vib list | grep -e nsx -e vsip

This might work too.
esxcfg-module -l | grep nsx

KVM

Ubuntu

dpkg --list | grep nsx

Redhat

rpm -qa | grep nsx

Check TEP and Hyperbus

Hyperbus is for containers.

vSphere

esxcli network ip interface ipv4 address list
Overlay (TEP and vmk10 (default vmk)). Hyperbus (vmk50 default vmk)).

Verify TCP/IP for TEP and hyberbus.
esxcli network ip netstack list
Vxlan is GENEVE in ESXi.

KVM

ifconfig

ovs-vsctl list Open_vSwitch

ovs-vsctl show
Ubuntu KVM hosts create two bridges.

nsx-managed
nsx-vswitch.0

Agents and Connectivity

ESXi

/etc/init.d/nsx-mpa status  
esxcli network ip connection list | grep 5671  
/etc/init.d/nsx-proxy status  
esxcli network ip connection list | grep 1235  

/etc/init.d/nsx-opsagent status

KVM

service nsx-mpa status  
netstat -nap | grep 5671  

service nsx-proxy status  
netstat -nap | grep 1235  

Alternatively, view the connection status using other CLI commands.

ESXi

type the esxcli command.
esxcli network ip connection list | grep 1235

KVM

Type the command netstat -anp –tcp | grep 1235.
user@host:~$ netstat -anp --tcp | grep 1235

Checking Communication from Host to Controller and Manager

ESXi

On an ESXi host using NSX-T CLI commands:

esxi-01.corp.local> get managers

– 192.168.110.19   Connected

esxi-01.corp.local>  
get controllers

Controller IP    Port     SSL         Status       Is Physical Master   Session State  Controller FQDN
192.168.110.16   1235   enabled     connected             true               up               NA

KVM

On a KVM host using NSX-T CLI commands:

kvm-01\>
get managers

– 192.168.110.19   Connected

kvm-01\>
get controllers  

 Controller IP    Port     SSL         Status       Is Physical Master   Session State  Controller FQDN
 192.168.110.16   1235   enabled     connected             true               up               NA

View details of N-VDS The N-VDS has its own command line tool, net-vdl2.
net-vdl2 -l

ESXi LIF MAC View The LIF (Logical Interface) vMAC to pMAC on ESXi host.
net-vdr -C -l
02:50:56:56:44:52 is always the vMAC for the LIFs for all DRs.

DFW Validation

ESXi

nsxt-vsip is the DFW module.

Check dvFilter for firewall rules.
summarize-dvfilter | grep <VM_NAME>

KVM

Use these to validate the distributed firewall Settings. View app firewall virtual interfaces.
ovs-appctl -t /var/run/openvswitch/nsxa-ctl dfw/vif

View firewall rules with containing addrsets.
ovs-appctl -t /var/run/openvswitch/nsxa-ctl dfw/rules <VIF_ID_NUMBER>

CLI Upgrade Confirmation

Post upgrades, use these commands to validate that NSX-T has been upgraded successfully and the correct NSX-T modules are installed.

vSphere

esxcli software vib list | grep nsx

KVM

Ubuntu

dpkg -l | grep nsx

Red Hat

rpm -qa | egrep 'nsx|openvwitch'

nsxcli

get version

Keep the conversation going on Twitter!