01 Feb 2020 by Simon Greaves
Here is a reference guide for some useful command line tools you can run on NSX-T.
NSX CLI (nsxcli) is the command line tool for troubleshooting NSX-T. It’s run in a non-root mode so you have to use the command structure available. For instance there is no grep, But you can use find and pipe instead.
You can use the nsxcli command line tool from various elements throughout the NSX-T deployment including NSX Manager, Edges and ESXi Transport Nodes.
SSH Access to Edges
FYI I used Edge VMs throughout this article.
Post deployment, open the console and verify the SSH service is stopped.
get service ssh
Start the SSH service.
start service ssh
Set the SSH service to autostart when the VM is powered on.
set service ssh start-on-boot
Verify that the SSH service is running and Start on boot is set to True.
get service ssh
Disable the command-line timeout.
set cli-timeout 0
Obtain routing information for the gateways.
Verify that the SERVICE_ROUTER_TIER0 service gateway appears with an associated VRF ID.
In the command output, VRF 2 is associated with SR-T0-GW. The VRF ID in your lab might be different.
Vrf is used to access the gateway virtual routing functions. For BGP and other Tier-0 services you access the Tier-0 service router (SR) function of the gateway.
verify the BGP state.
get bgp neighbor summary
Check the status is established. A status of Active means still setting up!
View further information on the BGP connection. Also shows whether the connection is established or not.
get bgp neighbor
View ipv4 bgp information.
get bgp ipv4
Exit the Tier-0 VRF service gateway mode.
get logical-routers vrf # get route
b \> is a route learned from a BGP peer.
For DR check the forwarder for similar routing information.
In the previous example I was connected to the SR to collect that forwarding information, but you can also get it from the DR. This is useful for seeing routing configuration for DR components throughout the environment, such as those on a Tier-1 DR.
Runs on Edges.
get dhcp servers
get dhcp ip-pools
get dhcp leases
The output shows the general load balancer configuration, including UUID and Virtual Server ID.
Copy the UUID and the Virtual Server ID values and paste them.
Verify the virtual server configuration.
get load-balancer UUID virtual-server <Virtual_Server_ID>
Verify the server pool configuration.
get load-balancer UUID pools
UUID is the value that you recorded for the load balancer.
Verify that the L2VPN session is active, identify the peers, and ensure that the tunnel status is up.
get ipsecvpn session active
Verify that the sessions are up.
get ipsecvpn session status
Check whether the ipsecvpn session is up between the local and remote peers.
get ipsecvpn session summary
Get the l2vpn session, tunnel, and IPSEC session numbers, and check that the status is UP.
get l2vpn sessions
Get statistical information of the local and remote peers, whether the status is UP, count of packets received, bytes received (RX), packets transmitted (TX), and packets dropped, malformed, or loops.
get l2vpn session stats
Get the session configuration information.
get l2vpn session config
To change the password of an account run:
set user <username> [password <password> [old-password \<old-password>]]
Use the following to set:
set auth-policy minimum-password-length <password-length>
UI and API authentication policies.
The UI and API local users have the same policy.
set auth-policy api lockout-period <lockout-period>
set auth-policy api lockout-reset-period <lockout-reset-period>
set auth-policy api max-auth-failures <auth-failures>
Set CLI authentication policy
set auth-policy cli lockout-period lockout-period <lockout-period>
get log-file policy.log
st en to enter engineering mode (root privileged mode)
st en tail /var/log/policy/policy.log
set logging-server <hostname-or-ip-address[:port]> proto \<protocol> level <level>
List all transport nodes associated with a logical switch.
get logical-switches <switch_UUID> transport-node-table
List all TEPs associated with a logical switch.
get logical-switches <switch_UUID> vtep
List MAC table associated with a logical switch.
get logical-switches <switch_UUID> mac-table
List ARP table associated with a logical switch.
get logical-switches <switch_UUID> arp-table
To deploy a manager to an existing cluster, get the cluster configuration ID, make a note of the existing Managers certificate thumbprint and use that to join the new node to the cluster. Finally get the cluster status to confirm the new host has joined.
get cluster config join <NSX-Manager-IP> cluster-id <cluster-id> username<NSX-Manager-username> password<NSX-Manager-password> thumbprint <NSX-Manager1's-thumbpint> get cluster status
OR via API
Several esxcli commands can be used to aid in NSX-T configuration.
esxcli network firewall ruleset set -r syslog -e true esxcli system syslog config set --loghost=<hostname-or-ip-address[:port]> esxcli system syslog reload
Can also use the nsxcli command set such as:
Show logical switch and N-VDS info
Display the status of the overlay tunnels
Note N-VDS’s used to be called host switches, hence the get host-switch command.
get-host switch <N-VDS_NAME> tunnel
Login as root
Create this file
Add this line to the file
systemctl restart rsyslog
nsxcli can also be used as outlined above. ESXi gives a bit more info as the kernel info is available in ESXi that isn’t there for KVM.
Can use CLI to setup network packet capture on:
start capture interface <interface-name> [file <filename>] [count <packet-count>] [expression <expression>]
set capture session <session-number> interface <port-uuid> direction <direction>
set capture session 1 interface fp-eth1 direction in
set capture session 1 expression src net 172.20.10.0/24
Removed captured session information with:
del capture session 1
Collect packets. Can send to a file.
Can Pipe it to view captured packets on the screen.
pktcap-uw | tcpdump -uw
get services get service <service name> get cluster status get configuration get managers
Check VIBs installed
esxcli software vib list | grep -e nsx -e vsip
This might work too.
esxcfg-module -l | grep nsx
dpkg --list | grep nsx
rpm -qa | grep nsx
Hyperbus is for containers.
esxcli network ip interface ipv4 address list
Overlay (TEP and vmk10 (default vmk)). Hyperbus (vmk50 default vmk)).
Verify TCP/IP for TEP and hyberbus.
esxcli network ip netstack list
Vxlan is GENEVE in ESXi.
ovs-vsctl list Open_vSwitch
Ubuntu KVM hosts create two bridges.
/etc/init.d/nsx-mpa status esxcli network ip connection list | grep 5671 /etc/init.d/nsx-proxy status esxcli network ip connection list | grep 1235
service nsx-mpa status netstat -nap | grep 5671
service nsx-proxy status netstat -nap | grep 1235
Alternatively, view the connection status using other CLI commands.
esxcli network ip connection list | grep 1235
Type the command netstat -anp –tcp | grep 1235.
netstat -anp --tcp | grep 1235
On an ESXi host using NSX-T CLI commands:
esxi-01.corp.local> get managers – 192.168.110.19 Connected
esxi-01.corp.local> get controllers Controller IP Port SSL Status Is Physical Master Session State Controller FQDN 192.168.110.16 1235 enabled connected true up NA
On a KVM host using NSX-T CLI commands:
kvm-01\> get managers – 192.168.110.19 Connected
kvm-01\> get controllers Controller IP Port SSL Status Is Physical Master Session State Controller FQDN 192.168.110.16 1235 enabled connected true up NA
View details of N-VDS
The N-VDS has its own command line tool,
ESXi LIF MAC
View The LIF (Logical Interface) vMAC to pMAC on ESXi host.
net-vdr -C -l
02:50:56:56:44:52 is always the vMAC for the LIFs for all DRs.
nsxt-vsip is the DFW module.
Check dvFilter for firewall rules.
summarize-dvfilter | grep <VM_NAME>
Use these to validate the distributed firewall Settings.
View app firewall virtual interfaces.
ovs-appctl -t /var/run/openvswitch/nsxa-ctl dfw/vif
View firewall rules with containing addrsets.
ovs-appctl -t /var/run/openvswitch/nsxa-ctl dfw/rules <VIF_ID_NUMBER>
Post upgrades, use these commands to validate that NSX-T has been upgraded successfully and the correct NSX-T modules are installed.
esxcli software vib list | grep nsx
dpkg -l | grep nsx
rpm -qa | egrep 'nsx|openvwitch'
Keep the conversation going on Twitter!Reply with Twitter