VMwareCLI

AUST IT Information Technology for everyone

Connecting Two NSX Instances with BGP
This quick article focusing on Connecting 2 NSX EDGEs via dynamic routing protocol. BGP (Border Gateway Protocol) in this case.

Connecting two NSXinstances using BGP is an essential practice in larger, distributed network environments. BGP helps with dynamic routing between multiple NSX instances, enabling them to share routing information and automatically adjust paths in case of network failures.

Initial requirements:
- Ensure both NSX instances are deployed and operational.
- The NSX Edge devices should be configured and ready to support BGP.
- IP connectivity between the two NSX Edge routers in place. Check connectivity with ICMP for example.
- A BGP ASN (Autonomous System Number) for each NSX instance wil be configured. Make sure you planning it properly. Use ASN from 65XXX.
Steps to Connect Two NSX Instances with BGP
1. Login to NSX Manager: Access the NSX Manager web interface of both NSX instances.
2. Configure BGP on NSX Edge: Navigate to the NSX Edge configuration page on both instances. You will need to configure the BGP settings on the Edge routers that are connected to each network.
  - Go to System > Routing > BGP.
  - Click Add to create a new BGP configuration.
3. Assign BGP ASN: On each NSX Edge, specify the BGP ASN (Autonomous System Number) unique to each instance. Make sure the ASNs are different for each NSX instance to avoid conflicts. Use ASN from 65XXX.
  
  Im my case I will use:
  - NSX Instance 1: ASN 65051
  - NSX Instance 2: ASN 65052
4. Set BGP Neighbor Configuration: Under the BGP configuration, you need to specify the neighbor IP address (the IP address of the remote (second) NSX Edge), as well as the remote ASN of the opposite NSX EDGE.
  
  In my LAB it is:
  - NSX Instance 1:
    - Neighbor IP: 172.17.5.2
    - Remote ASN: 65052
  - NSX Instance 2:
    - Neighbor IP: 172.17.5.1
    - Remote ASN: 65051
5. Configure Networks to Advertise: Decide which networks each NSX Edge will advertise to the other NSX instance. We can do rout-map filtering as well, if required. It allows to control exchanging the routes between EDGEs.
6. Enable BGP: Ensure you enable the BGP session on both NSX Edges. Check the "Enable BGP" box and save the configurations.
7. Verify BGP Session: Once the BGP session is established, verify the session status from the NSX Edge interface:
  - Go to System > Routing > BGP.
  - Ensure the BGP session is Established and that the advertised routes are visible under the Routing Table.
8. Monitor and Troubleshoot:
  - You can use tools like show bgp summary or show bgp neighbor from the NSX CLI for troubleshooting.
  - If the session does not come up, verify IP reachability between the two NSX Edges and check for any misconfigurations in ASNs or neighbor IPs.
  - In my experince it is easy task allow dynamically exchange routing information and automatically adjust paths based on network changes.
Always practice on the LAB before implementing in production.

Any questions, feel free to reach me out. Always happy to help :)
ICMP issue troubleshooting between esxi guests

Issue recently appeared that 2 guests out of 50 not response to ICMP from another vCenter VMs.

Source and destination on the same proadcast domain stratched accross all vCenters.

All VMs in the same vCenter can communicate with no issues.

Just 2 VM experiencing issues.

MAC address appeared on the source VM.

Checked mac address table on all equipements, looks ok.

Next step to do packetcap on vmnic of destination VM.
NSX Firewall Rules investigation

111
Enable RSS in the ESXi host

To take full advantage of the Intel® Converged Network Adapter's capabilities, enable Receive Side
Scaling (RSS) in the ESXi host to balance the CPU load across multiple cores. Refer to the VMware
performance paper VXLAN Performance Evaluation from VMware 5.1 for performance results
with Intel® Converged Network Adapter X520 with RSS enabled:

ESXi VSwitch | Firewall Performance Testing

Enable pnic RSS / mq:
in shell:
vmkload_mod -u ixgbe
vmkload_mod ixgbe RSS=”4,4,4,4,4,4″

in .vmx or advanced config:
ethernetX.pnicFeatures = “4”

allow multiple vnic thread for single vm:
ethernetX.ctxPerDev = “1”
sources:
http://blogs.vmware.com/performance/2015/04/network-improvements-vsphere-6-boost-performance-40g-nics.html
VMware CLI notes
Some useful NSX CLI commands:

System related:
- ESXi build and version numbers
  esxcli system version get
- Show hostname:
  hostname
- Show host IP address:
  hostname -i
- date and time of when ESXi was installed.
  esxcli system stats installtime get
- command to put ESXi in maintenance mode or take it out
  esxcli system maintenanceMode set –enable true
- Use this command to reboot or shutdown ESXi. The -d parameter is a countdown timer; minimum 10 seconds. ESXi must be in maintenance mode before you can use the command.
  esxcli system shutdown reboot -d 10 -r “Patch Updates”
- To display the current parameters state of particular module:
  esxcli system module parameters list -m ixgben
- check disk space on esxi:
  vdf -h
Storage section:
- The command generates a list of extents for each volume as well as the corresponding device name to UUID mapping
  esxcli storage vmfs extent list
- The command lists of all the volumes or datastores accessible by the ESXi host.
  esxcli storage filesystem list
iSCSI section:
- Print the mappings between vmhba names and /dev names (Print HBA devices with identifying information)
  esxcfg-scsidevs -a
- Print the mappings between vmhba names and /dev names (Print mappings for VMFS volumes to their Service Console partitions and vmhba names.)
  esxcfg-scsidevs -m | wc -l
- check...
  vmkchdev -l
- check...esxcli iscsi software get
- List of iSCSI adaptors, UID and states:
  esxcli software adaptor list
- To verify that the local iSCSI initiator has been able to connect to a certain iSCSI SAN we could filter for the TCP/3260 (default port for iSCSI Target) :
  esxcli network ip connection list | grep 3260
- Configuring monitoring using esxtop to monitor storage performance per HBA:
  esxtop -> f (to modify fields) -> Press b, c, d, e, h, and j to toggle the fields and press Enter. -> s and 2 (to change update time)
esxcfg-mpath -b

esxcli storage core device list -d naa.xxxxxxxxxxxx

esxcli storage vmfs lockmode list

esxcli system settings advanced list --option /VMFS3/HardwareAcceleratedLocking

Network section:
- Show IP addresse for all vmk's:
  esxcfg-vmknic -l
less vobd.log

esxcli network ip connections list | grep 1234
- Show all vmnic's settings:
  esxcfg-nics -l
- Show extended vmnic settings:
  esxcli network nic get -n vmnic1
- Show all vitrual switches
  esxcfg-vswitch -l
- Check vmnic statistic:
  esxcli network nic stats get -n vmnic2
- Displays the IPv4 configuration for all the interfaces present on ESXi
  esxcli network ip interface ipv4 get
Network section (vsish):
- Show statistic on card:
  cd /net/pNics/vmknic5 cat stats
Other sections:
- Search in archived logs (.gz):
  zcat vmkernel.*.gz | less
  zcat vmkernel.0.gz | head
  zcat vmkernel.0.gz | grep -i sence | egrep -v "0x5 0x20|0x4 0x24"
zcat vobd.*.gz | grep -i

vmware -vl

VIB sections:
- Show installed VIBs: esxcli software vib list
- Uninstall VIBs: esxcli software vib remove -n nsx-nsxv
- Install VIBs: esxcli software vib install -v /vmfs/volume/BronseNFS/install/VMware_bootbank_esx-nsxv_6.5.0-0.0.85...vib
- Upgrade VIBs: esxcli software vib updrade -v /vmfs/volume/BronseNFS/install/VMware_bootbank_esx-nsxv_6.5.0-0.0.85...vib
Software sections:
- Show module version:
  vmkload_mod -s bnx2i | grep Version
Packet capture:

pktcap-uw --vmk vmk0 --dir 1 -o /tmp/CAPTURE.pcap
pktcap-uw --uplink vmnic3 --dir 1 -o /tmp/CAPTURE.pcap
pktcap-uw --switchport 50331657 --dir 1 -o /tmp/CAPTURE.pcap

Capture dropped packets:
pktcap-uw --capture Drop --uplink vmnic7 --dir 0 -o /vmfs/volumes/XXX/tmp/CAPdc1vesx008vmnic7_in_d
- Compatability IO driver search:
  vmkchdev -l | grep vmn
  https://www.vmware.com/resources/compatibility/search.php?deviceCategory=io
- View the current ARP table using this command:esxcli network ip neighbor list
- Description: Displays the IPv4 configuration for all the interfaces present on ESXi.
  esxcli network ip interface ipv4 get
NSX related sections:
- To find the correct VIBs use URL:https://NSX_Manager/bin/vdn/nwfabric.properties
Virtual Machine related sections:
- Lists all the running VMs on the hosts. The World ID can be used with other commands to carry out various tasks related to VMs
  esxcli vm process list
VMware PowerCLI start notes

VMware PowerCLI 10.0.0 is the new release of PowerCLI and the first to become multi-platform: it adds support for Mac OS and Linux!

Thanks to Microsoft PowerShell Core 6.0, the open-sourced version of PowerShell that’s available on a number of operating systems, from Windows to Linux to Mac OS.
NSX: Uninstalling stuck in progress
Issue explanation:

I went to prepare a cluster but the hosts never fully installed.
I figured I'd uninstall and give it another try.
Now all the hosts in the cluster report "In Progress" with the cluster status "Uninstalling".
I've rebooted all hosts multiple times and checked that the VIBs have been removed.
```
esxcli software vib list
```
```
esx-vxlan
esx-vsip
esx-dvfilter-switch-security
```
The manager and controllers have been rebooted as well.

Solution:

The vibs had already been uninstalled but the task was still hung.
I was able to work around this issue by disconnecting each host and adding back to a different cluster.
I deleted the cluster that was stuck and was then able to prepare the hosts in a new cluster.
The only think I lost was VM folder structure.
The hosts installed without issue and have now been added into my transport zone.

Note:

1. Do not forget to have DNS server and add there:
- vCenter
- esxi's
- NSX Manager
- Controllers
2. Domain Controller is not compulsory but recommended.

Google AdSence

AUST IT - Computer help out of hours, when you need it most.

Find out why we do it for less.

About

AUST IT will help you resolve any technical support issues you are facing onsite or remotely via remote desktop 24/7. More...

Contacts

Reservoir, Melbourne,
3073, VIC, Australia

Phone: 0422 348 882

This email address is being protected from spambots. You need JavaScript enabled to view it.

Sydney: 0481 837 077

Connect

Join us in social networks to be in touch.

Connecting Two NSX Instances with BGP

Initial requirements:

Steps to Connect Two NSX Instances with BGP

Always practice on the LAB before implementing in production.

ICMP issue troubleshooting between esxi guests

NSX Firewall Rules investigation

Enable RSS in the ESXi host

VMware CLI notes

Some useful NSX CLI commands:

VMware PowerCLI start notes

NSX: Uninstalling stuck in progress

Popular tag

Who's Online

Google AdSence

AUST IT - Computer help out of hours, when you need it most.

About

Contacts

Connect

Newsletter