Caveats When Deploying Nexus9000v

As I’m building a VXLAN lab based on Nexus9000v, I ran into some caveats while deploying. Some things are related to ESX (vSphere) only while others apply to also other platforms.

The boot process for Nexus9000v is a bit special. It requires using a serial console to access switch prompt and from there booting the NX-OS image. There are a few steps to enable this in vCenter. For the VM that was deployed using the OVA, edit settings of the VM and go to Virtual Hardware -> Serial port 1:

Use the following settings:

• Use Network.
• Direction – Server.
• Port URI – telnet://<portnumber>.

Note that when selecting a port number, it must be a port above 1024.

Next, under VM Options, go to Advanced and select Edit Configuration…

Click Add Configuration Params and add the following entry:

efi.serialconsole.enabled with a value of TRUE:

The server also needs to have firewall ports open. This is done by going to Configure -> System -> Firewall:

Make sure that VM serial port is enabled.

Then, power on the device which will boot to Loader. Boot on the image stored on bootflash:

Loader Version 5.9.3.94

loader > dir                                                                   

bootflash::  

  .rpmstore
  nxos64-cs.10.2.5.M.bin
  bootflash_sync_list
  .swtam
  evt_log_snapshot
  log_profile.yaml
  eem_snapshots
  virtual-instance
  intersight
  scripts
  20230818_061159_poap_5022_init.log
  poap_retry_debugs.log
  nexuscloud

bootflash::  


loader > boot nxos64-cs.10.2.5.M.bin

The device will then boot:

Booting nxos64-cs.10.2.5.M.bin 
Trying diskboot 
 Filesystem type is ext2fs, partition type 0x83
Image valid
Boot Time: 8/18/2023  8:59:1
[    0.148954] RETBleed: WARNING: Spectre v2 mitigation leaves CPU vulnerable to RETBleed attacks, data leaks possible!
[    2.499348] check if this has MMC
[    2.500115] Register as MTD

Skip the POAP process by answering yes:

Waiting for system online status before starting POAP ...

Abort Power On Auto Provisioning [yes - continue with normal setup, skip - bypass password and basic configuration, no - continue with Power On Auto Provisioning] (yes/skip/no)[no]: yes
Disabling POAP.......Disabling POAP
2023 Aug 18 09:13:27 switch %$ VDC-1 %$ poap: Rolling back, please wait... (This may take 5-15 minutes) 
Disabling lldp

Configure the admin account:

         ---- System Admin Account Setup ----


Do you want to enforce secure password standard (yes/no) [y]: 

  Enter the password for "admin": 
 
  Confirm the password for "admin": 

Skip the basic setup:

---- Basic System Configuration Dialog VDC: 1 ----

This setup utility will guide you through the basic configuration of
the system. Setup configures only enough connectivity for management
of the system.

Please register Cisco Nexus9000 Family devices promptly with your
supplier. Failure to register may affect response times for initial
service calls. Nexus9000 devices must be registered to receive 
entitled support services.

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime
to skip the remaining dialogs.

 Would you like to enter the basic configuration dialog (yes/no): no

Once logged in, add a boot statement or it will boot to the Loader again:

switch# conf t
switch(config)# boot nxos nxos64-cs.10.2.5.M.bin
Performing image verification and compatibility check, please wait....
switch(config)# copy run start
[########################################] 100%
Copy complete, now saving to disk (please wait)...
Copy complete.

It took a while to get the device booted due to these caveats but now it’s working. I hope it’s easier for you when following the steps above.