🔧 Ethernet Debug Playbook
Quick Reference Guide for MRMAC/DCMAC & AXI 10G/25G Ethernet
MRMAC/DCMAC Debug
Multi-Rate MAC (MRMAC) & 600G Channelized Multirate Ethernet Subsystem (DCMAC)
📚 Driver Resources
🎯 Reference Designs & Examples
📅 Version & Support Timeline
🏗️ MRMAC Support History:
- 2021.1+: MRMAC support introduced (patches required from TRD)
- 2023.2+: WTS officially started supporting MRMAC
- It's possible to obtain patches from the official VCK190 Ethernet TRD repository when the version and configuration matches with TRD
2021.1+
2023.2 (WTS)
2024.1
2025.1
⚙️ Notable Configurations
🎯 4x25G Configuration (Standard)
- Standard quad configuration (WTS supported from 2023.2)
- Simpler clock setup
- Most common use case
- Reference: VCK190-Ethernet/2023.2/pl_4x25g_mrmac
🎯 3x25G Configuration
- Requires specific clock configuration
- Check GT lane assignments
- Not so common in Versal; some customers like to share the GT Quad with a different IP for example Aurora
- Reference: See above under Reference Designs & Examples
🎯 1x100G CAUI-4 Configuration
- Support from 2024.2
- Four lanes operating at 25.78125 Gbps
- Reference: VCK190-Ethernet/2024.2/pl_1x100g_mrmac
🔧 Common Issues & Solutions
🔴 Link Up Issues (Most Critical) ▼
⚡ Quick Link Up Troubleshooting Steps:
- Start with DAC (Direct Attached Cable) - Simplest setup to isolate issues
- Verify Clock in IBERT - Ensure GT clocks are stable and correct frequency
- Verify Device Tree - Check DTS bindings, compatible strings, and PHY configuration
GT Configuration Mismatch
Check GT reference clock, lane polarity, and protocol settings using IBERT
Critical
Device Tree Configuration
Verify compatible string, reg addresses, clocks, and PHY mode in DTS. An example system-user.dtsi is provided above in the Reference Designs section.
Config
Does Issuing GT RX datapath reset help?
The provided reference designs has AXI GPIOs connected to the reset port(s), use devmem command to toggle the reset.
Hardware
🟡 Performance Issues ▼
Throughput Degradation
Check for packet drops, buffer sizing, and DMA configuration
Performance
🟢 Driver Issues ▼
⚠️ Common Axienet Driver Errors:
At the time of this writing, the axienet driver errors out for 3 main reasons:
- ❌ GT Ref Clock Issue: When the GT ref clock is not available or correct
netdev_err(ndev, "MRMAC PLL lock not complete! Cross-check the MAC ref clock configuration\n"); - ❌ Link Partner Missing: When the link partner is not there
netdev_err(ndev, "MRMAC block lock not complete! Cross-check the MAC ref clock configuration\n"); - ❌ No RX Valid Control Code: When there is no recent RX valid control code
netdev_err(ndev, "MRMAC Link is down! No recent RX Valid Control Code\n");
Then we expect to see this message:
netdev_err(ndev, "MRMAC Link is down!\n");
However, when the driver is happy, we expect to see something like the example shown below.
MRMAC setup as <RATE>, for example at 10G:
[ 1307.699546] xilinx_axienet a4000000.mrmac eth0: MRMAC setup at 10000
Probe Failures
Check DT bindings, compatible strings, and resource allocation. Compare with working system-user.dtsi example.
Driver
Driver Patch Required
Some versions require patches from TRD. Check the driver patch example for PTP and other features.
Patch
MCDMA Kernel Configuration
If using MCDMA, ensure CONFIG_AXIENET_HAS_MCDMA is enabled in kernel configuration
Config
💻 Useful Commands
# Check interface status and capabilities
ethtool eth0
# View MRMAC driver messages
dmesg | grep -i mrmac
# View AXI Ethernet driver messages
dmesg | grep -i axienet
# Check link status
ip link show eth0
# Monitor detailed statistics
ethtool -S eth0
# View interface configuration
ifconfig eth0
📋 Debug Checklist
- ✅ Verify MRMAC version support (2021.1+ with patches, 2023.2+ WTS)
- ✅ Check if patches from TRD are applied (when version/config matches)
- ✅ Use DAC (Direct Attached Cable) for initial testing
- ✅ Verify GT clocks in IBERT - stable and correct frequency
- ✅ Validate Device Tree configuration against system-user.dtsi example
- ✅ Check compatible string matches driver expectation
- ✅ Verify PHY mode configuration (25gbase-r, etc.)
- ✅ If using MCDMA, ensure CONFIG_AXIENET_HAS_MCDMA is enabled in kernel config
- ✅ Check driver probe messages in dmesg
- ✅ Review driver patch example if custom features needed (PTP, etc.)
- ✅ Did it work in internal loopback mode? Try internal Near-end PMA loopback (no need to modify the CRDHOLD in this mode; easiest internal loopback mode to configure from the IBERT)
10G/25G Ethernet Subsystem Debug
10G/25G Ethernet Subsystem
📚 Driver Resources
🎯 Reference Designs & Examples
⚙️ Hardware Setup (Zynq MPSoC)
⚠️ Critical Hardware Checks for Zynq MPSoC:
- TX_DISABLE Jumper: Verify jumpers for TX_DISABLE are properly set
- Si570 Programming: The Si570 oscillator must be programmed to provide the correct reference clock for the GT
- GT Reference Clock: 156.25 MHz for 10G operation
⏰ Clock Requirements
- AXI Lite Clock: 100 MHz (config interface)
- AXI Stream Clock: 156.25 MHz (10G) / 390.625 MHz (25G)
- GT Reference Clock: 156.25 MHz
- DMA Clock: Match AXI Stream clock
🔧 Common Issues & Solutions
🔴 Link Up Issues (Most Critical) ▼
⚡ Quick Link Up Troubleshooting Steps:
- Start with DAC (Direct Attached Cable) - Simplest setup to isolate issues
- Check TX_DISABLE Jumpers (Zynq MPSoC) - Ensure jumpers are in correct position - Checkout the board user guide or schematics to find out board specifics
- Verify Si570 Programming (Zynq MPSoC) - Confirm reference clock is correctly programmed
- Verify Clock in IBERT (Versal) - Ensure GT clocks are stable and correct frequency
- Verify Device Tree - Check DTS bindings, compatible strings, and PHY configuration
GT Configuration Mismatch
Check GT reference clock, lane polarity, and protocol settings using IBERT
Critical
Device Tree Configuration
Verify compatible string, reg addresses, clocks, and PHY mode in DTS. An example system-user.dtsi is provided above in the Reference Designs section.
Config
🟡 DMA Issues ▼
DMA Reset Timeout
If DMA reset timeout persists, it may be necessary to increase the timeout value directly in the driver by patching it
Critical
Patch
🟢 Driver Issues ▼
Probe Failures
Check DT bindings, compatible strings, and resource allocation. Compare with working system-user.dtsi example.
Driver
MCDMA Kernel Configuration
If using MCDMA, ensure CONFIG_AXIENET_HAS_MCDMA is enabled in kernel configuration
Config
🔵 Hardware Issues ▼
GT Configuration
Check lane assignments, polarity, and reference clock. Use IBERT to verify.
Hardware
FIFO Underflow/Overflow
Check FIFO depth settings in IP configuration
Hardware
💻 Useful Commands
# Check interface status and capabilities
ethtool eth0
# View AXI Ethernet driver messages
dmesg | grep -i axienet
# View DMA driver messages
dmesg | grep -i dma
# Check link status
ip link show eth0
# Monitor detailed statistics
ethtool -S eth0
# Check device tree probe
dmesg | grep "xilinx_axienet"
# View interface configuration
ifconfig eth0
# Monitor DMA interrupts
cat /proc/interrupts | grep dma
📋 Debug Checklist
- ✅ Check TX_DISABLE jumpers (Zynq MPSoC boards)
- ✅ Verify Si570 is programmed correctly for GT reference clock
- ✅ Use DAC (Direct Attached Cable) for initial testing
- ✅ Verify GT clocks in IBERT - stable and correct frequency (156.25 MHz)
- ✅ Validate Device Tree configuration against system-user.dtsi example
- ✅ Check compatible string matches driver expectation
- ✅ Verify PHY mode configuration (xgmii, etc.)
- ✅ If using MCDMA, ensure CONFIG_AXIENET_HAS_MCDMA is enabled in kernel config
- ✅ Check driver probe messages in dmesg
- ✅ If DMA reset timeout persists, consider patching driver to increase timeout
