A few of us recently worked on a design that combined a Xilinx Zynq platform with the precision time protocol v2 (PTPv2, a.k.a. IEEE 1588-2008). You can find more details about the protocol here, but the summary is it can help synchronize multiple remote clocks to within (potentially) a few nanoseconds of one another in Linux using daemons like LinuxPTP (Yocto integration here). Other implementations of the spec. exist for Audio-video Broadcast, for example, that tighten the tolerances even further.

We were told by at least one source that there were concerns the Zynq in this configuration would not be able to converge to anything better than perhaps a few hundred nanoseconds. Per the wiki however, timing closer to 100 ns should be possible. Read on to see how we did.

Hardware PTP

One of the key requirements for getting down into that tight synchronization is the use of hardware timestamping. For a typical Zynq, this would normally mean utilizing the hardware TSU (time stamping unit) on the PS-connected GEM (i.e., connected to the ARM processor directly). However according to this Xilinx tech tip, that option has been deprecated since the built-in TSU isn’t up to the task.

So how can you proceed? In short, you use a PL-connected MAC/PHY that can support timestamping and a precise real-time clock (RTC).

MAC / PHY

The requirement of a PL-connected MAC was fairly straight-forward in our case. We were using the Avnet FMCv2 carrier card, which has a PL-attached SFP cage capable of the 1 Gbps operation we needed. As for the MAC logic, we used the Xilinx 2018.3 1G/2.5G AXI Ethernet Subsystem, which can be configured for IEEE 1588-2008 support as long as you select 1000BASE-X operation (see that wiki).

NOTE: Vivado 2018.3 has a (now known) bug in generating the subsystem which misconfigures a FIFO and leaves out an important logic element for controlling that FIFO. We’ve been told a fix is forthcoming. Please contact us if you need help.

Using this core, you will also need to enable the related Kernel drivers per the Wiki, for example:

CONFIG_XILINX_AXIEMAC=y
CONFIG_XILINX_AXIEMAC_HWTSTAMP=y

When you configure the MAC in this way, systemtime_* inputs are exposed for seconds and nanoseconds counters which you must drive with an RTC or some other logic that can be managed by the PTPv2 daemon.

In the picture, you can see how the RTC core’s time_* outputs are connected to the AXI Ethernet Subsystem (AES) systemtime_* inputs of the same name. So, next up is the RTC.

RTC

The real-time clock we chose is available on OpenCores, here. Firmware integration is straight-forward with this core, and the included test script provides a register map. That register map was easily translated into a Linux Kernel driver, and added to Geon’s meta-ha1588 Yocto layer (consequently, if anyone adds drivers for the other cores, please feel free to submit a pull request).

Carefully follow the README and your integration works a bit like this:

1. Add the layer to your environment
2. Set KERNEL_FEATURES_append = " ha1588.scc" somewhere appropriate, like local.conf or your machine definition.
3. Update your device tree source per the README
4. Compile and install the kernel.

NOTE: If you’re not using the linux-yocto kernel, you will need to append your kernel recipe with include recipes-kernel/linux/ha1588.inc.

At this point, you should have the following two devices: /dev/rtcN and /dev/ptpM, where N and M are enumerations as the devices came online. You can test this interface using the hwclock utility to get/set time.

Testing PTPv2

In our lab, we had several of these devices and other network hardware. We configured one device as a master clock (undisciplined) using LinuxPTP by running:

hwclock --systohc
ptp4l -m -q -i ethX

The HA1588 RTC is a “soft” RTC which resets on power-up / programming of the FPGA. Therefore we take the current system time and write it to the RTC first, which synchronizes our timestamping engine to the system time. The X is the enumeration of that NIC.

At the slave system, we ran: ptp4l -m -q -i ethY -s. The Y is the enumeration of the slave NIC, and you’ll notice we did not bother synchornizing system time to the RTC since we’ll be getting that by way of our PTP master.

After a few seconds, the timing of the two systems converged to less than 100 nanoseconds.

ptp4l[142.618]: master offset         36 s2 freq   -1858 path delay       134
ptp4l[144.619]: master offset          4 s2 freq   -1868 path delay       143
ptp4l[146.619]: master offset         14 s2 freq   -1863 path delay       143
ptp4l[148.621]: master offset         10 s2 freq   -1862 path delay       143
ptp4l[150.622]: master offset         -5 s2 freq   -1868 path delay       154
ptp4l[152.624]: master offset          4 s2 freq   -1865 path delay       157

Important: LinuxPTP is actually two daemons. The first, used above, is for synchronizing the timestamping engine (NIC to PTP). The second is for synchronizing something else to that engine (i.e., PTP to RTC device). In our case, those two devices are the same, so use of the second daemon would be redundant.

Conclusion

As one can see, it’s possible to achieve tight timing synchronization with a Zynq and PL-attached MAC when using an off-the-shelf IP core and free software from us here at Geon. If you would like to know more information about this implementation or have other timing concerns in your design, please don’t hesitate to contact us.