Testing and developing with 5G networks is often costly for one main reason, the testbed. Traditional 5G testbeds can cost upwards of $100,000 in hardware alone which prices out many organizations from learning about and contributing to 5G development. Geon has developed a testbed based on open source software and Commercial-Off-The-Shelf (COTS) hardware; lowering the high barrier of entry into 5G development. We based our testbed on Open Air Interfaces, Ettus Universal Software Radio Peripheral (USRP) Software Defined Radio (SDRs), and Google Pixel Phones
Kubernetes and the 5G core network
For the last few months, our team has been working with Open Air Interface (OAI), an open source 5G software project created for the purpose of providing a 3GPP-Compliant 5G Standalone (SA) tool set. OAI includes all the code necessary to run a fully virtual 5G core network both in a docker context and, more importantly to our team, a Kubernetes (K8s) context. Our testbed has three Dell desktops, all running on Ubuntu 18.04. We created a 3 node K8s cluster with one K8s control plane, which also hosts the gNB (Next Generation NodeB), and two worker nodes; see figure below. The OAI community provides a series of Helm charts that when combined, launch a fully operational core network in a K8s cluster. These charts enable our team to startup and tear down the core network with a single command in a matter of minutes while easily adjusting configurations that can be tracked and shared through our Git process. However, some updates were still required to these charts to standup an operational system. In the remainder of this post, we will detail those changes in the hopes of helping you get started with 5G development more quickly and easily.
RAN and UE devices
Geon’s testbed requires a core network to authenticate and administer connections, a gNB to act as our base station, and UE (User Equipment) to act as our users. The Helm charts that had been created by OAI were only built to support software simulation and could not support the B205-mini USRPs hardware that we intend to use for our use case. The fist step was to make some modifications to the software provided to get both a gNB K8s pod and a UE K8s pod running while using the hardware attached to their node. Geon also made use of the Ettus plugin to allow the cluster to manage resource allocation of SDRs. We were able to update the charts to be able to automatically detect our radio hardware and launch pods throughout the cluster, completing the requirements of having both a core network and the appropriate RAN and UE devices all running with basic Helm charts. This began the first few stages of testing.
Defining Success
The OAI documentation gave a few examples of a successful UE connection, based on the core network logs, but we had some extra steps in mind for our development purposes.
- The gNB must register with the AMF successfully
- The gNB must synchronize with the UE and assign it a UEID
- The AMF must register the UE
- The SMF must approve the UE for a PDU session and give it an IP within the network slice that had been configured
- The UE must be capable of pinging the internet through the interface provided by the core network
OAI SW UE
We first attempted to meet these requirements with a reference implementation using the OAI provided software UE implementation — no Pixel. We had to make a few updates to the Helm charts to ensure proper network slices and 5G bands were being used. This step was pretty turnkey for us and required only minor updates.
COTS UE
With our reference implementation operational, the next step was to replace the OAI software UE with a COTS UE, the Google Pixel. Step 1 of our previous test was easily accomplished because the gNB remained the same. Step 2 however presented some challenges. Even with strong signal link between the gNB and the Pixel, the gNB would drop the connection to the UE shortly after synchronization, occasionally making it to the beginning of step 3 but never establishing a PDU session. Since we decided not to not “root” our Pixel 5, we had limited visibility what was happening from the UE perspective — minimal debug and logging are available without custom software loads and configuration. Instead, we began looking at the RF environment in our testbed by making RF captures to gain insight into the behavior we were observing. The figure below shows a transmitted gNB synchronization block.
Additional captures showed that the Pixel would occasionally skip entire frames; see “Google Pixel 5 – Skipped Synchronization Frames” figure below. The gNB was configured to drop the connection of any UE that skipped responses to a synchronization block. Once we adjusted the gNB settings to allow for greater frame inactivity, step 2 was quickly accomplished.
Step 3 also required some updates to our configurations. Within the core network there is a database that holds all of the pre-registered UE’s and their authentication info. When a UE begins registration with the AMF, the AMF sends request to other parts of the core network to authenticate the UE based on the International Mobile Subscriber Identity (IMSI) provided. Similar to a commercial 5G carrier, we had to pre-register our Pixel in the core network, allowing the core network to successfully authenticate the UE. This involved modifying the database in the core network and also writing to the Pixel’s SIM card. Using examples found at the OpenCells website we retrieved the IMSI and other authentication info from the sim card, and updated the core networks database to include our Pixel as a registered user. With a quick restart to apply the changes, step 3 was accomplished.
An important note about the IMSI that we specified on the simcards, the IMSI had to begin with 00101 which specifies a PLMN(Public Land Mobile Network) for a test network. In our testing we found that this was required in order to put the Pixel into a mode that was compatible with the OAI network that was launched.
Step 4 proved challenging. We discovered that the network slicing configuration present in every config file, had been hand tailored by OAI to match their exact test scenarios. If a UE asked for a PDU session in a different network slice, the SMF found in the core network would see the mismatch and reject the UE’s request for a PDU session. While we couldn’t determine a way to dictate which network slice the Pixel asked for, we could modify the OAI charts further to include the slice that was being requested. A few trial and errors later, the SMF logged a successful chain of slicing matches and a PDU session was granted to our Pixel.
Finally we reached step 5 of our testing. Having received a PDU session from the SMF, the Pixel was assigned a valid IP address but still could not reach the internet. Our investigation focussed on the configured network slices in our core network. Each network slice is given a range of IP addresses to assign to UE’s in that slice and those address ranges are then passed to the User Plane Function (User Plane Function) as a set of interfaces on which the UE can communicate. Due to a mismatch in those ranges between the SMF and the UPF, the UE could receive an address in a valid range, given by the SMF, but all traffic would get lost on the wrong interface in the UPF. A Final configuration update and we were soon able to show successful pings through the UE, completing step 5 of our testing.
Once the network connection was happening with a good level of consistency a speed test of the test network was preformed with good results:
Supporting Code
If you would like to reproduce our results in your own lab the updates that Geon made to the OAI helm charts and kubernetes instalation tooling is published in this gitlab repository.
Next Steps
With a full 5G testbed at our disposal and the option of using commercial 5G equipment, Geon is now in a position to perform 5G testing and development from our own lab, enabling open source 5G testing and research.
