- Table of Contents
- What is EdgeNet?
- Prerequisites
- Installation
- Tutorials
- Components
- Concepts
- Development and Contributing
EdgeNet is a free and open-source cloud orchestration software extension that brings industry-standard cloud software Kubernetes to the network edge.
It is important to distinguish EdgeNet software and EdgeNet Testbed. If you are a user of EdgeNet Testbed please refer to the Testbed's website. This documentation is intended for users and developers of EdgeNet software (will be referred to as simply EdgeNet).
The EdgeNet's source code is hosted in the official Github repository.
To effectively understand how to use EdgeNet in your Kubernetes cluster, it is required to have a basic understanding of Kubernetes. For more information on how Kubernetes operates, you can refer to the Kubernetes documentation.
To deploy EdgeNet to your Kubernetes cluster, we recommend following the advanced installation tutorial. To use the command line tool fedmanctl we recommend following the fedmanctl installation tutorial.
If you want to remove EdgeNet from your cluster, refer to the removing EdgeNet from a Kubernetes cluster tutorial.
We have provided some of the tutorials below for using different functionalities:
- Registering a Tenant
- Creating a Node-level Slice for Your Tenant
- Subnamespace Creation for Your Tenant
- Configure User Permissions
- Role Request
- Cluster Role Request
We have also added tutorials for fedmanctl command line tool. They can be accessed here:
Old EdgeNet tutorials can be accessed under the /doc/tutorials/old folder. See the old tutorial's Readme.
We have devised 4 major categories of features that EdgeNet offers and grouped the corresponding CRDs. However, this classification does not fully differentiate CRDs. For instance, the Selective Deployment CRD takes the role in both federation and location-based node selection categories.
Here are the categories and their associated CRDs:
To facilitate federation capabilities, our EdgeNet project incorporates a command line utility called fedmanctl. This utility serves the purpose of enabling the federation of workload and manager Kubernetes clusters.
Currently, fedmanctl is undergoing active development, focusing primarily on implementing the core components related to federation features. While it is a work in progress, essential functionalities have already been incorporated. For the basic use case of federating a workload Kubernetes cluster, please consult the fedmanctl federation tutorial.
fedmanctl comprises two modules known as workload and manager. The workload module encompasses subcommands that initialize the federation capabilities of EdgeNet and generate a token for the manager cluster. This token contains sensitive information that grants external access to the workload cluster's API server. On the other hand, the manager subcommands are responsible for establishing a link between the workload cluster and the manager cluster using the token.
In the future, additional subcommands may be developed to establish connections between manager Kubernetes clusters.
To extend Kubernetes API, EdgeNet makes use of Kubernetes CRDs (custom resource definitions). CRDs allow us to define custom API objects to be manipulated by the Kubernetes API. These components and their properties are explained in components seciton.
A controller in Kubernetes listens for changes in an object. Then it tries to converge the desired state of the object with the current state. It is also possible to implement custom controllers that may contain business logic for custom objects. EdgeNet employs its custom controllers can be found under the pkg/controllers folder and uses the design of Kubernetes' sample-controller example. Additionally, custom controllers usually run inside the cluster in a production environment. The images of these custom controllers can be found on EdgeNet's DockerHub page.
The architecture of the EdgeNet custom resources and controllers runs in the control plane along with other Kubernetes components. To ensure better isolation of workloads, we advise using Kata Containers as the container runtime, but this is optional and any container runtime can be configured when creating the Kubernetes cluster. You can refer to official Kubernetes documentation on how to create Kubernetes clusters for more information.
EdgeNet uses the tenant custom resource to represent a tenant where each tenant is a customer that contracts for services on behalf of one or more users. EdgeNet provides multitenancy support to Kubernetes clusters it has installed. This means a tenant and its workloads cannot access resources, or objects reserved for other tenants or their workloads.
In some cases, we want to ensure the resources are reserved for a specific tenant. This requirement is satisfied by a mechanism called slicing, which is assigned to tenants by creating slice claims. There are two types to create two types of slices; Node-level slices allow the reservation of whole nodes just for a single tenant. Sub-node-level slices, on the other hand, allow granular resources on a selected node to be reserved.
There is also the subnamespace mechanism implemented in EdgeNet to ensure tenants create non-flat namespaces with specific resource quotas. The resource limitations are also propagated when a new subnamespace is added. This can be seen in the figure below. r represents the root namespace which has a specified quota of 100 units. Note that, r doesn't have a quota directly since it is an abstraction. However, each other namespace exists in the flat namespaces of Kubernetes thus, they also have a quota assigned to them. For example, when the two subnamespaces aa and ab are added to the subnamespace a, the 60-unit resource is divided by 3 to 20, 25, and 15 units.
Lastly, the tenants can have tenant resource quotas. Which puts a limit to a tenant's useable resource quota.
EdgeNet utilizes a custom resource known as selective deployment to facilitate the specification of a deployment's geographic area. This feature enables users to precisely define the geographical boundaries within which their deployment operates. To support this mechanism, it is essential to determine the geographic locations of the nodes involved.
To achieve this, EdgeNet employs a node labeler, which is responsible for assigning geographical labels to each node. The node labeler plays a crucial role in accurately identifying and categorizing the nodes based on their physical locations. By labeling the nodes accordingly, EdgeNet can effectively manage the selective deployment of resources and ensure that workloads are distributed within the specified geographic area.
The combination of the selective deployment custom resource and the node labeler enhances EdgeNet's capabilities in achieving targeted and geographically constrained deployments. This enables users to have greater control over the geographical distribution of their resources and optimize their system's performance based on specific requirements or constraints.
The incorporation of federation support within EdgeNet empowers multiple clusters to collaborate and distribute workloads efficiently. This feature aims to facilitate the creation of a flexible system by enabling clusters from different providers. By leveraging the federation capabilities, EdgeNet users can harness the advantages of geographically distributed resources and optimize their workload management.
However, it is important to note that the federation features are currently in the development phase and have not yet reached their full potential in the release-1.0 version of EdgeNet. The development team is actively working to enhance and refine these features to ensure their reliability and effectiveness.
If you are familiar with the Standard Go Project Layout used by other Kubernetes-related projects, you will easily be able to navigate the EdgeNet repository.
To get a sense of where we are heading, please see our planned features board. We follow an agile development approach, with two-week sprints, each one leading to a new psroduction version of the code. Our current sprint is one of the milestones, and you can see more near-term issues in our project backlog.
To start work, clone the latest release branch. If you add a new piece of code, please make sure you have prefaced it with the standard copyright notice and license information found in other places in the code. If you have an idea or an implementation you would like us to look at, please create a pull request for @bsenel to review.
Please refer to the contributing guides before creating a pull request.
For architectural design proposal please create a ADR located under /doc/adrs/.
We make sure the code works correctly by having high-quality unit tests. You can find the unit test guides for creating unit tests.
- The
masterbranch reflects the currently-deployed version of EdgeNet. - The latest
releasebranch is where we prepare the next EdgeNet release. Please use this branch for all of the pull requests.