The ZeroTier SDK is composed of two libraries: `libztcore` which is the platform-agnostic network hypervisor, and `libzt` which is the network hypervisor paired with a userspace network stack. `libzt` is a superset of `libztcore` and is distinguished by the fact that it exposes a standard [socket API](https://en.wikipedia.org/wiki/Berkeley_sockets) and simple network control API. The full source for these products can be found at [github.com/zerotier/libzt](https://github.com/zerotier/libzt) for the SDK and [github.com/zerotier/ZeroTierOne](https://github.com/zerotier/ZeroTierOne) for the desktop client. With these libraries the network stack and virtual link are exclusive to your app and traffic is fully encrypted via the [Salsa20](https://en.wikipedia.org/wiki/Salsa20) cipher. For a more in-depth discussion on the technical side of ZeroTier, check out our [Manual](https://www.zerotier.com/manual.shtml?pk_campaign=github_libzt)
The ZeroTier source code is open source and is licensed under the GNU GPL v3 (not LGPL). If you'd like to embed it in a closed-source commercial product or appliance, please e-mail contact@zerotier.com to discuss commercial licensing. Otherwise it can be used for free.
Before we dive into the technicals, the first thing to understand is that there are two API families to choose from and each is intended for a very different purpose:
`libztcore` Intended for raw performance. If your goal is simply moving frames as quickly as possible and you're willing to put in some extra work, is what you're looking. The API is described in `include/ZeroTierOne.h`. For an example of how this API is used, see the living documentation that is `src/Service.cpp`.
*NOTE: The remainder of this document will focus on the usage of the socket and control C API exposed in `include/ZeroTier.h` for `libzt`. For more information on the `libztcore` API see `include/ZeroTierOne.h`. We also provide bindings, frameworks, and packages for other languages and platforms in the `ports` directory and example applications using them in the `examples` directory.*
The next few sections explain how to use the control API. These functions are non-blocking and will return an error code specified in `include/ZeroTierConstants.h` and will result in the generation of callback events. It is your responsibility to handle these events.
At this stage, if a cryptographic identity for this node does not already exist, it will generate a new one and store it on disk, the node's address (commonly referred to as `nodeId`) will be derived from this identity and will be presented to you upon receiving the `ZTS_EVENT_NODE_ONLINE` shown below.
*NOTE: The first argument `path` is a path where you will allow ZeroTier to store its automatically-generated cryptographic identity files (`identity.public` and `identity.secret`), these files are your keys to communicating on the network. Keep them safe and keep them unique. If any two nodes are online using the same identities you will have a bad time. The second argument `userCallbackFunc` is a function that you specify to handle all generated events for the life of your program (see below):*
After calling `zts_start()` you will receive one or more of the following events:
```
ZTS_EVENT_NODE_OFFLINE
ZTS_EVENT_NODE_ONLINE
ZTS_EVENT_NODE_DOWN
ZTS_EVENT_NODE_IDENTITY_COLLISION
ZTS_EVENT_NODE_UNRECOVERABLE_ERROR
ZTS_EVENT_NODE_NORMAL_TERMINATION
```
After receiving `ZTS_EVENT_NODE_ONLINE` you will be allowed to join or leave networks.
At the end of your program or when no more network activity is anticipated, the user application can shut down the service with `zts_stop()`. However, it is safe to leave the service running in the background indefinitely as it doesn't consume much memory or CPU while at idle. `zts_stop()` is a non-blocking call and will itself issue a series of events indicating that various aspects of the ZeroTier service have successfully shut down.
It is worth noting that while `zts_stop()` will stop the service, but the user-space network stack will continue operating in a headless hibernation mode. This is intended behavior due to the fact that the network stack we've chosen doesn't currently support the notion of shutdown since it was initially designed for embedded applications that are simply switched off. If you do need a way to shut everything down and free all resources you can call `zts_free()`, but please note that calling this function will prevent all subsequent `zts_start()` calls from succeeding and will require a full application restart if you want to run the service again. The events `ZTS_EVENT_NODE_ONLINE` and `ZTS_EVENT_NODE_OFFLINE` can be seen periodically throughout the lifetime of your application depending on the reliability of your underlying network link, these events are lagging indicators and are typically only triggered every thirty (30) seconds.
Lastly, the function `zts_restart()` is provided as a way to restart the ZeroTier service along with all of its virtual interfaces. The network stack will remain online and undisturbed during this call. Note that this call will temporarily block until the service has fully shut down, then will return and you may then watch for the appropriate startup callbacks mentioned above.
Joining a ZeroTier virtual network is as easy as calling `zts_join(uint64_t networkId)`. Similarly there is a `zts_leave(uint64_t networkId)`. Note that `zts_start()` must be called and a `ZTS_EVENT_NODE_ONLINE` event must be received before these calls will succeed. After calling `zts_join()` any one of the following events may be generated:
```
ZTS_EVENT_NETWORK_NOT_FOUND
ZTS_EVENT_NETWORK_CLIENT_TOO_OLD
ZTS_EVENT_NETWORK_REQUESTING_CONFIG
ZTS_EVENT_NETWORK_OK
ZTS_EVENT_NETWORK_ACCESS_DENIED
ZTS_EVENT_NETWORK_READY_IP4
ZTS_EVENT_NETWORK_READY_IP6
ZTS_EVENT_NETWORK_DOWN
```
`ZTS_EVENT_NETWORK_READY_IP4` and `ZTS_EVENT_NETWORK_READY_IP6` are combinations of a few different events. They signal that the network was found, joined successfully, an IP address was assigned and the network stack's interface is ready to process traffic of the indicated type. After this point you should be able to communicate with peers on the network.
After successfully starting the service and joining a network, communicating with other nodes (peers) on that network is as easy as it would ordinarily be without ZeroTier. However, one thing to be aware of is the difference between relay and P2P modes. In the event that a direct connection cannot be established between your nodes, ZeroTier offers a free relaying service, this means that your nodes are reachable almost instantaneously but at a temporary performance cost. One should wait to send large amounts of traffic until a `ZTS_EVENT_PEER_P2P` is received for the node that you're interested in talking to. This event usually only takes a few seconds to appear after data has initially been sent. Similarly if after some time ZeroTier determines that a previously known path to one of your nodes is no longer available you will see a `ZTS_EVENT_PEER_RELAY` event.
One can use `zts_get_peer_status(uint64_t peerId)` to query the current reachability state of another node. This function will actually **return** the previously mentioned event values, plus an additional one called `ZTS_EVENT_PEER_UNREACHABLE` if no known direct path exists between the calling node and the remote node.
As mentioned in previous sections, the control API works by use of non-blocking calls and the generation of a few dozen different event types. Depending on the type of event there may be additional contextual information attached to the `zts_callback_msg` object that you can use. This contextual information will be housed in one of the following structures which are defined in `include/ZeroTier.h`:
In this callback function you can perform additional non-blocking API calls or other work. While not returning control to the service isn't forbidden (the event messages are generated by a separate thread) it is recommended that you return control as soon as possible as not returning will prevent the user application from receiving additional callback event messages which may be time-sensitive.
These events pertain to the state of the current node. This message type will arrive with a `zts_node_details` object accessible via `msg->node`. Additionally, one can query the status of the node with `zts_get_node_status()`, this will **return** the status as an integer value only.
These events pertain to the state of the indicated network. This event type will arrive with a `zts_network_details` object accessible via `msg->network`. If for example you want to know the number of assigned routes for your network you can use `msg->network->num_routes`. Similarly for the MTU, use `msg->network->mtu`. Additionally, one can query the status of the network with `zts_get_network_status(uint64_t networkId)`, this will **return** the status as an integer value only.
These events are triggered when the reachability status of a peer has changed, this can happen at any time. This event type will arrive with a `zts_peer_details` object for additional context. Additionally, one can query the status of the network with `zts_get_peer_status(uint64_t peerId)`, this will **return** the status as an integer value only.
These events are triggered when a direct path to a peer has been discovered or is now considered too old to be used. You will see these in conjunction with peer events. This event type will arrive with a `zts_physical_path` object for additional context.
These events are triggered when new addresses are assigned to the node on a particular virtual network. This event type will arrive with a `zts_addr_details` object for additional context.
These events aren't very important to the application developer but are important for debugging. These signal whether the userspace networking stack was brought up successfully. You can ignore these in most cases. This event type will arrive with no additional contextual information.
These events aren't very important to the application developer but are important for debugging. These signal whether the userspace networking stack was brought up successfully. You can ignore these in most cases. This event type will arrive with a `zts_netif_details` object for additional context.
Just as there are two APIs (socket and control), there are two sets of error codes. The control API (`zts_start()`, `zts_join()`, etc) errors defined in `include/ZeroTierConstants.h` are:
-`ZTS_ERR_OK`: Everything is ok
-`ZTS_ERR_INVALID_ARG`: An argument provided by the user application is invalid (e.g. out of range, NULL, etc)
-`ZTS_ERR_SERVICE`: The service isn't initialized or is for some reason currently unavailable. Try again.
-`ZTS_ERR_INVALID_OP`: For some reason this API operation is not permitted or doesn't make sense at this time.
-`ZTS_ERR_NO_RESULT`: The call succeeded, but no object or relevant result was available
-`ZTS_ERR_GENERAL`: General internal failure (memory allocation, null reference, etc)
The socket API error codes are defined in `doc/errno.h`
*NOTE: For Android/Java (or similar) which use JNI, the socket API's error codes are negative values*
The control API for `libzt` is thread safe and can be called at any time from any thread. There is a single internal lock guarding access to this API. The socket API is similar in this regard. Callback events are generated by a separate thread and are independent from the rest of the API's internal locking mechanism. Not returning from a callback event won't impact the rest of the API but it will prevent your application from receiving future events so it is in your application's best interest to perform as little work as possible in the callback function and promptly return control back to ZeroTier.
*Note: Internally, `libzt` will spawn a number of threads for various purposes: a thread for the core service, a thread for the network stack, a low priority thread to process callback events, and a thread for each network joined. The vast majority of work is performed by the core service and stack threads.*
Protocol and service statistics are available in debug builds of `libzt`. These statistics are detailed fully in the section of `include/ZeroTier.h` that is guarded by `LWIP_STATS`. The protocol constants are defined in `include/ZeroTierConstants.h`. An example usage is as follows:
The library form of ZeroTier can act as a network controller and in `libzt` this is controlled via the `zts_controller_*` API calls specified in `include/ZeroTier.h`. Currently controller mode is not available in the `iOS` and `macOS` framework builds.
