gfwleak/zhangyang-libzt
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updated included ZTO version

Browse Source
This commit is contained in:
Joseph Henry
2016-11-21 09:23:56 -08:00
parent 4774017b6f
commit 152938d7e7
405 changed files with 2622 additions and 91397 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
zerotierone/controller/README.md
View File

@@ -1,32 +0,0 @@
Network Controller Implementation
======
This folder contains code implementing the node/NetworkController.hpp interface to allow ZeroTier nodes to create and manage virtual networks.
### Building
By default this code is not built or included in the client. To build on Linux, BSD, or Mac add `ZT_ENABLE_NETWORK_CONTROLLER=1` to the make command line. You'll need the development headers for Sqlite3 installed. They ship as part of OSX and Xcode. On Linux or BSD you'll probably need to install a package.
### Running
When started, a controller-enabled build of ZeroTier One will automatically create and initialize a *controller.db* in its home folder. This is where all the controller's data and persistent state lives.
Since Sqlite3 supports multiple processes attached to the same database, it is safe to back up a running database with the command line *sqlite3* utility:
sqlite3 /path/to/controller.db .dump
In production ZeroTier runs this frequently and keeps many timestamped copies going back about a week. These are also backed up (encrypted) to Amazon S3 along with the rest of our data.
### Administrating
See service/README.md for documentation on the JSON API presented by this network controller implementation. Also see *nodejs-zt1-client* for a NodeJS JavaScript interface.
### Reliability
Network controllers can go offline without affecting already-configured members of running networks. You just won't be able to change anything and new members will not be able to join.
High-availability can be implemented through fail-over. A simple method involves making a frequent backup of the SQLite database (use the SQLite command line client to do this safely) and the network configuration master's working directory. Then, if the master goes down, another instance of it can rapidly be provisioned elsewhere. Since ZeroTier addresses are mobile, the new instance will quickly (usually no more than 30s) take over for the old one and service requests.
### Limits
A single network configuration master can administrate up to 2^24 (~16m) networks as per the ZeroTier protocol limit. There is no hard limit on the number of clients, though millions or more would impose significant CPU demands on a server. Optimizations could be implemented such as memoization/caching to reduce this.

2212
zerotierone/controller/SqliteNetworkController.cpp
View File

File diff suppressed because it is too large Load Diff

191
zerotierone/controller/SqliteNetworkController.hpp
View File

@@ -1,191 +0,0 @@
/*
* ZeroTier One - Network Virtualization Everywhere
* Copyright (C) 2011-2015 ZeroTier, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* --
*
* ZeroTier may be used and distributed under the terms of the GPLv3, which
* are available at: http://www.gnu.org/licenses/gpl-3.0.html
*
* If you would like to embed ZeroTier into a commercial application or
* redistribute it in a modified binary form, please contact ZeroTier Networks
* LLC. Start here: http://www.zerotier.com/
*/
#ifndef ZT_SQLITENETWORKCONTROLLER_HPP
#define ZT_SQLITENETWORKCONTROLLER_HPP
#include <stdint.h>
#include <sqlite3.h>
#include <string>
#include <map>
#include <vector>
#include "../node/Constants.hpp"
#include "../node/NetworkController.hpp"
#include "../node/Mutex.hpp"
#include "../osdep/Thread.hpp"
// Number of in-memory last log entries to maintain per user
#define ZT_SQLITENETWORKCONTROLLER_IN_MEMORY_LOG_SIZE 32
// How long do circuit tests last before they're forgotten?
#define ZT_SQLITENETWORKCONTROLLER_CIRCUIT_TEST_TIMEOUT 60000
namespace ZeroTier {
class Node;
class SqliteNetworkController : public NetworkController
{
public:
SqliteNetworkController(Node *node,const char *dbPath,const char *circuitTestPath);
virtual ~SqliteNetworkController();
virtual NetworkController::ResultCode doNetworkConfigRequest(
const InetAddress &fromAddr,
const Identity &signingId,
const Identity &identity,
uint64_t nwid,
const NetworkConfigRequestMetaData &metaData,
Buffer<8194> &netconf);
unsigned int handleControlPlaneHttpGET(
const std::vector<std::string> &path,
const std::map<std::string,std::string> &urlArgs,
const std::map<std::string,std::string> &headers,
const std::string &body,
std::string &responseBody,
std::string &responseContentType);
unsigned int handleControlPlaneHttpPOST(
const std::vector<std::string> &path,
const std::map<std::string,std::string> &urlArgs,
const std::map<std::string,std::string> &headers,
const std::string &body,
std::string &responseBody,
std::string &responseContentType);
unsigned int handleControlPlaneHttpDELETE(
const std::vector<std::string> &path,
const std::map<std::string,std::string> &urlArgs,
const std::map<std::string,std::string> &headers,
const std::string &body,
std::string &responseBody,
std::string &responseContentType);
// threadMain() for backup thread -- do not call directly
void threadMain()
throw();
private:
enum IpAssignmentType {
// IP assignment is a static IP address
ZT_IP_ASSIGNMENT_TYPE_ADDRESS = 0,
// IP assignment is a network -- a route via this interface, not an address
ZT_IP_ASSIGNMENT_TYPE_NETWORK = 1
};
unsigned int _doCPGet(
const std::vector<std::string> &path,
const std::map<std::string,std::string> &urlArgs,
const std::map<std::string,std::string> &headers,
const std::string &body,
std::string &responseBody,
std::string &responseContentType);
NetworkController::ResultCode _doNetworkConfigRequest(
const InetAddress &fromAddr,
const Identity &signingId,
const Identity &identity,
uint64_t nwid,
const NetworkConfigRequestMetaData &metaData,
Buffer<8194> &netconf);
static void _circuitTestCallback(ZT_Node *node,ZT_CircuitTest *test,const ZT_CircuitTestReport *report);
Node *_node;
Thread _backupThread;
volatile bool _backupThreadRun;
std::string _dbPath;
std::string _circuitTestPath;
std::string _instanceId;
// Circuit tests outstanding
struct _CircuitTestEntry
{
ZT_CircuitTest *test;
std::string jsonResults;
};
std::map< uint64_t,_CircuitTestEntry > _circuitTests;
// Last request time by address, for rate limitation
std::map< std::pair<uint64_t,uint64_t>,uint64_t > _lastRequestTime;
sqlite3 *_db;
sqlite3_stmt *_sGetNetworkById;
sqlite3_stmt *_sGetMember;
sqlite3_stmt *_sCreateMember;
sqlite3_stmt *_sGetNodeIdentity;
sqlite3_stmt *_sCreateOrReplaceNode;
sqlite3_stmt *_sGetMaxNodeHistoryNetworkVisitCounter;
sqlite3_stmt *_sAddNodeHistoryEntry;
sqlite3_stmt *_sDeleteOldNodeHistoryEntries;
sqlite3_stmt *_sGetActiveNodesOnNetwork;
sqlite3_stmt *_sGetNodeHistory;
sqlite3_stmt *_sGetEtherTypesFromRuleTable;
sqlite3_stmt *_sGetActiveBridges;
sqlite3_stmt *_sGetIpAssignmentsForNode;
sqlite3_stmt *_sGetIpAssignmentPools;
sqlite3_stmt *_sGetLocalRoutes;
sqlite3_stmt *_sCheckIfIpIsAllocated;
sqlite3_stmt *_sAllocateIp;
sqlite3_stmt *_sDeleteIpAllocations;
sqlite3_stmt *_sDeleteLocalRoutes;
sqlite3_stmt *_sGetRelays;
sqlite3_stmt *_sListNetworks;
sqlite3_stmt *_sListNetworkMembers;
sqlite3_stmt *_sGetMember2;
sqlite3_stmt *_sGetIpAssignmentPools2;
sqlite3_stmt *_sListRules;
sqlite3_stmt *_sCreateRule;
sqlite3_stmt *_sCreateNetwork;
sqlite3_stmt *_sGetNetworkRevision;
sqlite3_stmt *_sSetNetworkRevision;
sqlite3_stmt *_sGetIpAssignmentsForNode2;
sqlite3_stmt *_sDeleteRelaysForNetwork;
sqlite3_stmt *_sCreateRelay;
sqlite3_stmt *_sDeleteIpAssignmentPoolsForNetwork;
sqlite3_stmt *_sDeleteRulesForNetwork;
sqlite3_stmt *_sCreateIpAssignmentPool;
sqlite3_stmt *_sUpdateMemberAuthorized;
sqlite3_stmt *_sUpdateMemberActiveBridge;
sqlite3_stmt *_sDeleteMember;
sqlite3_stmt *_sDeleteAllNetworkMembers;
sqlite3_stmt *_sDeleteNetwork;
sqlite3_stmt *_sGetGateways;
sqlite3_stmt *_sDeleteGateways;
sqlite3_stmt *_sCreateGateway;
sqlite3_stmt *_sIncrementMemberRevisionCounter;
sqlite3_stmt *_sGetConfig;
sqlite3_stmt *_sSetConfig;
Mutex _lock;
};
} // namespace ZeroTier
#endif

127
zerotierone/controller/schema.sql
View File

@@ -1,127 +0,0 @@
CREATE TABLE Config (
k varchar(16) PRIMARY KEY NOT NULL,
v varchar(1024) NOT NULL
);
CREATE TABLE Network (
id char(16) PRIMARY KEY NOT NULL,
name varchar(128) NOT NULL,
private integer NOT NULL DEFAULT(1),
enableBroadcast integer NOT NULL DEFAULT(1),
allowPassiveBridging integer NOT NULL DEFAULT(0),
v4AssignMode varchar(8) NOT NULL DEFAULT('none'),
v6AssignMode varchar(8) NOT NULL DEFAULT('none'),
multicastLimit integer NOT NULL DEFAULT(32),
creationTime integer NOT NULL DEFAULT(0),
revision integer NOT NULL DEFAULT(1),
memberRevisionCounter integer NOT NULL DEFAULT(1)
);
CREATE TABLE AuthToken (
id integer PRIMARY KEY NOT NULL,
networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,
authMode integer NOT NULL DEFAULT(1),
useCount integer NOT NULL DEFAULT(0),
maxUses integer NOT NULL DEFAULT(0),
expiresAt integer NOT NULL DEFAULT(0),
token varchar(256) NOT NULL
);
CREATE INDEX AuthToken_networkId_token ON AuthToken(networkId,token);
CREATE TABLE Node (
id char(10) PRIMARY KEY NOT NULL,
identity varchar(4096) NOT NULL
);
CREATE TABLE NodeHistory (
nodeId char(10) NOT NULL REFERENCES Node(id) ON DELETE CASCADE,
networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,
networkVisitCounter INTEGER NOT NULL DEFAULT(0),
networkRequestAuthorized INTEGER NOT NULL DEFAULT(0),
requestTime INTEGER NOT NULL DEFAULT(0),
clientMajorVersion INTEGER NOT NULL DEFAULT(0),
clientMinorVersion INTEGER NOT NULL DEFAULT(0),
clientRevision INTEGER NOT NULL DEFAULT(0),
networkRequestMetaData VARCHAR(1024),
fromAddress VARCHAR(128)
);
CREATE INDEX NodeHistory_nodeId ON NodeHistory (nodeId);
CREATE INDEX NodeHistory_networkId ON NodeHistory (networkId);
CREATE INDEX NodeHistory_requestTime ON NodeHistory (requestTime);
CREATE TABLE Gateway (
networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,
ip blob(16) NOT NULL,
ipVersion integer NOT NULL DEFAULT(4),
metric integer NOT NULL DEFAULT(0)
);
CREATE UNIQUE INDEX Gateway_networkId_ip ON Gateway (networkId, ip);
CREATE TABLE IpAssignment (
networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,
nodeId char(10) REFERENCES Node(id) ON DELETE CASCADE,
type integer NOT NULL DEFAULT(0),
ip blob(16) NOT NULL,
ipNetmaskBits integer NOT NULL DEFAULT(0),
ipVersion integer NOT NULL DEFAULT(4)
);
CREATE UNIQUE INDEX IpAssignment_networkId_ip ON IpAssignment (networkId, ip);
CREATE INDEX IpAssignment_networkId_nodeId ON IpAssignment (networkId, nodeId);
CREATE TABLE IpAssignmentPool (
networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,
ipRangeStart blob(16) NOT NULL,
ipRangeEnd blob(16) NOT NULL,
ipVersion integer NOT NULL DEFAULT(4)
);
CREATE UNIQUE INDEX IpAssignmentPool_networkId_ipRangeStart ON IpAssignmentPool (networkId,ipRangeStart);
CREATE TABLE Member (
networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,
nodeId char(10) NOT NULL REFERENCES Node(id) ON DELETE CASCADE,
authorized integer NOT NULL DEFAULT(0),
activeBridge integer NOT NULL DEFAULT(0),
memberRevision integer NOT NULL DEFAULT(0),
PRIMARY KEY (networkId, nodeId)
);
CREATE INDEX Member_networkId_activeBridge ON Member(networkId, activeBridge);
CREATE INDEX Member_networkId_memberRevision ON Member(networkId, memberRevision);
CREATE TABLE Relay (
networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,
address char(10) NOT NULL,
phyAddress varchar(64) NOT NULL
);
CREATE UNIQUE INDEX Relay_networkId_address ON Relay (networkId,address);
CREATE TABLE Rule (
networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,
ruleNo integer NOT NULL,
nodeId char(10) REFERENCES Node(id),
sourcePort char(10),
destPort char(10),
vlanId integer,
vlanPcp integer,
etherType integer,
macSource char(12),
macDest char(12),
ipSource varchar(64),
ipDest varchar(64),
ipTos integer,
ipProtocol integer,
ipSourcePort integer,
ipDestPort integer,
flags integer,
invFlags integer,
"action" varchar(4096) NOT NULL DEFAULT('accept')
);
CREATE UNIQUE INDEX Rule_networkId_ruleNo ON Rule (networkId, ruleNo);

129
zerotierone/controller/schema.sql.c
View File

@@ -1,129 +0,0 @@
#define ZT_NETCONF_SCHEMA_SQL \
"CREATE TABLE Config (\n"\
" k varchar(16) PRIMARY KEY NOT NULL,\n"\
" v varchar(1024) NOT NULL\n"\
");\n"\
"\n"\
"CREATE TABLE Network (\n"\
" id char(16) PRIMARY KEY NOT NULL,\n"\
" name varchar(128) NOT NULL,\n"\
" private integer NOT NULL DEFAULT(1),\n"\
" enableBroadcast integer NOT NULL DEFAULT(1),\n"\
" allowPassiveBridging integer NOT NULL DEFAULT(0),\n"\
" v4AssignMode varchar(8) NOT NULL DEFAULT('none'),\n"\
" v6AssignMode varchar(8) NOT NULL DEFAULT('none'),\n"\
" multicastLimit integer NOT NULL DEFAULT(32),\n"\
" creationTime integer NOT NULL DEFAULT(0),\n"\
" revision integer NOT NULL DEFAULT(1),\n"\
" memberRevisionCounter integer NOT NULL DEFAULT(1)\n"\
");\n"\
"\n"\
"CREATE TABLE AuthToken (\n"\
" id integer PRIMARY KEY NOT NULL,\n"\
" networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,\n"\
" authMode integer NOT NULL DEFAULT(1),\n"\
" useCount integer NOT NULL DEFAULT(0),\n"\
" maxUses integer NOT NULL DEFAULT(0),\n"\
" expiresAt integer NOT NULL DEFAULT(0),\n"\
" token varchar(256) NOT NULL\n"\
");\n"\
"\n"\
"CREATE INDEX AuthToken_networkId_token ON AuthToken(networkId,token);\n"\
"\n"\
"CREATE TABLE Node (\n"\
" id char(10) PRIMARY KEY NOT NULL,\n"\
" identity varchar(4096) NOT NULL\n"\
");\n"\
"\n"\
"CREATE TABLE NodeHistory (\n"\
" nodeId char(10) NOT NULL REFERENCES Node(id) ON DELETE CASCADE,\n"\
" networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,\n"\
" networkVisitCounter INTEGER NOT NULL DEFAULT(0),\n"\
" networkRequestAuthorized INTEGER NOT NULL DEFAULT(0),\n"\
" requestTime INTEGER NOT NULL DEFAULT(0),\n"\
" clientMajorVersion INTEGER NOT NULL DEFAULT(0),\n"\
" clientMinorVersion INTEGER NOT NULL DEFAULT(0),\n"\
" clientRevision INTEGER NOT NULL DEFAULT(0),\n"\
" networkRequestMetaData VARCHAR(1024),\n"\
" fromAddress VARCHAR(128)\n"\
");\n"\
"\n"\
"CREATE INDEX NodeHistory_nodeId ON NodeHistory (nodeId);\n"\
"CREATE INDEX NodeHistory_networkId ON NodeHistory (networkId);\n"\
"CREATE INDEX NodeHistory_requestTime ON NodeHistory (requestTime);\n"\
"\n"\
"CREATE TABLE Gateway (\n"\
" networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,\n"\
" ip blob(16) NOT NULL,\n"\
" ipVersion integer NOT NULL DEFAULT(4),\n"\
" metric integer NOT NULL DEFAULT(0)\n"\
");\n"\
"\n"\
"CREATE UNIQUE INDEX Gateway_networkId_ip ON Gateway (networkId, ip);\n"\
"\n"\
"CREATE TABLE IpAssignment (\n"\
" networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,\n"\
" nodeId char(10) REFERENCES Node(id) ON DELETE CASCADE,\n"\
" type integer NOT NULL DEFAULT(0),\n"\
" ip blob(16) NOT NULL,\n"\
" ipNetmaskBits integer NOT NULL DEFAULT(0),\n"\
" ipVersion integer NOT NULL DEFAULT(4)\n"\
");\n"\
"\n"\
"CREATE UNIQUE INDEX IpAssignment_networkId_ip ON IpAssignment (networkId, ip);\n"\
"\n"\
"CREATE INDEX IpAssignment_networkId_nodeId ON IpAssignment (networkId, nodeId);\n"\
"\n"\
"CREATE TABLE IpAssignmentPool (\n"\
" networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,\n"\
" ipRangeStart blob(16) NOT NULL,\n"\
" ipRangeEnd blob(16) NOT NULL,\n"\
" ipVersion integer NOT NULL DEFAULT(4)\n"\
");\n"\
"\n"\
"CREATE UNIQUE INDEX IpAssignmentPool_networkId_ipRangeStart ON IpAssignmentPool (networkId,ipRangeStart);\n"\
"\n"\
"CREATE TABLE Member (\n"\
" networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,\n"\
" nodeId char(10) NOT NULL REFERENCES Node(id) ON DELETE CASCADE,\n"\
" authorized integer NOT NULL DEFAULT(0),\n"\
" activeBridge integer NOT NULL DEFAULT(0),\n"\
" memberRevision integer NOT NULL DEFAULT(0),\n"\
" PRIMARY KEY (networkId, nodeId)\n"\
");\n"\
"\n"\
"CREATE INDEX Member_networkId_activeBridge ON Member(networkId, activeBridge);\n"\
"CREATE INDEX Member_networkId_memberRevision ON Member(networkId, memberRevision);\n"\
"\n"\
"CREATE TABLE Relay (\n"\
" networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,\n"\
" address char(10) NOT NULL,\n"\
" phyAddress varchar(64) NOT NULL\n"\
");\n"\
"\n"\
"CREATE UNIQUE INDEX Relay_networkId_address ON Relay (networkId,address);\n"\
"\n"\
"CREATE TABLE Rule (\n"\
" networkId char(16) NOT NULL REFERENCES Network(id) ON DELETE CASCADE,\n"\
" ruleNo integer NOT NULL,\n"\
" nodeId char(10) REFERENCES Node(id),\n"\
" sourcePort char(10),\n"\
" destPort char(10),\n"\
" vlanId integer,\n"\
" vlanPcp integer,\n"\
" etherType integer,\n"\
" macSource char(12),\n"\
" macDest char(12),\n"\
" ipSource varchar(64),\n"\
" ipDest varchar(64),\n"\
" ipTos integer,\n"\
" ipProtocol integer,\n"\
" ipSourcePort integer,\n"\
" ipDestPort integer,\n"\
" flags integer,\n"\
" invFlags integer,\n"\
" \"action\" varchar(4096) NOT NULL DEFAULT('accept')\n"\
");\n"\
"\n"\
"CREATE UNIQUE INDEX Rule_networkId_ruleNo ON Rule (networkId, ruleNo);\n"\
""

8
zerotierone/controller/schema2c.sh
View File

@@ -1,8 +0,0 @@
#!/bin/bash
# Run this file to package the .sql file into a .c file whenever the SQL changes.
rm -f schema.sql.c
echo '#define ZT_NETCONF_SCHEMA_SQL \' >schema.sql.c
cat schema.sql | sed 's/"/\\"/g' | sed 's/^/"/' | sed 's/$/\\n"\\/' >>schema.sql.c
echo '""' >>schema.sql.c