Initial octo-ns

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -0,0 +1,156 @@
+# octo-ns
+
+## 背景及概念
+
+ octo-ns
+ 是美团octo服务治理体系中负责命名服务的系列模块，包括[c++ sdk](sdk/mns-sdk/docs/mns_sdk_api.md)、[java
+ sdk](sdk/mns-invoker)、
+ [基础代理sgagent](sg_agent)、[命名服务缓存nsc](nsc)、[健康检查服务scanner](scanner)、[zookeeper](https://zookeeper.apache.org/)
+ ，搭配管理平台 [octo-portal](https://github.com/Meituan-Dianping/octo-portal) 用户能够便捷的进行服务运营交互。
+
+ octo-ns
+ 基于服务描述信息（tags+weight）实现服务注册/发现、路由分组、负载均衡、健康检测等功能，利用这些服务治理功能用户可以专注于业务逻辑开发、能够追踪线上服务状态、快速处置异常风险。octo-ns提升了业务对于服务的运营能力，其总体架构图如下:
+
+<div align=center> <img src="./docs/image/ns.svg" width="500"> </div align=center>
+
+* 服务service
+
+  通过自定义接口提供给消费端的软件功能，octo目前支持rpc、http服务。
+
+* 服务名 servicename
+
+  服务标识，可映射到指定服务；octo体系中通过appkey字段表示服务名。
+
+* 服务提供者 service provider
+
+  是指提供服务的被调用一方。
+
+* 服务消费者 service consumer
+
+  是指发起对某个服务调用的一方。
+
+* 注册中心
+
+  注册中心存储服务描述信息用以实现命名服务整个功能。服务在启动时将服务信息上报到注册中心，服务消费者从注册中心获取对应的服务提供者信息。健康检查组件会根据对服务节点的检测结果更新注册中心服务状态信息。
+
+* 服务注册(service registry)
+
+  服务提供者启用后向注册中心上报服务信息的流程。
+
+* 服务发现(service discovery)
+
+  服务消费者获取服务提供者信息的流程。
+
+* 权重
+
+  浮点数，是负载均衡抉择服务端流量配比的重要指标；业务和路由规则会对权重进行调整。
+
+* 服务分组(service group)
+
+  是业务自定义的服务提供者和服务消费者之间的调用约束关系。
+
+* 健康检查(health check)
+
+  服务节点的端口/进程存活状态的检查。octo-ns体系中通过scanner执行健康扫描，并更新服务描述信息到注册中心。
+
+## 架构特点
+* 模块解耦功能拆分
+
+  通信框架(c++版本whale、java版本dorado)、服务治理代理(sgagent)、健康检查服务(scanner)、服务治理平台(octo-portal)。
+
+* 代理模式轻量化框架
+
+  服务治理策略实现下移到代理组件sgagent，精简框架功能，提升框架稳定性。
+
+ <div align=center> <img src="./docs/image/ns_trait.svg" width="500"> </div align=center>
+
+
+
+* 水平扩展的代理模块(sgagent)
+    如上所述，sgagent实现了服务治理如注册、发现、路由、策略等重要功能，简化服务消费者和服务提供者使用的通信框架。
+
+    服务治理代理从部署上分为本地和哨兵模式两种，当本地sdk到sgagent不可用时可切换到哨兵集群，保证服务平滑扩展。
+
+* 统一服务状态检查服务(scanner)
+
+   scanner以周期扫描形式对服务进行探活检测，并更新服务状态到注册中心。
+
+* 高可用
+
+   调用链容灾：octo-ns各组件的本地缓存机制，提升了整体系统的容灾能力。链路中后台的某个组件发生故障时，可以使用缓存正常提供服务治理功能。
+
+   哨兵模式：本地sgagent不可用时，降级（fallback）到哨兵集群进行服务治理。
+
+##  架构介绍
+
+<div align=center> <img src="./docs/image/ns_components.svg" width="500"> </div align=center>
+   octo-ns由sdk、sgagent、nsc、scanner和zookeeper组成，使用octo-portal管理端实现可视化服务营运操作，例如更改服务提供者状态、调整权重。服务提供者启动后，通信框架主动发起注册操作，上报服务描述信息接入octo-ns。接下来分模块对octo-ns进行介绍。
+
+
+### 服务治理代理sgagent
+
+  **服务注册/发现**：这是命名服务的基本功能，注册是提供者上报服务描述信息；发现是消费者根据appkey获取对应服务描述信息。分为增量和通用两种方式，通用方式是指添加provider的服务描述信息到对应appkey下面，为服务列表增加新的成员；增量方式是指一个包含多种servicename接口类型的apppkey，通过在注册中心建立servicename和appkey的双向映射，可实现servicename粒度的服务注册/发现功能。
+
+  **服务分组**：根据分组信息，sgagent调整服务节点权重，定制化流量负载。用户可根据需求自定义分组规则，服务分组需要进行事先配置，启动后方能生效。不同分组规则根据优先级排序，优先级高者生效。执行服务分组规则时，对于服务列表中匹配失败的节点进行降权处理，流量极低概率进入降权后的节点（未来管理页面将提供可选按钮，决定降权还是删除规则外服务节点）。目前支持的服务分组策略如下：
+
+  a.自定义路由分组：在服务端配置该策略后，可定向指定上游打入下游规则范围内的服务节点
+
+  b.同机房优先分组：开启后，服务列表中与调用方非同机房的服务节点被降权
+
+  c.同中心优先分组：开启后，服务列表中与调用方非同中心的服务节点被降权
+
+  d.同城市路由分组：开启后，服务列表中与调用方非同城的服务节点被降权
+
+  **标签隔离**：依据服务节点中标识的不同标签，实现灵活的流量和链路隔离。服务注册时在服务描述信息中携带标签信息，服务发现时过滤服务列表中持有标签的服务提供者。例如：用以隔离测试的泳道标签，用以单元化部署和灰度链路的单元化标签。
+
+###  缓存服务nsc
+
+ 缓存注册到octo-ns的服务描述信息，一方面提供批量服务数据缓存与预处理功能，提升服务发现性能；另一方面增强服务治理链路的容灾能力，本地sgagent不可用的场景下该模块通过http形式提供远程哨兵服务列表。
+###  服务健康scanner
+
+  健康检查模块负责实时检查注册中心的服务提供者健康状况，一旦发现服务提供者不可用就将其置为未启动状态，及时摘除流量；当服务提供者恢复正常时，scanner会及时将服务提供者的状态置为正常状态，恢复流量。服务整个生命周期内，octo-ns利用scanner实现自动化的流量摘除和恢复，无需业务手动介入。为了提高检查的准确性，scanner支持octo自定义的心跳协议来进行应用层检查，octo体系内的通信框架默认支持，无需额外配置。
+
+### 功能流程
+
+* 服务注册
+
+   业务进程启动后，通信框架发起服务注册，依赖sgagent上报服务描述信息到注册中心。本地sgagent失效时，sdk通过nsc服务获取远程sgagent哨兵机器列表信息，注册请求fallback到哨兵集群完成整个流程。服务提供者服务描述信息数据存入注册中心后，可从octo-portal管理页面进行查阅和修改。
+
+
+<div align=center> <img src="./docs/image/registry_sucess.png" width="500"> </div align=center>
+
+* 服务发现
+
+  octo-ns支持基于http和rpc的服务发现，服务消费者可通过调用sgagent服务发现接口获取服务提供者列表。sgagent会优先从nsc
+  进行服务发现，从nsc服务发现失败时会降级从zookeeper获取服务提供者数据。
+
+* 健康检查
+
+  scanner运行时会不断扫描注册中心的服务节点，依次检查每个服务提供者状态情况，按需修改服务描述信息。
+
+<div align=center> <img src="./docs/image/scanner_traits.png" width="400"> </div align=center>
+
+##  使用文档
+
+* [快速开始](/docs/ns-quick-start.md)
+
+
+##  未来规划
+
+* 注册中心ap化：将注册中心从目前基于zookeeper的cp系统演进为ap系统，在保证最终一致性的前提下以可用性为优先目标。
+* service mesh：sgagent借鉴时下盛行的mesh理念，进一步简化通信框架和SDK功能逻辑。
+
+
+
+**Copyright and License**
+
+[Apache 2.0 License](/LICENSE)
+
+
+#####  联系我们
+
+ * Mail: [email protected]
+- [**Issues**](https://github.com/Meituan-Dianping/octo-ns/issues)
+
+
+