Hindsight

Hindsight -- 轻量级的数据处理骨架。「Hindsight - light weight data processing skeleton」

  • 所有者: mozilla-services/hindsight
  • 平台: Docker, Linux
  • 許可證: Mozilla Public License 2.0
  • 分類:
  • 主題:
  • 喜歡:
    0
      比較:

Github星跟蹤圖

Hindsight

概述

Hindsight 是一个基于 lua 沙盒 项目的 C 语言数据处理基础设施。我已经收到几个关于更轻量级和更快的数据管道的咨询,并有交付保证,以取代 Heka。Hindsight 就是那个围绕着同样的 lua 沙盒提供 "至少一次" 交付语义的轻量级骨架。骨架由 扩展包 补充,包括数百个数据结构、算法、插件、解析器和语法。由于核心基础架构(Hindsight 和 Lua 沙盒)稳定且变化不频繁,因此扩展库是目前大部分活跃开发的地方。还有一个 Hindsight 管理界面,用于监控、调试和插件管理(你可以在这里查看运行中的实例:hsadmin)。

构建

先决条件

CMake 构建指令

git clone https://github.com/mozilla-services/hindsight.git
cd hindsight
mkdir release
cd release
# Linux
cmake -DCMAKE_BUILD_TYPE=release ..
make
ctest
cpack -G TGZ # (DEB|RPM|ZIP)
# Cross platform support is planned but not supported yet

默认情况下,hindsight 是针对 OpenSSL 进行链接的,并配置为在库中设置锁定回调以确保正确的线程操作。如果不需要这个功能,可以使用 cmake 构建选项 -DWITHOUT_OPENSSL=true 来禁用这个功能,例如,如果你不使用任何使用 OpenSSL 的沙盒/模块,并且不想要这个依赖关系。

发布

  • 主分支是当前的版本,在任何时候都被认为是稳定的。
  • 新版本的发布频率可以达到每两周一次(我们的冲刺周期)。唯一的例外是一个高优先级的补丁。
  • 所有活跃的工作都会被标记为冲刺里程碑,并在项目仪表盘中进行跟踪。
  • 新版本会在冲刺结束后的第二天发布。
    • dev 分支中的版本会被更新
    • 变化被合并到主文件中
    • 一个新的标签被创建

Docker 映像

Docker 映像是由 master 和 dev 分支构建的,可以拉取,或者使用 Dockerfile 构建。

请注意,这里构建的 Docker 映像只是一个只包含 lua_sandbo x和 hindsight 的裸映像。要想获得包含所有扩展的功能更全的映像,请参见 extensions repo 的 Docker 映像。

贡献

  • 所有的拉取请求都必须针对开发分支,不允许直接提交到主分支。
  • 所有非琐碎的贡献都应该从提交问题开始(如果是新功能,请在做任何工作之前提出你的设计/方法,因为不是所有的功能请求都被接受)。


主要指標

概覽
名稱與所有者mozilla-services/hindsight
主編程語言C
編程語言CMake (語言數: 5)
平台Docker, Linux
許可證Mozilla Public License 2.0
所有者活动
創建於2015-03-09 23:05:07
推送於2022-06-04 21:03:30
最后一次提交
發布數19
最新版本名稱v0.16.0 (發布於 )
第一版名稱v0.12.7 (發布於 )
用户参与
星數666
關注者數59
派生數54
提交數249
已啟用問題?
問題數103
打開的問題數18
拉請求數83
打開的拉請求數2
關閉的拉請求數17
项目设置
已啟用Wiki?
已存檔?
是復刻?
已鎖定?
是鏡像?
是私有?

Hindsight

Overview

Hindsight is a C based data processing infrastructure based on the
lua sandbox project. I have
received several inquiries about a lighter weight and faster data pipeline with
delivery guarantees to replace Heka.
Hindsight is that light weight skeleton around the same lua sandbox offering
'at least once' delivery semantics. The skeleton is supplemented by
extension packages
including hundreds of data structures, algorithms, plugins, parsers and
grammars. The extensions repository is where most of the active development is
happening now as the core infrastructure (Hindsight and the Lua Sandbox)
is stable and changes infrequently. There is also a Hindsight Administration UI
available for monitoring, debugging and plugin management (you can check out a
running instance here: hsadmin)

Build

Prerequisites

CMake Build Instructions

git clone https://github.com/mozilla-services/hindsight.git
cd hindsight
mkdir release
cd release

# Linux
cmake -DCMAKE_BUILD_TYPE=release ..
make
ctest
cpack -G TGZ # (DEB|RPM|ZIP)

# Cross platform support is planned but not supported yet

By default hindsight is linked against OpenSSL and configured to set locking callbacks in the
library to ensure proper threaded operation. If this functionality is not desired the cmake
build option -DWITHOUT_OPENSSL=true can be used to disable this, for example if you are not
using any sandboxes/modules that make use of OpenSSL and do not want the dependency.

Releases

  • The master branch is the current release and is considered stable at all
    times.
  • New versions can be released as frequently as every two weeks (our sprint
    cycle). The only exception would be for a high priority patch.
  • All active work is flagged with the sprint milestone and tracked in the
    project dashboard.
  • New releases occur the day after the sprint finishes.
    • The version in the dev branch is updated
    • The changes are merged into master
    • A new tag is created

Docker Images

Docker images are constructed from the
master and dev branches and can be pulled, or built using the Dockerfile.

Note that the Docker image built here is only a bare bones image containing just lua_sandbox
and hindsight. For a more full featured image that also contains all of the extensions, see
the Docker image for the extensions
repo.

Contributions

  • All pull requests must be made against the dev branch, direct commits to
    master are not permitted.
  • All non trivial contributions should start with an issue being filed (if it is
    a new feature please propose your design/approach before doing any work as not
    all feature requests are accepted).