Hindsight

Hindsight -- 轻量级的数据处理骨架。「Hindsight - light weight data processing skeleton」

  • Owner: mozilla-services/hindsight
  • Platform: Docker, Linux
  • License:: Mozilla Public License 2.0
  • Category::
  • Topic:
  • Like:
    0
      Compare:

Github stars Tracking Chart

Hindsight

概述

Hindsight 是一个基于 lua 沙盒 项目的 C 语言数据处理基础设施。我已经收到几个关于更轻量级和更快的数据管道的咨询,并有交付保证,以取代 Heka。Hindsight 就是那个围绕着同样的 lua 沙盒提供 "至少一次" 交付语义的轻量级骨架。骨架由 扩展包 补充,包括数百个数据结构、算法、插件、解析器和语法。由于核心基础架构(Hindsight 和 Lua 沙盒)稳定且变化不频繁,因此扩展库是目前大部分活跃开发的地方。还有一个 Hindsight 管理界面,用于监控、调试和插件管理(你可以在这里查看运行中的实例:hsadmin)。

构建

先决条件

CMake 构建指令

git clone https://github.com/mozilla-services/hindsight.git
cd hindsight
mkdir release
cd release
# Linux
cmake -DCMAKE_BUILD_TYPE=release ..
make
ctest
cpack -G TGZ # (DEB|RPM|ZIP)
# Cross platform support is planned but not supported yet

默认情况下,hindsight 是针对 OpenSSL 进行链接的,并配置为在库中设置锁定回调以确保正确的线程操作。如果不需要这个功能,可以使用 cmake 构建选项 -DWITHOUT_OPENSSL=true 来禁用这个功能,例如,如果你不使用任何使用 OpenSSL 的沙盒/模块,并且不想要这个依赖关系。

发布

  • 主分支是当前的版本,在任何时候都被认为是稳定的。
  • 新版本的发布频率可以达到每两周一次(我们的冲刺周期)。唯一的例外是一个高优先级的补丁。
  • 所有活跃的工作都会被标记为冲刺里程碑,并在项目仪表盘中进行跟踪。
  • 新版本会在冲刺结束后的第二天发布。
    • dev 分支中的版本会被更新
    • 变化被合并到主文件中
    • 一个新的标签被创建

Docker 映像

Docker 映像是由 master 和 dev 分支构建的,可以拉取,或者使用 Dockerfile 构建。

请注意,这里构建的 Docker 映像只是一个只包含 lua_sandbo x和 hindsight 的裸映像。要想获得包含所有扩展的功能更全的映像,请参见 extensions repo 的 Docker 映像。

贡献

  • 所有的拉取请求都必须针对开发分支,不允许直接提交到主分支。
  • 所有非琐碎的贡献都应该从提交问题开始(如果是新功能,请在做任何工作之前提出你的设计/方法,因为不是所有的功能请求都被接受)。


Main metrics

Overview
Name With Ownermozilla-services/hindsight
Primary LanguageC
Program languageCMake (Language Count: 5)
PlatformDocker, Linux
License:Mozilla Public License 2.0
所有者活动
Created At2015-03-09 23:05:07
Pushed At2022-06-04 21:03:30
Last Commit At
Release Count19
Last Release Namev0.16.0 (Posted on )
First Release Namev0.12.7 (Posted on )
用户参与
Stargazers Count666
Watchers Count59
Fork Count54
Commits Count249
Has Issues Enabled
Issues Count103
Issue Open Count18
Pull Requests Count83
Pull Requests Open Count2
Pull Requests Close Count17
项目设置
Has Wiki Enabled
Is Archived
Is Fork
Is Locked
Is Mirror
Is Private

Hindsight

Overview

Hindsight is a C based data processing infrastructure based on the
lua sandbox project. I have
received several inquiries about a lighter weight and faster data pipeline with
delivery guarantees to replace Heka.
Hindsight is that light weight skeleton around the same lua sandbox offering
'at least once' delivery semantics. The skeleton is supplemented by
extension packages
including hundreds of data structures, algorithms, plugins, parsers and
grammars. The extensions repository is where most of the active development is
happening now as the core infrastructure (Hindsight and the Lua Sandbox)
is stable and changes infrequently. There is also a Hindsight Administration UI
available for monitoring, debugging and plugin management (you can check out a
running instance here: hsadmin)

Build

Prerequisites

CMake Build Instructions

git clone https://github.com/mozilla-services/hindsight.git
cd hindsight
mkdir release
cd release

# Linux
cmake -DCMAKE_BUILD_TYPE=release ..
make
ctest
cpack -G TGZ # (DEB|RPM|ZIP)

# Cross platform support is planned but not supported yet

By default hindsight is linked against OpenSSL and configured to set locking callbacks in the
library to ensure proper threaded operation. If this functionality is not desired the cmake
build option -DWITHOUT_OPENSSL=true can be used to disable this, for example if you are not
using any sandboxes/modules that make use of OpenSSL and do not want the dependency.

Releases

  • The master branch is the current release and is considered stable at all
    times.
  • New versions can be released as frequently as every two weeks (our sprint
    cycle). The only exception would be for a high priority patch.
  • All active work is flagged with the sprint milestone and tracked in the
    project dashboard.
  • New releases occur the day after the sprint finishes.
    • The version in the dev branch is updated
    • The changes are merged into master
    • A new tag is created

Docker Images

Docker images are constructed from the
master and dev branches and can be pulled, or built using the Dockerfile.

Note that the Docker image built here is only a bare bones image containing just lua_sandbox
and hindsight. For a more full featured image that also contains all of the extensions, see
the Docker image for the extensions
repo.

Contributions

  • All pull requests must be made against the dev branch, direct commits to
    master are not permitted.
  • All non trivial contributions should start with an issue being filed (if it is
    a new feature please propose your design/approach before doing any work as not
    all feature requests are accepted).