Logging operator
Logging operator for Kubernetes based on Fluentd and Fluent-bit.
The Logging operator automates the deployment and configuration of a Kubernetes logging pipeline. The operator deploys and configures a Fluent Bit daemonset on every node to collect container and application logs from the node file system. Fluent Bit queries the Kubernetes API and enriches the logs with metadata about the pods, and transfers both the logs and the metadata to Fluentd. Fluentd receives, filters, and transfer logs to multiple outputs. Your logs will always be transferred on authenticated and encrypted channels.
What is this operator for?
This operator helps you bundle logging information with your applications: you can describe the behavior of your application in its charts, the Logging operator does the rest.
Feature highlights
- Namespace isolation
- Native Kubernetes label selectors
- Secure communication (TLS)
- Configuration validation
- Multiple flow support (multiply logs for different transformations)
- Multiple output support (store the same logs in multiple storage: S3, GCS, ES, Loki and more...)
- Multiple logging system support (multiple fluentd, fluent-bit deployment on the same cluster)
Architecture
You can define outputs (destinations where you want to send your log messages, for example, Elasticsearch, or and Amazon S3 bucket), and flows that use filters and selectors to route log messages to the appropriate outputs. You can also define cluster-wide outputs and flows, for example, to use a centralized output that namespaced users cannot modify.
You can configure the Logging operator using the following Custom Resource Descriptions.
- logging - Represents a logging system. Includes
FluentdandFluent-bitconfiguration. Specifies thecontrolNamespace. Fluentd and Fluent-bit will be deployed in thecontrolNamespace - output - Defines an Output for a logging flow. This is a namespaced resource. See also
clusteroutput. - flow - Defines a logging flow with
filtersandoutputs. You can specifyselectorsto filter logs by labels. Outputs can beoutputorclusteroutput. This is a namespaced resource. See alsoclusterflow. - clusteroutput - Defines an output without namespace restriction. Only effective in
controlNamespace. - clusterflow - Defines a logging flow without namespace restriction.
See the detailed CRD documentation.
Quickstart
Follow these quickstart guides to try out the Logging operator!
Install
Deploy Logging Operator with Kubernetes manifests or Helm chart.
Caution: The master branch is under heavy development. Use releases instead of the master branch to get stable software.
Support
If you are using the Logging operator in a production environment and require commercial support, contact Banzai Cloud, the company backing the development of the Logging operator. If you are looking for the ultimate observability tool for multi-cluster Kubernetes infrastructures to automate the collection, correlation, and storage of logs and metrics, check out One Eye.
If you encounter any problems that the documentation does not address, open an issue or talk to us on the Banzai Cloud Slack channel #logging-operator.
Documentation
You can find the complete documentation of the Logging operator on the Banzai Cloud Documentation Page :blue_book:
Contributing
If you find this project useful, help us:
- Support the development of this project and star this repo! :star:
- If you use the Logging operator in a production environment, add yourself to the list of production adopters.:metal:
- Help new users with issues they may encounter :muscle:
- Send a pull request with your new features and bug fixes :rocket:
For more information, read the developer documentation.
License
Copyright (c) 2017-2020 Banzai Cloud, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.