🚀 KubeSphere 3.1.1 is now available as it extends Kubernetes from the cloud to the edge. Read the introduction for 3.1.x → close
  • English
    • English
    • 简体中文
    1. 1.Experience account: demo1 / Demo123
    2. 2.This account is only allowed to view parts of UI
    3. 3.It's recommended that install KubeSphere in your environment
  •  Star
  1. Documentation / 
  2. Cluster Administration / 
  3. Cluster Settings / 
  4. Log Collection / 
  5. Introduction

Cluster Administration

Last updated:
    Edit

    Introduction to Log Collection

    KubeSphere provides a flexible log collection configuration method. Powered by FluentBit Operator, users can easily add, modify, delete, enable or disable Elasticsearch, Kafka and Fluentd receivers. Once a receiver is added, logs will be sent to this receiver.

    This tutorial gives a brief introduction about the general steps of adding log receivers in KubeSphere.

    Prerequisites

    • You need an account granted a role including the permission of Cluster Management. For example, you can log in to the console as admin directly or create a new role with the permission and assign it to an account.

    • Before adding a log receiver, you need to enable any of the logging, events or auditing components. For more information, see Enable Pluggable Components.

    Add a Log Receiver for Container Logs

    To add a log receiver:

    1. Log in to the web console of KubeSphere as admin.

    2. Click Platform in the top-left corner and select Cluster Management.

      Note

      If you have enabled the multi-cluster feature, you can select a specific cluster.

    3. Go to Log Collection under Cluster Settings in the sidebar.

    4. Click Add Log Receiver on the Logging tab.

      Note

      • At most one receiver can be added for each receiver type.
      • Different types of receivers can be added simultaneously.

    Add Elasticsearch as a log receiver

    A default Elasticsearch receiver will be added with its service address set to an Elasticsearch cluster if logging, events, or auditing is enabled in ClusterConfiguration.

    An internal Elasticsearch cluster will be deployed to the Kubernetes cluster if neither externalElasticsearchUrl nor externalElasticsearchPort is specified in ClusterConfiguration when logging, events or auditing is enabled. The internal Elasticsearch cluster is for testing and development only. It is recommended that you configure an external Elasticsearch cluster for production.

    Log searching relies on the internal or external Elasticsearch cluster configured.

    If the default Elasticsearch log receiver is deleted, refer to Add Elasticsearch as a Receiver to add a new one.

    Add Kafka as a log receiver

    Kafka is often used to receive logs and serves as a broker to other processing systems like Spark. Add Kafka as a Receiver demonstrates how to add Kafka to receive Kubernetes logs.

    Add Fluentd as a log receiver

    If you need to output logs to more places other than Elasticsearch or Kafka, you can add Fluentd as a log receiver. Fluentd has numerous output plugins which can forward logs to various destinations such as S3, MongoDB, Cassandra, MySQL, syslog, and Splunk. Add Fluentd as a Receiver demonstrates how to add Fluentd to receive Kubernetes logs.

    Add a Log Receiver for Events or Auditing Logs

    Starting from KubeSphere v3.0.0, the logs of Kubernetes events and the auditing logs of Kubernetes and KubeSphere can be archived in the same way as container logs. The tab Events or Auditing on the Log Collection page will appear if events or auditing is enabled accordingly in ClusterConfiguration. You can go to the corresponding tab to configure log receivers for Kubernetes events or Kubernetes and KubeSphere auditing logs.

    Container logs, Kubernetes events and Kubernetes and KubeSphere auditing logs should be stored in different Elasticsearch indices to be searched in KubeSphere. The index prefixes are:

    • ks-logstash-log for container logs
    • ks-logstash-events for Kubernetes events
    • ks-logstash-auditing for Kubernetes and KubeSphere auditing logs

    Turn a Log Receiver on or Off

    You can turn a log receiver on or off without adding or deleting it. To turn a log receiver on or off:

    1. On the Log Collection page, click a log receiver and go to the receiver’s detail page.

    2. Click More and select Change Status.

    3. Select Activate or Close to turn the log receiver on or off.

    4. A log receiver’s status will be changed to Close if you turn it off, otherwise the status will be Collecting on the Log Collection page.

    Modify or Delete a Log Receiver

    You can modify a log receiver or delete it:

    1. On the Log Collection page, click a log receiver and go to the receiver’s detail page.

    2. Edit a log receiver by clicking Edit or Edit YAML from the drop-down list.

    3. Delete a log receiver by clicking Delete Log Receiver.


    Thanks for the feedback. If you have a specific question about how to use KubeSphere, ask it on Slack. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.

    Previous : Cluster Visibility and Authorization
    What’s on this Page