Configuration Library

Use the configuration library -- seriously!

Formatting & Linting

Use Jsonnet tooling for formatting (jsonnetfmt ) and linting (jsonnet-lint) configuration files. These can be added to CI pipelines for team-wide consistency.

Organization

Configuration files should be organized by pipeline and resource using this hierarchical folder structure: root/[pipeline]/[resource]/

This hierarchy supports three layers of configuration:

global: Configurations used in multiple pipelines, stored in root/foo.libsonnet.
regional: Configurations used in multiple resources of a single pipeline, stored in root/[pipeline]/foo.libsonnet.
local: Configurations used in one resource of a single pipeline, stored in root/[pipeline]/[resource]/foo.libsonnet.

Further segmentation of files at the local level is recommended if users want to logically group configurations or if a single configuration becomes too large (the larger it is, the harder it is to understand).

For example, configurations for transforming events into the Elastic Common Schema (ECS) are easier to manage if they are logically grouped according to the ECS data model (e.g., client.* fields are in client.libsonnet , process.* fields are in process.libsonnet , user.* fields are in user.libsonnet , etc.).