Concepts

Introduction

A trace test, in the context of Mtracer, is any mt.yaml file that uses the defined format to invoke an action within the system under observation and verify its effects on the traces collected by the observability backend.

In other words, a trace test is a system test that focuses on analyzing the trace generated by applications following a specific action, such as an HTTP or gRPC call.

Structure of a Trace Test

An mt.yaml test file that defines a trace test consists of several sections, each with a specific purpose.

Example of an mt.yaml file defining a trace test:

name: "Dice microservice test gRPC"
description: "Quick test for the dice microservice to verify that it generates the expected trace spans when receiving a gRPC request."
setupCommands:
  - type: "shell"
    cmd: "echo 'Setting up test environment'"
    cleanupCmd:
      cmd: "echo 'Cleaning up test environment'"
trigger:
  type: "grpc"
  args:
    serverAddress: "localhost:5051"
    descriptorSource:
      type: "file"
      protoPath: "dice.proto"
    method: "dice.DiceService.RollDice"
    metadata:
      authorization: "Bearer localToken"
    data:
      rollerName: "Alice"
      rollTime: "2023-10-01T12:00:00Z"
      rollDuration: "10s"
waitBeforeFetch: 10s
timeout: 120s
retryDelay: 1s
lastSpan:
  serviceName: "even-or-odd-service"
  operationName: "even-or-odd-span"
  spanKind: "internal"
  spanStatus: "unset"
expectedProperties:
    spanCount: 7
    errorCount: 0
    maxDuration: 30s
    minDuration: 1s
expectedTraces:
  - ordered: yes
    checker: contains
    spans:
      - serviceName: "grpc-server"
        operationName: "grpc.RollDice"
        spanKind: "server"
        spanStatus: "unset"
        maxDuration: 2s
        minDuration: 1us
      - serviceName: "nats-consumer"
        operationName: "nats-consume"
        spanKind: "consumer"
        spanStatus: "unset"
        maxDuration: 1s
      - serviceName: "dice-service"
        operationName: "dice-server"
        spanKind: "server"
        spanStatus: "unset"
      - serviceName: "even-or-odd-service"
        operationName: "even-or-odd-span"
        spanKind: "internal"
        spanStatus: "unset"
  - ordered: no
    spans:
      - serviceName: "dice-service"
        operationName: "dice-server"
        spanKind: "server"
        spanStatus: "unset"
      - serviceName: "nats-consumer"
        operationName: "nats-consume"
        spanKind: "consumer"
        spanStatus: "unset"
      - serviceName: "even-or-odd-service"
        operationName: "even-or-odd-span"
        spanKind: "internal"
        spanStatus: "unset"
assertions:
  - name: "Check error count and span status"
    type: "cel"
    queries:
      errorCheck: "trace.errorCount == 0"
      spanStatusCheck: "trace.spans.all(s, s.spanStatus == 'unset')"
postExecChecks:
  - name: "Check if there are any rolls for Alice in the database"
    type: "sql"
    args:
      dsn: "postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable"
      query: "(SELECT COUNT(*) FROM rolls WHERE player_name = 'Alice') > 0"
  - name: "Check if the dice service is running"
    type: "shell"
    args:
      cmd: "echo 'Hello'"
      expectedExitCode: 0

Test Name and Description

The name and description sections are used to identify the test and provide a brief description of its purpose. This information is useful for quickly understanding the test’s goal and for effectively organizing tests.

Setup Commands

The setupCommands section is optional and allows you to define a series of commands to run before and after the test execution to set up and clean up the environment. These commands are crucial for ensuring the test runs properly and repeatedly.

For more details on setup commands, see the Setup Commands section.

Trigger

The trigger section is mandatory and defines the action that initiates the trace within the system under test.

For more details on triggers, see the Trigger section.

Wait Before Fetch

The waitBeforeFetch parameter is optional. It specifies a time interval to wait between the execution of the trigger and the beginning of the trace fetching process from the observability backend. This interval is important to ensure that all spans generated following the trigger are correctly collected and analyzed.

This field must be expressed using the Go Duration String format.
The default value is 5s.

Timeout

The timeout parameter is optional and specifies the maximum time to wait when fetching the trace from the observability backend. Once this time expires, the latest collected trace is used for verification. If no trace has been collected within the specified time, the test fails.
The default value is 60s.

Retry Delay

The retryDelay parameter is optional and specifies the time interval to wait between one attempt to fetch the trace and the next. This interval is important to avoid overloading the observability backend with overly frequent fetch requests.
The default value is 1s.

Last Span

The lastSpan section is optional and allows you to specify the characteristics of the final span expected in the trace generated following the trigger. This section is useful for understanding when the trace can be considered complete and ready for analysis, allowing you to begin verifying expectations on the traces as soon as the last expected span is collected, without necessarily waiting for the timeout.

Expected Trace Properties

The expectedProperties section is optional and allows you to specify some general properties you expect to find within the trace generated following the trigger.
Below is a table showing the general properties that can be specified in this section:

Expected Traces

The expectedTraces section is optional and defines the expectations regarding the trace generated following the trigger. In this section, you can specify one or more expected traces, each with its own characteristics and conditions to verify.

For more details, see the Expected Trace section.

Assertions

The assertions section is optional and allows you to define a series of assertions to verify on the collected trace or individual spans.

For more details on assertions, see the Assertions section.

Post-Execution Checks

The postExecChecks section is optional and allows you to define a series of checks to run after the test execution to verify that the environment has been correctly modified following the trigger. These checks are fundamental to ensuring the test had the desired effect on the system under test.

For more details on post-execution checks, see the Post-Execution Checks section.