ECS Logs

Ingesting ECS Logs with KubeSense

KubeSense supports ingesting logs from Amazon ECS (Elastic Container Service) for both EC2 and Fargate launch types. This enables unified observability across your containerized workloads.

Prerequisites

Before you begin, ensure you have:

ECS cluster running (EC2 or Fargate)
Tasks configured to send logs to CloudWatch Logs
KubeSense aggregator deployed and accessible
AWS permissions to create Firehose streams and configure log drivers

ECS EC2 Launch Type

For ECS tasks running on EC2 instances, you can use LogSensor installed on the host or ingest via CloudWatch.

Method 1: LogSensor on EC2 Host (Recommended)

Install LogSensor on each EC2 instance in your ECS cluster to directly read container logs.

Step 1: Install LogSensor on EC2

Use the LogSensor VM agent installation:

# Create config.yaml
cat << EOF > config.yaml
AGGREGATOR_HOST: <KUBESENSE_AGGREGATOR_LB>
AGGREGATOR_PORT: "30054"
METRICS_STORE_HOST: <KUBESENSE_METRICS_SCRAPPER_LB>
METRICS_STORE_PORT: "30060"
SENSOR_INSTALLATION_TYPE: legacy
HOST_NAME: <HOST_NAME>
LOG_ENABLED: true
METRICS_ENABLED: true
CLUSTER_NAME: <ECS_CLUSTER_NAME>
LOG_PATHS:
  - service: ecs-containers
    path: /var/lib/docker/containers
    type: file
ATTRIBUTES:
  region: us-east-1
  launch_type: ec2
EOF

# Install LogSensor
bash <(wget -qO- https://docker.kubesense.ai/setup.sh) sensor config.yaml

Step 2: Configure Log Paths

LogSensor will automatically discover and read:

Docker container logs from /var/lib/docker/containers
ECS task logs
Application logs mounted as volumes

Method 2: CloudWatch Logs via Firehose

Configure ECS tasks to use the awslogs log driver and forward via Firehose.

Step 1: Configure ECS Task Definition

Update your task definition to use CloudWatch Logs:

{
  "family": "my-ecs-task",
  "containerDefinitions": [
    {
      "name": "my-app",
      "image": "my-app:latest",
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/my-ecs-task",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ]
}

Step 2: Create Firehose Stream

Follow the same process as CloudWatch Logs integration:

Create Firehose stream with HTTP Endpoint destination
Set endpoint to KubeSense aggregator Firehose endpoint
Create subscription filter for /ecs/* log groups

ECS Fargate Launch Type

For Fargate tasks, logs must go through CloudWatch Logs.

Step 1: Configure Fargate Task Definition

{
  "family": "my-fargate-task",
  "requiresCompatibilities": ["FARGATE"],
  "networkMode": "awsvpc",
  "cpu": "256",
  "memory": "512",
  "containerDefinitions": [
    {
      "name": "my-app",
      "image": "my-app:latest",
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/fargate/my-app",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ]
}

Step 2: Create Firehose Stream

Create Firehose stream for ECS Fargate logs:

aws firehose create-delivery-stream \
  --delivery-stream-name kubesense-ecs-fargate-logs \
  --http-endpoint-destination \
  --http-endpoint-destination-configuration \
    EndpointConfiguration="Url=https://<KUBESENSE_AGGREGATOR_HOST>:<PORT>/firehose",\
    AccessKey="<KUBESENSE_API_TOKEN>",\
    ContentEncoding="GZIP",\
    BufferingHints={SizeInMBs=5,IntervalInSeconds=60},\
    RequestConfiguration="{\"ContentEncoding\":\"GZIP\",\"Headers\":{\"env_name\":\"ecs-fargate-prod\"}}"

Step 3: Create Subscription Filters

For each ECS log group:

aws logs put-subscription-filter \
  --log-group-name "/ecs/fargate/my-app" \
  --filter-name "kubesense-ecs-fargate" \
  --filter-pattern "" \
  --destination-arn "arn:aws:firehose:region:account:deliverystream/kubesense-ecs-fargate-logs" \
  --role-arn "arn:aws:iam::account:role/KubeSenseCloudWatchFirehoseRole"

Alternative: FireLens Log Router

For more advanced log routing, use AWS FireLens:

Step 1: Add FireLens Container

{
  "family": "my-ecs-task",
  "containerDefinitions": [
    {
      "name": "log-router",
      "image": "amazon/aws-for-fluent-bit:latest",
      "essential": true,
      "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
          "enable-ecs-log-metadata": "true"
        }
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/firelens",
          "awslogs-region": "us-east-1"
        }
      }
    },
    {
      "name": "my-app",
      "image": "my-app:latest",
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "http",
          "Host": "<KUBESENSE_AGGREGATOR_HOST>",
          "Port": "30052",
          "URI": "/logs",
          "Header_key": "Authorization",
          "Header_value": "Bearer <KUBESENSE_API_TOKEN>",
          "Format": "json"
        }
      }
    }
  ]
}

Configure KubeSense Aggregator

Update aggregator configuration for ECS logs:

aggregator:
  customSources:
    enabled: true
    sources:
      ecs_firehose_logs:
        type: aws_kinesis_firehose
        address: 0.0.0.0:443
        access_keys:
          - "<KUBESENSE_API_TOKEN>"
      ecs_http_logs:
        type: http_server
        address: 0.0.0.0:30052
        decoding:
          codec: json
  enrichment:
    enabled: true
    addECSMetadata: true
    addDockerMetadata: true

Log Enrichment

KubeSense aggregator enriches ECS logs with:

Task metadata: Task ID, task definition, cluster name
Container information: Container name, image, labels
ECS metadata: Launch type, platform version
Docker metadata: Container ID, image tags

Monitoring and Verification

Check ECS task logs: Verify tasks are generating logs
Monitor CloudWatch: Check log groups for ECS logs
Verify Firehose: Check delivery metrics
Check KubeSense dashboard: Verify logs appear with ECS metadata
Review aggregator logs: Check for ingestion errors

Troubleshooting

EC2 Launch Type Issues

Verify LogSensor installation: Check LogSensor is running on EC2 hosts
Check Docker socket access: Ensure LogSensor can access Docker logs
Verify log paths: Check that log paths are correct
Review permissions: Ensure LogSensor has read access to container logs

Fargate Launch Type Issues

Verify log driver: Check task definition uses awslogs driver
Check CloudWatch permissions: Ensure task execution role has CloudWatch permissions
Verify log groups: Check log groups exist and are accessible
Review Firehose subscription: Ensure subscription filters are active

Missing Metadata

Check enrichment config: Verify aggregator enrichment is enabled
Review log format: Ensure logs include ECS metadata
Check FireLens config: If using FireLens, verify metadata is enabled

Best Practices

Use LogSensor for EC2: Direct log collection is more efficient than CloudWatch
Organize log groups: Use consistent naming for ECS log groups
Set retention: Configure CloudWatch Logs retention appropriately
Monitor costs: CloudWatch and Firehose can generate significant costs
Use FireLens for advanced routing: For complex log routing requirements
Tag resources: Use ECS tags for better log organization and filtering

Cost Considerations

CloudWatch Logs: Charged per GB ingested and stored
Firehose: Charged per GB processed
LogSensor: No additional AWS costs (direct collection)
Data transfer: Consider transfer costs between AWS and KubeSense

Conclusion

ECS logs integration provides comprehensive observability for both EC2 and Fargate launch types, enabling unified log analysis across all your containerized workloads.

ECS Logs

ON THIS PAGE