Skip to main content

observability with prometheus (authentification and TLS configuration)

How to Secure Prometheus with HTTPS and Authentication

Prometheus is a powerful monitoring tool, but by default, it does not include built-in authentication or transport layer security (TLS). This means anyone with network access can read your metrics, and if data is transmitted without encryption, it can be intercepted by attackers. To secure your Prometheus instance, you must configure both basic authentication and HTTPS. Here's an updated step-by-step guide to help you.

1. Why Secure Prometheus?

Prometheus is often used to monitor critical infrastructure, making it a valuable target for attackers. By default, anyone can access your Prometheus dashboard and view sensitive metrics if authentication and encryption are not configured. Without TLS encryption, even if authentication is enabled, the credentials and data can be intercepted over the network.

To ensure a minimum level of security, you must configure both HTTPS and authentication. These configurations:

  • Protect your metrics from being intercepted using TLS encryption.
  • Restrict access to authorized users only via authentication.

2. Configuration Overview

Prometheus uses the --web.config.file flag to specify a YAML configuration file. This file allows you to configure:

  • TLS settings: For encrypted HTTPS communication.
  • Basic authentication: To restrict access using usernames and passwords.

3. Setting Up HTTPS (TLS)

Step 1: Prepare Your Certificates

You will need:

  • A certificate file (cert_file).
  • A private key file (key_file).

You can generate a self-signed certificate for testing purposes or obtain a certificate from a trusted Certificate Authority (CA).

Generate a Self-Signed Certificate:

openssl req -x509 -newkey rsa:4096 -keyout prometheus.key -out prometheus.crt -days 365 -nodes

Save these files (prometheus.key and prometheus.crt) in a secure location accessible to Prometheus.

Step 2: Configure TLS in web-config.yml

Create a configuration file named web-config.yml with the following content:

tls_server_config:
  cert_file: "/path/to/prometheus.crt"
  key_file: "/path/to/prometheus.key"
  min_version: "TLS12" # Ensures only secure protocols are used
  client_auth_type: "RequireAndVerifyClientCert" # Optional: Enforces client authentication

Step 3: Run Prometheus with HTTPS

Launch Prometheus using the --web.config.file flag to specify the configuration file:

prometheus --config.file=prometheus.yml --web.config.file=web-config.yml

Step 4: Verify HTTPS

Visit your Prometheus instance at https://<server>:<port> in a browser. Ensure that the connection is secured using HTTPS.

4. Enabling Basic Authentication

Step 1: Add Users to web-config.yml

In the same web-config.yml file, add the basic_auth_users section. User passwords must be hashed using bcrypt.

Generate a bcrypt hash:

htpasswd -nBC 12 admin

This command generates a bcrypt hash for the username admin. Add the generated hash to your web-config.yml:

basic_auth_users:
  admin: "$2y$12$7G3Hk1eONF7Pz2C6plTtNeRnlh7g9gXVsLIDtOFHgWhKnHVcEu8.e"

Step 2: Apply the Configuration

Restart Prometheus with the updated configuration:

prometheus --config.file=prometheus.yml --web.config.file=web-config.yml

Step 3: Test Authentication

Now, when you access Prometheus via a browser, you should see a login prompt asking for your username and password.


Example: Prometheus prompts for authentication when secured.







5. Enhancing Security with HTTP Headers

To add an additional layer of security, configure HTTP headers in web-config.yml to protect against common vulnerabilities like clickjacking or MIME sniffing.

http_server_config:
  headers:
    Strict-Transport-Security: "max-age=63072000; includeSubDomains; preload"
    X-Content-Type-Options: "nosniff"
    X-Frame-Options: "deny"

6. Example web-config.yml

Below is an example configuration combining HTTPS, basic authentication, and security headers:

tls_server_config:
  cert_file: "/path/to/prometheus.crt"
  key_file: "/path/to/prometheus.key"
  min_version: "TLS12"

http_server_config:
  headers:
    Strict-Transport-Security: "max-age=63072000; includeSubDomains; preload"
    X-Content-Type-Options: "nosniff"
    X-Frame-Options: "deny"

basic_auth_users:
  admin: "$2y$12$7G3Hk1eONF7Pz2C6plTtNeRnlh7g9gXVsLIDtOFHgWhKnHVcEu8.e"

7. Key Considerations

  • TLS Performance: HTTPS introduces slight overhead but is essential for security.
  • Certificate Renewal: Ensure your certificates are kept up-to-date.
  • Configuration Reload: Prometheus automatically reloads its configuration for HTTPS and authentication on each HTTP request, making it easy to manage updates.

8. Conclusion

By configuring HTTPS and authentication, you significantly enhance the security of your Prometheus instance. This setup ensures encrypted connections and restricts access to authorized users only. For production environments, it is recommended to:

  • Use certificates from a trusted CA.
  • Deploy Prometheus behind a reverse proxy like NGINX for added security and scalability.

Implementing these steps ensures your Prometheus metrics remain secure and inaccessible to unauthorized users.

Comments

Post a Comment

Popular posts from this blog

Observability with grafana and prometheus (SSO configutation with active directory)

How to Set Up Grafana Single Sign-On (SSO) with Active Directory (AD) Grafana is a powerful tool for monitoring and visualizing data. Integrating it with Active Directory (AD) for Single Sign-On (SSO) can streamline access and enhance security. This tutorial will guide you through the process of configuring Grafana with AD for SSO. Prerequisites Active Directory Domain : Ensure you have an AD domain set up. Domain: bazboutey.local AD Server IP: 192.168.170.212 Users: grafana (for binding AD) user1 (to demonstrate SSO) we will end up with a pattern like this below Grafana Installed : Install Grafana on your server. Grafana Server IP: 192.168.179.185 Administrator Privileges : Access to modify AD settings and Grafana configurations. Step 1: Configure AD for LDAP Integration Create a Service Account in AD: Open Active Directory Users and Computers. Create a user (e.g., grafana ). Assign this user a strong password (e.g., Grafana 123$ ) and ensure it doesn’t expire. Gather Required AD D...
Post a Comment
Read more

Deploying a Scalable Monitoring Stack Lab on AWS using Terraform and Ansible

Deploying a Scalable Monitoring Stack Lab on AWS using Terraform and Ansible Introduction Effective monitoring is a cornerstone of cloud infrastructure management, ensuring high availability and performance. This guide provides a professional walkthrough on deploying Prometheus , Grafana , and Node Exporter on AWS using Terraform for infrastructure provisioning and Ansible for configuration management. This lab will create a prometheus server and a grafana server, It will install node exporter on both server. You should be able to see the metrics in grafana, we already install a node exporter dashboard for the user. The diagram below will give you an idea of what the architecture will look like If you want to replicate this lab, you can find the complete code repository here: GitHub - MireCloud Terraform Infra .  Infrastructure Setup with Terraform 1. Creating a Dedicated VPC To ensure isolation, we define a VPC named Monitoring with a CIDR block of 10.0.0.0/16 . reso...
Post a Comment
Read more

Building a Static Website on AWS with Terraform

The Journey to a Fully Automated Website Deployment A few weeks ago, I found myself needing to deploy a simple static website . Manually setting up an S3 bucket, configuring permissions, and linking it to a CloudFront distribution seemed like a tedious process. As someone who loves automation, I decided to leverage Terraform to simplify the entire process. Why Terraform? Infrastructure as Code (IaC) is a game-changer. With Terraform, I could:  Avoid manual setup errors  Easily reproduce and  Automate security best practices Instead of clicking through AWS settings, I wrote a few Terraform scripts and deployed everything in minutes. Let me walk you through how I did it!  Architecture Overview The architecture consists of three main components: User:  The end user accesses the website via a CloudFront URL.  CloudFront Distribution:  Acts as a content delivery network (CDN) to distribute content efficiently, reduce latency, and enhance security. It ...
Post a Comment
Read more