Skip to main content

observability with prometheus (authentification and TLS configuration)

How to Secure Prometheus with HTTPS and Authentication

Prometheus is a powerful monitoring tool, but by default, it does not include built-in authentication or transport layer security (TLS). This means anyone with network access can read your metrics, and if data is transmitted without encryption, it can be intercepted by attackers. To secure your Prometheus instance, you must configure both basic authentication and HTTPS. Here's an updated step-by-step guide to help you.

1. Why Secure Prometheus?

Prometheus is often used to monitor critical infrastructure, making it a valuable target for attackers. By default, anyone can access your Prometheus dashboard and view sensitive metrics if authentication and encryption are not configured. Without TLS encryption, even if authentication is enabled, the credentials and data can be intercepted over the network.

To ensure a minimum level of security, you must configure both HTTPS and authentication. These configurations:

  • Protect your metrics from being intercepted using TLS encryption.
  • Restrict access to authorized users only via authentication.

2. Configuration Overview

Prometheus uses the --web.config.file flag to specify a YAML configuration file. This file allows you to configure:

  • TLS settings: For encrypted HTTPS communication.
  • Basic authentication: To restrict access using usernames and passwords.

3. Setting Up HTTPS (TLS)

Step 1: Prepare Your Certificates

You will need:

  • A certificate file (cert_file).
  • A private key file (key_file).

You can generate a self-signed certificate for testing purposes or obtain a certificate from a trusted Certificate Authority (CA).

Generate a Self-Signed Certificate:

openssl req -x509 -newkey rsa:4096 -keyout prometheus.key -out prometheus.crt -days 365 -nodes

Save these files (prometheus.key and prometheus.crt) in a secure location accessible to Prometheus.

Step 2: Configure TLS in web-config.yml

Create a configuration file named web-config.yml with the following content:

tls_server_config:
  cert_file: "/path/to/prometheus.crt"
  key_file: "/path/to/prometheus.key"
  min_version: "TLS12" # Ensures only secure protocols are used
  client_auth_type: "RequireAndVerifyClientCert" # Optional: Enforces client authentication

Step 3: Run Prometheus with HTTPS

Launch Prometheus using the --web.config.file flag to specify the configuration file:

prometheus --config.file=prometheus.yml --web.config.file=web-config.yml

Step 4: Verify HTTPS

Visit your Prometheus instance at https://<server>:<port> in a browser. Ensure that the connection is secured using HTTPS.

4. Enabling Basic Authentication

Step 1: Add Users to web-config.yml

In the same web-config.yml file, add the basic_auth_users section. User passwords must be hashed using bcrypt.

Generate a bcrypt hash:

htpasswd -nBC 12 admin

This command generates a bcrypt hash for the username admin. Add the generated hash to your web-config.yml:

basic_auth_users:
  admin: "$2y$12$7G3Hk1eONF7Pz2C6plTtNeRnlh7g9gXVsLIDtOFHgWhKnHVcEu8.e"

Step 2: Apply the Configuration

Restart Prometheus with the updated configuration:

prometheus --config.file=prometheus.yml --web.config.file=web-config.yml

Step 3: Test Authentication

Now, when you access Prometheus via a browser, you should see a login prompt asking for your username and password.


Example: Prometheus prompts for authentication when secured.







5. Enhancing Security with HTTP Headers

To add an additional layer of security, configure HTTP headers in web-config.yml to protect against common vulnerabilities like clickjacking or MIME sniffing.

http_server_config:
  headers:
    Strict-Transport-Security: "max-age=63072000; includeSubDomains; preload"
    X-Content-Type-Options: "nosniff"
    X-Frame-Options: "deny"

6. Example web-config.yml

Below is an example configuration combining HTTPS, basic authentication, and security headers:

tls_server_config:
  cert_file: "/path/to/prometheus.crt"
  key_file: "/path/to/prometheus.key"
  min_version: "TLS12"

http_server_config:
  headers:
    Strict-Transport-Security: "max-age=63072000; includeSubDomains; preload"
    X-Content-Type-Options: "nosniff"
    X-Frame-Options: "deny"

basic_auth_users:
  admin: "$2y$12$7G3Hk1eONF7Pz2C6plTtNeRnlh7g9gXVsLIDtOFHgWhKnHVcEu8.e"

7. Key Considerations

  • TLS Performance: HTTPS introduces slight overhead but is essential for security.
  • Certificate Renewal: Ensure your certificates are kept up-to-date.
  • Configuration Reload: Prometheus automatically reloads its configuration for HTTPS and authentication on each HTTP request, making it easy to manage updates.

8. Conclusion

By configuring HTTPS and authentication, you significantly enhance the security of your Prometheus instance. This setup ensures encrypted connections and restricts access to authorized users only. For production environments, it is recommended to:

  • Use certificates from a trusted CA.
  • Deploy Prometheus behind a reverse proxy like NGINX for added security and scalability.

Implementing these steps ensures your Prometheus metrics remain secure and inaccessible to unauthorized users.

Comments

Post a Comment

Popular posts from this blog

FastAPI Instrumentalisation with prometheus and grafana Part1 [Counter]

welcome to this hands-on lab on API instrumentation using Prometheus and FastAPI! In the world of modern software development, real-time API monitoring is essential for understanding usage patterns, debugging issues, and ensuring optimal performance. In this lab, we’ll demonstrate how to enhance a FastAPI-based application with Prometheus metrics to monitor its behavior effectively. We’ve already set up the lab environment for you, complete with Grafana, Prometheus, and a PostgreSQL database. While FastAPI’s integration with databases is outside the scope of this lab, our focus will be entirely on instrumentation and monitoring. For those interested in exploring the database integration or testing , you can review the code in our repository: FastAPI Monitoring Repository . What You’ll Learn In this lab, we’ll walk you through: Setting up Prometheus metrics in a FastAPI application. Instrumenting API endpoints to track: Number of requests HTTP methods Request paths Using Grafana to vi...
Post a Comment
Read more

Join Ubuntu 20.04 to Active Directory with SSSD and SSH Access

Join Ubuntu 20.04 to Active Directory with SSSD and SSH Access  Overview This guide walks you through joining an Ubuntu 20.04 machine to an Active Directory domain using SSSD, configuring PAM for AD user logins over SSH, and enabling automatic creation of home directories upon first login. We’ll also cover troubleshooting steps and verification commands. Environment Used Component Value Ubuntu Client       ubuntu-client.bazboutey.local Active Directory FQDN   bazboutey.local Realm (Kerberos)   BAZBOUTEY.LOCAL AD Admin Account   Administrator Step 1: Prerequisites and Package Installation 1.1 Update system and install required packages bash sudo apt update sudo apt install realmd sssd libnss-sss libpam-sss adcli \ samba-common-bin oddjob oddjob-mkhomedir packagekit \ libpam-modules openssh-server Step 2: Test DNS and Kerberos Configuration Ensure that the client can resolve the AD domain and discover services. 2.1 Test domain name resol...
Post a Comment
Read more

Observability with grafana and prometheus (SSO configutation with active directory)

How to Set Up Grafana Single Sign-On (SSO) with Active Directory (AD) Grafana is a powerful tool for monitoring and visualizing data. Integrating it with Active Directory (AD) for Single Sign-On (SSO) can streamline access and enhance security. This tutorial will guide you through the process of configuring Grafana with AD for SSO. Prerequisites Active Directory Domain : Ensure you have an AD domain set up. Domain: bazboutey.local AD Server IP: 192.168.170.212 Users: grafana (for binding AD) user1 (to demonstrate SSO) we will end up with a pattern like this below Grafana Installed : Install Grafana on your server. Grafana Server IP: 192.168.179.185 Administrator Privileges : Access to modify AD settings and Grafana configurations. Step 1: Configure AD for LDAP Integration Create a Service Account in AD: Open Active Directory Users and Computers. Create a user (e.g., grafana ). Assign this user a strong password (e.g., Grafana 123$ ) and ensure it doesn’t expire. Gather Required AD D...
Post a Comment
Read more