Skip to main content

FastAPI Instrumentalisation with prometheus and grafana Part1 [Counter]

welcome to this hands-on lab on API instrumentation using Prometheus and FastAPI!

In the world of modern software development, real-time API monitoring is essential for understanding usage patterns, debugging issues, and ensuring optimal performance. In this lab, we’ll demonstrate how to enhance a FastAPI-based application with Prometheus metrics to monitor its behavior effectively.

We’ve already set up the lab environment for you, complete with Grafana, Prometheus, and a PostgreSQL database. While FastAPI’s integration with databases is outside the scope of this lab, our focus will be entirely on instrumentation and monitoring. For those interested in exploring the database integration or testing , you can review the code in our repository:
FastAPI Monitoring Repository.

What You’ll Learn

In this lab, we’ll walk you through:

  1. Setting up Prometheus metrics in a FastAPI application.
  2. Instrumenting API endpoints to track:
    • Number of requests
    • HTTP methods
    • Request paths
  3. Using Grafana to visualize these metrics.
  4. Focusing on Prometheus Counters, one of the key metric types.

Why Prometheus?

Prometheus is a powerful open-source monitoring system that allows you to collect and query metrics from your applications. It offers various types of metrics to track different behaviors:

  • Counters: Measure cumulative values that only increase or reset to zero (e.g., number of requests).
  • Histograms: Track distributions of data (e.g., request durations).
  • Gauges: Record values that can increase or decrease (e.g., memory usage).
  • Summaries: Aggregate and summarize values over time (e.g., latency percentiles).

In this lab, we’ll start with Counters and dive into the other types in subsequent parts.

Part 1 Overview: Counters

In Part 1, we’ll:

  • Implement a simple CRUD API using FastAPI.
  • Add Prometheus Counter metrics to track:
    • Total number of API requests.
    • Breakdown by request path and HTTP method.
  • Set up Grafana dashboards to visualize these metrics.

This lab is designed to give you hands-on experience with monitoring APIs using Prometheus. As we progress through the three parts, you’ll gain a comprehensive understanding of how to instrument APIs with various metric types, enabling better observability and performance insights.

Let’s dive into Part 1 and start building!

This revised version makes the content flow smoothly while emphasizing the key points, keeping readers engaged and excited to move forward. Let me know if you'd like more adjustments!

Prerequisites

Before starting, ensure the following are ready:

  • A running PostgreSQL database.
  • Python environment with the necessary libraries installed:
    • FastAPI
    • SQLAlchemy
    • Prometheus-client
  • Prometheus installed and running to scrape metrics.
  • Basic understanding of FastAPI and Prometheus concepts.

Lab Objectives

By the end of this lab, you will:

  1. Create a FastAPI application with CRUD functionality.
  2. Integrate Prometheus metrics into the application.
  3. Expose metrics such as request counts categorized by paths and HTTP methods.
  4. Visualize and monitor the API using Prometheus.

Overview of the Code

The following code implements a FastAPI application, which serves as the base for our API. It includes CRUD operations on a PostgreSQL database and instrumentation with Prometheus counters.

from fastapi import FastAPI , Response, status, HTTPException, Depends, Request
from fastapi.params  import Body
from pydantic import BaseModel
from random import randrange
from sqlalchemy.orm import Session
import sys
import os
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import psycopg
import time
from psycopg import connect, ClientCursor 
from psycopg.rows import dict_row
from sqlalchemy_projects import models
from sqlalchemy_projects.database import engine, SessionLocal, get_db
from prometheus_client import Counter, start_http_server,Histogram, Summary

# Initialize Prometheus metrics
REQUESTS_BY_PATH = Counter('http_requests_total_FastAPI', 'Total number of requests', labelnames=['path', 'method'])
ERRORS = Counter('http_errors_total_fastAPI', 'Total number of errors', labelnames=['status_code'])

start_http_server(7000)

# Create database models
models.Base.metadata.create_all(bind=engine)

app = FastAPI()

# Define your routes
class Post(BaseModel):
    title: str
    content: str

@app.get("/")
def read_root():
    REQUESTS_BY_PATH.labels("/", 'get').inc()
    return {"Hello": "Welcome to my FastAPI practice World!!!"}

@app.get("/sqlalchemy")
def test_posts(db: Session = Depends(get_db)):
    posts = db.query(models.Post).all()
    REQUESTS_BY_PATH.labels("/sqlalchemy", 'get').inc()
    return {"data": posts}

@app.get("/posts")
def get_posts(db: Session = Depends(get_db)):
    posts = db.query(models.Post).all()
    REQUESTS_BY_PATH.labels("/posts", 'get').inc()
    return {"data": posts}

@app.post("/posts", status_code=status.HTTP_201_CREATED)
def create_new_post(post: Post, db: Session = Depends(get_db)):
    new_post = models.Post(**post.dict())
    db.add(new_post)
    db.commit()
    db.refresh(new_post)
    REQUESTS_BY_PATH.labels("/posts", 'post').inc()
    return {"data": new_post}

@app.get("/posts/{id}")
def get_post(id: int, db: Session = Depends(get_db)):
    post = db.query(models.Post).filter(models.Post.id == id).first()
    if not post:
        ERRORS.labels('404').inc()
        raise HTTPException(status_code=404, detail=f"The post with id: {id} was not found")
    REQUESTS_BY_PATH.labels("/posts/{id}", 'get').inc()
    return {"post details": post}

@app.delete("/posts/{id}", status_code=status.HTTP_204_NO_CONTENT)
def delete_post(id: int, db: Session = Depends(get_db)):
    REQUESTS_BY_PATH.labels("/posts/id",'delete').inc()
    post = db.query(models.Post).filter(models.Post.id == id)

    if post.first() == None:
        ERRORS.labels('404').inc()
        raise HTTPException(status_code=404, detail= f"The post with id: {id} was not found")      
    post.delete(synchronize_session=False) 
    db.commit()

    return Response(status_code=status.HTTP_204_NO_CONTENT)

@app.put("/posts/{id}")
def update_post(id: int, post: Post, db: Session = Depends(get_db)):
    post_query = db.query(models.Post).filter(models.Post.id == id)
    existing_post = post_query.first()

    if not existing_post:
        ERRORS.labels('404').inc()
        raise HTTPException(status_code=404, detail=f"The post with id: {id} was not found")

    post_query.update(post.dict(), synchronize_session=False)
    db.commit()
 

Key Components of the Lab Code:

  1. Database Connection with SQLAlchemy:

    • Models and database session are managed using SQLAlchemy.
    • PostgreSQL serves as the database backend.

  2. CRUD Endpoints:

    • GET /posts: Retrieve all posts.
    • POST /posts: Create a new post.
    • GET /posts/{id}: Retrieve a specific post by ID.
    • DELETE /posts/{id}: Delete a specific post by ID.
    • PUT /posts/{id}: Update a specific post by ID.

  3. Prometheus Instrumentation:

  • The Prometheus-client library is used to track metrics.
  • A Counter metric (REQUESTS_BY_PATH) is defined to count HTTP requests by their path and HTTP method.
  • Metrics are incremented for each endpoint call using labels for path and method.

  1. Endpoints for CRUD Operations:

    • Each CRUD endpoint is connected to the database and uses SQLAlchemy for interaction.
    • The Prometheus counter is updated at the start of each endpoint execution.

  2. Metrics Exposure:

    • Prometheus metrics are exposed through the built-in HTTP server at port 7001 (using start_http_server).
    • Prometheus can scrape these metrics by configuring its prometheus.yml to point to this endpoint.

Code Walkthrough

Below is a step-by-step explanation of the lab code.

1. Initialization

The app initializes with:

  • Importing required libraries (FastAPI, Prometheus-client, SQLAlchemy).
  • Connecting to the PostgreSQL database using SQLAlchemy.
from fastapi import FastAPI
from prometheus_client import Counter, start_http_server
from sqlalchemy.orm import Session
from sqlalchemy_projects.database import get_db
from sqlalchemy_projects import models

2. Prometheus Metrics

A Counter metric is defined to track:

  • Total HTTP requests.
  • Labels: request path and HTTP method.
REQUESTS_BY_PATH = Counter('http_requests_total', 'Total number of requests', labelnames=['path', 'method'])
start_http_server(7001)

3. CRUD Operations

Endpoints are created for each operation:

  • Retrieve All Posts (GET /posts): Queries all posts from the database and increments the request counter.
@app.get("/posts")
def get_posts(db: Session = Depends(get_db)):
    posts = db.query(models.Post).all()
    REQUESTS_BY_PATH.labels("/posts", "get").inc()
    return {"data": posts}
  • Create Post (POST /posts): Adds a new post to the database and increments the request counter.
@app.post("/posts", status_code=status.HTTP_201_CREATED)
def create_new_post(post: Post, db: Session = Depends(get_db)):
    new_post = models.Post(**post.model_dump())
    db.add(new_post)
    db.commit()
    db.refresh(new_post)
    REQUESTS_BY_PATH.labels("/posts", "post").inc()
    return {"data": new_post}
  • Retrieve Specific Post (GET /posts/{id}): Fetches a single post by its ID and raises a 404 error if not found.
@app.get("/posts/{id}")
def get_post(id: int, db: Session = Depends(get_db)):
    post = db.query(models.Post).filter(models.Post.id == id).first()
    if not post:
        raise HTTPException(status_code=404, detail=f"The post with id {id} was not found")
    REQUESTS_BY_PATH.labels("/posts/id", "get").inc()
    return {"data": post}
  • Update Post (PUT /posts/{id}): Updates a post's content using its ID.
@app.put("/posts/{id}")
def update_post(id: int, post: Post, db: Session = Depends(get_db)):
    post_query = db.query(models.Post).filter(models.Post.id == id)
    existing_post = post_query.first()
    if not existing_post:
        raise HTTPException(status_code=404, detail=f"The post with id {id} was not found")
    post_query.update(post.model_dump(), synchronize_session=False)
    db.commit()
    REQUESTS_BY_PATH.labels("/posts/id", "update").inc()
    return {"data": post_query.first()}
  • Delete Post (DELETE /posts/{id}): Deletes a post using its ID.
@app.delete("/posts/{id}")
def delete_post(id: int, db: Session = Depends(get_db)):
    post = db.query(models.Post).filter(models.Post.id == id)
    if not post.first():
        raise HTTPException(status_code=404, detail=f"The post with id {id} was not found")
    post.delete(synchronize_session=False)
    db.commit()
    REQUESTS_BY_PATH.labels("/posts/id", "delete").inc()
    return Response(status_code=status.HTTP_204_NO_CONTENT)

Running the Application

  1. Start the API Server: Run the FastAPI application, which starts the server at port 8000.

  2. Prometheus Metrics: The Prometheus metrics are exposed at port 7000.

  3. Monitor with Prometheus: Configure Prometheus to scrape the metrics endpoint by adding this to the prometheus.yml file:

scrape_configs:
  - job_name: "fastapi"
    static_configs:
      - targets: ["localhost:7000"]
  1. Test API Endpoints: Use tools like curl, Postman, or a web browser to interact with the API and generate metrics.
  2. After testing the different endpoint, you can verify your metrics on port 7000


GRAFANA Dashboard

With all the metrics displays, you can now create dashboards in grafana




Conclusion

This lab demonstrates how to:

  • Build a FastAPI application with CRUD operations.
  • Integrate Prometheus to track and expose metrics.
  • Leverage monitoring tools to observe API behavior.

Prometheus instrumentation is a powerful way to gain insights into the performance and usage of APIs, ensuring they remain reliable and efficient.

Comments

Post a Comment

Popular posts from this blog

Observability with grafana and prometheus (SSO configutation with active directory)

How to Set Up Grafana Single Sign-On (SSO) with Active Directory (AD) Grafana is a powerful tool for monitoring and visualizing data. Integrating it with Active Directory (AD) for Single Sign-On (SSO) can streamline access and enhance security. This tutorial will guide you through the process of configuring Grafana with AD for SSO. Prerequisites Active Directory Domain : Ensure you have an AD domain set up. Domain: bazboutey.local AD Server IP: 192.168.170.212 Users: grafana (for binding AD) user1 (to demonstrate SSO) we will end up with a pattern like this below Grafana Installed : Install Grafana on your server. Grafana Server IP: 192.168.179.185 Administrator Privileges : Access to modify AD settings and Grafana configurations. Step 1: Configure AD for LDAP Integration Create a Service Account in AD: Open Active Directory Users and Computers. Create a user (e.g., grafana ). Assign this user a strong password (e.g., Grafana 123$ ) and ensure it doesn’t expire. Gather Required AD D...
Post a Comment
Read more

Deploying a Scalable Monitoring Stack Lab on AWS using Terraform and Ansible

Deploying a Scalable Monitoring Stack Lab on AWS using Terraform and Ansible Introduction Effective monitoring is a cornerstone of cloud infrastructure management, ensuring high availability and performance. This guide provides a professional walkthrough on deploying Prometheus , Grafana , and Node Exporter on AWS using Terraform for infrastructure provisioning and Ansible for configuration management. This lab will create a prometheus server and a grafana server, It will install node exporter on both server. You should be able to see the metrics in grafana, we already install a node exporter dashboard for the user. The diagram below will give you an idea of what the architecture will look like If you want to replicate this lab, you can find the complete code repository here: GitHub - MireCloud Terraform Infra .  Infrastructure Setup with Terraform 1. Creating a Dedicated VPC To ensure isolation, we define a VPC named Monitoring with a CIDR block of 10.0.0.0/16 . reso...
Post a Comment
Read more

Building a Static Website on AWS with Terraform

The Journey to a Fully Automated Website Deployment A few weeks ago, I found myself needing to deploy a simple static website . Manually setting up an S3 bucket, configuring permissions, and linking it to a CloudFront distribution seemed like a tedious process. As someone who loves automation, I decided to leverage Terraform to simplify the entire process. Why Terraform? Infrastructure as Code (IaC) is a game-changer. With Terraform, I could:  Avoid manual setup errors  Easily reproduce and  Automate security best practices Instead of clicking through AWS settings, I wrote a few Terraform scripts and deployed everything in minutes. Let me walk you through how I did it!  Architecture Overview The architecture consists of three main components: User:  The end user accesses the website via a CloudFront URL.  CloudFront Distribution:  Acts as a content delivery network (CDN) to distribute content efficiently, reduce latency, and enhance security. It ...
Post a Comment
Read more