While helping at the Cloud Foundry booth at KubeCon Paris 2024, I saw a lot of interest in open source Cloud Native Buildpacks. And it makes sense. As everybody jumps on the platform engineering train, operation teams are looking for the building blocks that will allow them to standardize their application lifecycle regardless of the stack. Buildpacks, hosted by the Linux Foundation, offer features that fit well with the platform engineering trend; they will enable standardization of the containerizing process and provide a consistent developer experience no matter the stack while providing solid performance and security features.

Buildpacks have been around for over a decade now — with Heroku creating the concept in 2011 — and many practitioners have already worked with them at some point in their careers. One question kept coming from KubeCon attendees: is it still hard to install APM (Application Performance Monitoring)?

Those who have been using Buildpacks for a while will know that integrating an APM agent was complicated. Cloud Foundry maintainer Tim Downey explained that “users had to run a separate buildpack — along with application’s one — that supplied the APM binary. This buildpack was either chained alongside the application’s regular start command or used by CF sidecar processes”.

But that’s no longer the case. In this article, I will show how to easily add an APM — taking OpenTelemetry as an example — to a Python application.

Setup a Python Application

If you already have a Python application that you can play with, skip this section; if not, read on.

Let’s clone a repository that contains a bunch of sample applications and navigate to the Python application folder:

git clone https://github.com/sylvainkalache/sample-web-apps cd sample-web-apps/python/

12 | git clone https://github.com/sylvainkalache/sample-web-appscd sample-web-apps/python/
---|---

As you can see, it’s a simple Flask application.

$ cat my-app.py from flask import Flask, request, render_template import gunicorn import platform import subprocess app = Flask(name) @app.route("/") def hello(): return "Hello, World!\n" + "Python version: " + platform.python_version() + "\n"

1234567891011 | $ cat my-app.pyfrom flask import Flask, request, render_templateimport gunicornimport platformimport subprocess app = Flask(name) @app.route("/")def hello(): return "Hello, World!\n" + "Python version: " + platform.python_version() + "\n"
---|---

Add OpenTelemetry to Our Application

OpenTelemetry is an open-source framework for collecting telemetry data such as metrics, logs, and traces from any application. The collected data can be sent to tools such as Prometheus for metrics, Jaeger and Zipkin for tracing, and ELK Stack (Elasticsearch, Logstash, Kibana) for logs.

First, you will need to import these basic OpenTelemetry libraries.

from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor

123 | from opentelemetry import tracefrom opentelemetry.sdk.trace import TracerProviderfrom opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
---|---

Then, we need to define an OpenTelemetry tracer and an exporter. In this case, I will keep it simple and export the collected data to the console for the sake of demonstration.

Set up the TracerProvider trace.set_tracer_provider(TracerProvider()) # Initialize the ConsoleSpanExporter console_exporter = ConsoleSpanExporter() # Set up SimpleSpanProcessor to use ConsoleSpanExporter span_processor = SimpleSpanProcessor(console_exporter) trace.get_tracer_provider().add_span_processor(span_processor) # Get a tracer tracer = trace.get_tracer(name)

123456789101112 | # Set up the TracerProvidertrace.set_tracer_provider(TracerProvider()) # Initialize the ConsoleSpanExporterconsole_exporter = ConsoleSpanExporter() # Set up SimpleSpanProcessor to use ConsoleSpanExporterspan_processor = SimpleSpanProcessor(console_exporter)trace.get_tracer_provider().add_span_processor(span_processor) # Get a tracertracer = trace.get_tracer(name)
---|---

Now, we need to tell OpenTelemetry to trace something; add this line to initiate a new tracing span to the hello method.

with tracer.start_as_current_span("my_tracer"):

1 | with tracer.start_as_current_span("my_tracer"):
---|---

Here is what your my-app.py file should look like:

from flask import Flask, request, render_template import gunicorn import platform import subprocess from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor app = Flask(name) # Set up the TracerProvider trace.set_tracer_provider(TracerProvider()) # Initialize the ConsoleSpanExporter console_exporter = ConsoleSpanExporter() # Set up SimpleSpanProcessor to use ConsoleSpanExporter span_processor = SimpleSpanProcessor(console_exporter) trace.get_tracer_provider().add_span_processor(span_processor) # Get a tracer tracer = trace.get_tracer(name) @app.route("/") def hello(): with tracer.start_as_current_span("hello_world"): return "Hello, World!\n" + "Python version: " + platform.python_version() + "\n"

123456789101112131415161718192021222324252627 | from flask import Flask, request, render_templateimport gunicornimport platformimport subprocessfrom opentelemetry import tracefrom opentelemetry.sdk.trace import TracerProviderfrom opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor app = Flask(name) # Set up the TracerProvidertrace.set_tracer_provider(TracerProvider()) # Initialize the ConsoleSpanExporterconsole_exporter = ConsoleSpanExporter() # Set up SimpleSpanProcessor to use ConsoleSpanExporterspan_processor = SimpleSpanProcessor(console_exporter)trace.get_tracer_provider().add_span_processor(span_processor) # Get a tracertracer = trace.get_tracer(name) @app.route("/")def hello(): with tracer.start_as_current_span("hello_world"): return "Hello, World!\n" + "Python version: " + platform.python_version() + "\n"
---|---

Last, to ensure that Buildpacks detects and installs the OpenTelemetry libraries and dependencies, add the lines below to your requirements.txt file.

opentelemetry-api opentelemetry-sdk opentelemetry-instrumentation opentelemetry-instrumentation-flask

1234 | opentelemetry-apiopentelemetry-sdkopentelemetry-instrumentationopentelemetry-instrumentation-flask
---|---

Pack It and Run It

The beauty of Buildpack is that with one pack CLI command, you will create a production-ready OCR image from our application. At the root of the Python application folder, run the following pack command.

pack build my-python-app --builder paketobuildpacks/builder-jammy-base

1 | pack build my-python-app --builder paketobuildpacks/builder-jammy-base
---|---

The container was created, and you can run it using the docker command below:

docker run -ti -p 5000:8000 -e PORT=8000 my-python-app

1 | docker run -ti -p 5000:8000 -e PORT=8000 my-python-app
---|---

Now, in another tab, query your application using curl.

$ curl 0:5000 Hello, World! Python version: 3.10.14

123 | $ curl 0:5000Hello, World!Python version: 3.10.14
---|---

Your application should output the following, and we can see the OpenTelemetry output displayed.

[2024-05-01 21:35:18 +0000] [1] [INFO] Starting gunicorn 22.0.0 [2024-05-01 21:35:18 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1) [2024-05-01 21:35:18 +0000] [1] [INFO] Using worker: sync [2024-05-01 21:35:18 +0000] [22] [INFO] Booting worker with pid: 22 { "name": "hello_world", "context": { "trace_id": "0xebb54bc5f7340b22c237a2c73af2a266", "span_id": "0x7a02dd7f80e16c16", "trace_state": "[]" }, "kind": "SpanKind.INTERNAL", "parent_id": null, "start_time": "2024-05-01T21:35:32.803533Z", "end_time": "2024-05-01T21:35:32.803574Z", "status": { "status_code": "UNSET" }, "attributes": {}, "events": [], "links": [], "resource": { "attributes": { "telemetry.sdk.language": "python", "telemetry.sdk.name": "opentelemetry", "telemetry.sdk.version": "1.24.0", "service.name": "unknown_service" }, "schema_url": "" } }

12345678910111213141516171819202122232425262728293031 | [2024-05-01 21:35:18 +0000] [1] [INFO] Starting gunicorn 22.0.0[2024-05-01 21:35:18 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)[2024-05-01 21:35:18 +0000] [1] [INFO] Using worker: sync[2024-05-01 21:35:18 +0000] [22] [INFO] Booting worker with pid: 22{ "name": "hello_world", "context": { "trace_id": "0xebb54bc5f7340b22c237a2c73af2a266", "span_id": "0x7a02dd7f80e16c16", "trace_state": "[]" }, "kind": "SpanKind.INTERNAL", "parent_id": null, "start_time": "2024-05-01T21:35:32.803533Z", "end_time": "2024-05-01T21:35:32.803574Z", "status": { "status_code": "UNSET" }, "attributes": {}, "events": [], "links": [], "resource": { "attributes": { "telemetry.sdk.language": "python", "telemetry.sdk.name": "opentelemetry", "telemetry.sdk.version": "1.24.0", "service.name": "unknown_service" }, "schema_url": "" }}
---|---

As you can see, it is now very easy to install an APM in an application packaged with Buildpack. The same principle applies to most APMs, including New Relic and Datadog. For those reluctant to use open source Buildpacks because of that, it’s time to reconsider!