Quick start using Docker 🐳¶

Prerequisites¶

SPRUCE accepts AWS CUR reports (Parquet), Azure cost details (CSV), or FOCUS exports from either provider — see Generate usage reports for how to produce them. This tutorial uses an AWS CUR, generated via Data Exports and stored on S3 as Parquet files; commands for the other inputs are shown below.

For this tutorial, we will assume that you copied the S3 files to your local file system. You can do this with the AWS CLI

aws s3 cp s3://{bucket}/{prefix}/{data_export_name}/data/ curs --recursive

You will also need to have Docker installed.

With Docker¶

Pull the latest Docker image with

docker pull ghcr.io/digitalpebble/spruce

This retrieves a Docker image containing Apache Spark as well as the SPRUCE jar.

The command below processes the data locally by mounting the working directory as a volume. The input reports are assumed to be in a directory called curs.

AWS CURAWS FOCUSAzure FOCUS

docker run --rm -v $(pwd):/workspace -w /workspace \
ghcr.io/digitalpebble/spruce \
-i curs -o output

docker run --rm -v $(pwd):/workspace -w /workspace \
ghcr.io/digitalpebble/spruce \
-i focus -o output -f FOCUS

docker run --rm -v $(pwd):/workspace -w /workspace \
ghcr.io/digitalpebble/spruce \
-i focus -o output -p AZURE -f FOCUS

The -i parameter specifies the location of the directory containing the input reports. The -o parameter specifies the location of enriched Parquet files generated in output.

The option -c allows to specify a JSON configuration file to override the default settings. The options -p (cloud provider) and -f (report format) are described in Quick start using Apache Spark.

The directory output contains an enriched copy of the input reports. See Explore the results to understand what the output contains.

Reporting with Docker¶

The same image contains the reporting tools, so you can generate a report or explore the enriched output interactively without installing Python.

Generate a static report (format inferred from the suffix: .md, .html or .pdf):

docker run -it --rm -v $(pwd):/workspace -w /workspace \
ghcr.io/digitalpebble/spruce \
report -i output -o report.html

Launch the interactive dashboard, then open http://localhost:8501:

docker run --rm -p 8501:8501 -v $(pwd):/workspace -w /workspace \
ghcr.io/digitalpebble/spruce \
dashboard -i output

The image version determines both the SPRUCE jar and the reporting scripts, so when enriching and reporting are done at different times, use the same explicit tag (e.g. ghcr.io/digitalpebble/spruce:1.1) for both steps rather than relying on latest.