Modules configuration

The enrichment modules are configured in a file called default-config.json. This file is included in the JAR and looks like this:

{
  "modules": [
    {
      "className": "com.digitalpebble.spruce.modules.RegionExtraction"
    },
    {
      "className": "com.digitalpebble.spruce.modules.ccf.Storage",
      "config": {
        "hdd_coefficient_tb_h": 0.65,
        "ssd_coefficient_tb_h": 1.2
      }
    },
    {
      "className": "com.digitalpebble.spruce.modules.Networking",
      "config": {
        "network_coefficients_kwh_gb": {
          "intra": 0.001,
          "inter": 0.0015,
          "extra": 0.059
        }
      }
    },
    {
      "className": "com.digitalpebble.spruce.modules.boavizta.BoaviztAPIstatic"
    },
    {
      "className": "com.digitalpebble.spruce.modules.Serverless",
      "config": {
        "memory_coefficient_kwh": 0.0000598,
        "x86_cpu_coefficient_kwh": 0.0088121875,
        "arm_cpu_coefficient_kwh": 0.00191015625
      }
    },
    {
      "className": "com.digitalpebble.spruce.modules.ccf.Accelerators",
      "config": {
        "gpu_utilisation_percent": 50
      }
    },
    {
      "className": "com.digitalpebble.spruce.modules.ecologits.BedrockEcoLogits",
      "config": {
        "input_token_ratio": 0.5
      }
    },
    {
      "className": "com.digitalpebble.spruce.modules.PUE",
      "config": {
        "default": 1.15
      }
    },
    {
      "className": "com.digitalpebble.spruce.modules.Water"
    },
    {
      "className": "com.digitalpebble.spruce.modules.electricitymaps.AverageCarbonIntensity"
    },
    {
      "className": "com.digitalpebble.spruce.modules.OperationalEmissions",
      "config": {
        "powerSupplyEfficiency": 1.04,
        "powerTransmissionLosses": 1.08
      }
    }
  ]
}

This determines which modules are used and in what order but also configures their behaviour. For instance, the Networking module uses different coefficients for intra-region, inter-region, and external data transfers, all configurable via the network_coefficients_kwh_gb map.

Change the configuration

In order to use a different configuration, for instance to replace a module with another one, or change their configuration (like the network coefficient above), you simply need to write a json file with your changes and pass it as an argument to the Spark job with ‘-c’.