HaaS (HEAT as a Service)

The HEAT tool (www.heatwalkingcycling.org) uses an R package to which data collected in the UI is passed to conduct all calculations. This experimental API allows to access the HEAT model (R package) directly. Currently, “business logic” (i.e. validation of input data) is only available through the UI, therefore all manually provided model inputs should be introduced with great care.

Steps which happen in UI

To understand the model’s functions, a very simple outline/description is provided below:

  1. The UI collects “scenario” details (aka use case criteria) (incl. geographic location, selected modes of transport, type of assessment, etc.). These UI inputs are prefixed with ucc_ (for use case criteria). In the UI all validity checks and warnings are carried out - here and only here at the present moment (incl. maximum levels of active travel, minimum population size, etc.)
  2. These inputs with values are passed to the prepare_generic_data_relevant() function which builds a data.frame with all needed background data for the selected “scenario”. Data.frame is returned to UI, so that some default values (fetched from built-in background data) can be reviewed and optionally reset in the UI.
  3. Once all inputs in UI are collected and the Results page is reached, the model receives again all inputs and required background values (i.e. an extended list of inputs from 1) passing these to the calc_results() function (also default values from 2 are used). The model is run and results are calculated.

API

The API methods replicate UI behaviour. For now the following endpoints are available:

/apiv1/<model:character>/<method:character>/?output_format=format<character>

where:

  • model allows to specify which model should be used. Available values:
  • method specifies which “function” of the model should be executed:
    • prepare_gdr executes prepare_generic_data_relevant() function. Needed input: webapp_inputs (more later).
    • calc_results executes calc_results() function. Needed inputs: webapp_input, generic_data_relevant (more later).
    • results executes prepare_generic_data_relevant() and calc_results() – wrapper to reduce requests send to API (if no changes in data.frame with default values are needed). Needed inputs: webapp_inputs (more later).
  • format (optional, if not provided - default: jsonl-serialized) is responsible for output/returned format of data. Possible values:
    • jsonl-serialized - description of this format in the next section. This is default output format if not specified;
    • json;
    • rds - only version 2 of RDS is supported.

API - request and inputs

At the present moment only POST HTTP method is handled by all endpoints.

The endpoints need input data which is sent as a part of a request via POST body. There are two formats accepted to send input data:

  • jsonl-serialized
  • json

Since conversion to json might not always reflect R object perfectly (for example mapping of columns in data.frame with NA-s only) custom jsonl-serialized was added. This is a form produced by the jsonlite::serializeJSON() function. To be able to process a request in that form, the content type of POST request has to be set to jsonl-serialized.

jsonl-serialized is recommended since some HEAT data.frames store empty columns which are needed to not break the structure of processing steps.

In the future RDS format will be added to the list of available options of formats of input data.

Inputs:

  • webapp_inputs - this is a RDS file which should be downloaded from the “Export page” of the UI. It stores UI inputs with values.

  • Codebooks: To understand the meaning of the input variables (i.e. “Name” column) refer to the codebook files stored here:

    • HEAT_codebook_UI_inputvars_loops_expanded.csv (in a new window): this file includes all UI input variables available in the tool. “question_varname” corresponds to the “Name” column in webapp_inputs. The codebook file includes the “human readable” question_text, question_explanation, etc. , as well as HEAT default values (“value”), where applicable, among some other information.
    • HEAT_codebook_default_background_parameters.csv (in a new window): this file includes the parameters not visible to users in the UI. Here the column “paramter_name” corresponds to the “name” column in the webapp_input file. The “human readable” explanation is in “parameter_description”, and “value” and “unit” inform what value is used in the tool.

The values of all webapp_inputs can be modified before they are sent to the API. Please modify these with great care. Since the business logic is not a part of the model itself, improper combination of values or variables (inputs) won’t be detected, resulting in incorrect results. The general rule is that any ucc_ UI inputs should not be modified manually (since these shape the UI pages and the layout of input fields, and the corresponding collection of UI inputs with values and the data structures used by the model). Any other values of inputs can be changed - such as numeric values. Still, constraints which are available in the UI (as warnings or errors) should be respected to get reliable results from the model. Note that webapp_inputs as RDS has to be downloaded from the UI corresponding to the model version used (at the present moment only heat_5_0 is available). Please use “Inspect” tool in the web browser to get the names of the inputs. * generic_data_relevant - this is an output of prepare_gdr API “method”. Can be used as input for calc_results API “method”.

API - security

To be able to receive any data from API user cookie has to have proper value. To request the cookie send an email to: .

API - returned data clarifications

  • prepare_gdr API method returns list().
  • calc_results and results API methods return list() with nested data.frame()-s. Results in the form presented in the UI (i.e. adjusted to presentation layer) are available as results key, while results_raw stores raw results without any presentation processing. The rest of keys store intermediate results of the model - which are not a part of this document.

API - limits

There is a strict limit of 5 requests per minute per IP - this includes all requests to https://api.heatwalkingcycling.org/apiv1/ path with all “sub-paths” (“sub-directories”).

API - examples

To work with API httr package can be used.

Input and returned data as json

# read RDS exported from UI
webapp_input <- readRDS("exported.rds")

# to get data prepared by "results"
# input data as "json"
# returned data as "json"
resp <- 
  httr::POST("https://api.heatwalkingcycling.org/apiv1/heat_5_0/results?output_format=json",
                    httr::set_cookies("user" = "cookie_value"),
                    body = jsonlite::toJSON(list("webapp_input" = webapp_input)),
                    encode = "multipart")
                    
# convert from "json" to R format
from_json <- jsonlite::fromJSON(httr::content(resp, "text"))

Input and returned data as jsonl-serialized

# read RDS exported from UI
webapp_input <- readRDS("exported.rds")

# to get data prepared by "results"
# input data as "jsonl-serialized"
# returned data as "jsonl-serialized"
resp <- 
  httr::POST("https://api.heatwalkingcycling.org/apiv1/heat_5_0/results?output_format=jsonl_serialized",
                    httr::set_cookies("user" = "cookie_value"),
                    body = jsonlite::serializeJSON(list("webapp_input" = webapp_input)),
                    httr::content_type("application/jsonl-serialized"),
                    encode = "multipart")
                    
# convert from "jsonl-serialized" to R format
from_serialized_json <- jsonlite::unserializeJSON(httr::content(resp, "text"))

Input and returned data as rds, two inputs

# read RDS exported from UI
webapp_input <- readRDS("exported.rds")

# to get data prepared by "prepare_gdr"
# input data as "json"
# returned data as "rds"
resp <- 
  httr::POST("https://api.heatwalkingcycling.org/apiv1/heat_5_0/prepare_gdr?output_format=rds",
                    httr::set_cookies("user" = "cookie_value"),
                    body = jsonlite::toJSON(list("webapp_input" = webapp_input)),
                    encode = "multipart")

# convert to R
from_rds <- unserialize(httr::content(resp, "raw"))

# to get data prepared by "calc_results"
# input data as "json", two inputs - "generic_data_relevant" from previous step 
# returned data as "rds"
resp <- 
  httr::POST("https://api.heatwalkingcycling.org/apiv1/heat_5_0/calc_results?output_format=rds",
                    httr::set_cookies("user" = "cookie_value"),
                    body = jsonlite::toJSON(
                      list("webapp_input" = webapp_input,
                           "generic_data_relevant" = from_rds)),
                    encode = "multipart")

# convert to R
from_rds <- unserialize(httr::content(resp, "raw"))

Input and returned data mix of formats, two inputs

# read RDS exported from UI
webapp_input <- readRDS("exported.rds")

# to get data prepared by "prepare_gdr"
# input data as "jsonl_serialized"
# returned data as "jsonl_serialized"
resp <- 
  httr::POST("https://api.heatwalkingcycling.org/apiv1/heat_5_0/prepare_gdr?output_format=jsonl_serialized",
                    httr::set_cookies("user" = "cookie_value"),
                    body = jsonlite::serializeJSON(list("webapp_input" = webapp_input)),
                    httr::content_type("application/jsonl-serialized"),
                    encode = "multipart")

# convert to R
from_serialized_json <- jsonlite::unserializeJSON(httr::content(resp, "text"))

# to get data prepared by "calc_results"
# Input data as "jsonl_serialized", two inputs - "generic_data_relevant" from 
# previous step.
# returned data as "rds"
resp <- 
  httr::POST("https://api.heatwalkingcycling.org/apiv1/heat_5_0/calc_results?output_format=rds",
                    httr::set_cookies("user" = "cookie_value"),
                    body = jsonlite::serializeJSON(
                      list("webapp_input" = webapp_input,
                           "generic_data_relevant" = from_serialized_json)),
                    httr::content_type("application/jsonl-serialized"),
                    encode = "multipart")
# convert to R
from_rds <- unserialize(httr::content(resp, "raw"))

Disclaimer

HaaS is in an experimental stage - still it should work without any problems. Only potential issues are connected with data passing via API (JSON <-> R data mappings) or improper modifications of input values, especially from UI “Scenario” sections.

Please verify sample (random 10%?) of requests by comparing results returned by HaaS and produced in UI (for the same scenario).