Dataframes endpoint

POST
/v2/dataframes

Add dataframes into the storage backend

Add dataframes into the storage backend.

Name

In

Type

Description

dataframes

body

list

List of dataframes to add.

Request Example

In the body:

{
    "dataframes": [
        {
            "period": {
                "begin": "20190723T122810Z",
                "end": "20190723T132810Z"
            },
            "usage": {
                "metric_one": [
                    {
                        "vol": {
                            "unit": "GiB",
                            "qty": 1.2
                        },
                        "rating": {
                            "price": 0.04
                        },
                        "groupby": {
                            "group_one": "one",
                            "group_two": "two"
                        },
                        "metadata": {
                            "attr_one": "one",
                            "attr_two": "two"
                        }
                    }
                ],
                "metric_two": [
                    {
                        "vol": {
                            "unit": "MB",
                            "qty": 200.4
                        },
                        "rating": {
                            "price": 0.06
                        },
                        "groupby": {
                            "group_one": "one",
                            "group_two": "two"
                        },
                        "metadata": {
                            "attr_one": "one",
                            "attr_two": "two"
                        }
                    }
                ]
            }
        },
        {
            "period": {
                "begin": "20190823T122810Z",
                "end": "20190823T132810Z"
            },
            "usage": {
                "metric_one": [
                    {
                        "vol": {
                            "unit": "GiB",
                            "qty": 2.4
                        },
                        "rating": {
                            "price": 0.08
                        },
                        "groupby": {
                            "group_one": "one",
                            "group_two": "two"
                        },
                        "metadata": {
                            "attr_one": "one",
                            "attr_two": "two"
                        }
                    }
                ],
                "metric_two": [
                    {
                        "vol": {
                            "unit": "MB",
                            "qty": 400.8
                        },
                        "rating": {
                            "price": 0.12
                        },
                        "groupby": {
                            "group_one": "one",
                            "group_two": "two"
                        },
                        "metadata": {
                            "attr_one": "one",
                            "attr_two": "two"
                        }
                    }
                ]
            }
        }
    ]
}

Status codes

Success

Code

Reason
204 - No Content

Request was successful even though no content is to be returned.

Error

Code

Reason
400 - Bad Request

Invalid request.
401 - Unauthorized

Unauthenticated user.
403 - Forbidden

Forbidden operation for the authentified user.
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

No content is to be returned.

GET
/v2/dataframes

Get dataframes from the storage backend

Get dataframes from the storage backend.

Name

In

Type

Description

limit (Optional)

query

int

For pagination. The maximum number of results to return.

offset (Optional)

query

int

For pagination. The index of the first element that should be returned.

begin (Optional)

query

iso8601 timestamp

Begin of the period for which the dataframes are required.

end (Optional)

query

iso8601 timestamp

End of the period for which the dataframes are required.

filters (Optional)

query

dict

Optional filters.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
401 - Unauthorized

Unauthenticated user.
403 - Forbidden

Forbidden operation for the authentified user.
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

Name

In

Type

Description

total

body

int

Total of datapoints matching the query parameters.

dataframes

body

list

List of dataframes matching the query parameters.

Response Example

{
    "total": 3,
    "dataframes": [
        {
            "usage": {
                "metric_one": [
                    {
                        "vol": {
                            "unit": "GiB",
                            "qty": 1.2
                        },
                        "rating": {
                            "price": 0.04
                        },
                        "groupby": {
                            "group_one": "one",
                            "group_two": "two"
                        },
                        "metadata": {
                            "attr_one": "one",
                            "attr_two": "two"
                        }
                    }
                ],
                "metric_two": [
                    {
                        "vol": {
                            "unit": "GiB",
                            "qty": 1.2
                        },
                        "rating": {
                            "price": 0.04
                        },
                        "groupby": {
                            "group_one": "one",
                            "group_two": "two"
                        },
                        "metadata": {
                            "attr_one": "one",
                            "attr_two": "two"
                        }
                    }
                ]
            },
            "period": {
                "begin": "2019-07-23T12:28:10+00:00",
                "end": "2019-07-23T13:28:10+00:00"
            }
        },
        {
            "usage": {
                "volume.size": [
                    {
                        "vol": {
                            "unit": "GiB",
                            "qty": 1.9
                        },
                        "rating": {
                            "price": 3.8
                        },
                        "groupby": {
                            "project_id": "8ace6f139a1742548e09f1e446bc9737",
                            "user_id": "b28fd3f448c34c17bf70e32886900eed",
                            "id": "be966c6d-78a0-42cf-bab9-e833ed996dee"
                        },
                        "metadata": {
                            "volume_type": ""
                        }
                    }
                ]
            },
            "period": {
                "begin": "2019-08-01T01:00:00+00:00",
                "end": "2019-08-01T02:00:00+00:00"
            }
        }
    ]
}

Scope state endpoint

GET
/v2/scope

Get the status of several scopes

Returns the status of several scopes.

Name

In

Type

Description

collector (Optional)

query

string

Filter on collector.

fetcher (Optional)

query

string

Filter on fetcher.

limit (Optional)

query

int

For pagination. The maximum number of results to return.

offset (Optional)

query

int

For pagination. The index of the first element that should be returned.

scope_id (Optional)

query

string

Filter on scope.

scope_key (Optional)

query

string

Filter on scope_key.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
403 - Forbidden

Forbidden operation for the authentified user.
404 - Not Found

Not found
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

Name

In

Type

Description

collector

body

string

Collector for the given scope

fetcher

body

string

Fetcher for the given scope

state

body

iso8601 timestamp

State of the scope. This variable represents the last processed timestamp for the storage state element. It is DEPRECATED, and it will be removed in upcoming releases. The alternative is last_processed_timestamp.

last_processed_timestamp

body

iso8601 timestamp

It represents the last processed timestamp for the storage state element.

scope_id

body

string

Scope

scope_key

body

string

Scope key for the given scope

active

body

bool

Defines if a scope should be processed or not; True means that CloudKitty must process the scope.

Response Example

{
    "results": [
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "7a7e5183264644a7a79530eb56e59941",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-09 10:00:00",
            "active": true
        },
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "9084fadcbd46481788e0ad7405dcbf12",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-08 03:00:00",
            "active": true
        },
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "1f41d183fca5490ebda5c63fbaca026a",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-06 22:00:00",
            "active": true
        }
    ]
}
PUT
/v2/scope

Reset the status of several scopes

Reset the status of several scopes.

Name

In

Type

Description

state

body

iso8601 timestamp

State of the scope. This variable represents the last processed timestamp for the storage state element. It is DEPRECATED, and it will be removed in upcoming releases. The alternative is last_processed_timestamp.

last_processed_timestamp

body

iso8601 timestamp

It represents the last processed timestamp for the storage state element.

collector (Optional)

body

string

Filter on collector.

fetcher (Optional)

body

string

Filter on fetcher.

scope_id (Optional)

body

string

Filter on scope.

scope_key (Optional)

body

string

Filter on scope_key.

all_scopes (Optional)

body

bool

Confirmation whether all scopes must be reset

Status codes

Success

Code

Reason
202 - Accepted

Request has been accepted for asynchronous processing.

Error

Code

Reason
400 - Bad Request

Invalid request.
403 - Forbidden

Forbidden operation for the authentified user.
404 - Not Found

Not found
405 - Method Not Allowed

The method is not allowed for the requested URL.
PATCH
/v2/scope

Patch a scope

Patches/updates a scope.

Name

In

Type

Description

collector (Optional)

query

string

Filter on collector.

fetcher (Optional)

query

string

Filter on fetcher.

limit (Optional)

query

int

For pagination. The maximum number of results to return.

offset (Optional)

query

int

For pagination. The index of the first element that should be returned.

scope_id (Optional)

query

string

Filter on scope.

scope_key (Optional)

query

string

Filter on scope_key.

active (Optional)

body

bool

Defines if a scope should be processed or not; True means that CloudKitty must process the scope.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
403 - Forbidden

Forbidden operation for the authentified user.
404 - Not Found

Not found
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

Name

In

Type

Description

collector

body

string

Collector for the given scope

fetcher

body

string

Fetcher for the given scope

state

body

iso8601 timestamp

State of the scope. This variable represents the last processed timestamp for the storage state element. It is DEPRECATED, and it will be removed in upcoming releases. The alternative is last_processed_timestamp.

scope_id

body

string

Scope

scope_key

body

string

Scope key for the given scope

active

body

bool

Defines if a scope should be processed or not; True means that CloudKitty must process the scope.

Response Example

{
    "results": [
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "7a7e5183264644a7a79530eb56e59941",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-09 10:00:00",
            "active": true
        },
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "9084fadcbd46481788e0ad7405dcbf12",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-08 03:00:00",
            "active": true
        },
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "1f41d183fca5490ebda5c63fbaca026a",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-06 22:00:00",
            "active": true
        }
    ]
}
POST
/v2/scope

Create a scope

Create a scope.

Name

In

Type

Description

collector (Optional)

query

string

Filter on collector.

fetcher (Optional)

query

string

Filter on fetcher.

scope_id (Optional)

query

string

Filter on scope.

scope_key (Optional)

query

string

Filter on scope_key.

active (Optional)

body

bool

Defines if a scope should be processed or not; True means that CloudKitty must process the scope.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
403 - Forbidden

Forbidden operation for the authentified user.
404 - Not Found

Not found
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

Name

In

Type

Description

scope_id

body

string

Scope

scope_key

body

string

Scope key for the given scope

fetcher

body

string

Fetcher for the given scope

collector

body

string

Collector for the given scope

state

body

iso8601 timestamp

State of the scope. This variable represents the last processed timestamp for the storage state element. It is DEPRECATED, and it will be removed in upcoming releases. The alternative is last_processed_timestamp.

last_processed_timestamp

body

iso8601 timestamp

It represents the last processed timestamp for the storage state element.

active

body

bool

Defines if a scope should be processed or not; True means that CloudKitty must process the scope.

scope_activation_toggle_date

body

iso8601 timestamp

It represents the last time the scope was activated/deactivated via the PATCH API.

Response Example

{
    "results": [
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "7a7e5183264644a7a79530eb56e59941",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-09 10:00:00",
            "active": true
        },
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "9084fadcbd46481788e0ad7405dcbf12",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-08 03:00:00",
            "active": true
        },
        {
            "collector": "gnocchi",
            "fetcher": "keystone",
            "scope_id": "1f41d183fca5490ebda5c63fbaca026a",
            "scope_key": "project_id",
            "last_processed_timestamp": "2019-05-06 22:00:00",
            "active": true
        }
    ]
}

Summary endpoint

GET
/v2/summary

Get a rating summary

Get a rating summary for one or several tenants.

Name

In

Type

Description

limit (Optional)

query

int

For pagination. The maximum number of results to return.

offset (Optional)

query

int

For pagination. The index of the first element that should be returned.

begin (Optional)

query

iso8601 timestamp

Begin of the period for which the summary is required.

end (Optional)

query

iso8601 timestamp

End of the period for which the summary is required.

groupby (Optional)

query

list of strings

Optional attributes to group the summary by.

filters (Optional)

query

dict

Optional filters. These filters accept multiple query parameters. To use this option with multiple parameters, repeat it as many time as desired in the query string. For instance, to restrict the result to only two projects, add to the query string filters=project_id:<project_id_one>&filters=project_id:<project_id_two>. Bear in mind that this string must be URL escaped. Therefore, it becomes filters=project_id%3A<project_id_one>&filters=project_id%3A<project_id_two>.

custom_fields (Optional)

query

list of strings

Optional attributes to customize the summary GET API response. When using this parameter, users can create custom reports. The default behavior is to list the sum of the quantity and the sum of the price, which is projected as rate field. The default value for the custom_fields parameter is SUM(qty) AS qty, SUM(price) AS rate. One can customize this field as they wish with InfluxDB queries. The following statements "select", "from", "drop", "delete", "create", "alter", "insert", "update" are not allowed though. For instance, if one wants to retrieve the quantity field as the last value of the quantity, and not the sum (this is quite interesting when generating reports for storage values), the user can send the parameter as last(qty) AS qty, SUM(price) AS rate. To discover all possible fields that one can work with, the user can also use * as a parameter.

Currently this feature only works for Influx storage backend. It (the feature) depends on the storage backend driver to support it. If the user tries to set this configuration while using other storage backends, it will be ignored.

response_format (Optional)

query

string

Optional attribute to define the object structure used in the response. Both responses will be JSON objects. Possible values are table or object.

The default value is table object structure, where one has the attributes total, which indicates the total number of entries in the response; results, which is a list of lists, where the nested list contains the values of each entry; and, columns, which is the attribute that describes all of the available columns. Then, each index in this list (columns) corresponds to the metadata of the values in the results list.

The structure for the object option uses a dictionary. The response still has the total attribute. However, in the results attribute, one will find a list of objects, instead of a list of lists of values that we see in the table option. This facilitates the processing of some use cases.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
403 - Forbidden

Forbidden operation for the authentified user.
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

The response has the following default format (response_format=’table’):

{
    "columns": [
        "begin",
        "end",
        "qty",
        "rate",
        "group1",
        "group2",
    ],
    "results": [
        [
            "2019-06-01T00:00:00Z",
            "2019-07-01T00:00:00Z",
            2590.421676635742,
            1295.210838317871,
            "group1",
            "group2",
        ]
    ],
    "total": 4
}

total is the total amount of found elements. columns contains the name of the columns for each element of results. The columns are the four mandatory ones (begin, end, qty, rate) along with each attribute the result is grouped by.

format is the response format. It can be “table” or “object”. The default response structure is “table”, which is presented above. The object structure uses the following pattern.

{
    "results": [
        {"begin": "2019-06-01T00:00:00Z",
         "end": "2019-07-01T00:00:00Z",
         "qty": 2590.421676635742,
         "rate": 1295.210838317871,
         "group1": "group1",
         "group2": "group2",
        },
    ],
    "total": 4
}

Note

It is also possible to group data by time, in order to obtain timeseries. In order to do this, group by time. No extra column will be added, but you’ll get one entry per collect period in the queried timeframe. See examples below.

Name

In

Type

Description

begin

body

iso8601 timestamp

Begin of the period for the item.

end

body

iso8601 timestamp

End of the period for the item.

qty

body

float

Qty for the item in the specified period.

rate

body

float

Rate for the item in the specified period.

Response Example

Grouping by time and project_id:

curl "http://cloudkitty-api:8889/v2/summary?groupby=time&groupby=project_id&limit=3"

{
  "total": 232,
  "columns": [
    "begin",
    "end",
    "qty",
    "rate",
    "project_id"
  ],
  "results": [
    [
      "2019-10-01T06:00:00+02:00",
      "2019-10-01T07:00:00+02:00",
      3.5533905029296875,
      1.7766952514648438,
      "84631866b2d84db49b29828052bdc287"
    ],
    [
      "2019-10-01T07:00:00+02:00",
      "2019-10-01T08:00:00+02:00",
      3.5533905029296875,
      1.7766952514648438,
      "84631866b2d84db49b29828052bdc287"
    ],
    [
      "2019-10-01T08:00:00+02:00",
      "2019-10-01T09:00:00+02:00",
      3.5533905029296875,
      1.7766952514648438,
      "84631866b2d84db49b29828052bdc287"
    ]
  ]
}

curl "http://cloudkitty-api:8889/v2/summary?filters=project_id%3Afe9c35372db6420089883805b37a34af&groupby=type&groupby=project_id"

{
    "columns": [
        "begin",
        "end",
        "qty",
        "rate",
        "project_id",
        "type"
    ],
    "results": [
        [
            "2019-06-01T00:00:00Z",
            "2019-07-01T00:00:00Z",
            2590.421676635742,
            1295.210838317871,
            "fe9c35372db6420089883805b37a34af",
            "image.size"
        ],
        [
            "2019-06-01T00:00:00Z",
            "2019-07-01T00:00:00Z",
            1354,
            3625,
            "fe9c35372db6420089883805b37a34af",
            "instance"
        ],
        [
            "2019-06-01T00:00:00Z",
            "2019-07-01T00:00:00Z",
            502,
            502,
            "fe9c35372db6420089883805b37a34af",
            "ip.floating"
        ],
        [
            "2019-06-01T00:00:00Z",
            "2019-07-01T00:00:00Z",
            175.9,
            351.8,
            "fe9c35372db6420089883805b37a34af",
            "volume.size"
        ]
    ],
    "total": 4
}

Task schedule endpoint

CloudKitty has a task endpoint /v2/task/<type_of_task>, which allows operators to schedule administrative tasks, such as reprocessing.

Currently, the only task available is the reprocessing one, which is avaiable via the following endpoints.

  • POST /v2/task/reprocesses – to create a reprocessing task.

  • GET /v2/task/reprocesses/<path_scope_id> – to retrieve a reprocessing task.

  • GET /v2/task/reprocesses – to retrieve all reprocessing task.

POST
`/v2/task/reprocesses`

Create a reprocessing task

The endpoint used to schedule a reprocessing task. The scheduled tasks are loaded to execution once every processing cycle, as defined in the CloudKitty period configuration.

Name

In

Type

Description

scope_ids

body

string

The scope IDs to reprocess. Must be comma-separated to schedule more than one.

start_reprocess_time

body

iso8601 timestamp

The start date for the reprocessing task.

end_reprocess_time

body

iso8601 timestamp

The end date for the reprocessing task.

reason

body

string

The reason for the reprocessing to take place.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
403 - Forbidden

Forbidden operation for the authentified user.
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

We will return an empty object as the response in case of success:

{}

Example

curl -s -X POST "https://<cloudkitty_server_and_port_here>/v2/task/reprocess" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -H "X-Auth-Token: ${ACCESS_TOKEN_KEYSTONE}" -H "Content-Type: application/json" -d '{"reason": "Reprocessing test", "scope_ids": "<Some scope ID>", "start_reprocess_time": "2021-06-01 00:00:00+00:00", "end_reprocess_time": "2021-06-01 23:00:00+00:00"}'

The scope IDs can be retrieved via “/v2/scope” API, which is the API that one can use to list all scopes, and their status.

GET
`/v2/task/reprocesses/`

Retrieve a reprocessing task

The endpoint used to retrieve a reprocessing task. By using this endpoint, one can for instance check the progress of the reprocessing tasks.

Name

In

Type

Description

path_scope_id

path

string

The scope ID to retrieve.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
403 - Forbidden

Forbidden operation for the authentified user.
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

We will return the scope data in case of a valid scope ID:

{"scope_id": "scope ID goes here",
 "reason": "The reason for this reprocessing for this scope",
 "start_reprocess_time": "2021-06-01 00:00:00+00:00",
 "end_reprocess_time": "2021-07-01 00:00:00+00:00",
 "current_reprocess_time": "2021-06-06 00:00:00+00:00"}

Example

curl -s -X GET "https://<cloudkitty_server_and_port_here>/v2/task/reprocesses/<scope ID goes here>" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -H "X-Auth-Token: ${ACCESS_TOKEN_KEYSTONE}"
GET
`/v2/task/reprocesses`

Retrieve all reprocessing tasks

The endpoint used to retrieve all reprocessing tasks. By using this endpoint, one can retrieve all reprocessing tasks scheduled for a scope.

Name

In

Type

Description

scope_ids (Optional)

query

string

The scope IDs one wants to retrieve the reprocessing tasks of. If not informed, all reprocessing tasks, for all scopes are retrieved.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
403 - Forbidden

Forbidden operation for the authentified user.
405 - Method Not Allowed

The method is not allowed for the requested URL.

Response

We will return the scope data in case of a valid scope ID:

[{"scope_id": "scope ID goes here",
 "reason": "The reason for this reprocessing for this scope",
 "start_reprocess_time": "2021-06-01 00:00:00+00:00",
 "end_reprocess_time": "2021-07-01 00:00:00+00:00",
 "current_reprocess_time": "2021-06-06 00:00:00+00:00"}]

Example

curl -s -X GET "https://<cloudkitty_server_and_port_here>/v2/task/reprocesses" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -H "X-Auth-Token: ${ACCESS_TOKEN_KEYSTONE}"

Rating modules endpoint

GET
/v2/rating/modules

Get the list of modules

Returns the list of all rating modules loaded. This method does not require any parameter.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
401 - Unauthorized

Unauthenticated user.
403 - Forbidden

Forbidden operation for the authentified user.

Response

Name

In

Type

Description

modules

body

list

List of modules.

Response Example

 {
   "modules": [
     {
         "module_id": "noop"
         "description": "Dummy test module.",
         "enabled": true,
         "hot-config": false,
         "priority": 1
     },
     {
         "module_id": "hashmap",
         "description": "HashMap rating module.",
         "enabled": false,
         "hot-config": true,
         "priority": 2
     }
   ]
 }
GET
/v2/rating/modules/

Get one module

Returns the details of one specific module. This method does not require any parameter.

Status codes

Success

Code

Reason
200 - OK

Request was successful.

Error

Code

Reason
400 - Bad Request

Invalid request.
401 - Unauthorized

Unauthenticated user.
403 - Forbidden

Forbidden operation for the authentified user.
404 - Not Found

Not found

Response

Name

In

Type

Description

module_id

body

string

The id of the module

description

body

string

A quick description of the module

enabled

body

bool

Boolean representing if the module is enabled

hot_config

body

bool

Boolean representing if the module supports hot-config

priority

body

int

Priority of the module, relative to other modules

Response Example

{
  "module_id": "sample_id",
  "description": "Sample extension",
  "enabled": true,
  "hot-config": false,
  "priority": 2
}