NAV
shell

Introduction

Documentation for CloudKarafka API.

Details

Get instance connection details

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/details

The above command returns JSON structured like this:

{
  "name": "test-speedcar",
  "hostname": "test-speedcar.srvs.cloudkafka.com",
  "subnet": "10.128.0.0/24",
  "kafka_version": "1.1.0",
  "connections": {
    "plaintext": {
      "port": 9092
    },
    "ssl": {
      "port": 9093
    },
    "sasl_scram": {
      "port": 9094,
      "username": "aiqasdasda",
      "password": "MFOmvlRPyFKR8asdasdasdasasdas6K"
    }
  },
  "servers": [
    {
      "name": "test-speedcar-01",
      "hostname": "test-speedcar-01.srvs.cloudkafka.com",
      "private_ip": "10.128.0.15",
      "region": "amazon-web-services::us-east-1"
    }
  ]
}

The connection sasl_scram might not be available to you if you are running a Kafka cluster with Kafka version older than 0.11.0.2.

HTTP Request

GET https://api.cloudkarafka.com/api/details

Certificates

Get CA certificate

  curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/ca/ca_cert.pem > ca_cert.pem

The above command returns the ca certificate as plaintext which can be written to file

HTTP Request

GET https://api.cloudkarafka.com/api/ca/ca_cert.pem

Sign the certificate

  curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -H "Content-type: application/json" \
  -d '{
    "csr": "the certificate request"
  }' \
  https://api.cloudkarafka.com/api/ca/sign > signed_cert.pem

The above command returns the signed certificate as plaintext which can be written to file

HTTP Request

POST https://api.cloudkarafka.com/api/ca/sign

Request Parameters

Parameter Description
csr The content of the certificate signing request

Cluster

Get cluster status

Useful to check status of a rolling cluster restart.

HTTP Request

GET https://api.cloudkarafka.com/api/api/cluster/status

  curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/api/cluster/status

The above command returns JSON structured like this:

{
  "name": "test-speedcar",
  "ready": true,
  "configured": true
}

Response attributes

Parameter Description
name The cluster name
ready false when the cluster is being created
configured false when rolling cluster restart is in progress

Initiate rolling cluster restart

Triggers a rolling cluster restart, where the Kafka service is restarted on one server at a time. The cluster needs to be ready and configured for the command to proceed.

HTTP Request

POST https://api.cloudkarafka.com/api/api/cluster/restart

  curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -X POST \
  https://api.cloudkarafka.com/api/api/cluster/restart

The above command does not return a body when successful

Response

The response is 201 Created if the rolling cluster restart was initiated successfully, otherwise 400 Bad Request with an error.

Alarms

Information about both alarms, notifications and recipient. When creating a new instace, a set of default alarms and recipient will be created.

Recipient: There will always be a default recipient created, using the team email set during sign up. This recipient is used to receive notifications from the default alarms.

Alarm: There will always be four default alarms created, cpu, memory, disk and notice alarms.

Create recipient

To use another recipient than default recipient, a new one needs to be created.

Available recipients types:

HTTP Request

POST https://api.cloudkarafka.com/api/alarms/recipients

Example of email recipient

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "type=email&value=notification@example.com&name=low"
  https://api.cloudkarafka.com/api/alarms/recipients

This creates an email recipient for notification@example.com that can be used to receive alarm notifications. The returned JSON structured look like this:

{
  "id": 2,
  "type": "email",
  "value": "notification@example.com",
  "name": "Low",
  "options": {},
}

Request body parameters

Parameter Required Type  Description
type true string The type of notification to be sent
 value true string Endpoint to where the notification will be sent
name false string Optional, name for the recipient

List recipients

Retrieve all recipients for an instance that can receive notifications

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/alarms/recipients

The above command returns JSON structured like this:

[
  {
    "id": 1,
    "type": "email",
    "value": "team@84codes.com",
    "name": "Default",
    "options": null
  },
  {
    "id": 2,
    "type": "email",
    "value": "notification@example.com",
    "name": "Low",
    "options": null
  }
]

HTTP Request

GET https://api.cloudkarafka.com/api/alarms/recipients

Retrieve specific recipient

Use the id of a recipient to retrieve it

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/alarms/recipients/1

The above command returns JSON structured like this:

{
  "id": 1,
  "type": "email",
  "value": "team@example.com",
  "name": "Default",
  "options": null
}

HTTP Request

GET https://api.cloudkarafka.com/api/alarms/recipients/<id>

Update recipient

Update a specific recipient with new information

HTTP Request

PUT https://api.cloudkarafka.com/api/alarms/recipients/<id>

Example of email recipient

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "type=email&value=alarms@example.com&name=High"
  https://api.cloudkarafka.com/api/alarms/recipients/2

This updates an email recipient for notification@example.com that can be used to receive alarm notifications. The returned JSON structured look like this:

{
  "id": 2,
  "type": "email",
  "value": "alarm@example.com",
  "name": "High",
  "options": {},
}

URL Parameters

Parameter  Description
id  The recipient identifier

Request body parameters

Parameter Required Type  Description
type true string The type of notification to be sent
 value true string Endpoint to where the notification will be sent
name false string Optional, name for the recipient

Delete recipient

Use the id of a specific recipient to remove it

curl -XDELETE -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/alarms/recipients/2

HTTP Request

DELETE https://api.cloudkarafka.com/api/alarms/recipients/<id>

URL Parameters

Parameter Description
 id Use the id of the recipient that you want to delete

Create alarm

Available alarms types:

Name Type Dedicated Shared Description
CPU cpu yes - Alarm if CPU usage is above a certain level for a period of time
Memory memory yes - Alarm if memory usage is above a certain level for a period of time
Disk space disk yes - Alarm id disk usage is above a certain level for a period of time
Net split netsplit yes - Alarm if net split occurs
Server unreachable server_unreachable yes - Alarm if server is unreachable
 Notice notice yes yes Notifications on events such as planned or emergency maintenance

HTTP Request

POST https://api.cloudkarafka.com/api/alarms

Request (CPU alarm)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "type=cpu&enabled=true&value_threshold=90&time_threshold=600&recipients=[1]"
  https://api.cloudkarafka.com/api/alarms

This sets the alarm to trigger if the CPU usage is above 90% for 10 minutes or more.

{
  "type": "cpu",
  "enabled": true,
  "value_threshold": 90,
  "time_threshold": 600,
  "recipients": [1]
}
Parameter Type Description
type string The alarm type, valid types see table Available alarms types above
enabled bool Enable or disable the alarm
value_threshold int The value threshold to trigger the alarm
time_threshold int For how long (in seconds) the value_threshold should be active before trigger alarm
recipients []int Recipients identifiers for which should receive notifications when the alarm trigger

Request (Memory alarm)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "type=memory&enabled=true&value_threshold=90&time_threshold=600&recipients=[1]"
  https://api.cloudkarafka.com/api/alarms

This sets the alarm to trigger if the memory usage is above 90% for 10 minutes or more.

{
  "type": "memory",
  "enabled": true,
  "value_threshold": 90,
  "time_threshold": 600,
  "recipients": [1]
}
Parameter Type Description
type string The alarm type, valid types see table Available alarms types above
enabled bool Enable or disable the alarm
value_threshold int The value threshold to trigger the alarm
time_threshold int For how long (in seconds) the value_threshold should be active before trigger alarm
recipients []int Recipients identifiers for which should receive notifications when the alarm trigger

Request (Disk space alarm)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "type=disk&enabled=true&value_threshold=5&time_threshold=600&recipients=[1]"
  https://api.cloudkarafka.com/api/alarms

This sets the alarm to trigger if available disk space is above 5Gb for 10 minutes or more.

{
  "type": "cpu",
  "enabled": true,
  "value_threshold": 5,
  "time_threshold": 600,
  "recipients": [1]
}
Parameter Type Description
type string The alarm type, valid types see table Available alarms types above
enabled bool Enable or disable the alarm
value_threshold int The value threshold to trigger the alarm
time_threshold int For how long (in seconds) the value_threshold should be active before trigger alarm
recipients []int Recipients identifiers for which should receive notifications when the alarm trigger

Request (Notice)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "type=notice&enabled=true&recipients=[1]"
  https://api.cloudkarafka.com/api/alarms

Notifies recipients on events such as planned or emergency maintenance

{
  "type": "notice",
  "enabled": true,
  "recipients": [1]
}
Parameter Type Description
type string The alarm type, valid types see table Available alarms types above
enabled bool Enable or disable the alarm
recipients []int Recipients identifiers for which should receive notifications when the alarm trigger

Request (Server unreachable)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "type=server_unreachable&enabled=true&recipients=[1]"
  https://api.cloudkarafka.com/api/alarms

Notifies recipients if the server is unreachable

{
  "type": "server_unreachable",
  "enabled": true,
  "recipients": [1]
}
Parameter Type Description
type string The alarm type, valid types see table Available alarms types above
enabled bool Enable or disable the alarm
recipients []int Recipients identifiers for which should receive notifications when the alarm trigger

List alarms

Retrive all alarms used for the instance.

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/alarms

Example of the default created alarms:

[
    {
        "time_threshold": 600,
        "value_threshold": 90,
        "id": 7395,
        "type": "cpu",
        "recipients": [],
        "enabled": true
    },
    {
        "time_threshold": 600,
        "value_threshold": 90,
        "id": 7396,
        "type": "memory",
        "recipients": [],
        "enabled": true
    },
    {
        "time_threshold": 600,
        "value_threshold": 5,
        "id": 7397,
        "type": "disk",
        "recipients": [],
        "enabled": true
    },
    {
        "id": 7398,
        "type": "notice",
        "recipients": [],
        "enabled": true
    }
]

HTTP Request

GET https://api.cloudkarafka.com/api/alarms

Retrieve specific alarm

Use the id of a specific alarm to retrieve it.

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/alarms/7395

The above command returns JSON structured like this:

{
  "time_threshold": 600,
  "value_threshold": 90,
  "id": 7395,
  "type": "cpu",
  "recipients": [1],
  "enabled": true
}

HTTP Request

GET https://api.cloudkarafka.com/api/alarms/<id>

Update alarm

Use the id of a specific alarm to update it

HTTP Request

PUT https://api.cloudkarafka.com/api/alarms/<id>

Request Parameters (CPU alarm)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "type=cpu&enabled=true&value_threshold=90&time_threshold=600&recipients=[1]"
  https://api.cloudkarafka.com/api/alarms

This sets the alarm to trigger if the CPU usage is above 90% for 10 minutes or more.

Parameter Type Description
type string The alarm type, valid types see table Available alarms types above
enabled bool Enable or disable the alarm
value_threshold int The value threshold to trigger the alarm
time_threshold int For how long (in seconds) the value_threshold should be active before trigger alarm
recipients []int Recipients identifiers for which should receive notifications when the alarm trigger

For more examples, see under the section about creating alarm.

Delete alarm

Use the id for a specific alarm to remove it.

curl -XDELETE -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/alarms/7395

HTTP Request

DELETE https://api.cloudkarafka.com/api/alarms/<id>

URL Parameters

Parameter Description
id Use the id of the alarm that you want to delete

Integrations

List log integrations

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/integrations/logs

The above command returns JSON structured like this:

[
  {
    "id":10,
    "type":"papertrail",
    "config": {
      "url":"logs.papertrail.com:2123"
    },
    "error": "",
    "account_id": "99ed6d99-fjsd-7d7d-8ujd-2f53asf34weaws"
  }
]

HTTP Request

GET https://api.cloudkarafka.com/api/integrations/logs

Add log integration

Available log integrations are:

HTTP Request

POST https://api.cloudkarafka.com/api/integrations/logs/<SYSTEM>

URL Parameters

Parameter Description
SYSTEM The logs system to integrate to, any of these: papertrail, loggly, logentries or splunk

Request Parameters (cloudwatchlog)

Need to create an IAM user with programmatic permissions:

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "region=us-west-1&access_key_id=87656789&secret_access_key=09876tyui9876yui98" \
  https://api.cloudkarafka.com/api/integrations/logs/cloudwatchlog
Parameter Description
region The AWS region to use
access_key_id The access key id to use
secret_access_key The secret that goes with the key id

Request Parameters (papertrail)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "url=logs.papertrail.com:2123" \
  https://api.cloudkarafka.com/api/integrations/logs/papertrail
Parameter Description
url The URL to push the logs to

Request Parameters (loggly)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "token=XXXXXXXXX" \
  https://api.cloudkarafka.com/api/integrations/logs/loggly
Parameter Description
token The token used for authentication

Request Parameters (logentries)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "token=XXXXXXXXXXXXX" \
  https://api.cloudkarafka.com/api/integrations/logs/logentries
Parameter Description
token A TCP token for authentication

Request Parameters (splunk)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "token=XXXXX&host_port=innput-prd-p-rdq96mdbptj4.cloud.splunk.com:8080" \
  https://api.cloudkarafka.com/api/integrations/logs/splunk
Parameter Description
token The token used for authentication
host_port Destination to send the logs

Delete log integration

curl -XDELETE -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/integrations/logs/636

HTTP Request

DELETE https://api.cloudkarafka.com/api/integrations/logs/<INT_ID>

URL Parameters

Parameter Description
INT_ID The ID of the integration to delete

List metrics integrations

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/integrations/metrics

The above command returns JSON structured like this:

[
  {
    "id":10,
    "type":"datadog",
    "config": {
      "api_key":"THE_API_KEY"
    },
    "error": "",
    "account_id": "99ed6d99-fjsd-7d7d-8ujd-2f53asf34weaws"
  }
]

HTTP Request

GET https://api.cloudkarafka.com/api/integrations/metrics

Add metrics integration

Available metrics integrations are:

HTTP Request

POST https://api.cloudkarafka.com/api/integrations/metrics/<SYSTEM>

URL Parameters

Parameter Description
SYSTEM The logs system to integrate to

Request Parameters (CloudWatch)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "region=us-west-1&access_key_id=87656789&secret_access_key=09876tyui9876yui98" \
  https://api.cloudkarafka.com/api/integrations/metrics/cloudwatch
Parameter Description
region The AWS region to use
access_key_id The access key id to use (Needs to have the PutMetricData permission)
secret_access_key The secret that goes with the key id

Request Parameters (Librato)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "email=my@email.com&api_key=9876thjio90876tyu" \
  https://api.cloudkarafka.com/api/integrations/metrics/librato
Parameter Description
email The email registered at librato
api_key The API key

Request Parameters (DataDog)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "api_key=XXXXXXXXXXXXX" \
  https://api.cloudkarafka.com/api/integrations/metrics/datadog
Parameter Description
api_key The API key

Request Parameters (NewRelic)

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "license_key=XXXXXXXXXXXXXXXXX" \
  https://api.cloudkarafka.com/api/integrations/metrics/newrelic
Parameter Description
license_key The license key for NewRelic

Delete metric integration

curl -XDELETE -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/integrations/metrics/636

HTTP Request

DELETE https://api.cloudkarafka.com/api/integrations/metrics/<INT_ID>

URL Parameters

Parameter Description
INT_ID The ID of the integration to delete

VPC

List peering requests

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/vpc-peering

The above command returns JSON structured like this:

{
  "rows": [
    {
      "tag_set": [],
      "vpc_peering_connection_id": "pcx-98yi98u",
      "requester_vpc_info": {
        "owner_id": "76567890987",
        "vpc_id": "vpc-876thui8",
        "cidr_block": "10.128.0.0/24"
      },
      "accepter_vpc_info": {
        "owner_id": "987687678",
        "vpc_id": "vpc-i876thuio",
        "cidr_block": "10.0.0.0/16"
      },
      "status": {
        "code": "active",
        "message": "Active"
      },
      "routes": [
        {
          "destination": "10.0.0.0/16",
          "state": "active"
        }
      ]
    }
  ],
  "vpc": {
    "id": "vpc-876thui8",
    "name": "test-speedcar",
    "subnet": "10.128.0.0/24"
  }
}

HTTP Request

GET https://api.cloudkarafka.com/api/vpc-peering

Create peering request

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  -d "peer_account_id=1234567890123456&peer_vpc_id=vpc-0123456789123&peer_subnet=10.20.30.0%2F24" 
  https://api.cloudkarafka.com/api/vpc-peering

This endpoint creates a peering request from the VPC hosting your cluster to the VPC you specify in the request.

HTTP Request

POST https://api.cloudkarafka.com/api/vpc-peering

Request Parameters (AWS)

Parameter Description
peer_account_id Your account ID at the cloud provider
peer_vpc_id Your VPC ID
peer_subnet Peer subnet, whole or part of your VPC subnet

Request Parameters (GCP)

Parameter Description
peer_network_uri Your GCE network uri
peer_subnet Your VPC subnet

Delete peering request

curl -XDELETE -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx \
  https://api.cloudkarafka.com/api/vpc-peering/78

HTTP Request

DELETE https://api.cloudkarafka.com/api/vpc-peering/<PCX_ID>

URL Parameters

Parameter Description
PCX_ID The ID of the peering connection

Authentication

Authentication is done by sending your API key in the user field for Basic Auth

curl -u :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx \
  https://api.cloudkarafka.com/api/api/details

You can find your API here on the details page for your instance.

Formats

All API end points support form FormData and JSON in the request. You need to format the request accordingly and if you send the request as JSON be sure to add the content type header Content-type: application/json otherwise the server won't be able to parse your request.

Status Codes

Error Code Meaning
400 Bad Request -- Your request is invalid.
401 Unauthorized -- Your API key is wrong.
403 Forbidden -- The kitten requested is hidden for administrators only.
404 Not Found -- The specified kitten could not be found.
405 Method Not Allowed -- You tried to access a kitten with an invalid method.
429 Too Many Requests -- Rate limiting, you have requested too many kittens in a given amount of time.
500 Internal Server Error -- We had a problem with our server. Try again later.