Custom Integration for DSR

The custom integration allows you to extend the DSR handling workflow with your own code. This is especially useful for applying custom business logic, handling internal home-brew systems or preventing external connections to internal databases. This is extremely easy to implement with your own API endpoint (such as a cloud function) or even no-code automation tools, such as Zapier or Integromat.

The custom integration works by allowing you to register API endpoints that will get called as part of the DSR workflow. Just like Mine's built-in integrations, custom integration supports any request type (Delete, Copy...), allows previewing user data, and supports advanced operations such as handling failures and performing long-running async operations.

The custom integration supports the following operations. Each operation requires an endpoint URL in order to be used:

Operation	Description
Delete	An endpoint designed to handle any request that is mapped to the delete workflow. It supports both async and sync flows.
Copy	An endpoint designed to handle any request that is mapped to the copy workflow. It supports both async and sync flows.
Preview	This is an optional endpoint allowing you to handle record previews when handling tickets through the UI. This is handled in a synchronous way.

Note: while Preview is optional, at least one of the copy or delete endpoints has to be defined.

Adding Custom Integrations

Custom integration is available for custom systems in your data inventory. To more than one custom integration simply add multiple custom systems.

When multiple custom integrations are used, the integrationId field is used for updating the status of a specific integration.

In the following sections we will describe each operation and how it works.

Delete

When a delete workflow is executed, MineOS makes an API call to the URL set in the Delete URL field. The HTTP request is of the following form:

POST https://example.com/endpoint HTTP/1.1
X-Mine-Signature: 4r4ClKnSeGWvTGQzMCUcEEK9uF528mvOlN/jtKMQ==
Content-type: application/json

{
  "EventId": "123456789abcdefghijklm",
  "EventType": "WebhookSlim",
  "isTest": "false",
  "ticketInfo": ...
}

The receiving endpoint is expected to return one of the following responses:

200 OK - Indicates the deletion process has been completed successfully and will be marked as 'completed' (synchronous flow).
202 Accepted - Indicates the deletion request has started processing will be marked as 'in progress' (async flow). Once finished, you should use the Update custom integration status to update the integration status to Completed/Failed/etc.

Failing to respond with 200/202, or returning any other non 2xx status, will cause MineOS to keep retrying the request.

Handling Failure

When a delete operation isn't successful you should update the integration status to Failed by using the Update custom integration status endpoint.

Copy

When a copy workflow is executed, MineOS makes an API call to the URL set in the Copy URL field. The HTTP request is of the following form:

POST https://example.com/endpoint HTTP/1.1
X-Mine-Signature: 4r4ClKnSeGWvTGQzMCUcEEK9uF528mvOlN/jtKMQ==
Content-type: application/json

{
  "traceId": "123456789abcdefghijklm",
  "integrationId": "0c36xnykgewwbzfukh1jkq",
  "isTest": true
  ...
}

The receiving endpoint is expected to return one of the following responses:

200 OK - Indicates the data collection process has been completed successfully and will be marked as 'completed' (synchronous flow). The response contains any JSON data that was collected.
202 Accepted - Indicates the data collection request has started and will be marked as 'in progress' (async flow). Once finished, you should use the Update custom integration status endpoint to update the integration status to Completed/Failed/etc and provide the data that was collected.

Failing to respond with 200/202, or returning any other non 2xx status, will cause MineOS to keep retrying the request.

Providing Collected Data - Sync flow

The receiving endpoint is expected to return 200 OK with a JSON response body including any user data that should be collected as part of the copy workflow. The response JSON can be of any format, it has no schema. Any information returned by this endpoint is collected by the system and sent to the requesting user.

Providing Collected Data - Async Flow

Collected data is sent to MineOS when calling the Update custom integration status endpoint, in one of two ways:

JSON data
File URL

Preview and User Search

When a any workflow is about to get executed, MineOS makes an API call to the URL set in the Preview URL field. The HTTP request is of the following form:

POST https://example.com/endpoint HTTP/1.1
X-Mine-Signature: 4r4ClKnSeGWvTGQzMCUcEEK9uF528mvOlN/jtKMQ==
Content-type: application/json

{
    "traceId": "123456789abcdefghijklm",
    "integrationId": "0c36xnykgewwbzfukh1jkq",
    "isTest": true
    ...
}

The receiving endpoint is expected to return 200 OK with a JSON response body that includes a summary of user data records found. This data will not be shared with the requesting user, and is used as internal information to help you choose how to handle the request.

The response JSON must satisfy match the following structure:

{
  "records": [
    {
      "name": "firstRecord",
      "properties": [
        {
          "name": "fieldName",
          "value": "fieldValue"
        },
        {
          "name": "anotherName",
          "value": "anotherValue"
        }
      ]
    },
    ...
  ]
}

📘
Notes

You can returns as many records as you want

You can only return up to 3 properties for each record.

All property types should be string.

The request object does not always exist in a preview request.

Request Payload

Every time MineOS makes an HTTP call to your endpoint, the following JSON payload is sent as the request body:

{
    // usefull for logging
    "traceId": "123456789abcdefghijklm",
    // used for the CustomIntegrationStatus endpoint
    "integrationId": "0c36xnykgewwbzfukh1jkq",
    // indicates this is just a test event, actual processing should be skipped
    "isTest": true,
    // the request object is not always available:
    // its available when handling a DSR but not handling when handling a user search
    "request": {
        // used for the CustomIntegrationStatus endpoint
        "id": "ABCDEFGHIJKLMNOPQRSTUV",
        // type of request. Possible values: Delete,GetCopy,DoNotSell,Undetermined,RightToEdit,DoNotMail
        "type": "Delete",
        // the channel from which the request was sent. Possible values: Form,MineApp,Api,EmailForwarding,Manual
        "source": "Form",
        // the main domain of the workspace to which the request was sent
        "domain": "test.com",
        "createdAt": "2023-07-28T11:35:21.7528109Z"
    },
    "userInfo": {
        "name": "Test User",
        "email": "[email protected]",
        "isVerified": false,
        "countryOfResidence": "TestCountry",
        // fields set through UpdateMetadata endpoint
        "customFields": {
            "testCustomKey": "Custom value"
        }
    }
}

Logging And Storing Information

At any time while processing a request, you might want to store information on the request:

Logging/documenting important business information, for example: the type of customer, the underlying system etc. This can be done with Add a note API. In the UI this information is visible in the notes section on the right side.
Storing metadata/state information, for example: internal user Id, URI of related resources or other metadata about the handling flow. This can be done with Update metadata API. This information is not visible in the UI but can be retrieved with Get metadata API.

Security

When receiving requests to your endpoint, its best to perform a security check before processing the event. There are a number of supported security mechanisms, you can use any one of them, or a combination of more than one:

API Key - the requests can include a header with a name and value of your choice. By setting a secret value as a custom header, you can verify in your code that the header contains the value your would expect, similar to using an API Key. This is the best and easiest option.
IP Whitelisting - we use static IPs that you can whitelist in your firewall / ACL. See the list of IPs below.
Verifying signature - this is another option instead of the API Key. Each request MineOS makes to your endpoint contains a HTTP header called: x-mine-signature. This header contains the event signature, which can be verified to make sure the request originated from your MineOS account. See example code for verifying the signature:

var key = "YOUR VERIFICATION KEY";
var data = "JSON PAYLOAD OF THE EVENT";

var hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
var rawSignature = hmac.ComputeHash(Encoding.UTF8.GetBytes(data));
var signature = BitConverter.ToString(rawSignature).Replace("-", "").ToLower();

var key = "YOUR VERIFICATION KEY";
var data = "JSON PAYLOAD OF THE EVENT";

var crypto = require('crypto');
var signature = crypto.createHmac('SHA256', key).update(data).digest('hex');

To test weather your implementation is correct, you can use an online HMAC calculator, like this one.

MineOS IPs:

34.77.7.236
35.187.97.141
34.76.247.90
34.140.71.114