Source files

The Relay service includes functionality to publish source files used during the data extraction portion of a poll. For example, a poll for ios_messages.messages will look at the sms.db file from an iOS backup to retrieve message data – using source file publishing the sms.db file itself will be returned alongside the extracted data and any relevant attachment files.

Enabling source file publishing

Source file publishing can be enabled on a per-subscription basis, via its poll_payload attribute, or through an organisation-wide or key-level configuration. All use the same configuration key: publish_source_files

Creating a subscription with source file publishing enabled

curl https://ricloud-api.reincubate.com/subscriptions \
  -X POST \
  -H 'Authorization: Token <your key_token>' \
  -H 'Content-Type: application/json' \
  -d '{
  "session": "<session ID>",
  "source": "<child source ID>",
  "poll_payload": {
    "data_types": ["ios_messages.messages"],
    "publish_source_files": true
  }
}'

Configuring source file publishing at the organisation level

curl https://ricloud-api.reincubate.com/organisation \
  -X POST \
  -H 'Authorization: Token <your key_token>' \
  -H 'Content-Type: application/json' \
  -d '{
  "config": {
    "publish_source_files": true
  }
}'

The above requires an organisation admin key.

Configuring source file publishing at the key level

curl https://ricloud-api.reincubate.com/key \
  -X POST \
  -H 'Authorization: Token <your key_token>' \
  -H 'Content-Type: application/json' \
  -d '{
  "config": {
    "publish_source_files": true
  }
}'

Organisation-wide configurations can be overridden by key-level configurations, which can be helpful when source files are only of interest for production usage. Similarly, both can be overridden by the poll payload parameter – allowing you to specify whether source files should be published on a case-by-case basis.

Note that enabling features via organisation or key configuration will retroactively apply the changed behaviour to existing subscriptions.

Working with source file results

When source file publishing is enabled the API will track any files touched as part of the extraction process, publish them to your storage bucket, and create a result object associated with the current poll.

A reference to the source files related to a particular set of data is included in the metadata attribute of the data result itself. For example:

{
  "data": [{
    "id": "a1b2c3d4",
    "data_type": "message",
    "conversation_id": "w6x7y8z9",
    "handle": "Pal",
    "type": "SMS",
    "text": "Hi, how’s it going?",
    "attachments": [{
          "type": "image",
          "file_id": "a492be9a8dadabdc774286116232420224f14976",
          "file_path": "Library/SMS/Attachments/de/14/33A09D66-287E-4B64-A874-1A1234BD7E61/IMG_0006.PNG",
          "filename": "IMG_0006.PNG"
    }],
    "group_handles": [
      "Pal",
      "renate@reincubate.com"
    ],
    "from_me": false,
    "deleted": false,
    "date": "2020-01-01T00:00:00.000000Z"
  }],
  "metadata": {
    "count": 1,
    "filters": null,
    "source_files": [{
      "file_id": "3d0d7e5fb2ce288813306e4d4636395e047a3d28",
      "domain": "HomeDomain",
      "file_path": "Library/SMS/sms.db",
      "filename": "sms.db",
      "size": 123456,
      "date_last_modified": "2020-01-02T00:00:00.000000Z"
    }]
  }
}

As with attachment file results, the file_id of a source file entry corresponds to the identifier attribute of the result it is related to. Therefore, this is also the name of the file written to your storage bucket.