Get Credentials

Get credentials

After signing in, you can create credentials that can be used to call the Platform APIs.

Catalog Service API (1.0)

Download OpenAPI specification:Download

Catalog is the system of record for data location and lineage within Adobe Experience Platform. Catalog Service does not contain the actual files or directories that contain the data. Instead, it holds the metadata and description of those files and directories.

Catalog acts as a metadata store or "catalog" where you can find information about your data within Experience Platform.

Use Catalog to answer the following questions: Where is my data located? At what stage of processing is this data? What systems or processes have acted on my data? What errors occurred during processing? If successful, how much data was processed?

Related documentation:
- Catalog Service documentation
Visualize API calls with Postman (a free, third-party software):
API paths:
- PLATFORM Gateway URL: https://platform.adobe.io/
- Base path for this API: /data/foundation/catalog
- Example of a complete path: https://platform.adobe.io/data/foundation/catalog/batches
Required headers:
- All calls require the headers Authorization, x-gw-ims-org-id, and x-api-key. For more information on how to obtain these values, see the authentication tutorial.
- All resources in Experience Platform are isolated to specific virtual sandboxes. All requests to Platform APIs require the header x-sandbox-name whose value is the all-lowercase name of the sandbox the operation will take place in (for example, "prod"). See the sandboxes overview for more information.
- All requests with a payload in the request body (such as POST, PUT, and PATCH calls) must include the header Content-Type with a value of application/json.
API error handling:
- Refer to the Experience Platform API troubleshooting guide for FAQs, API status codes, and request header errors.
Sorting and filtering responses:
- When performing listing (GET) requests in the Catalog Service API, you can use query parameters to sort and filter responses.

Batches

Batches allow users to understand which operations and applications have been performed on objects tracked by the system.

Retrieve a list of batches

This request retrieves a list of batches for operations and applications that have been performed on objects tracked by the system.

Request

query Parameters

limit	integer Default: 20 Limit response to a specified positive number of objects. The maximum number of objects that can be returned is 100.
createdAfter	integer <int64> Exclusively filter records created after this timestamp.
createdBefore	integer <int64> Exclusively filter records created before this timestamp.
created	integer <int64> Filter by the Unix timestamp (in milliseconds) when this object was persisted.
createdClient	string Filter by the ID of the IMS client that created this object.
updated	integer <int64> Filter by the Unix timestamp (in milliseconds) for the time of last modification.
createdUser	string Filter by the ID of the user who created this object.
updatedUser	string Filter by the ID of the user who changed this object.
version	string Filter by Semantic version of the account. Updated when the object is modified.
tags	string Filter response by the specified tags.
status	string Filter by the current (mutable) status of the batch.
recordCount	integer <int64> Filter by the total number of data records (rows or documents) processed in the batch.
failedRecordCount	integer <int64> Filter by the number of records that could not be processed in the batch.
size	integer <int64> The number of bytes processed in the batch.
started	integer <int64> Filter by the Unix timestamp (in milliseconds) when the batch processing action was started. Subtracting "started" from "completed" will yield the total processing time.
completed	integer <int64> Filter by the Unix timestamp (in milliseconds) when the batch processing action was completed. Subtracting "started" from "completed" will yield the total processing time.
startAfter	integer <int64> Filter for batches with availability dates that start after the specified timestamp.
startBefore	integer <int64> Filter for batches with availability dates that start before the specified timestamp.
endAfter	integer <int64> Filter for batches with availability dates that end after the specified timestamp.
endBefore	integer <int64> Filter for batches with availability dates that end before the specified timestamp.
batch	string Filter on the ID of the batch.
externalId	string Filter on the ID of the external object.
dataSet	string Filter on the ID of the dataset.
orderBy	string Sort parameter and direction for sorting the response. Ex. orderBy=asc:created,updated. This was previously called sort.
start	integer Default: 0 Specifies the offset of the pages of results. To get the first page of results, set the value to `start=0`.
property	string Regex used to filter objects in the response.
properties	string A comma separated allowlist of top-level object properties to be returned in the response. Used to cut down the number of properties and amount of data returned in the response bodies.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.

Responses

200

A list of the batch responses is returned.

400

A bad request.

403

Access is forbidden.

404

The batch responses were not found.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

get/batches

Response samples

application/json

{"5911f88ae2f4bf657c5a8cb5": {"imsOrg": "4F3BB22C5631222A7F000101@AdobeOrg",
"created": 1494349962314,
"createdClient": "MCDPCatalogServiceStage",
"createdUser": "MCDPCatalogServiceStage@AdobeID",
"updatedUser": "MCDPCatalogServiceStage@AdobeID",
"updated": 1494349963467,
"externalId": "/adobe/bulkingest/5af9ee67e4045c1b4945c209",
"status": "success",
"errors": [{"code": "err-1494349963436"
}
],
"version": "1.0.3",
"availableDates": {"startDate": 1337,
"endDate": 4000
},
"relatedObjects": [{"type": "batch",
"id": "foo_batch"
},
{"type": "dataSet",
"id": "foo_dataSet"
},
{"type": "dataSetView",
"id": "foo_dataSetView"
},
{"type": "dataSetFile",
"id": "foo_dataSetFile"
},
{"type": "expressionBlock",
"id": "foo_expressionBlock"
},
{"type": "service",
"id": "foo_service"
},
{"type": "serviceDefinition",
"id": "foo_serviceDefinition"
}
],
"metrics": {"foo": 1337
},
"tags": {"foo_bar": ["stuff"
],
"bar_foo": ["woo",
"baz"
],
"foo/bar/foo-bar": ["weehaw",
"wee:haw"
]
},
"inputFormat": {"format": "parquet",
"delimiter": ".",
"quote": "`",
"escape": "\\",
"nullMarker": "",
"header": "true",
"charset": "UTF-8"
}
}
}

Create a new batch

This request creates a new batch for operations and applications that have been performed on objects tracked by the system.

Request

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.
Content-Type required	string application/json

Request Body schema: application/json
required

The batch that will be created.

started	integer <int64> The Unix timestamp (in milliseconds) when the Batch processing action was started.
completed	integer <int64> The Unix timestamp (in milliseconds) when the Batch processing action was completed. Completed - Started should yield the total processing time.
	object Indicates that a batch was replayed or restated.
status required	string The current (mutable) status of this Batch. Enum: "processing" "success" "failure" "queued" "retrying" "stalled" "aborted" "abandoned" "inactive" "failed" "loading" "loaded" "staged" "active" "staging" "deleted"
recordCount	integer <int64> The total number of data records (rows/documents) processed in this Batch.
failedRecordCount	integer <int64> The number of records that could not be processed in this Batch.
	Array of objects (errors) An array of objects that include the error code, rows the error appears and a human-readable description of the error.
size	integer <int64> Number of bytes processed in this Batch.
	object Describes what date range of data is available in the Batch. Null if dates aren't relevant for data related to this Batch.
	Array of objects (relatedObjects) List of associated objects for this batch, such as datasets or other batches.
metrics	object Contains metrics related to this batch. Metric names are determined by the producer of the object since each batch may want to record metrics that are relevant to the process
	object Tags are values associated with a particular object, these are generally used by external systems for marking an object in a way that it understands. Normally, tags are not used for internal Catalog business logic.
	object (inputFormat) Metadata describing the INGEST format of the data being loaded (e.g. Parquet, CSV, JSON).

Responses

201

Array[ @/batches/batchId ]

400

The batch is invalid.

403

Access is forbidden.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

post/batches

Request samples

Payload

application/json

{"started": 1494349963122,
"completed": 1494349963467,
"replay": {"predecessors": ["5d01230fc78a4e4f8c0c6b387b4b8d1c"
],
"reason": "Deletion of privacy data."
},
"status": "success",
"recordCount": 45,
"failedRecordCount": 2,
"errors": [{"code": "err-1494349963436",
"rows": ["3-5"
],
"description": "This resource is read-only and cannot be modified."
}
],
"size": 512,
"availableDates": {"startDate": 1337,
"endDate": 4000
},
"relatedObjects": [{"type": "batch",
"id": "foo-batch",
"tag": "input",
"status": "processing",
"errors": [{"code": "err-1494349963436",
"rows": ["3-5"
],
"description": "This resource is read-only and cannot be modified."
}
],
"metrics": {"foo": 1337
}
}
],
"metrics": { },
"tags": {"property1": ["string"
],
"property2": ["string"
]
},
"inputFormat": {"format": "parquet",
"isMultiLineJson": true,
"delimiter": ".",
"quote": "`",
"escape": "\\",
"nullMarker": "",
"header": "true",
"charset": "UTF-8"
}
}

Response samples

application/json

["string"
]

Look up the unique values stored in a field

This endpoint retrieves the unique values stored in a field, as specified by the ID.

Request

path Parameters

FIELD

required

string

Field to get unique values for. Note: This endpoint will be deprecated soon.

query Parameters

limit	integer Default: 20 Limit response to a specified positive number of objects. The maximum number of objects that can be returned is 100.
createdAfter	integer <int64> Exclusively filter records created after this timestamp.
createdBefore	integer <int64> Exclusively filter records created before this timestamp.
created	integer <int64> Filter by the Unix timestamp (in milliseconds) when this object was persisted.
createdClient	string Filter by the ID of the IMS client that created this object.
updated	integer <int64> Filter by the Unix timestamp (in milliseconds) for the time of last modification.
createdUser	string Filter by the ID of the user who created this object.
updatedUser	string Filter by the ID of the user who changed this object.
version	string Filter by Semantic version of the account. Updated when the object is modified.
status	string Filter by the current (mutable) status of the batch.
recordCount	integer <int64> Filter by the total number of data records (rows or documents) processed in the batch. 100 objects is the maximum that can be returned by requests.
failedRecordCount	integer <int64> Filter by the number of records that could not be processed in the batch.
size	integer <int64> The number of bytes processed in the batch.
started	integer <int64> Filter by the Unix timestamp (in milliseconds) when the batch processing action was started. Subtracting "started" from "completed" will yield the total processing time.
completed	integer <int64> Filter by the Unix timestamp (in milliseconds) when the batch processing action was completed. Subtracting "started" from "completed" will yield the total processing time.
startAfter	integer <int64> Filter for batches with availability dates that start after the specified timestamp.
startBefore	integer <int64> Filter for batches with availability dates that start before the specified timestamp.
endAfter	integer <int64> Filter for batches with availability dates that end after the specified timestamp.
endBefore	integer <int64> Filter for batches with availability dates that end before the specified timestamp.
batch	string Filter on the ID of the batch.
externalId	string Filter on the ID of the external object.
dataSet	string Filter on the ID of the dataset.
transform	string Filter on the ID of the transform.
orderBy	string Sort parameter and direction for sorting the response. Ex. orderBy=asc:created,updated. This was previously called sort.
start	integer Default: 0 Specifies the offset of the pages of results. To get the first page of results, set the value to `start=0`.
property	string Regex used to filter objects in the response.
properties	string A comma separated allowlist of top-level object properties to be returned in the response. Used to cut down the number of properties and amount of data returned in the response bodies.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.

Responses

200

A list of unique values for a batch.

400

A bad request has been submitted.

403

Access is forbidden.

404

The batch is not found.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

get/batches/uniques/{FIELD}

Response samples

application/json

["5911f88ae2f4bf657c5a8cb5",
"5911f88ae2f4bf657c5a8cb4",
"5911f88ae2f4bf657c5a8cb3"
]

Look up a batch

This request retrieves a specific batch based on the {BATCH_ID} entered.

Request

path Parameters

BATCH_ID

required

string

The ID of the batch you are performing the operation on.

query Parameters

aggregate

boolean

If true, then return the aggregated status, metrics, and errors in the root level status, metrics, and errors (default is false).

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.

Responses

200

The batch object for the specified batch ID is returned.

400

A bad request.

403

Access is forbidden.

404

The specified batch was not found.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

get/batches/{BATCH_ID}

Response samples

application/json

{"5911f88ae2f4bf657c5a8cb5": {"imsOrg": "5911f88ae2f4bf657c5a8cb5",
"created": 1494349962314,
"createdClient": "MCDPCatalogServiceStage",
"createdUser": "MCDPCatalogServiceStage@AdobeID",
"updatedUser": "MCDPCatalogServiceStage@AdobeID",
"updated": 1494349963467,
"externalId": "/adobe/bulkingest/5af9ee67e4045c1b4945c209",
"status": "success",
"errors": [{"code": "err-1494349963436"
}
],
"version": "1.0.3",
"availableDates": {"startDate": 1337,
"endDate": 4000
},
"relatedObjects": [{"type": "batch",
"id": "foo_batch"
},
{"type": "dataSet",
"id": "foo_dataSet"
},
{"type": "dataSetView",
"id": "foo_dataSetView"
},
{"type": "dataSetFile",
"id": "foo_dataSetFile"
},
{"type": "expressionBlock",
"id": "foo_expressionBlock"
},
{"type": "service",
"id": "foo_service"
},
{"type": "serviceDefinition",
"id": "foo_serviceDefinition"
}
],
"metrics": {"foo": 1337
},
"tags": {"foo_bar": ["stuff"
],
"bar_foo": ["woo",
"baz"
],
"foo/bar/foo-bar": ["weehaw",
"wee:haw"
]
},
"inputFormat": {"format": "parquet",
"delimiter": ".",
"quote": "`",
"escape": "\\",
"nullMarker": "",
"header": "true",
"charset": "UTF-8"
}
}
}

Update a batch

This request updates a specific batch based on the {BATCH_ID} specified.

Request

path Parameters

BATCH_ID

required

string

The ID of the batch you are performing the operation on.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.
if-match	string Set to verify the valid versions of document to be modified by matching the updated date.
if-none-match	string Set to verify the invalid versions of document to be modified by matching the updated date.
Content-Type required	string application/json

Request Body schema: application/json
required

The batch information to be updated.

started	integer <int64> The Unix timestamp (in milliseconds) when the Batch processing action was started.
completed	integer <int64> The Unix timestamp (in milliseconds) when the Batch processing action was completed. Completed - Started should yield the total processing time.
	object Indicates that a batch was replayed or restated.
status required	string The current (mutable) status of this Batch. Enum: "processing" "success" "failure" "queued" "retrying" "stalled" "aborted" "abandoned" "inactive" "failed" "loading" "loaded" "staged" "active" "staging" "deleted"
recordCount	integer <int64> The total number of data records (rows/documents) processed in this Batch.
failedRecordCount	integer <int64> The number of records that could not be processed in this Batch.
	Array of objects (errors) An array of objects that include the error code, rows the error appears and a human-readable description of the error.
size	integer <int64> Number of bytes processed in this Batch.
	object Describes what date range of data is available in the Batch. Null if dates aren't relevant for data related to this Batch.
	Array of objects (relatedObjects) List of associated objects for this batch, such as datasets or other batches.
metrics	object Contains metrics related to this batch. Metric names are determined by the producer of the object since each batch may want to record metrics that are relevant to the process
	object Tags are values associated with a particular object, these are generally used by external systems for marking an object in a way that it understands. Normally, tags are not used for internal Catalog business logic.
	object (inputFormat) Metadata describing the INGEST format of the data being loaded (e.g. Parquet, CSV, JSON).

Responses

200

Array[ @/batches/batchId ]

400

A bad request.

403

Access is forbidden.

404

Not found

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

put/batches/{BATCH_ID}

Request samples

Payload

application/json

{"started": 1494349963122,
"completed": 1494349963467,
"replay": {"predecessors": ["5d01230fc78a4e4f8c0c6b387b4b8d1c"
],
"reason": "Deletion of privacy data."
},
"status": "success",
"recordCount": 45,
"failedRecordCount": 2,
"errors": [{"code": "err-1494349963436",
"rows": ["3-5"
],
"description": "This resource is read-only and cannot be modified."
}
],
"size": 512,
"availableDates": {"startDate": 1337,
"endDate": 4000
},
"relatedObjects": [{"type": "batch",
"id": "foo-batch",
"tag": "input",
"status": "processing",
"errors": [{"code": "err-1494349963436",
"rows": ["3-5"
],
"description": "This resource is read-only and cannot be modified."
}
],
"metrics": {"foo": 1337
}
}
],
"metrics": { },
"tags": {"property1": ["string"
],
"property2": ["string"
]
},
"inputFormat": {"format": "parquet",
"isMultiLineJson": true,
"delimiter": ".",
"quote": "`",
"escape": "\\",
"nullMarker": "",
"header": "true",
"charset": "UTF-8"
}
}

Response samples

application/json

["string"
]

Create a new batch

This request creates a new batch for operations and applications that have been performed on objects tracked by the system, using the ID value of an existing dataset.

Request

path Parameters

BATCH_ID

required

string

The ID of the batch you are performing the operation on.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.
Content-Type required	string application/json

Request Body schema: application/json
required

Batch to be created.

started	integer <int64> The Unix timestamp (in milliseconds) when the Batch processing action was started.
completed	integer <int64> The Unix timestamp (in milliseconds) when the Batch processing action was completed. Completed - Started should yield the total processing time.
	object Indicates that a batch was replayed or restated.
status required	string The current (mutable) status of this Batch. Enum: "processing" "success" "failure" "queued" "retrying" "stalled" "aborted" "abandoned" "inactive" "failed" "loading" "loaded" "staged" "active" "staging" "deleted"
recordCount	integer <int64> The total number of data records (rows/documents) processed in this Batch.
failedRecordCount	integer <int64> The number of records that could not be processed in this Batch.
	Array of objects (errors) An array of objects that include the error code, rows the error appears and a human-readable description of the error.
size	integer <int64> Number of bytes processed in this Batch.
	object Describes what date range of data is available in the Batch. Null if dates aren't relevant for data related to this Batch.
	Array of objects (relatedObjects) List of associated objects for this batch, such as datasets or other batches.
metrics	object Contains metrics related to this batch. Metric names are determined by the producer of the object since each batch may want to record metrics that are relevant to the process
	object Tags are values associated with a particular object, these are generally used by external systems for marking an object in a way that it understands. Normally, tags are not used for internal Catalog business logic.
	object (inputFormat) Metadata describing the INGEST format of the data being loaded (e.g. Parquet, CSV, JSON).

Responses

201

Array[ @/batches/batchId ]

400

A bad request.

403

Access is forbidden.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

post/batches/{BATCH_ID}

Request samples

Payload

application/json

{"started": 1494349963122,
"completed": 1494349963467,
"replay": {"predecessors": ["5d01230fc78a4e4f8c0c6b387b4b8d1c"
],
"reason": "Deletion of privacy data."
},
"status": "success",
"recordCount": 45,
"failedRecordCount": 2,
"errors": [{"code": "err-1494349963436",
"rows": ["3-5"
],
"description": "This resource is read-only and cannot be modified."
}
],
"size": 512,
"availableDates": {"startDate": 1337,
"endDate": 4000
},
"relatedObjects": [{"type": "batch",
"id": "foo-batch",
"tag": "input",
"status": "processing",
"errors": [{"code": "err-1494349963436",
"rows": ["3-5"
],
"description": "This resource is read-only and cannot be modified."
}
],
"metrics": {"foo": 1337
}
}
],
"metrics": { },
"tags": {"property1": ["string"
],
"property2": ["string"
]
},
"inputFormat": {"format": "parquet",
"isMultiLineJson": true,
"delimiter": ".",
"quote": "`",
"escape": "\\",
"nullMarker": "",
"header": "true",
"charset": "UTF-8"
}
}

Response samples

application/json

["string"
]

Delete a batchDeprecated

Note: This endpoint has been deprecated. Please use the delete a batch endpoint within the batch ingestion API.

Request

path Parameters

BATCH_ID

required

string

The ID of the batch you are performing the operation on.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.

Responses

200

Array[ @/batches/batchId ]

400

A bad request.

403

Access is forbidden.

404

The batch was not found.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

delete/batches/{BATCH_ID}

Response samples

application/json

["string"
]

Update one or more batch attributes

This request allows you to update specified attributes of an existing batch.

Request

path Parameters

BATCH_ID

required

string

The ID of the batch you are performing the operation on.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.
if-match	string Set to verify the valid versions of document to be modified by matching the updated date.
Content-Type required	string application/json
if-none-match	string Set to verify the invalid versions of document to be modified by matching the updated date.

Request Body schema: application/json
required

The batch that is to be updated.

started	integer <int64> The Unix timestamp (in milliseconds) when the Batch processing action was started.
completed	integer <int64> The Unix timestamp (in milliseconds) when the Batch processing action was completed. Completed - Started should yield the total processing time.
	object Indicates that a batch was replayed or restated.
status	string The current (mutable) status of this Batch. Enum: "processing" "success" "failure" "queued" "retrying" "stalled" "aborted" "abandoned" "inactive" "failed" "loading" "loaded" "staged" "active" "staging" "deleted"
recordCount	integer <int64> The total number of data records (rows/documents) processed in this Batch.
failedRecordCount	integer <int64> The number of records that could not be processed in this Batch.
	Array of objects (errors) An array of objects that include the error code, rows the error appears, and a human-readable description of the error.
size	integer <int64> Number of bytes processed in this Batch.
	object Describes what date range of data is available in the Batch. Null if dates aren't relevant for data related to this Batch.
	Array of objects (relatedObjects) List of associated objects for this batch, such as datasets or other batches.
metrics	object Contains metrics related to this Batch. Metric names are determined by the producer of the object since each Batch may want to record metrics that are relevant to the process
	object Tags are values associated with a particular object, these are generally used by external systems for marking an object in a way that it understands. Normally tags are not used for internal Catalog business logic
	object (inputFormat) Metadata describing the INGEST format of the data being loaded (e.g. Parquet, CSV, JSON).

Responses

200

Array[ @/batches/batchId ]

400

A bad request.

403

Access is forbidden.

404

The specified batch was not found.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

patch/batches/{BATCH_ID}

Request samples

Payload

application/json

{"started": 1494349963122,
"completed": 1494349963467,
"replay": {"predecessors": ["62a31e96ba86e52a2f729a8d"
],
"reason": "Deletion of privacy data"
},
"status": "processing",
"recordCount": 50,
"failedRecordCount": 2,
"errors": [{"code": "err-1494349963436",
"rows": ["3-5"
],
"description": "This resource is read-only and cannot be modified."
}
],
"size": 512,
"availableDates": {"startDate": 1337,
"endDate": 4000
},
"relatedObjects": [{"type": "batch",
"id": "foo-batch",
"tag": "input",
"status": "processing",
"errors": [{"code": "err-1494349963436",
"rows": ["3-5"
],
"description": "This resource is read-only and cannot be modified."
}
],
"metrics": {"foo": 1337
}
}
],
"metrics": {"foo": 1337
},
"tags": {"property1": ["string"
],
"property2": ["string"
]
},
"inputFormat": {"format": "parquet",
"isMultiLineJson": true,
"delimiter": ".",
"quote": "`",
"escape": "\\",
"nullMarker": "",
"header": "true",
"charset": "UTF-8"
}
}

Response samples

application/json

["string"
]

Batched requests

Batched requests allow users to supply an array of objects in the request payload, representing what would normally be individual requests.

Retrieve a list of batched requestsDeprecated

This request allows you to retrieve a list of batched requests which can be ordered as required. For example, the list can be ordered by created date in ascending order.

Request

query Parameters

orderBy	string Sort parameter and direction for sorting the response. Ex. orderBy=asc:created,updated. This was previously called sort.
limit	integer Default: 20 Limit response to a specified positive number of objects. The maximum number of objects that can be returned is 100.
start	integer Default: 0 Specifies the offset of the pages of results. To get the first page of results, set the value to `start=0`.
property	string Regex used to filter objects in the response.
properties	string A comma separated allowlist of top-level object properties to be returned in the response. Used to cut down the number of properties and amount of data returned in the response bodies.

Responses

200

serviceResources

404

Not found

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

get/

Response samples

application/json

{"dataSource": "foo_dataSource",
"dataSet": "foo_dataSet",
"transform": "string"
}

Create a new batched requestDeprecated

Note: This endpoint has been deprecated. This request creates a new batched request which allows you to supply an array of objects in the request payload, representing what would normally be individual requests.

Request

header Parameters

Content-Type

required

string

application/json

Request Body schema: application/json
required

The relative path of the resource.

Array

id	string
method	string
headers	object Example: "headers": {"if-match": "1.0.6"}
resource	string
body	object

Responses

201

Array[ @/resource/resourceId ]

400

A bad request.

403

Access is forbidden.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

post/

Request samples

Payload

application/json

[{"id": "string",
"method": "string",
"headers": {"if-match": "1.0.6"
},
"resource": "string",
"body": { }
}
]

Response samples

application/json

["string"
]

Datasets

Datasets are the building blocks for data transformation and tracking in Catalog Service.

Retrieve a list of datasets

This request lets you retrieve a list of datasets which are the building blocks for data transformation and tracking in Catalog Service.

Request

query Parameters

version	string Filter by Semantic version of the account. Updated when the object is modified.
created	integer <int64> Filter by the Unix timestamp (in milliseconds) when this object was persisted.
updated	integer <int64> Filter by the Unix timestamp (in milliseconds) for the time of last modification.
createdClient	string Filter by the ID of the IMS client that created this object.
createdUser	string Filter by the ID of the user who created this object.
updatedUser	string Filter by the ID of the user who changed this object.
name	string Filter on the name of the dataset.
description	string Filter on the description of the dataset.
enableErrorDiagnostics	boolean A field that provides the ability to opt in to generating diagnostic files for the errors while ingesting data.
orderBy	string Sort parameter and direction for sorting the response. Ex. orderBy=asc:created,updated. This was previously called sort.
limit	integer Default: 20 Limit response to a specified positive number of objects. The maximum number of objects that can be returned is 100.
start	integer Default: 0 Specifies the offset of the pages of results. To get the first page of results, set the value to `start=0`.
property	string Regex used to filter objects in the response.
properties	string A comma separated allowlist of top-level object properties to be returned in the response. Used to cut down the number of properties and amount of data returned in the response bodies.
tags	string Filter response by the specified tags.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.

Responses

200

A list of datasets is returned.

400

A bad request.

403

Access is forbidden.

404

The datasets are not found.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

get/dataSets

Response samples

application/json

{"5ab540d0864cf0267448ead4": {"version": "1.0.0",
"imsOrg": "4F3BB22C5631222A7F000101@AdobeOrg",
"name": "Acme accounts dataset - 1521827915638",
"created": 1521828048923,
"updated": 1521828208046,
"createdClient": "platform_foundation_catalog",
"createdUser": "platform_foundation_catalog@AdobeID",
"updatedUser": "platform_foundation_catalog@AdobeID",
"tags": {"foo": ["bar",
"foos",
"ball"
],
"adobe/touchpoint/appliedTransformations": ["CLUSTERED:FOO"
]
},
"viewId": "5ab54170864cf0267448ead5",
"basePath": "s3://bar/ball/baz",
"files": "@/dataSetFiles?dataSetId=5ab540d0864cf0267448ead4",
"observableSchema": { }
}
}

Create a new dataset

This request creates a new dataset.

Request

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.
Content-Type required	string application/json

Request Body schema: application/json
required

Information about the new dataset you are creating.

name	string A descriptive, human-readable name for this dataset.
description	string A longer text description of the dataset.
	object Tags are values associated with a particular object, these are generally used by external systems for marking an object in a way that it understands. Normally tags are not used for internal Catalog business logic
enableErrorDiagnostics	boolean This field provides the ability to opt in to generating diagnostic files for the errors while ingesting data.
observableSchema	object observableSchema stores a JSON Schema object that is a valid subset (including the null case) of the XDM model specified by `schemaRef`.
	object Metadata describing the dataset files associated with this DatasetView.
schema	string This is the uri for the XDM model
	object (schemaRef) This is the full URL for XDM schema path. This field will be derived from schema field, if it is not present. This field will be used to annotate XDM schema version used in the dataSet or dataSetView.
dataSourceId	integer The ID of the datasource. It must be unique, along with its name.

Responses

201

The dataset has been successfully created.

400

A bad request.

403

Access is forbidden.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

post/dataSets

Request samples

Payload

application/json

{"name": "Profile-enabled sample dataset",
"description": "This is a sample dataset that is enabled for Profile.",
"schemaRef": {"id": "https://ns.adobe.com/{TENANT_ID}/schemas/31670881463308a46f7d2cb09762715",
"contentType": "application/vnd.adobe.xed-full-notext+json; version=1"
},
"tags": {"unifiedIdentity": ["enabled: true"
],
"unifiedProfile": ["enabled: true",
"isUpsert: true"
]
}
}

Response samples

application/json

["string"
]

Look up a dataset

This request allows you to retrieve a dataset using the {DATASET_ID}.

Request

path Parameters

DATASET_ID

required

string

The ID of the dataset.

query Parameters

properties

string

A comma separated allowlist of top-level object properties to be returned in the response. Used to cut down the number of properties and amount of data returned in the response bodies.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.

Responses

200

The dataset for the specified dataset ID is returned.

400

A bad request.

403

Access is forbidden.

404

The dataset was not found.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

get/dataSets/{DATASET_ID}

Response samples

application/json

{"5ab540d0864cf0267448ead4": {"version": "1.0.0",
"imsOrg": "4F3BB22C5631222A7F000101@AdobeOrg",
"name": "Acme accounts dataset - 1521827915638",
"enableErrorDiagnostics": true,
"created": 1521828048923,
"updated": 1521828208046,
"createdClient": "platform_foundation_catalog",
"createdUser": "4F3BB22C5631222A7F000101@AdobeOrg",
"updatedUser": "platform_foundation_catalog@AdobeID",
"tags": {"foo": ["bar",
"foos",
"ball"
],
"adobe/touchpoint/appliedTransformations": ["CLUSTERED:FOO"
]
},
"viewId": "5ab54170864cf0267448ead5",
"basePath": "s3://bar/ball/baz",
"files": "@/dataSetFiles?dataSetId=5ab540d0864cf0267448ead4",
"observableSchema": { }
}
}

Update a dataset

This request allows you to update a specific dataset. The dataset that requires updating can be retrieved using the {DATASET_ID}.

Request

path Parameters

DATASET_ID

required

string

The ID of the dataset.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.
if-match	string Set to verify the valid versions of document to be modified by matching the updated date.
if-none-match	string Set to verify the invalid versions of document to be modified by matching the updated date.
Content-Type required	string application/json

Request Body schema: application/json
required

Data set field(s) to be updated.

name	string A descriptive, human-readable name for this dataset.
description	string A longer text description of the dataset.
	object Tags are values associated with a particular object, these are generally used by external systems for marking an object in a way that it understands. Normally tags are not used for internal Catalog business logic
enableErrorDiagnostics	boolean This field provides the ability to opt in to generating diagnostic files for the errors while ingesting data.
observableSchema	object observableSchema stores a JSON Schema object that is a valid subset (including the null case) of the XDM model specified by `schemaRef`.
	object Metadata describing the dataset files associated with this DatasetView.
schema	string This is the uri for the XDM model
	object (schemaRef) This is the full URL for XDM schema path. This field will be derived from schema field, if it is not present. This field will be used to annotate XDM schema version used in the dataSet or dataSetView.
dataSourceId	integer The ID of the datasource. It must be unique, along with its name.

Responses

200

Array[ @/dataSets/dataSetId ]

400

A bad request.

403

Access is forbidden.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

put/dataSets/{DATASET_ID}

Request samples

Payload

application/json

{"name": "Acme accounts dataset - 1521827915638",
"description": "A dataset for accounts held by Acme.",
"tags": {"foo_bar": ["stuff"
],
"bar_foo": ["woo",
"baz"
],
"foo/bar/foo-bar": ["weehaw",
"wee:haw"
]
},
"enableErrorDiagnostics": true,
"observableSchema": { },
"fileDescription": {"format": "parquet",
"delimiters": ["s"
],
"quotes": "`",
"escapes": ["\\"
],
"nullMarkers": "",
"header": "true",
"charset": "UTF-8"
},
"schema": "string",
"schemaRef": {"id": "https://ns.adobe.com/{TENANT_ID}/schemas/142afb78d8b368a5ba97a6cc8fc7e033",
"contentType": "application/vnd.adobe.xed+json;version=1"
},
"dataSourceId": 0
}

Response samples

application/json

["string"
]

Delete a dataset

This request deletes a dataset using the {DATASET_ID}.

Request

path Parameters

DATASET_ID

required

string

The ID of the dataset.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.

Responses

200

Array[ @/dataSets/dataSetId ]

400

A bad request.

403

Access is forbidden.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

delete/dataSets/{DATASET_ID}

Response samples

application/json

["string"
]

Update a dataset's attributes

This request allows you to update specified attributes of an existing DataSetView.

Request

path Parameters

DATASET_ID

required

string

The ID of the dataset.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.
if-match	string Set to verify the valid versions of document to be modified by matching the updated date.
if-none-match	string Set to verify the invalid versions of document to be modified by matching the updated date.
Content-Type required	string application/json

Request Body schema:
application/json
required

Dataset field(s) to be updated.

name	string A descriptive, human-readable name for this dataset.
description	string A longer text description of the dataset.
	object Tags are values associated with a particular object, these are generally used by external systems for marking an object in a way that it understands. Normally tags are not used for internal Catalog business logic
enableErrorDiagnostics	boolean This field provides the ability to opt in to generating diagnostic files for the errors while ingesting data.
observableSchema	object observableSchema stores a JSON Schema object that is a valid subset (including the null case) of the XDM model specified by `schemaRef`.
	object Metadata describing the dataset files associated with this DatasetView.
schema	string This is the uri for the XDM model
	object (schemaRef) This is the full URL for XDM schema path. This field will be derived from schema field, if it is not present. This field will be used to annotate XDM schema version used in the dataSet or dataSetView.
dataSourceId	integer The ID of the datasource. It must be unique, along with its name.

Responses

200

Array[ @/dataSets/dataSetId ]

400

A bad request.

403

Access is forbidden.

404

Not found

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

patch/dataSets/{DATASET_ID}

Request samples

Payload

{"name": "Acme accounts dataset - 1521827915638",
"description": "A dataset for accounts held by Acme.",
"tags": {"foo_bar": ["stuff"
],
"bar_foo": ["woo",
"baz"
],
"foo/bar/foo-bar": ["weehaw",
"wee:haw"
]
},
"enableErrorDiagnostics": true,
"observableSchema": { },
"fileDescription": {"format": "parquet",
"delimiters": ["s"
],
"quotes": "`",
"escapes": ["\\"
],
"nullMarkers": "",
"header": "true",
"charset": "UTF-8"
},
"schema": "string",
"schemaRef": {"id": "https://ns.adobe.com/{TENANT_ID}/schemas/142afb78d8b368a5ba97a6cc8fc7e033",
"contentType": "application/vnd.adobe.xed+json;version=1"
},
"dataSourceId": 0
}

Response samples

application/json

["string"
]

Retrieve a dataset's credentialsDeprecated

This request fetches credentials for the requested dataset ID.

Request

path Parameters

required

string

Object ID

query Parameters

batchId

string

If wishing to write, the batchId used during writing. Required for building a stage path.

header Parameters

Authorization required	string The access token which can be copied from your Experience Platform integration, prefixed with "Bearer". For more information on how to obtain this value, visit the authentication tutorial.
x-api-key required	string The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-gw-ims-org-id required	string The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the authentication tutorial.
x-sandbox-name required	string The name of the sandbox in which the operation will take place. See the sandboxes overview documentation for more information.
x-user-token	string For calls which are proxied by catalog, the access token for the user on whose behalf this call is made.
accept-encoding	string Used to signal the intent to consume compressed response bodies. Supported encodings: gzip.

Responses

200

Credential object given dataSet.

400

A bad request.

403

Access is forbidden.

500

An internal server error has ocurred.

default

An unexpected error has ocurred.

get/dataSets/{id}/credentials

Response samples

application/json

{"dataSetId": "5abd49645591445e1ba04f87",
"path": "https://platform.adobe.io/data/foundation/dataset/datasets/5abd49645591445e1ba04f87/",
"stagePath": "platformdataset/stage/5d01230fc78a4e4f8c0c6b387b4b8d1c",
"credentials": {"clientKey": "string",
"clientId": "string",
"tenantId": "string",
"resourceGroupName": "string",
"subscriptionId": "string",
"name": "string",
"adlsType": "string",
"adlsUri": "string",
"hadoopUri": "string"
}
}

dataSetId	string Filter by a datasetId. A list can be provided by separating the values with comma. The number of datasets requested is limited by the max-http-header-size config.
lastBatchId	string Filter by the ID of the last related batch of a dataset Example: lastBatchId=ca12b29612bf4052872edad59573703c
lastBatchStatus	string Filter by the status of the last related batch of the dataset Example: lastBatchStatus=success
lastSuccessfulBatchId	string Filter by the ID of the last successful batch related to a dataset Example: lastSuccessfulBatchId=ca12b29612bf4052872edad59573703c
lastFailedBatchId	string Filter by the ID of the last failed batch related to a dataset Example: lastFailedBatchId=ca12b29612bf4052872edad59573703a
created	integer <int64> Filter by the Unix timestamp (in milliseconds) when this object was persisted. Example: created=1541534444286
updated	integer <int64> Filter by the Unix timestamp (in milliseconds) for the time of last modification. Example: updated=1541534444286
orderBy	string Sort parameter and direction for sorting the response. Ex. orderBy=asc:created,updated. This was previously called sort. Example: orderBy=orderBy=asc:created,updated
limit	integer Default: 20 Limit response to a specified positive number of objects. The maximum number of objects that can be returned is 100.
start	integer Default: 0 Specifies the offset of the pages of results. To get the first page of results, set the value to `start=0`.
properties	string A comma separated allowlist of top-level object properties to be returned in the response. Used to cut down the number of properties and amount of data returned in the response bodies.

Get credentials

Catalog Service API (1.0)

Batches

Retrieve a list of batches

query Parameters

header Parameters

Create a new batch

header Parameters

Request Body schema: application/jsonrequired

Look up the unique values stored in a field

path Parameters

query Parameters

header Parameters

Look up a batch

path Parameters

query Parameters

header Parameters

Update a batch

path Parameters

header Parameters

Request Body schema: application/jsonrequired

Create a new batch

path Parameters

header Parameters

Request Body schema: application/jsonrequired

Delete a batchDeprecated

path Parameters

header Parameters

Update one or more batch attributes

path Parameters

header Parameters

Request Body schema: application/jsonrequired

Batched requests

Retrieve a list of batched requestsDeprecated

query Parameters

Create a new batched requestDeprecated

header Parameters

Request Body schema: application/jsonrequired

Datasets

Retrieve a list of datasets

query Parameters

header Parameters

Create a new dataset

header Parameters

Request Body schema: application/jsonrequired

Look up a dataset

path Parameters

query Parameters

header Parameters

Update a dataset

path Parameters

header Parameters

Request Body schema: application/jsonrequired

Delete a dataset

path Parameters

header Parameters

Update a dataset's attributes

path Parameters

header Parameters

Request Body schema: application/jsonapplication/json-patch+jsonapplication/jsonrequired

Retrieve a dataset's credentialsDeprecated

path Parameters

query Parameters

header Parameters

Tags

Retrieve a list of aggregated valuesDeprecated

path Parameters

header Parameters

List last batches

query Parameters

header Parameters

Request Body schema: application/json
required

Request Body schema: application/json
required

Request Body schema: application/json
required

Request Body schema: application/json
required

Request Body schema: application/json
required

Request Body schema: application/json
required

Request Body schema: application/json
required

Request Body schema:
application/json
required