Setting up Marketo Lead Activity Data Stream with Adobe I/O Events

These instructions describe how to set up and get started using Adobe I/O Events for Marketo lead-based activities and change events.

Introduction

The Lead Activity Data Stream provides near real-time streaming of audit tracking Lead Activities where large volumes of Lead Activities can be sent to a customer’s external system. Streams enable customers to effectively audit Lead-related events, usage patterns, provide views into Lead changes and trigger processes and workflows based upon the different types of Lead Events.

Setup Lead Activity Data Stream in Marketo

Data Streams are available to those that have purchased a Marketo Engage Performance Tier Package. Once a Performance Tier agreement is in place, work with your TAM and Customer Engineering team to enable this Data Stream for your subscription.

We typically just need to know the MunchkinId for the subscription, and the associated Adobe OrgId, which enables access to the Adobe IO Developer Console.

For the Lead Activity Data Stream, we will also need to know the specific types of events that you would like to subscribe to, which can be found in the Event List below.

Setup Adobe I/O

See Getting Started with Adobe I/O Events

For basic instructions for this use case, starting from console.adobe.io:

When prompted, click the designated button to proceed

• Select Create new project

  

• Select Add event

  

• Filter by Experience Cloud

• Select Marketo Lead Activity Data Stream

  

• Subscribe to the lead-driven change events of your choosing

  

• Set up OAuth Server-to-Server Credentials

  

• Set up Event Registration

  

  • Provide a name and description for this event subscription

  • Optionally choose whether to enable Webhook or Runtime action

    • Enable Webhook

      • We recommend batch over single webhooks
      • For Webhook URL a public https endpoint must be provided
      • The endpoint must be able to handle get and post requests
      • The get request must respond with the challenge query if it exists
      • The post request must respond that it received the message or the webhook will re-attempt to send several times before giving up and automatically disabling the webhook sends

    • Enable Runtime action

      • See Setting up your Runtime Environment
      • Select a pre-made runtime action/runtime namespace

• After Saving

  

  • Verify that the Status is Active
  • If Webhook was selected, verify that it successfully passed the challenge without errors

Developer Guidelines

When setting up a project to subscribe to events, there are three ways to interact with those event subscriptions in order to receive the events. The first is Journaling, which provides a pull model in which events can be pulled via API and stores up to 7 days of past events. The second is Webhooks, which can be configured to send events either as single events or batched to a webhook endpoint in near real-time with the event occurrence. Third is Runtime, where you can set up your own custom function within Adobe that events will automatically run through near-real time.

Journaling

Getting Started with Journaling

Important Takeaways:

• Stores up to 7 days of history
• Can be iterated through from any previous event within the history
• Will still receive and store events even if webhook is failing
• Useful for fetching events that were missed due to webhook issues or for a pulling mechanism instead of webhook push

Webhooks

Getting Started with Adobe I/O Events Webhooks

Webhook Endpoint Requirements:

• Handle GET and POST requests

• Respond with a 200-type response within a reasonable time period

• Challenge Request

  • GET request with challenge query parameter
  • Must respond with value of challenge query parameter

• Webhook Events

  • POST request with JSON data body with one or more events
  • Recommended to set up webhook as batch

Event Data Structure

Events are structured in JSON format using the CloudEvents spec

Example Event (batch)

[
  {
    "event_id": "ec2c6092-1e94-45e9-b2dd-0b6bad0baa6f",
    "specversion": "1.0",
    "type": "com.adobe.platform.marketo.activity.standard.openemail",
    "source": "urn:marketo_activity_stream",
    "id": "a1764870-4f4b-4f8c-82d6-59b0fff2d39c",
    "time": "2024-12-17T18:39:33Z",
    "datacontenttype": "application/json",
    "data": {
      "munchkinId": "123-ABC-456",
      "leadId": "1234",
      "activityDate": "2024-12-17T18:39:33Z",
      "activityTypeId": 10,
      "activityType": "Open Email",
      "activityLogItemId": 1234567890,
      "primaryAttributeValueId": 1234,
      "primaryAttributeValue": "Attribute Value",
      "attributes": [
        {
          "name": "Campaign Run ID",
          "dataType": "integer",
          "value": 1234
        },
        {
          "name": "Platform",
          "dataType": "string",
          "value": "Platform Name"
        },
        {
          "name": "Device",
          "dataType": "string",
          "value": "Device Name"
        },
        {
          "name": "Mailing ID",
          "dataType": "object",
          "value": 1234
        },
        {
          "name": "User Agent",
          "dataType": "string",
          "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
        },
        {
          "name": "Is Mobile Device",
          "dataType": "boolean",
          "value": false
        }
      ]
    },
    "recipient_client_id": "<your_client_id>"
  },
  {
    "event_id": "97980214-3c8f-4964-88a3-75b42833a1f9",
    "specversion": "1.0",
    "type": "com.adobe.platform.marketo.activity.standard.visitwebpage",
    "source": "urn:marketo_activity_stream",
    "id": "a434d7ee-fe91-4b96-aaf2-61f1c6f4c46e",
    "time": "2024-12-17T18:42:49Z",
    "datacontenttype": "application/json",
    "data": {
      "munchkinId": "123-ABC-456",
      "leadId": "1234",
      "activityDate": "2024-12-17T18:42:49Z",
      "activityTypeId": 1,
      "activityType": "Visit Webpage",
      "activityLogItemId": 1234567890,
      "primaryAttributeValueId": 1234,
      "primaryAttributeValue": "Attribute Value",
      "attributes": [
        {
          "name": "Client IP Address",
          "dataType": "string",
          "value": "11.22.33.44"
        },
        {
          "name": "Query Parameters",
          "dataType": "string",
          "value": ""
        },
        {
          "name": "Referrer URL",
          "dataType": "string",
          "value": ""
        },
        {
          "name": "User Agent",
          "dataType": "string",
          "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
        },
        {
          "name": "Webpage ID",
          "dataType": "object",
          "value": 1234
        },
        {
          "name": "Webpage URL",
          "dataType": "string",
          "value": "/lp/123-ABC-456/MyLandingPage.html"
        }
      ]
    },
    "recipient_client_id": "<your_client_id>"
  }
]

Example Event (single)

{
  "event_id": "4109a990-da28-4c5b-bad9-22df8903ed9a",
  "specversion": "1.0",
  "type": "com.adobe.platform.marketo.activity.standard.clicklink",
  "source": "urn:marketo_activity_stream",
  "id": "570b455e-203f-4983-91a8-5a03f597fda5",
  "time": "2024-12-17T18:43:56Z",
  "datacontenttype": "application/json",
  "data": {
    "munchkinId": "123-ABC-456",
    "leadId": "1234",
    "activityDate": "2024-12-17T18:43:56Z",
    "activityTypeId": 3,
    "activityType": "Click Link",
    "activityLogItemId": 1234567890,
    "primaryAttributeValueId": 1234,
    "primaryAttributeValue": "Attribute Value",
    "attributes": [
      {
        "name": "Link ID",
        "dataType": "object",
        "value": 1234
      },
      {
        "name": "Client IP Address",
        "dataType": "string",
        "value": "11.22.33.44"
      },
      {
        "name": "Query Parameters",
        "dataType": "string",
        "value": ""
      },
      {
        "name": "Referrer URL",
        "dataType": "string",
        "value": "https://marketo.com/lp/123-ABC-456/MyLandingPage.html"
      },
      {
        "name": "User Agent",
        "dataType": "string",
        "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
      },
      {
        "name": "Webpage ID",
        "dataType": "object",
        "value": 1234
      }
    ]
  },
  "recipient_client_id": "<your_client_id>"
}

Data Field Definitions

Many of the fields are common across the different types of events. The data object will contain the specific details of the event. In particular, the attributes array will contain a subset of all possible attributes (or none at all!) depending on the event type, and the event context.

The type field will always be the Event Type name, all lowercase with spaces removed, appended to the string "com.adobe.platform.marketo.activity.standard". For example, for the "Open Email" Event Type, the type string will be "com.adobe.platform.marketo.activity.standard.openemail".

All customer-defined activity types will be delivered with an identical event type "com.adobe.platform.marketo.activity.custom", with no suffix.

The data field contains the following data:

And finally, all Attribute objects in the attributes array will contain the following fields:

Event List

These are all of the currently supported Lead Activity Event Types in alphabetical order. If you have a "firehose" endpoint configured that is subscribed to more than one event type, you can identify the type of event in three different ways.

Here are examples using the "Open Email" event type:

1. Text match on the activityType field under the data object (e.g. payload.data.activityType = "Open Email")
2. Number match on the actitityTypeId field under the data object (e.g. payload.data.activityTypeId = 10)
3. Text match on the type field with the concatenated, lower-cased Event Type (e.g. payload.type = "com.adobe.platform.marketo.activity.standard.openemail")

Note - This is a snapshot listing of most available events. There may be some events that don't show up or no longer exist.

Debug

Debug Tracing

Once you have successfully completed your setup and event subscription registration, events should start being stored in the journal. In addition, if you have webhooks or runtime set up, the events will go through those flows. From the project's page in the event registration details, you should see a tab for Debug Tracing. For webhooks, this will show a record of failed and successful challenge attempts as well as webhook attempts. Each request includes the request/response details to help debug.

There is also an Event Browser tab which lets you manually look at events in the Journal from the Developer Console UI