data_intake Array

The data_intake array is optional and only used to define how to set up inbound data that is pushed (using a webhook) or pulled (using API polling) into ActiveCampaign from your system. The top-level data_intake array contains one or more data_intake objects. An application can define more than one data_intake, each dedicated to a specific inbound event. Each data_intake object describes how to manage inbound data for a specific event or external resource.

There are currently two types of data_intake objects supported, webhook and polling. A webhook-based data_intake defines how the ActiveCampaign platform will create, delete, and optionally update a webhook event in the system you are integrating. A polling-based data_intake is an alternative solution for executing API requests at scheduled intervals.

🚧

Webhook vs. Polling

If webhooks are available, this is the preferred integration method. Webhooks typically offer a more "real-time" experience and require less processing overhead. Only use polling if the system you are integrating does not support webhooks, or none of the existing webhooks support your integration use case.

Webhook

A webhook is a way of registering a callback between two systems. Events are pushed from one to another, often as they occur in near real-time. A data_intake defined as "type": "webhook" will create a new webhook URL in ActiveCampaign to accept incoming data. This URL is passed to the external API to register the webhook. When a webhook event occurs, the integrating system will post data to the registered webhook URL hosted at ActiveCampaign. A webhook data_intake object defines how to create, delete, and (optionally) update webhook registrations using your platform's API.

A webhook data_intake object contains the following properties and objects.

name Property

The identifier for the data_intake object. This name must be unique across all data_intake objects within the same app. It is recommended to use the event name defined by the webhook API. For example, "name": "form_webhook" or "name": "event.account.created".

type Property

The type of data_intake object. For a webhook, the value must be set to "webhook".

scope Property

The scope property defines the granularity of webhooks supported by the integrating system. The scope can be set to one of the following values.

📘

When a webhook scope is connection or application, additional information is required to process the incoming data.

resource_id Property

The resource_id property is required for webhooks with a "connection" or "application" scope. It is a !jq comand that helps the integration locate the correct field within the webhook payload to use as the external resource id.

This value must be present in your webhook payload for our system to deliver data to the intended destination. The value of this field should match the resource id defined in the describe_selection.

account_id Property

The account_id property is similar to the resource_id but is only required for "application" webhooks that contain data for multiple users.

create Object

The create object defines how to create a webhook on your system so event data can be pushed to ActiveCampaign. The create object defines how the ActiveCampaign platform will call your system's API to register the webhook URL. The new URL created in ActiveCampaign to receive the inbound data is available using the ${webhook.url} variable.

In the following example, the create object uses the !pipe command to chain together several commands. The !http command is used to submit the webhook URL in the body of an HTTP POST. The !jq command parses the response returned by the API. The !save command is used to store the webhook id to use in the future to update or delete the webhook.

"create": {
  "!pipe": [
    "!http": {
      "body": {
        "url": "${webhook.url}"
      },
      "method": "POST",
      "path": "/path/to/your/webhook/create/api"
    }, 
    {
      "!jq": ".body | {webhook_id: .id}"
    },
    {
      "!save": {
        "scope": "workflow"
      }
    }
  ]
}

📘

The scope used in this example depends on the scope defined for the data_intake. This value may be different for your system.

update Object (optional)

The update object defines how to update an existing webhook in your system. The associated webhook may need to be updated to continue receiving data, such as when a customer changes the workflow data mapping. It is optional and depends on the publisher API definition. The following example uses the webhook_id stored in the workflow scope from the previous create example.

"update": {
  "!http": {
    "method": "PUT",
    "path": "/path/to/your/webhook/update/api",
    "body": {
      "id": "${data.workflow.webhook_id}"
    }
  }
}

delete Object

The delete step defines how to remove a previously created webhook to stop pushing unneeded event data to ActiveCampaign. The following example uses the webhook_id stored in the workflow scope from the previous create example.

"delete": {
    "!http": {
      "method": "DELETE",
      "path": "/path/to/your/webhook/delete/${data.workflow.webhook_id}"
    }
  }

verification Object (optional)

Webhook verification is a way to evaluate the validity and authenticity of a webhook URL. This enables a third-party system to register and send webhook payloads with confidence. Many API's (including Asana, Meta, LinkedIn, Zoom, and WhatsApp) require validation and periodic re-validation to ensure that a webhook is genuine, and originates from ActiveCampaign.

Your CX application can support webhook verification by adding a verification section to your data_intake definition. For example, please see the sample LinkedIn webhook verification definition below:

"data_intake": [
    {
      "name": "event_lead_created",
      "type": "webhook",
      "scope": "application",
      "verification": {
        "_comment": "LinkedIn requires an initial manual handshake. Then will automatically revalidate every 2 hours",
        "!switch": {
          "jq": "if .query_params.challengeCode then 0 else 1 end",
          "cases": [
            {
              "!pipe": [
                {
                  "!jq": "{value_to_encrypt: .query_params.challengeCode}"
                },
                {
                  "!sha": {
                    "secret": "12345_thisIsASecret_54321",
                    "algorithm": "sha256"
                  }
                },
                {
                  "!respond": {
                    "code": 200,
                    "headers": {
                      "Content-Type": "application/json"
                    },
                    "body": {
                      "challengeCode": "${piped_content.1.value_to_encrypt}",
                      "challengeResponse": "${piped_content.2.hash}"
                    }
                  }
                }
              ]
            },
            {
              "_comment": "Process webhook payload as normal",
              "!respond": {
                "process_payload": true
              }
            }
          ]
        }
      }
    }
  ]

Types of webhook verification HTTP requests:

• Validation request: The first HTTP request to verify the webhook URL before registration.
• Re-validation request: The subsequent HTTP requests to verify the webhook URL after registration.
• Payload request: The HTTP request with the webhook payload, used to create the Contact in ActiveCampaign.

All three of the above HTTP requests send data to ActiveCampaign via the webhook URL, with subtle variations:

• Third-party system webhook URL validation flow leverage the application's client secret as the secret key along with a universally known hash algorithm such as the HMACSHA256 algorithm to generate and/or validate your CX application's response to a challenge code.
• Third-party systems normally send the challenge code as a part of the validation or re-validation request headers, query parameters, or body.

📘

The verification step accepts any of the commands available. We recommend using the !switch command with a !pipe containing !jq, !sha and !respond commands.

verification Object Example Explanation:

1. In the above-mentioned example LinkendIn verification object, we defined a !switchcommand with a!pipecontaining!jq, !sha and !respond commands.
2. You can access the challenge code value in the jq property of the !switch command as follows:
   • .headers.challengeCode when the challenge code is sent as a header
   • .query_params.challengeCode when the challenge code is sent as a query
   • .body.challengeCode when the challenge code is sent in the body
3. Inside a !pipe command, use !jq command to extract the value_to_encrypt (challenge code) from the incoming validation or re-validation request.

🚧

The challenge code key should match the key sent to the webhook URL.

If http://test-webhook-url?challenge_code=12345 is the webhook URL, then the challenge code value can be accessed in the switch command’s jq property as: .query_params.challenge_code

4. Use !sha command to generate the challenge code hash.
5. Use !respond command to define the HTTP response expected by the third-party system.

Polling

Not all APIs or inbound sources support webhooks. A data_intake defined as "type": "polling" can request data from your system at regular intervals. A polling configuration has specific lifecycle events for state management, initial_setup, pre_poll, request, and post_poll. request is the only event that is required. The other events may be needed for polling to work correctly with your system.

Polling State Management

It is important to understand that polling is defined by lifecycle events in order to maintain a consistent state between events. State variables allow the platform to persist data between sync executions, as well as between its lifecycle events and request iterations.

With any of the following lifecycle events, the data returned from the last execution is examined, and if a polling_state key is found its data will be saved and made available to subsequent lifecycle events.

When writing to the polling_state key it is essential to include all known keys and their current value. Saving the polling_state replaces the entire object, not just individual keys.

Example of returning the current polling_state without change: { polling_state: ${polling_state} }

Example of returning an updated polling_state: { polling_state: { keyA: ${polling_state.keyA}, keyB: "New Value"} }

The current polling state can be accessed in your commands using the ${polling_state} variable. To access a specific value you stored in the polling state, for example, a key named last_update, you can use ${polling_state.last_update}.

type Property

Defines the type of data_intake. For API polling, the value must be set to "polling".

name Property

The unique identifier for the data_intake. It is recommended to include the name of the resource requested from your API. For example, "name": "import-customers".

initial_setup Object

Polling takes advantage of lifecycle events and a special polling_state object. The initial_setup command runs only once when a customer configures or resets a workflow that uses polling. The most common use of this command is to initialize any polling_state keys and values for a first run. In some cases, that may involve fetching information from your system. All data returned by initial_setup except for polling_state will be ignored.

"initial_setup": {
  "!pipe": [
    {
      "!http": {
        "method": "GET",
        "url": "https://example.api/object/${custom_data::form_field.value}"
      }
    },
    {
      "!jq": ".body | { polling_state: { keyA: .keyA, keyB: \"fixed start value\" }}"
    },
  ]
},

pre_poll Object

Similar to initial_setup, pre_poll is used to prepare any polling_state data for the current execution cycle. This command will run once per cycle, before the request command. All data returned by pre_poll except for polling_state will be ignored.

"pre_poll": {
  "!pipe": [
    {
      "!http": {
        "method": "GET",
        "url": "https://example.api/object/${custom_data::form_field.value}"
      }
    },
    {
      "!jq": ".body | { polling_state: { keyA: .keyA, keyB: ${polling_state::keyB}}}"
    },
  ]
},

request Object

The request is the foundation for the polling process. The request is used to:

• Make outbound requests, looping until an exit condition
• Update state variables for tokens/parameters based on each response
• Return a final result to be used by the application workflow

"request": {
  "!pipe": [
    {
      "!http": {
        "method": "GET",
        "url": "https://example.api/object/${custom_data.form_field.value}?keyA=${polling_state.keyA}"
      }
    },
    {
      "!jq": ".body | { results: .data, polling_state: { keyA: .keyA, keyB: ${polling_state.keyB}, iterate: false} }"
    },
  ]
},

Because the request is iterable, a formatted object needs to be returned as the final command result. This object must return a results object, an iteration flag, and the updated polling_state object. In the previous example, a !jq command is used to create and return the required results object as the final step of the request.

• results: A jq selection to return an array of data from the !http response. Each item in the array will be passed to the workflow’s data_pipline.
• iterate: Any jq logic that results in a boolean to let the execution know if more data needs to be fetched, such as a paginated response.
• polling_state: The updated polling state will be available to other commands and events.

🚧

Pagination parameters, syncTokens, and other response-driven changes to variables are to be self-managed in the state. Do not rely on the pagination mechanism defined in the top-level api object, if any, for polling requests.

request Exit Conditions

The request command will complete and re-execute until one of the following exit conditions is met.

• response.iterate == false
• Total requests executed in this sync cycle exceed 100
• Total of response.results[] from combined requests in this sync cycle exceed 1,000
• response.results[] is null or empty
• response.results matches the previous request

post_poll Object

The post_poll configuration is run once per cycle after all request iterations have been completed. This lifecycle event is useful for cleanup, or perhaps an outbound request to notify a 3rd-party API endpoint. It could also be used to replicate the initial_setup for polls that don't maintain a long-term state. The format of post_poll is the same as pre_poll and initial_setup.