Skip to content

Learn · Guides

Connect a data feed

Point the platform at a data source and watch the first events land.

Checked against the product on · written for people writing code

A feed is a door data comes through. When the feed is running, every record it brings in becomes an event about a watched thing, and your rules weigh it.

This takes about five minutes. At the end you will see events arriving in the live tail.

Before you start

  • An account with the admin role in the work area you are adding the feed to.
  • The source: a file location, an endpoint, a mailbox, or a telemetry stream.
  • The credential the source needs, if it needs one.

Add the feed

console.dmzagent.com/feeds

Watch

Data feeds

1All kindsAll statesAdd feed
2
NameKindScheduleLast runState
Plant telemetryOpenTelemetrystreaming4 seconds agoHealthy
Support inboxEmailevery 5 minutes2 minutes agoHealthy
Nightly exportCSVdaily 02:009 hours agoHealthy
Figure. The data feeds screen, with three feeds already running. The data feeds list with three connected feeds, their kinds, schedules and last-run times, and an Add feed button.

  1. Open Data feeds and select Add feed.1

  2. Pick the kind that matches your source.

    The kind decides which fields appear next. A field shows only when the kind and source location call for it.

  3. Fill in the source and the schedule.

    A CSV feed takes a source location of upload, url, s3, or sftp. A streaming source takes a token.

  4. Choose a sync mode.

    Reload replaces everything. Append adds new rows. Upsert matches on a key and updates in place. Replace swaps the whole set in one step.

  5. Select Test connection, then Create feed.

    The test reads a sample without writing anything.

console.dmzagent.com/feeds

Watch

Add feed

1
Kind
CSVThe shape of the data coming in.
Name
Nightly export
Source location
URLupload · url · s3 · sftp
URL
https://example.com/exports/latest.csv
Schedule
daily 02:00
Sync mode
upsertreload · append · upsert · replace
2CancelTest connectionCreate feed
Figure. The add-feed form for a scheduled CSV pull. The add-feed form with kind, name, source location, schedule and sync mode fields, and a Create feed button.

Fill in the form

  1. Set the kind, name, source and schedule.1

  2. Select Create feed.2

    The feed starts on its schedule. A streaming feed starts at once.

Watch it land

console.dmzagent.com/feeds

Watch

Plant telemetry · live

1Receiving. 42 events in the last minute.
2
TimeWatched thingKindPayload
12:04:11pump-12observationreading 41.2 F
12:04:07pump-12observationreading 40.9 F
12:04:02pump-09observationreading 38.4 F
Figure. The live tail. Each row is one event, with the watched thing it belongs to. The live feed tail showing three events arriving, each with a timestamp, subject and payload summary.

  1. Open the feed and read the banner.1

    It reports the rate of arrival for the last minute.

  2. Read the tail.2

    Each row names the watched thing the event was filed under.

Note. A watched thing is named subject:<division>:<type>:<slug>. The type is one of chat, sensor, lead, ticket, or journey, and it decides how events are grouped into a story.

When something goes wrong

What you seeWhat it meansWhat to do
The test connection fails with 401 or 403The credential is wrong or expired.Re-enter the credential. For OAuth sources, reconnect the account.
The feed runs and the tail stays emptyThe source returned no new records for the sync mode.Switch the sync mode to reload for one run, then switch back.
Events land under one watched thingThe key field that names the watched thing is missing.Set the key field in the feed mapping.
The state reads Backing offThe source rejected several pulls in a row.Read the run history on the feed for the response the source gave.
Ask the assistant to wire this in →Open your organization