This is a Singer tap that produces JSON-formatted data following the Singer spec.
See the getting-started guide:
https://github.com/singer-io/getting-started
This section dives into basic usage of tap-zuora by walking through extracting
data from the api.
Create a config file containing the zuora credentials, e.g.:
{
"api_type": "REST",
"username": "abcd",
"password": "1234",
"sandbox": "true",
"european": "false",
"start_date": "2022-10-10T00:00:00Z"
}The tap can be invoked in discovery mode to find the available zuora objects.
$ tap-zuora --config config.json --discover
A discovered catalog is output, with a JSON-schema description of each table. A source table directly corresponds to a Singer stream.
Sync uses the catalog supplied to the tap; upgrading the tap does not refresh a
connection's saved catalog. After upgrading, run discovery again and update the
connection's saved catalog, preserving its stream and field selections. The new
catalog makes Deleted nullable for streams that support deletion tracking and
omits it for streams that do not.
After the catalog update, verify the target schema for affected tables. The tap
emits the updated Singer schema but does not alter target database constraints.
Whether an existing target column is made nullable or removed automatically is
target-specific. If it remains NOT NULL or otherwise conflicts with the new
schema, apply the target's supported schema migration manually before resuming
the load.
In sync mode, tap-zuora consumes the catalog and looks for streams that have been
marked as selected in their associated metadata entries.
Redirect output from the tap's discovery mode to a file so that it can be modified:
$ tap-zuora --config config.json --discover > catalog.jsonThen edit catalog.json to make selections. The stream's metadata entry (associated
with "breadcrumb": []) gets a top-level selected flag, as does its columns' metadata
entries.
[
{
"breadcrumb": [],
"metadata": {
"valid-replication-keys": [
"UpdatedOn"
],
"table-key-properties": [
"Id"
],
"forced-replication-method": "INCREMENTAL",
+ "selected": "true"
}
},
]Optionally, also create a state.json file. current_stream is an optional attribute used for identifying the last object to be synced in case the job is interrupted mid-stream. The next run would begin where the last job left off.
{
"current_stream": "Account",
"bookmarks": {
"AchNocEventLog": {"UpdatedOn": "2022-10-15T00:00:00Z"},
"PaymentMethodTransactionLog": {"TransactionDate": "2022-10-15T00:00:00Z"}
}
}With a catalog.json that describes field and table selections, the tap can be invoked in sync mode:
$ tap-zuora --config config.json --catalog catalog.json --state state.jsonMessages are written to standard output following the Singer specification. The resultant stream of JSON data can be consumed by a Singer target.
Copyright © 2017 Stitch