This is a Singer tap that produces JSON-formatted data following the Singer spec.
This tap:
-
Pulls raw data from the freshdesk API.
-
Extracts the following resources from Freshdesk::
-
Outputs the schema for each resource
-
Incrementally pulls data based on the input state
- Primary keys: ['id']
- Replication strategy: INCREMENTAL
- Primary keys: ['id']
- Replication strategy: INCREMENTAL
- Primary keys: ['id']
- Replication strategy: INCREMENTAL
- Primary keys: ['id']
- Replication strategy: INCREMENTAL
- Primary keys: ['id']
- Replication strategy: INCREMENTAL
- Primary keys: ['id']
- Replication strategy: INCREMENTAL
- Primary keys: ['id']
- Replication strategy: FULL_TABLE
- Primary keys: ['id']
- Replication strategy: FULL_TABLE
- Primary keys: ['id']
- Replication strategy: FULL_TABLE
Freshdesk recommends avoiding page numbers above 500.
The tap therefore permits at most 500 page requests per pagination run,
including each child stream's run for a parent object. Empty or partial pages
terminate pagination normally, including on page 500. If page 500 is full and
another page would be needed, the tap logs an error and raises
freshdeskPaginationLimitError with the stream name and limit. The sync fails
instead of silently truncating data or requesting pages indefinitely.
This safety ceiling does not change the existing recovery for the Tickets API's 300-page limit.
-
Install
Clone this repository, and then install using setup.py. We recommend using a virtualenv:
> virtualenv -p python3 venv > source venv/bin/activate > python setup.py install OR > cd .../tap-freshdesk > pip install -e .
-
Get your Freshdesk API Key
Login to your Freshdesk account, navigate to your profile settings page, and save "Your API Token", you'll need it for the next step.
-
Create your tap's
config.jsonfile. The tap config file for this tap should include these entries: Create a JSON file calledconfig.jsoncontaining the api token you just found and the subdomain to your Freshdesk account. Setdomainto only the account subdomain label (for example,subdomainforsubdomain.freshdesk.com). Do not include the scheme, a path, or the full hostname.{ "api_key": "your-api-token", "domain": "subdomain", "start_date": "2017-01-17T20:32:05Z" }Optionally, also create a `state.json` file. `currently_syncing` is an optional attribute used for identifying the last object to be synced in case the job is interrupted mid-stream. The next run would begin where the last job left off. ```json { "currently_syncing": "agents", "bookmarks": { "companies": "2019-09-27T22:34:39.000000Z", "contacts": "2019-09-28T15:30:26.000000Z" } } -
Run the Tap in Discovery Mode This creates a catalog.json for selecting objects/fields to integrate:
tap-freshdesk --config config.json --discover > catalog.jsonSee the Singer docs on discovery mode here
-
Run the Tap in Sync Mode (with catalog) and write out to state file
For Sync mode:
> tap-freshdesk --config tap_config.json --catalog catalog.json > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
To load to json files to verify outputs:
> tap-freshdesk --config tap_config.json --catalog catalog.json | target-json > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
To pseudo-load to Stitch Import API with dry run:
> tap-freshdesk --config tap_config.json --catalog catalog.json | target-stitch --config target_config.json --dry-run > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
-
Test the Tap
While developing the freshdesk tap, the following utilities were run in accordance with Singer.io best practices: Pylint to improve code quality
> pylint tap_freshdesk -d missing-docstring -d logging-format-interpolation -d too-many-locals -d too-many-argumentsPylint test resulted in the following score:
Your code has been rated at 9.67/10
> tap-freshdesk --config tap_config.json --catalog catalog.json | singer-check-tap > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
Unit tests may be run with the following.
python -m pytest --verboseNote, you may need to install test dependencies.
pip install -e .'[dev]'
Copyright © 2017 Stitch