Sprinkle supports a wide range of data sources. A list of data sources will be shown in the Create datasource tab. In this case, on selecting the FTP tab, it requires the user to name it.
Post naming, it routes the user to the configuration page.Now the user needs to fill in the credentials such as Host,Port,User,Password before testing the connection and updating.
Optimising Incremental Ingestion in FTP datasource
Also users can select Yes or No to Optimize Incremental Ingestion. If optimize is Yes, all the datasets will undergo full ingestion on every Sunday or every night. If optimize is No, data will be ingesting incrementally and it never goes under complete ingestion.
Next in the datasets tab, users can add tables. In this step the user needs to give the table name, mode of ingestion, file type, directory path. After all these are set, the user can click on create.
Run and Schedule
In the Run and Schedule tab, the concurrency (number of tables that can run in parallel, a maximum of 7) can be set preferentially before running the job. The status of the job will be updated in the tab below once it’s complete. The jobs can also be set to run automatically by enabling autorun. By default, the frequency is set to every night. Frequency can be changed by clicking on More --> Autorun-->Change Frequency.