July 1, 2021, 9:50pm
🙋 Feature Request:
Add errorlevel / exit code override when empty file is being skipped and no other errors encountered.
Expect a return code of 0 when skipping a file
Return errorlevel / exit code of 1 when skipping a file on Linux. Windows will give a return code of 0.
💁 Possible Solution:
Change the default behavior or provide an override flag
When Data Loader is leveraged in automated processes there may be many cases where an attempt to load an empty file would be made. In this context it should provide a successful return code.
Examples / Screenshots:
July 6, 2021, 10:39pm
Tested out the return code on a local ubuntu VM and got the correct output of 0 when skipping a file. Will need to further investigate the specific setup/scenario that results in not getting 0.
cgiles@ubuntu1:~/data_loader$ ./flowobjects_data_loader --file files/empty_file.tsv --sf_account intricity --sf_username cgiles --sf_password '*********' --sf_database cgiles --sf_warehouse workshop_wh --sf_role sysadmin --sf_schema TPCH_SF10 --sf_table LINEITEM --sf_file_format upload_performance_test --sf_stage_type TABLE --header_rows 1 --remove_split_dir
FlowObjects Data Loader. Version 2.3.0
Copyright (c) 2018-2021, Intricity, LLC. All rights reserved.
Licensed for use by: Chase Giles ~ firstname.lastname@example.org :: expires in 177 day(s)
2021-07-06 22:32:50,682 - Application - INFO - Application started.
2021-07-06 22:32:50,682 - Application - INFO - Connecting to Snowflake Database
2021-07-06 22:32:51,357 - Application - INFO - Resuming Warehouse workshop_wh
2021-07-06 22:32:51,560 - Application - INFO - Statement executed successfully.
2021-07-06 22:32:51,560 - Application - INFO - Starting Transaction FILE_UPLOAD
2021-07-06 22:32:51,561 - Application - INFO - CREATING TRANSACTION FILE_UPLOAD
2021-07-06 22:32:51,693 - Application - INFO - Statement executed successfully.
2021-07-06 22:32:51,793 - Application - INFO - SKIPPING EMPTY FILE: empty_file.tsv
2021-07-06 22:32:52,068 - Application - INFO - Statement executed successfully.
cgiles@ubuntu1:~/data_loader$ echo $?
@cgiles thank you for the feature request and the additional testing.
We’ll put in a
flag that would leave the
exit code as
when an empty file is encountered, assuming of course that no other errors were encountered.
@cgiles Data Loader pre-release of version 2.3.1 is available which has an optional CLI flag
no_error_on_empty which when used will generate an
when an empty file is encountered.