On empty file, allow returning ERRORLEVEL 0 instead of 1

Snowflake Data Loader Feature Requests
,
1

🙋 Feature Request:

Add errorlevel / exit code override when empty file is being skipped and no other errors encountered.

:thinking: Expected Behavior:

Expect a return code of 0 when skipping a file

:hushed: Current Behavior:

Return errorlevel / exit code of 1 when skipping a file on Linux. Windows will give a return code of 0.

💁 Possible Solution:

Change the default behavior or provide an override flag

:flashlight: Context:

When Data Loader is leveraged in automated processes there may be many cases where an attempt to load an empty file would be made. In this context it should provide a successful return code.

:computer: Examples / Screenshots:

2

Tested out the return code on a local ubuntu VM and got the correct output of 0 when skipping a file. Will need to further investigate the specific setup/scenario that results in not getting 0.

cgiles@ubuntu1:~/data_loader$ ./flowobjects_data_loader --file files/empty_file.tsv --sf_account intricity --sf_username cgiles --sf_password '*********' --sf_database cgiles --sf_warehouse workshop_wh --sf_role sysadmin --sf_schema TPCH_SF10 --sf_table LINEITEM --sf_file_format upload_performance_test --sf_stage_type TABLE --header_rows 1 --remove_split_dir

FlowObjects Data Loader. Version 2.3.0
Copyright (c) 2018-2021, Intricity, LLC.  All rights reserved.
https://flowobjects.com

Licensed for use by: Chase Giles ~ cgiles@intricity.com :: expires in 177 day(s)

2021-07-06 22:32:50,682 - Application - INFO - Application started.
2021-07-06 22:32:50,682 - Application - INFO - Connecting to Snowflake Database
2021-07-06 22:32:51,357 - Application - INFO - Resuming Warehouse workshop_wh
2021-07-06 22:32:51,560 - Application - INFO - Statement executed successfully.
2021-07-06 22:32:51,560 - Application - INFO - Starting Transaction FILE_UPLOAD
2021-07-06 22:32:51,561 - Application - INFO - CREATING TRANSACTION FILE_UPLOAD
2021-07-06 22:32:51,693 - Application - INFO - Statement executed successfully.
2021-07-06 22:32:51,793 - Application - INFO - SKIPPING EMPTY FILE: empty_file.tsv
2021-07-06 22:32:52,068 - Application - INFO - Statement executed successfully.
cgiles@ubuntu1:~/data_loader$ echo $?
0
3

@cgiles thank you for the feature request and the additional testing.

We’ll put in a flag that would leave the errorlevel / exit code as 0 when an empty file is encountered, assuming of course that no other errors were encountered.

4

@cgiles Data Loader pre-release of version 2.3.1 is available which has an optional CLI flag no_error_on_empty which when used will generate an errorlevel / exit of 0 instead of 1 when an empty file is encountered.