Using the Bulk Export Utility

This section describes the options to use the Bulk Export Utility. To start using the Bulk Export Utility, do the following:

There are two versions of this utility, a Linux and a Windows dwexport.exe version. Determine which version to use based on what operating system you have. Only Linux commands will be used as examples in this page. For Windows commands, replace ./dwexport with dwexport.exe

  1. Open your terminal.
  2. Navigate to the location of the Bulk Export Utility folder.
  3. Enter the below utility commands using any of the following options as required. For a complete list of bulk export options, see Bulk Export Utility.
    CommandDescription
    ./dwexport -u username@telit.comExports all the records (api_logs, apps, attributes and so on) from all the collections of your default org.

    The bulk exports should be shared within the organization to avoid huge multiple bulk exports.

    ./dwexport -u username@telit.com -j = falseExports all the records from all the collections in your default org and the output is not a valid json instead it outputs each reacord as{..} {..} {..}.
    ./dwexport -u username@telit.com -o demo-noaaExports all the records from all the collections in the org DEMO-NOAA.
    ./dwexport -u username@telit.com -c api_logs,logsExports all the records from the api_logs and logs collections in your default org.
    ./dwexport -u username@telit.com -r resume.jsonThis will continue the export starting from last records that was previously exported from all the collections in the your default org.
    ./dwexport -u username@telit.com -f /my/prefered/directory/Exports all the records from all the collections in your default org to the specified directory.
    ./dwexport -u username@telit.com -s 2018-01-01T00:00:01ZExports all the records since the beginning of 2018.
    ./dwexport -u username@telit.com -e 2018-01-01T00:00:01ZExports all the records up to the beginning of 2018.
    ./dwexport -u username@telit.com -s 2018-01-01T00:00:01Z -e 2018-08-01T23:59:59ZExports all the records that are in between the range (start and end date/time) provided.
    ./dwexport -u username@telit.com -o demo-noaa -f /blah/Bulk_Export_Utility/example/ -c alarms,attributes,locations,properties,things,triggersExports all the records in the listed collections from the DEMO-NOAA org and stores the exported files in the specified directory.

    If the example folder already contains exported data, the remainder of the data can be exported by adding -r resume.json to the command.

    ./dwexport -u username@telit.com -csv=trueExports only the alarms, attributes, locations and properties in CSV format.

    Overrides the -j flag.

    ./dwexport -u username@telit.com -t 507f1f77bcf86cd799439011,507f191e810c19729de860eaExports all the properties and attributes of the two thing IDs provided.
    ./dwexport -u username@telit.com -t 507f1f77bcf86cd799439011,507f191e810c19729de860ea -p property1,property2 Exports all the property1 and property2 values of the two thing IDs provided..
    ./dwexport -u username@telit.com -t 507f1f77bcf86cd799439011,507f191e810c19729de860ea -a attribute1 Exports all the attribute1 values of the two thing IDs provided..
    After bulk exporting of all the records and it is possible to use the resume.json to get the update of the previous records.
  4. Press Enter to execute the command.
    You will be prompted to enter the password.
  5. Enter the Password.  The password is the same password used to login to the portal.  For more information about the password, see Logging into the Management Portal.
    After authentication the Bulk export starts and exports all the records as specified in the command. Once the bulk export is complete, the local web app will stop receiving updates and you will get a similar prompt as below that the export is complete.
  6. To view the number of records exported, go to the URL(http://localhost:8090/bulk_export/).  You will see a similar screen as below.
Syntax

./dwexport -u [ emailaddress ] [ -o | -c | -r | -f | -s | -e | -j | -csv] export parameters

Options Description
-u The username to login with  (The same username used to log into the Portal).
-o [OPTIONAL] The name of an Org that the user will bulk export from. If excluded, the default Org you belong to will be used.
-c [OPTIONAL] A comma-separated list of the specific collections the user wants to export. If excluded, all collections will be exported.
-r [OPTIONAL] The resume.json file to resume an export. If excluded, the export will start from the beginning. This can be used in conjunction with an end timestamp (-e)
-f [OPTIONAL] The directory in which the user wants the export files to go to. If excluded, the export files will appear in the same directory as the utility.
-s

[OPTIONAL] The starting timestamp, in RFC3339 format, for the desired range of records to export.
-e [OPTIONAL] The ending timestamp, in RFC3339 format, for the desired range of records to export. If a starting timestamp is given, but an ending one is not, the present time will be used. If an ending timestamp is given, but a starting is not, a time of '0' will be used.

While using -s or -e, the date range values specified are encoded into a hexadecimal format and compared against the record IDs. Each records ID contains the date/time when the record is created. This can be different from any other recorded timestamp provided to the command that created the record
-l [OPTIONAL] Specifies the time range of last data values to export. i.e. last 7d, last 12h etc.
-j [OPTIONAL] Untoggles the JSON array output.
-csv [OPTIONAL] Toggle the output to be in CSV format.

Only supported for alarms, attributes, locations and properties.

-t [OPTIONAL] A comma separated list of up to five things IDs to export property and/or attributes for.

This will automatically be exported in csv format and does not support resume functionality.

-p [OPTIONAL] A comma separated list of the properties to be exported for things in the thing list provided.
-a [OPTIONAL] A comma separated list of the attributes to be exported for things in the thing list provided.

If you do not include any json or csv flags, it will automatically download everything as a valid json array [ {..}, {..}, {..} ], but if you do not want it to be a valid json array, you can do -j=false and it will output each reacord as {..} {..} {..}.

Related Topics Link IconRelated Topics