MySQL :: HeatWave User Guide :: 5.8.7 File Name Format for Exported Query Result Files

HeatWave User Guide / ... / File Name Format for Exported Query Result Files

5.8.7 File Name Format for Exported Query Result Files

Exported query results are in CSV, Parquet, or JSON (As of MySQL 9.3.1) format. The number of output files and size of individual files depends on the amount of data generated by the query.

The names for the output files are auto-generated and based on the following format:

file_prefix/universal unique number/nnode_ID-tthread_ID-cChunk_ID.compression.file_format

File Name Parameters

File_prefix: The user-provided prefix for the file name.
Universal unique number: The unique ID generated for each query.
Node_ID: The node ID of the file.
Thread_ID: The thread ID of the file.
Chunk_ID: The chunk ID of the file.
Compression: The file compression type selected by the user.
File_format: The output file format selected by the user.

Export File Prefix

After the query results are exported to Object Storage, the export file prefix is added to the rpd_query_stats table. See The rpd_query_stats Table. The export file prefix includes the file prefix and the unique ID.

To view the export file prefix for all export queries, run the following command:

mysql>SELECT QUERY_ID,EXPORT_FILE_PREFIX from performance_schema.rpd_query_stats WHERE EXPORT_FILE_PREFIX is NOT NULL;

To view the export file prefix for the most recent export, run the following command:

mysql>SELECT /*+set_var(use_secondary_engine=off) */  EXPORT_FILE_PREFIX FROM performance_schema.rpd_query_stats 
      WHERE EXPORT_FILE_PREFIX is NOT NULL ORDER BY QUERY_ID DESC LIMIT 1;

Sample Object Names

In the following example, these object names are generated for the user-defined prefix:

outfile-basic
universal unique number: 976b11a5-4499-11ef-acb7-0200171701b

The export file prefix outfile-basic/976b11a5-4499-11ef-acb7-0200171701b is appended to the filenames.

Refer to the following file names:

universal unique number: 976b11a5-4499-11ef-acb7- 0200171701b (unique number generated for each query), 
compression: snappy,
format : parquet
  - outfile-basic/976b11a5-4499-11ef-acb7-0200171701b/n0-t1-c1.snappy.parquet
  - outfile-basic/976b11a5-4499-11ef-acb7-0200171701b/n0-t0-c1.snappy.parquet
  - outfile-basic/976b11a5-4499-11ef-acb7-0200171701b/n0-t0-c2.snappy.parquet
  - outfile-basic/976b11a5-4499-11ef-acb7-0200171701b/n0-t1-c2.snappy.parquet

PREV HOME UP NEXT