fre.app.remap_pp_components.remap_pp_components module

Remap/move components that will be post-processed from one convention, such as history files, to an updated output directory structure

fre.app.remap_pp_components.remap_pp_components.chunk_to_legacy(iso_dura: str) → str

Print Bronx-style frequency given an ISO8601 duration

Parameters:

iso_dura (str) – chunk

Returns:

bronx-style frequency

Return type:

str

fre.app.remap_pp_components.remap_pp_components.create_dir(out_dir: str, comp: str, freq: str, chunk: str, ens: str, dir_ts: bool) → str

Create the output directory structure

Parameters:

  • out_dir (str) – output directory

  • comp (str) – component that will be post-processed

  • freq (str) – frequency

  • chunk (str) – chunk

  • ens (str) – ensemble member

  • dir_ts (boolean) – directory time series workaround

Returns:

output directory structure

Return type:

str

fre.app.remap_pp_components.remap_pp_components.freq_to_date_format(iso_freq: str) → str

Print legacy Bronx-like date template format given a frequency (ISO 8601 duration)

Parameters:

iso_freq (str) – frequency

Raises:

ValueError – if there is an unknown frequency

Returns:

legacy bronx-like date template

Return type:

str

fre.app.remap_pp_components.remap_pp_components.freq_to_legacy(iso_dura: str) → str

Print Bronx-style frequency given an ISO8601 duration

Parameters:

iso_dura – frequency

Raises:

ValueError – if ISO duration can not be converted to Bronx-style frequency

Returns:

bronx-style frequency

Return type:

str

fre.app.remap_pp_components.remap_pp_components.get_chunk(comp_info: dict) → List[str]

Return the chunk size

Parameters:

comp_info (dict) – dictionary of information about requested component

Returns:

list of chunk sizes

Return type:

list

fre.app.remap_pp_components.remap_pp_components.get_freq(comp_info: dict) → List[str]

Return the frequency

Parameters:

comp_info (dict) – dictionary of information about requested component

Returns:

list of frequencies

Return type:

list

fre.app.remap_pp_components.remap_pp_components.get_sources(comp_info: dict, product: str) → List[str]

Retrieve source name for a component

Parameters:

  • comp_info (dict) – dictionary of information about requested component

  • product (str) – static, ts, or av

Returns:

list of sources associated with a pp component

Return type:

list

fre.app.remap_pp_components.remap_pp_components.get_varlist(comp_info: dict, product: str, req_source: str, src_vars: dict) → List[str]

Retrieve variables listed for a component; save in dictionary for use later

Parameters:

  • comp_info (dict) – dictionary of information about requested component

  • product (str) – static, ts, or av

  • req_source (str) – source being looped over

  • src_vars (dict) – dictionary of variables associated with source name

Raises:

ValueError – if there are no static sources, but the product is set to static

Returns:

list of variables associated with source name

Return type:

list

fre.app.remap_pp_components.remap_pp_components.remap_pp_components(input_dir: str, output_dir: str, begin_date: str, current_chunk: str, product: str, component: str, copy_tool: str, yaml_config: str, ts_workaround: bool, ens_mem: str)

Remap netcdf files to an updated output directory structure

Parameters:

  • input_dir (str) – input directory

  • output_dir (str) – output directory

  • begin (ISO string format) – date to begin post-processing

  • current_chunk (str) – current chunk to post-process

  • component (str) – component that will be post-processed

  • product (str) – variable to define time series or time averaging

  • ts_workaround (boolean) – time series workaround

  • ens_mem (str) – ensemble member number

  • copy_tool (str) – tool to use for copying files

  • yaml_config (str) – yaml configuration file

Raises:

ValueError –

  • if no input files are found

  • if no offline diagnostic file is found

fre.app.remap_pp_components.remap_pp_components.search_files(product: str, var: list, source: str, freq: str, current_chunk: str, begin: str) → List[str]

Pattern match and search for the correct files in the chunk directory

Parameters:

  • product (str) – ts, av or static

  • var (list of strings) – variables

  • source (str) – source history files for post-processed component

  • begin (ISO string format) – date to begin post-processing

  • current_chunk (str) – current chunk to post-process

  • freq (str) – frequency

Raises:

ValueError –

  • if specified variable can not be found

  • if product is not ts or av when frequency is not 0

Returns:

list of files found

Return type:

array

fre.app.remap_pp_components.remap_pp_components.truncate_date(date: str, freq: str) → str
Print a date string to a truncated precision.

  • Accepts a date and frequency

  • Outputs a date string with suitably reduced precision

  • Test cases: ‘19790101T0000Z P1D’, ‘19800101T0000Z P1M’, ‘19790101T0000Z PT0.5H’

  • Output using cylc (shared.sh calls in job logs): ‘19790101’, ‘198001’, ‘1979010100’

Parameters:

  • date (ISO string format) – date to begin post-processing

  • freq (str) – frequency

Returns:

truncated date string

Return type:

str

fre.app.remap_pp_components.remap_pp_components.verify_dirs(in_dir: str, out_dir: str)

Verify that the input and output directories exists and are directories

Parameters:

  • output_dir (str) – output directory

  • input_dir (str) – input directory

Raises:

ValueError –

  • input directory invalid

  • output directory invalid