Partitioning

class partitioning.Partitioning.Partitioning(hi, zi, freq, length, df, PreProcessing, argsQC={}, sampledEventsStats=False, argsOut={}, argsQThres={})

Bases: object

Initializes the Partitioning class.

Parameters:

• hi (float) – Canopy height (m).

• zi (float) – Eddy covariance measurement height (m).

• freq (int) – Sampling frequency (Hz).

• length (int) – Length of the time series (in minutes).

• df (pandas.DataFrame) –

  DataFrame with data (e.g., 30min intervals, but any length works), each variable in a column. If raw data is used, pre-processing is first implemented following these steps:

  • Quality control (removing outliers, despiking, flags of instruments, etc)

  • Rotation of coordinates (double rotation) for velocity components u, v, w measured by CSAT

  • Density corrections for instantaneous fluctuations of CO2 (c_p) and H2O (q_p) measured by open-gas analyser (“instantaneous” WPL correction) based on the paper: Detto, M. and Katul, G. G., 2007. “Simplified expressions for adjusting higher-order

    turbulent statistics obtained from open path gas analyzers”. Boundary-Layer Meteorology, 10.1007/s10546-006-9105-1

  • Turbulent fluctuations, here denoted as primed quantities (“_p”), are computed

  • Air temperature (T) and virtual temperature (Tv) computed from the sonic temperature (Ts)

  Raw data requires the following variables and units:

  • index : datetime

  • w : velocity in the z direction (m/s)

  • u : velocity in the x direction (m/s)

  • v : velocity in the y direction (m/s)

  • Ts : sonic temperature (Celsius)

  • P : pressure (kPa)

  • CO2 : carbon dioxide density (mg/m3)

  • H2O : water vapor density (g/m3)

  After pre-processing, the following additional variables are created:

  • w_p : fluctuations of velocity in the z direction (m/s)

  • u_p : fluctuations of velocity in the x direction (m/s)

  • v_p : fluctuations of velocity in the y direction (m/s)

  • T : thermodynamic air temperature (Celsius)

  • Tv : virtual temperature (Celsius)

  • co2_p : fluctuations of carbon dioxide density (mg/m3) - (corrected for external densities (WPL) if needed)

  • h2o_p : fluctuations of water vapor density (g/m3) - (corrected for external densities (WPL) if needed)

  • Ts_p : fluctuations of sonic air temperature (Celsius)

  • Tv_p : fluctuations of virtual temperature (Celsius)

• PreProcessing (bool, optional) – Indicates if pre-processing is necessary. If True, all pre-processing steps are implemented to raw data. If False, pre-processing is ignored and partitioning is immediately applied. In this case, the input files must contain all pre-processed variables listed above.

• argsQC (dict, optional) –

  Contains options to be used during pre-processing regarding fluctuation extraction and if density corrections are necessary. All options have default values, but can be modified if needed.

  density_correction - bool

  True if density corrections are necessary (open gas analyzer); False (closed or enclosed gas analyzer).

  fluctuations - str

  Describes the type of operation used to extract fluctuations: ‘BA’: block average ‘LD’: Linear detrending ‘FL’: Filter low frequencies. Requires filtercut to indicate the cutoff time in minutes.

  filtercut - int

  Cutoff time in minutes for the low-pass filter. Only used if method is ‘FL’.

  maxGapsInterpolate - int

  Number of consecutive gaps that will be interpolated.

  RemainingData - int

  Percentage (0-100) of the time series that should remain after pre-processing. If less than this quantity, partitioning is not implemented.

  time_lag_correction - bool

  If True, a time lag correction is applied to the CO2 and H2O time series relative to the W time series.

  max_lag_seconds - int

  Maximum time lag in seconds to consider for correlation. Defaults to 5 seconds.

  type_lag - str

  Specifies the type of lag to consider. Options are ‘positive’, ‘negative’, or ‘both’. Defaults to ‘positive’. ‘Positive’ means that CO2 and H2O lag behind W as expected in closed-path systems when the tube delays the signal.

  saveplotlag - bool

  If True, saves a plot of the cross-correlation function between the CO2 and H2O time series with respect to the W time series in the subfolder TimeLagCorrelationFigures.

  outfolder - str

  If an outfolder is given the plots of the cross-correlation are saved there. If not, the current working directory.

  UnitBorders - dict

  Define data range in between the median of the data has to be, otherwise an error is raised. For each column in the input data, one key in the dictionary containing a tuple (min, max) is necessary. Units are: m/s for u, v, w, Celsius for Ts, mg/m3 for co2, g/m3 for h2o, and kPa for P. Example: “UnitBorders”:{“Ts”: (0,70), “co2”: (200, 1500),”h2o”: (0, 50), “P”: (60, 150)}

  PhysicalBounds - dict

  Define data range in between the values have to be, otherwise the individual values are set to NaN. For each column in the input data, one key in the dictionary containing a tuple (min, max) is necessary. Units are: m/s for u, v, w, Celsius for Ts, mg/m3 for co2, g/m3 for h2o, and kPa for P. Example: “PhysicalBounds”:{“u”: (-20, 20),”v”: (-20, 20),”w”: (-20, 20),”Ts”: (-10, 50), “co2”: (0, 1500), “h2o”: (0, 40), “P”: (60, 150)}

• sampledEventsStats (bool, optional) – If True the time fraction and time scale of sampled events within each quadrant are calculated.

• argsOut (dict, optional) –

  Contains options in which units the results are given. Defaults are that the output is in mass based units. Possible to activate all simultanously.

  energetic_units - bool

  True if the H2O flux shall be provided in energetic units in W/m2

  mass_units - bool

  True if the CO2 and H2O flux shall be provided as mass flux: g/(m2 s) for h2o and mg/(m2 s) for co2.

  molar_units - bool

  True if the CO2 and H2O flux shall be provided as molar flux: mmol/(m2 s)

• argsQThres (dict, optional) –

  Contains the quadrant thresholds stating which amount of data needs to be present within each quadrant to partition the fluxes. Also the settings regarding the hyperbolic threshold and about the time scale of sampled events can be given here.

  cec_per_points_Q1Q2 - int

  For CEC more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  cec_per_points_each - int

  For CEC if less or at least % of data is within one of Q1 or Q2 the flux is contributed to the other quadrant.

  cecw_per_points_Q1Q2 - int

  For CECw more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  cecw_per_points_each - int

  For CECw if less or at least % of data is within one of Q1 or Q2, no partitioning is performed.

  mrea_per_points_Q1Q2 - int

  For MREA more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  mrea_per_points_each - int

  For MREA if less or at least % of data are within one of Q1 or Q2 the flux is contributed to the other quadrant.

  cea_per_points_Q1Q2 - int

  For CEA more % of data needs to be present within the four quadrants Q1 and Q2 for both up- and downdrafts, otherwise no partitioning is performed.

  cea_per_points_each - int

  For CEA more % of data needs to be in each of the necessary four quadrants Q1 and Q2 for both up- and downdrafts, no partitioning is performed.

  t_scale_gap_threshold - int

  For the time scale of sampled events, the minimum amount of datapoints to define a new conditionally sampled event.

Notes: Available Partitioning Methods

• Conditional Eddy Covariance (CEC)

• Modified Relaxed Eddy Accumulation (MREA)

• Flux Variance Similarity (FVS)

• Conditional Eddy Accumulation (CEA)

• Conditional Eddy Covariance + WUE (CECw)

CEC, CEA, and MREA only need time series of w_p, co2_p, h2o_p. The remaining quantities (e.g., P, T, Tv, etc.) are only needed if the water use efficiency (WUE) is computed for the FVS and CECw method. Alternatively, an external WUE can be used; in this case, FVS and CECw will only need time series of w_p, h2o_p, co2_p.

TScale(sdata)

Calculates the mean time scale in seconds of sampled events

Main reference: - Thomas et al., 2008 (Agr For Met).

Estimating daytime subcanopy respiration from conditional sampling methods applied to multi-scalar high frequency turbulence time series https://www.sciencedirect.com/science/article/pii/S0168192308000737

Parameters:

sdata (pandas.DataFrame) –

Data already subsetted to only contain data of a specific quadrant Needs to have the column “num_idx” which was a continous numeric index

of the continously increasing timestamp before exclusion of NaN and before subsetting.

Returns:

t_scale – Mean time scale in seconds of an event within the sdata

Return type:

float (in case sdata is empty np.nan)

TurbulentStats()

# Calculate turbulent statistics (scales, standard deviations, and correlations).

Returns:

‘ustar’: float

friction velocity [m/s]

’cstar’: float

scale for CO2 [mg/m3]

’qstar’: float

scale for H2O [g/m3]

’tstar’: float

scale for temperature [K]

’zeta’: float

Monin-Obukhov stability parameter

’std_t’: float

standard deviation of temperature [K]

’std_q’: float

standard deviation of H2O [g/m3]

’std_c’: float

standard deviation of CO2 [mg/m3]

’std_w’: float

standard deviation of w [m/s]

’std_u’: float

standard deviation of u [m/s]

’std_v’: float

floatstandard deviation of v [m/s]

’rqc’: float

correlation between H2O and CO2

’rqt’: float

correlation between H2O and temperature

’rct’: float

correlation between CO2 and temperature

’H’: float

sensible heat flux [W/m2]

In case of self.argsOut.get(“energetic_units”):

’LE’: float

latent heat flux [W/m2]

In case of self.argsOut.get(“mass_units”):

’ET_m’: float

evapotranspiration, covariance between w and h2o [g/(m2 s)]

’Fc’: float

carbon dioxide flux, covariance between w and CO2 [mg/m2/s]

In case of self.argsOut.get(“molar_units”):

’ET_a’: float

evapotranspiration [mmol/(m2 s)]

’Fc_a’: float

carbon dioxide flux [mmol/(m2 s)]

Return type:

dict

WaterUseEfficiency(ppath='C3', methodsWue=True)

Calculates water use efficiency in kg_co2/kg_h2o.

Main references: - Scanlon and Sahu 2008, Water Resources Research

“On the correlation structure of water vapor and carbon dioxide in the atmospheric surface layer: A basis for flux partitioning”

• Parts of the code were adapted from Skaggs et al. 2018, Agr For Met “Fluxpart: Open source software for partitioning carbon dioxide and water vapor fluxes” https://github.com/usda-ars-ussl/fluxpart

• Optimization model for W from Scanlon et al., 2019, Agr. For. Met. “Correlation-based flux partitioning of water vapor and carbon dioxide fluxes: Method simplification and estimation of canopy water use efficiency”

Parameters:

• ppath (str) – Type of photosynthesis (‘C3’ or ‘C4’).

• methodsWue (bool or dict, default True) –

  If True, all available methods are used. If False, no method is used. If dict, the specified methods are used.

  If True, the corresponding method is calculated

  const_ppm : bool const_ratio : bool linear : bool sqrt : bool opt : bool

• Models –

  Computes the water use efficiency (eq A1 in Scanlon and Sahu, 2008):

  • wue = 0.65 * (c_c - c_s) / (q_c - q_s)

  • c_c (kg/m3) and q_c (kg/m3) are near canopy concentrations of CO2 and H2O

    • Estimated from log profiles (eq A2a in Scanlon and Sahu, 2008).

  • c_s (kg/m3) and q_s (kg/m3) are stomata concentrations of CO2 and H2O

    • q_s is assumed to be at saturation.

    • c_s is parameterized from different models (Skaggs et al., 2018; Scanlon et al., 2019).

  The following models for c_s are implemented

  const_ppm:

  • Concentrations in kg/m3 are computed from a constant value in ppm.

  • Values from Campbell and Norman, 1998, p. 150. Campbell, G. S. and Norman, J. M. (1998). An Introduction to Environmental Biophysics. Springer, New York, NY.

  • c_s = 280 ppm (C3 plants).

  • c_s = 130 ppm (C4 plants).

  const_ratio:

  • The ratio of near canopy and stomata CO2 concentrations is assumed constant (c_s/c_c = constant).

  • Constants from Sinclair, T. R., Tanner, C. B., and Bennett, J. M. (1984). Water-use efficiency in crop production. BioScience, 34(1):36–40.

  • c_s/c_c = 0.70 for C3 plants.

  • c_s/c_c = 0.44 for C4 plants.

  linear:

  • The ratio of near canopy and stomata CO2 concentrations is a linear function of VPD.

  • Based on the results of Morison, J. I. L. and Gifford, R. M. (1983). Stomatal sensitivity to carbon dioxide and humidity. Plant Physiology, 71(4):789–796. Estimated constants from Skaggs et al (2018).

  • c_s/c_c = a - b * D

  • a, b = 1, 1.6*10-4 Pa-1 for C3 plants.

  • a, b = 1, 2.7*10-4 Pa-1 for C4 plants.

  • D (Pa) is vapor pressure deficit based on leaf-temperature.

  sqrt:

  • The ratio of near canopy and stomata CO2 concentrations is proportional to the 1/2 power of VPD.

  • Model by Katul, G. G., Palmroth, S., and Oren, R. (2009). Leaf stomatal responses to vapour pressure deficit under current and CO2-enriched atmosphere explained by the economics of gas exchange. Plant, Cell & Environment, 32(8):968–979.

  • c_s/c_c = 1 - sqrt(1.6 * lambda * D / c_c)

  • lambda = 22e-9 kg-CO2 / m^3 / Pa for C3 plants (from Skaggs et al., 2018).

  • Not available for C4 plants.

  opt:

  • Optimization model proposed by Scanlon et al (2019).

  • Does not need extra parameters.

  • Only available for C3 plants.

Returns:

self.wue – Dictionary containing the water use efficiency from different methods: - ‘const_ppm’: float

WUE from constant ppm [kg_co2/kg_h2o].

• ’const_ratio’: float

  WUE from constant ratio [kg_co2/kg_h2o].

• ’linear’: float

  WUE from linear model [kg_co2/kg_h2o].

In case of C3 plants additional available is: - ‘sqrt’: float

WUE from sqrt model [kg_co2/kg_h2o].

• ’opt’: float

  WUE from optimization model [kg_co2/kg_h2o].

Return type:

dict

partCEA(H=0.0)

Implements the Conditional Eddy Accumulation method proposed by Zahn et al. 2024.

Numerical Investigation of Observational Flux Partitioning Methods for Water Vapor and Carbon Dioxide

Parameters:

H (float, optional) – Hyperbolic threshold, by default 0.0.

Attributes

Contains all the flux components and status of the calculation.

in case of self.sampledEventsStats:

• Q1tfrac_cea - float

  Time fraction of sampled events within Quadrant 1.

• Q2tfrac_cea - float

  Time fraction of sampled events within Quadrant 2.

• Q1tscale_cea - float

  Mean time scale of sampled events within Quadrant 1.

• Q2tscale_cea - float

  Mean time scale of sampled events within Quadrant 2.

in case of self.argsOut.get(“energetic_units”):

• Tcea - float

  Plant transpiration (W/m2).

• Ecea - float

  Soil/surface evaporation (W/m2).

in case of self.argsOut.get(“mass_units”):

• Ecea_m - float

  Soil/surface evaporation (g/(m2 s)).

• Tcea_m - float

  Plant transpiration (g/(m2 s)).

• Pcea - float

  Plant net photosynthesis* (mg/m2/s).

• Rcea - float

  Soil/surface respiration (mg/m2/s).

in case of self.argsOut.get(“molar_units”):

• Ecea_a - float

  Soil/surface evaporation (mmol/(m2 s)).

• Tcea_a - float

  Plant transpiration (mmol/(m2 s)).

• Pcea_a - float

  Plant net photosynthesis* (mmol/m2/s).

• Rcea_a - float

  Soil/surface respiration (mmol/m2/s).

• statuscea - str

  Status of the calculation.

Type:

self.fluxesCEA

Notes

This component represents carboxylation minus photorespiration and leaf respiration; therefore, it is different from gross primary productivity.

partCEC(H=0.0)

Implements the Conditional Eddy Covariance method proposed by Zahn et al. 2021.

Direct Partitioning of Eddy-Covariance Water and Carbon Dioxide Fluxes into Ground and Plant Components.

Parameters:

H (float, optional) – Hyperbolic threshold, by default 0.0.

Attributes

Contains all the flux components and status of the calculation.

in case of self.sampledEventsStats:

• Q1tfrac_cec - float

  Time fraction of sampled events within Quadrant 1.

• Q2tfrac_cec - float

  Time fraction of sampled events within Quadrant 2.

• Q1tscale_cec - float

  Mean time scale of sampled events within Quadrant 1.

• Q2tscale_cec - float

  Mean time scale of sampled events within Quadrant 2.

in case of self.argsOut.get(“energetic_units”):

• Tcec - float

  Plant transpiration (W/m2).

• Ecec - float

  Soil/surface evaporation (W/m2).

in case of self.argsOut.get(“mass_units”):

• Ecec_m - float

  Soil/surface evaporation (g/(m2 s)).

• Tcec_m - float

  Plant transpiration (g/(m2 s)).

• Pcec - float

  Plant net photosynthesis* (mg/m2/s).

• Rcec - float

  Soil/surface respiration (mg/m2/s).

in case of self.argsOut.get(“molar_units”):

• Ecec_a - float

  Soil/surface evaporation (mmol/(m2 s)).

• Tcec_a - float

  Plant transpiration (mmol/(m2 s)).

• Pcec_a - float

  Plant net photosynthesis* (mmol/m2/s).

• Rcec_a - float

  Soil/surface respiration (mmol/m2/s).

Type:

self.fluxesCEC

• statuscec - str

  Status of the calculation.

Notes

This component represents carboxylation minus photorespiration and leaf respiration; therefore, it is different from gross primary productivity.

partCECw(W, H=0.0)

Implements Conditional Eddy Covariance + water use efficiency proposed by Zahn et al. 2024.

Numerical Investigation of Observational Flux Partitioning Methods for Water Vapor and Carbon Dioxide

Parameters:

W (float, optional) – Water use efficiency, by default 0.

Attributes

Contains all the flux components and status of the calculation. Dictionary with the following flux components:

in case of self.sampledEventsStats:

• Q1tfrac_cecw - float

  Time fraction of sampled events within Quadrant 1.

• Q2tfrac_cecw - float

  Time fraction of sampled events within Quadrant 2.

• Q1tscale_cecw - float

  Mean time scale of sampled events within Quadrant 1.

• Q2tscale_cecw - float

  Mean time scale of sampled events within Quadrant 2.

in case of self.argsOut.get(“energetic_units”):

• Tcecw - float

  Plant transpiration (W/m2).

• Ececw - float

  Soil/surface evaporation (W/m2).

in case of self.argsOut.get(“mass_units”):

• Ececw_m - float

  Soil/surface evaporation (g/(m2 s)).

• Tcecw_m - float

  Plant transpiration (g/(m2 s)).

• Pcecw - float

  Plant net photosynthesis* (mg/m2/s).

• Rcecw - float

  Soil/surface respiration (mg/m2/s).

in case of self.argsOut.get(“molar_units”):

• Ececw_a - float

  Soil/surface evaporation (mmol/(m2 s)).

• Tcecw_a - float

  Plant transpiration (mmol/(m2 s)).

• Pcecw_a - float

  Plant net photosynthesis* (mmol/m2/s).

• Rcecw_a - float

  Soil/surface respiration (mmol/m2/s).

• statuscecw - str

  Status of the calculation.

Type:

self.fluxesCECw

Notes

This component represents carboxylation minus photorespiration and leaf respiration; therefore, it is different from gross primary productivity.

partFVS(W)

Implements the Flux Variance Similarity Theory proposed by Scanlon et al., 2019.

Direct Partitioning of GPP and Re in a Subalpine Forest Ecosystem.

Parameters:

W (float, optional [kg_co2/kg_h2o]) – Water use efficiency, by default 0.

Attributes

Contains all the flux components and status of the calculation.

in case of self.argsOut.get(“energetic_units”):

• Tfvs - float

  Plant transpiration (W/m2).

• Efvs - float

  Soil/surface evaporation (W/m2).

in case of self.argsOut.get(“mass_units”):

• Efvs_m - float

  Soil/surface evaporation (g/(m2 s)).

• Tfvs_m - float

  Plant transpiration (g/(m2 s)).

• Pfvs - float

  Plant net photosynthesis* (mg/m2/s).

• Rfvs - float

  Soil/surface respiration (mg/m2/s).

in case of self.argsOut.get(“molar_units”):

• Efvs_a - float

  Soil/surface evaporation (mmol/(m2 s)).

• Tfvs_a - float

  Plant transpiration (mmol/(m2 s)).

• Pfvs_a - float

  Plant net photosynthesis* (mmol/m2/s).

• Rfvs_a - float

  Soil/surface respiration (mmol/m2/s).

• statusfvs - str

  Status of the calculation.

Type:

self.fluxesFVS

Notes

This component represents carboxylation minus photorespiration and leaf respiration; therefore, it is different from gross primary productivity.

partREA(H=0)

Implements the Modified Relaxed Eddy Accumulation proposed by Thomas et al., 2008 (Agr For Met).

Estimating daytime subcanopy respiration from conditional sampling methods applied to multi-scalar high frequency turbulence time series https://www.sciencedirect.com/science/article/pii/S0168192308000737 New contraints defined in Zahn et al (2021).

Parameters:

H (float, optional) – Hyperbolic threshold, by default 0.0.

Attributes

Dictionary with the following flux components:

in case of self.sampledEventsStats:

• Q1tfrac_mrea - float

  Time fraction of sampled events within Quadrant 1.

• Q2tfrac_mrea - float

  Time fraction of sampled events within Quadrant 2.

• Q1tscale_mrea - float

  Mean time scale of sampled events within Quadrant 1.

• Q2tscale_mrea - float

  Mean time scale of sampled events within Quadrant 2.

in case of self.argsOut.get(“energetic_units”):

• Tmrea - float

  Plant transpiration (W/m2).

• Emrea - float

  Soil/surface evaporation (W/m2).

in case of self.argsOut.get(“mass_units”):

• Emrea_m - float

  Soil/surface evaporation (g/(m2 s)).

• Tmrea_m - float

  Plant transpiration (g/(m2 s)).

• Pmrea - float

  Plant net photosynthesis* (mg/m2/s).

• Rmrea - float

  Soil/surface respiration (mg/m2/s).

in case of self.argsOut.get(“molar_units”):

• Emrea_a - float

  Soil/surface evaporation (mmol/(m2 s)).

• Tmrea_a - float

  Plant transpiration (mmol/(m2 s)).

• Pmrea_a - float

  Plant net photosynthesis* (mmol/m2/s).

• Rmrea_a - float

  Soil/surface respiration (mmol/m2/s).

Type:

self.fluxesREA

• statusmrea - str

  Status of the calculation.

Notes

This component represents carboxylation minus photorespiration and leaf respiration; therefore, it is different from gross primary productivity.

wrapper

partitioning.wrapper.CallPartitioning(filei, siteDetails, argsQC={}, argsOut={}, methods=True, methodsWue=True, statistics={'TurbStats': True}, argsQThres={}, loadfnct='NormLoad', loadkwargs={}, versatile_loadkwargs={})

Calls different partitioning methods and returns the data and units.

Parameters:

• filei (str) – String with path to file being loaded and partitoned.

• siteDetails (dict) –

  Dictionary with details about the measurement site.

  hiint/float,

  Canopy mean height in meters

  ziint/float,

  EC measurement height in meters

  freqint/float,

  EC measurement frequency in Hz

  lengthint/float,

  length of data file in minutes

  PreProcessingbool

  If pre-processing takes place.

  ppathstr

  Type of photosynthesis (‘C3’ or ‘C4’), for WUE calculation.

• argsQC (dict) –

  Contains options to be used during pre-processing regarding fluctuation extraction and if density corrections are necessary. All options have default values, but can be modified if needed.

  density_correctionbool

  True if density corrections are necessary (open gas analyzer); False (closed or enclosed gas analyzer).

  fluctuationsstr

  Describes the type of operation used to extract fluctuations: ‘BA’: block average ‘LD’: Linear detrending ‘FL’: Filter low frequencies. Requires filtercut to indicate the cutoff time in minutes.

  filtercutint

  Cutoff time in minutes for the low-pass filter. Only used if method is ‘FL’.

  maxGapsInterpolateint

  Number of consecutive gaps that will be interpolated.

  RemainingDataint

  Percentage (0-100) of the time series that should remain after pre-processing. If less than this quantity, partitioning is not implemented.

  saveprocessedbool

  If True, the pre-processed data is saved to a CSV file in the subfolder ProcessedData.

  time_lag_correctionbool

  If True, a time lag correction is applied to the CO2 and H2O time series relative to the W time series.

  max_lag_secondsint

  Maximum time lag in seconds to consider for correlation. Defaults to 5 seconds.

  type_lagstr

  Specifies the type of lag to consider. Options are ‘positive’, ‘negative’, or ‘both’. Defaults to ‘positive’. ‘Positive’ means that CO2 and H2O lag behind W as expected in closed-path systems when the tube delays the signal.

  saveplotlag - bool

  If True, saves a plot of the cross-correlation function between the CO2 and H2O time series with respect to the W time series in the subfolder TimeLagCorrelationFigures.

  outfolderstr

  If an outfolder is given the plots of the cross-correlation and the pre-processed files are saved there in the subfolders TimeLagCorrelationFigures or ProcessedData. If not, the current working directory.

  UnitBorders - dict

  Define data range in between the median of the data has to be, otherwise an error is raised. For each column in the input data, one key in the dictionary containing a tuple (min, max) is necessary. Units are: m/s for u, v, w, Celsius for Ts, mg/m3 for co2, g/m3 for h2o, and kPa for P. Example: “UnitBorders”:{“Ts”: (0,70), “co2”: (200, 1500),”h2o”: (0, 50), “P”: (60, 150)}

  PhysicalBounds - dict

  Define data range in between the values have to be, otherwise the individual values are set to NaN. For each column in the input data, one key in the dictionary containing a tuple (min, max) is necessary. Units are: m/s for u, v, w, Celsius for Ts, mg/m3 for co2, g/m3 for h2o, and kPa for P. Example: “PhysicalBounds”:{“u”: (-20, 20),”v”: (-20, 20),”w”: (-20, 20),”Ts”: (-10, 50), “co2”: (0, 1500), “h2o”: (0, 40), “P”: (60, 150)}

• argsOut (dict) –

  Contains options in which units the results are given. Defaults are that the output is in mass based units. Possible to activate all simultanously.

  energetic_unitsbool

  True if the H2O flux shall be provided in energetic units in W/m2

  mass_unitsbool

  True if the CO2 and H2O flux shall be provided as mass flux: g/(m2 s) for h2o and mg/(m2 s) for co2.

  molar_unitsbool

  True if the CO2 and H2O flux shall be provided as molar flux: mmol/(m2 s)

• methods (bool or dict, default True) –

  If True, all available partitioning methods are used. If False, no method is used. If dict, the specified methods are used.

  If True, the corresponding method is calculated

  MREA : bool CEC : bool CECw : bool CEA : bool FVS : bool

• methodsWue (bool or dict, default True) –

  If True, all available water use efficicency methods are used. If False, no method is used. If dict, the specified methods are used.

  If True, the corresponding method is calculated

  const_ppm : bool const_ratio : bool linear : bool sqrt : bool opt : bool

• statistics (bool or dict, default {"TurbStats":True}) –

  If True all possible statistics are calculated. If False no statistics are calculated. If dict, the specified statistics are calculated. If True, the corresponding method is calculated Keys not used, are set to False

  TurbStatsbool

  If True basic general turbulence statistics are calculated.

  steadynessbool

  If True, Foken’s stationarity test is implemented to check if the data is stationary. If False, the test is not implemented. The test is only informative and does not remove data, which is left to the user’s discretion.

  sampledEventsbool

  If True the time fraction and time scale of sampled events within each quadrant are calculated.

• argsQThres (dict) –

  Contains the quadrant thresholds stating which amount of data needs to be present within each quadrant to partition the fluxes.

  cec_per_points_Q1Q2int

  For CEC more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  cec_per_points_eachint

  For CEC if less or at least % of data is within one of Q1 or Q2 the flux is contributed to the other quadrant.

  cecw_per_points_Q1Q2int

  For CECw more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  cecw_per_points_eachint

  For CECw if less or at least % of data is within one of Q1 or Q2 the flux is contributed to the other quadrant.

  mrea_per_points_Q1Q2int

  For MREA more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  mrea_per_points_eachint

  For MREA if less or at least % of data are within one of Q1 or Q2 the flux is contributed to the other quadrant.

  cea_per_points_Q1Q2int

  For CEA more % of data needs to be present within the four quadrants Q1 and Q2 for both up- and downdrafts, otherwise no partitioning is performed.

  cea_per_points_eachint

  For CEA more % of data needs to be in each of the necessary four quadrants Q1 and Q2 for both up- and downdrafts, no partitioning is performed.

  t_scale_gap_thresholdint

  For the time scale of sampled events, the minimum amount of datapoints to define a new conditionally sampled event.

  Hfloat or dict

  Hyperbolic threshold criteria. If not specified 0 is used for all methods. If float: MREA, CEC, CEA, CECw get calculated using this threshold. If dict:

  Hyperbolic threshold per method used. If for a method no threshold is defined its set to 0.

  MREAfloat

  Hyperbolic threshold for MREA

  CECfloat

  Hyperbolic threshold for CEC

  CEAfloat

  Hyperbolic threshold for CEA

  CECwfloat

  Hyperbolic threshold for CECw

loadfnctstr

Function name as a string used for loading the data. Available options are:

“VersatileLoad”

loading, renaming and recalculations can be done with this function. it basically makes the other loading functions useless apart from their shorter notation.

“NormLoad”

basically pd.read_csv, pass the arguments to read the data as loadkwargs. the index gets to be the timestamp

“LoadBmmflux”

custom function to read the BMMFlux high-frequency output files. BMMFlux is the EddyCovariance Software of the Micrometeorology Group in Bayreuth.

See appendix of Thomas, C. K., Law, B. E., Irvine, J., Martin, J. G., Pettijohn, J. C., & Davis, K. J. (2009):

Seasonal hydrology explains interannual and seasonal variation in carbon and water exchange in a semiarid mature ponderosa pine forest in central Oregon. Journal of Geophysical Research: Biogeosciences, 114(G4). https://doi.org/10.1029/2009JG001010

loadkwargsdict

Arguments passed to the pd.read_csv() function in case of NormLoad and VersatileLoad.

versatile_loadkwargsdict

Arguments passed to the VersatileLoad function.

timestamp_colstr or list of str, optional

• If a list: Combines split columns (e.g., [‘Year’, ‘Month’, ‘Day’]) into a datetime index.

• If a string: Converts that specific column into the datetime index.

• If None: Converts the default DataFrame index to datetime.

convert_gasesbool, default False

If True, converts ‘co2’ and ‘h2o’ from mmol/m³ to mg/m³ and g/m³ respectively.

rename_colsdict, optional

A dictionary mapping old column names to new ones (e.g., {“Pressure”: “P”}).

select_colslist of str, optional

A list of specific columns to keep. All other columns will be dropped.

Returns:

• datun (dict) – Dictionary with partitioned data and units.

• Keys

• —-

• data (dict) – Partitioned data, each key:value pair corresponds to one return value of the partioning functions.

• units (dict) – The corresponding unit, as well as key:value pair.

partitioning.wrapper.LoadBmmflux(path, loadkwargs, **kwargs)

Loading csv data, optimized for BMMFlux high-frequency output files.

Parameters:

• path (str) – Path to the csv file to be loaded.

• loadkwargs (dict) – Not used.

• **kwargs (further arguments) – ignored

Returns:

df – The loaded data.

Return type:

pandas.DataFrame

partitioning.wrapper.NormLoad(path, loadkwargs, **kwargs)

Loading csv data.

Parameters:

• path (str) – Path to the csv file to be loaded.

• loadkwargs (dict) – Arguments passed to pd.read_csv. Options include i.a. header, index_col, usecols, names, na_values, skiprows

• kwargs (further arguments) – ignored

Returns:

df – The loaded data.

Return type:

pandas.DataFrame

partitioning.wrapper.VersatileLoad(path, loadkwargs=None, timestamp_col=None, convert_gases=False, rename_cols=None, select_cols=None)

A versatile CSV loading function that accommodates standard formats, multi-column high-frequency timestamps, gas unit conversions, and column filtering.

Parameters:

• path (str) – Path to the csv file to be loaded.

• loadkwargs (dict, optional) – Arguments passed directly to pd.read_csv (e.g., header, skiprows, na_values).

• timestamp_col (str or list of str, optional) –

  • If a list: Combines split columns (e.g., [‘Year’, ‘Month’, ‘Day’]) into a datetime index.

  • If a string: Converts that specific column into the datetime index.

  • If None: Converts the default DataFrame index to datetime.

• convert_gases (bool, default False) – If True, converts ‘co2’ and ‘h2o’ from mmol/m³ to mg/m³ and g/m³ respectively.

• rename_cols (dict, optional) – A dictionary mapping old column names to new ones (e.g., {“Pressure”: “P”}).

• select_cols (list of str, optional) – A list of specific columns to keep. All other columns will be dropped.

Returns:

df – The loaded data.

Return type:

pandas.DataFrame

partitioning.wrapper.process(siteDetails, infolder, outfolder, loadpattern='*.csv', outname='PartitioningResults', argsQC={}, argsOut={}, methods=True, methodsWue=True, statistics={'TurbStats': True}, argsQThres={}, loadfnct='NormLoad', loadkwargs={}, versatile_loadkwargs={}, logginglevel=20)

Loading the raw data, (optionally) pre-process, partition and save the results.

Parameters:

• siteDetails (dict) –

  Dictionary with details about the measurement site.

  hiint/float,

  Canopy mean height in meters

  ziint/float,

  EC measurement height in meters

  freqint/float,

  EC measurement frequency in Hz

  lengthint/float,

  length of data file in minutes

  PreProcessingbool

  If pre-processing takes place.

  ppathstr

  Type of photosynthesis (‘C3’ or ‘C4’), for WUE calculation.

• str (outname -) – Path to the folder where the input data is located. Needs to end with slash or backslash.

• str – Path to the folder where the output data is located. Needs to end with slash or backslash.

• str – Pattern in the filename to match for loading files.

• "*.csv" (default) –

  Pattern in the filename to match for loading files.

• str – Filename for the output data, excluding file ending.

• "PartitioningResults" (default) –

  Filename for the output data, excluding file ending.

  argsQCdict

  Contains options to be used during pre-processing regarding fluctuation extraction and if density corrections are necessary. All options have default values, but can be modified if needed.

  density_correctionbool

  True if density corrections are necessary (open gas analyzer); False (closed or enclosed gas analyzer).

  fluctuationsstr

  Describes the type of operation used to extract fluctuations: ‘BA’: block average ‘LD’: Linear detrending ‘FL’: Filter low frequencies. Requires filtercut to indicate the cutoff time in minutes.

  filtercutint

  Cutoff time in minutes for the low-pass filter. Only used if method is ‘FL’.

  maxGapsInterpolateint

  Number of consecutive gaps that will be interpolated.

  RemainingDataint

  Percentage (0-100) of the time series that should remain after pre-processing. If less than this quantity, partitioning is not implemented.

  saveprocessedbool

  If True, the pre-processed data is saved to a CSV file in the subfolder ProcessedData.

  time_lag_correctionbool

  If True, a time lag correction is applied to the CO2 and H2O time series relative to the W time series.

  max_lag_secondsint

  Maximum time lag in seconds to consider for correlation. Defaults to 5 seconds.

  saveplotlag - bool

  If True, saves a plot of the cross-correlation function between the CO2 and H2O time series with respect to the W time series in the subfolder TimeLagCorrelationFigures.

  type_lagstr

  Specifies the type of lag to consider. Options are ‘positive’, ‘negative’, or ‘both’. Defaults to ‘positive’. ‘Positive’ means that CO2 and H2O lag behind W as expected in closed-path systems when the tube delays the signal.

  UnitBorders - dict

  Define data range in between the median of the data has to be, otherwise an error is raised. For each column in the input data, one key in the dictionary containing a tuple (min, max) is necessary. Units are: m/s for u, v, w, Celsius for Ts, mg/m3 for co2, g/m3 for h2o, and kPa for P. Example: “UnitBorders”:{“Ts”: (0,70), “co2”: (200, 1500),”h2o”: (0, 50), “P”: (60, 150)}

  PhysicalBounds - dict

  Define data range in between the values have to be, otherwise the individual values are set to NaN. For each column in the input data, one key in the dictionary containing a tuple (min, max) is necessary. Units are: m/s for u, v, w, Celsius for Ts, mg/m3 for co2, g/m3 for h2o, and kPa for P. Example: “PhysicalBounds”:{“u”: (-20, 20),”v”: (-20, 20),”w”: (-20, 20),”Ts”: (-10, 50), “co2”: (0, 1500), “h2o”: (0, 40), “P”: (60, 150)}

  argsOutdict

  Contains options in which units the results are given. Defaults are that the output is in mass based units. Possible to activate all simultanously.

  energetic_unitsbool

  True if the H2O flux shall be provided in energetic units in W/m2

  mass_unitsbool

  True if the CO2 and H2O flux shall be provided as mass flux: g/(m2 s) for h2o and mg/(m2 s) for co2.

  molar_unitsbool

  True if the CO2 and H2O flux shall be provided as molar flux: mmol/(m2 s)

  methodsbool or dict, default True

  If True, all available partitioning methods are used. If False, no method is used. If dict, the specified methods are used.

  If True, the corresponding method is calculated

  MREA : bool CEC : bool CECw : bool CEA : bool FVS : bool

  methodsWuebool or dict, default True

  If True, all available water use efficicency methods are used. If False, no method is used. If dict, the specified methods are used.

  If True, the corresponding method is calculated

  const_ppm : bool const_ratio : bool linear : bool sqrt : bool opt : bool

  statisticsbool or dict, default {“TurbStats”:True}

  If True all possible statistics are calculated. If False no statistics are calculated. If dict, the specified statistics are calculated.

  If True, the corresponding method is calculated Keys not used, are set to False

  TurbStatsbool

  If True basic general turbulence statistics are calculated.

  steadynessbool

  If True, Foken’s stationarity test is implemented to check if the data is stationary. If False, the test is not implemented. The test is only informative and does not remove data, which is left to the user’s discretion.

  sampledEventsbool

  If True the time fraction and time scale of sampled events within each quadrant are calculated.

  argsQThresdict

  Contains the quadrant thresholds stating which amount of data needs to be present within each quadrant to partition the fluxes. Also the settings regarding the hyperbolic threshold and about the time scale of sampled events can be given here.

  cec_per_points_Q1Q2int

  For CEC more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  cec_per_points_eachint

  For CEC if less or at least % of data is within one of Q1 or Q2 the flux is contributed to the other quadrant.

  cecw_per_points_Q1Q2int

  For CECw more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  cecw_per_points_eachint

  For CECw if less or at least % of data is within one of Q1 or Q2 the flux is contributed to the other quadrant.

  mrea_per_points_Q1Q2int

  For MREA more % of data needs to be present within quadrant 1 and 2 to partition. Otherwise no partitioning is performed.

  mrea_per_points_eachint

  For MREA if less or at least % of data are within one of Q1 or Q2 the flux is contributed to the other quadrant.

  cea_per_points_Q1Q2int

  For CEA more % of data needs to be present within the four quadrants Q1 and Q2 for both up- and downdrafts, otherwise no partitioning is performed.

  cea_per_points_eachint

  For CEA more % of data needs to be in each of the necessary four quadrants Q1 and Q2 for both up- and downdrafts, no partitioning is performed.

  t_scale_gap_thresholdint

  For the time scale of sampled events, the minimum amount of datapoints to define a new conditionally sampled event.

  Hfloat or dict

  Hyperbolic threshold criteria. If not specified 0 is used for all methods. If float: MREA, CEC, CEA, CECw get calculated using this threshold. If dict:

  Hyperbolic threshold per method used. If for a method no threshold is defined its set to 0.

  MREAfloat

  Hyperbolic threshold for MREA

  CECfloat

  Hyperbolic threshold for CEC

  CEAfloat

  Hyperbolic threshold for CEA

  CECwfloat

  Hyperbolic threshold for CECw

  loadfnctstr

  Function name as a string used for loading the data. Available options are:

  ”VersatileLoad”

  loading, renaming and recalculations can be done with this function. it basically makes the other loading functions useless apart from their shorter notation.

  ”NormLoad”

  basically pd.read_csv, pass the arguments to read the data as loadkwargs. the index gets to be the timestamp

  ”LoadBmmflux”

  custom function to read the BMMFlux high-frequency output files. BMMFlux is the EddyCovariance Software of the Micrometeorology Group in Bayreuth.

  See appendix of Thomas, C. K., Law, B. E., Irvine, J., Martin, J. G., Pettijohn, J. C., & Davis, K. J. (2009):

  Seasonal hydrology explains interannual and seasonal variation in carbon and water exchange in a semiarid mature ponderosa pine forest in central Oregon. Journal of Geophysical Research: Biogeosciences, 114(G4). https://doi.org/10.1029/2009JG001010

  loadkwargsdict

  Arguments passed to the pd.read_csv() function in case of NormLoad and VersatileLoad.

  versatile_loadkwargsdict

  Arguments passed to the VersatileLoad function.

  timestamp_colstr or list of str, optional

  • If a list: Combines split columns (e.g., [‘Year’, ‘Month’, ‘Day’]) into a datetime index.

  • If a string: Converts that specific column into the datetime index.

  • If None: Converts the default DataFrame index to datetime.

  convert_gasesbool, default False

  If True, converts ‘co2’ and ‘h2o’ from mmol/m³ to mg/m³ and g/m³ respectively.

  rename_colsdict, optional

  A dictionary mapping old column names to new ones (e.g., {“Pressure”: “P”}).

  select_colslist of str, optional

  A list of specific columns to keep. All other columns will be dropped.

  logginglevelint, optional, default 20

  The logging threshold for the root logger. Defaults to logging.INFO (20). Common values are logging.DEBUG (10), logging.INFO (20), or logging.WARNING (30).

Saves

df_datapandas.DataFrame

Processed and partioned data as csv-file with metadata header.

returns:

df_data – Processed and partioned data

rtype:

pandas.DataFrame