API Reference¶

This section provides an auto-generated API reference for the core components of the odoo-data-flow library.

Command-Line Interface (`main`)¶

This module contains the main click-based command-line interface.

odoo-data-flow¶

Odoo Data Flow: A tool for importing, exporting, and processing data.

odoo-data-flow [OPTIONS] COMMAND [ARGS]...

Options

--version¶: Show the version and exit.

-v, --verbose¶: Enable verbose, debug-level logging.

--log-file <log_file>¶: Path to a file to write logs to, in addition to the console.

export¶

Runs the data export process.

odoo-data-flow export [OPTIONS]

Options

-c, --config <config>¶

Configuration file for connection parameters.

Default:: conf/connection.conf

--file <filename>¶: Required Output file path.

--model <model>¶: Required Odoo model to export from.

--fields <fields>¶: Required Comma-separated list of fields to export.

--domain <domain>¶: Odoo domain filter as a list string.

--worker <worker>¶: Number of simultaneous connections.

--size <batch_size>¶: Number of records to process per batch.

-s, --sep <separator>¶: CSV separator character.

--context <context>¶: Odoo context as a dictionary string.

--encoding <encoding>¶: Encoding of the data file.

import¶

Runs the data import process.

odoo-data-flow import [OPTIONS]

Options

-c, --config <config>¶

Configuration file for connection parameters.

Default:: conf/connection.conf

--file <filename>¶: Required File to import.

--model <model>¶: Odoo model to import into. If not provided, it’s inferred from the filename.

--worker <worker>¶: Number of simultaneous connections.

--size <batch_size>¶: Number of lines to import per connection.

--skip <skip>¶: Number of initial lines to skip.

--fail¶: Run in fail mode, retrying records from the .fail.csv file.

-s, --sep <separator>¶: CSV separator character.

--groupby <split>¶: Column to group data by to avoid concurrent updates.

--ignore <ignore>¶: Comma-separated list of columns to ignore.

--check¶: Check if records are imported after each batch.

--context <context>¶: Odoo context as a dictionary string.

--o2m¶: Special handling for one-to-many imports.

--encoding <encoding>¶: Encoding of the data file.

migrate¶

Performs a direct server-to-server data migration.

odoo-data-flow migrate [OPTIONS]

Options

--config-export <config_export>¶: Required Path to the source Odoo connection config.

--config-import <config_import>¶: Required Path to the destination Odoo connection config.

--model <model>¶: Required The Odoo model to migrate.

--domain <domain>¶: Domain filter to select records for export.

--fields <fields>¶: Required Comma-separated list of fields to migrate.

--mapping <mapping>¶: A dictionary string defining the transformation mapping.

--export-worker <export_worker>¶: Number of workers for the export phase.

--export-batch-size <export_batch_size>¶: Batch size for the export phase.

--import-worker <import_worker>¶: Number of workers for the import phase.

--import-batch-size <import_batch_size>¶: Batch size for the import phase.

path-to-image¶

Converts columns with local file paths into base64 strings.

odoo-data-flow path-to-image [OPTIONS] FILE

Options

-f, --fields <fields>¶: Required Comma-separated list of fields to convert from path to base64.

--path <path>¶: Image path prefix. Defaults to the current working directory.

--out <out>¶: Name of the resulting output file.

Arguments

FILE¶: Required argument

url-to-image¶

Downloads content from URLs in columns and converts to base64.

odoo-data-flow url-to-image [OPTIONS] FILE

Options

-f, --fields <fields>¶: Required Comma-separated list of fields with URLs to convert to base64.

--out <out>¶: Name of the resulting output file.

Arguments

FILE¶: Required argument

workflow¶

Run post-import processing workflows.

odoo-data-flow workflow [OPTIONS] COMMAND [ARGS]...

invoice-v9¶

Runs the legacy Odoo v9 invoice processing workflow.

odoo-data-flow workflow invoice-v9 [OPTIONS]

Options

-c, --config <config>¶

Path to the connection configuration file.

Default:: conf/connection.conf

--action <actions>¶

Workflow action to run. Can be specified multiple times. Defaults to ‘all’.

Options:: tax | validate | pay | proforma | rename | all

--field <field>¶: Required The source field containing the legacy invoice status.

--status-map <status_map_str>¶: Required Dictionary string mapping Odoo states to legacy states. e.g., “{‘open’: [‘OP’]}”

--paid-date-field <paid_date_field>¶: Required The source field containing the payment date.

--payment-journal <payment_journal>¶: Required The database ID of the payment journal.

--max-connection <max_connection>¶: Number of parallel threads.

Transformation Processor (`lib.transform`)¶

This module contains the main Processor class used for data transformation.

This module contains the core Processor class for transforming data.

class odoo_data_flow.lib.transform.Processor(filename=None, separator=';', encoding='utf-8', header=None, data=None, preprocess=<function Processor.<lambda>>, **kwargs)¶

Core class for reading, transforming, and preparing data for Odoo.

Parameters:

filename (str | None)
separator (str)
encoding (str)
header (list[str] | None)
data (list[list[Any]] | None)
preprocess (Callable[[list[str], list[list[Any]]], tuple[list[str], list[list[Any]]]])
kwargs (Any)

check(check_fun, message=None)¶

Runs a data quality check function against the loaded data.

Parameters:

check_fun (Callable[[...], bool]) – The checker function to execute.
message (str | None) – An optional custom error message to display on failure.

Returns:

True if the check passes, False otherwise.

Return type:

bool

split(split_fun)¶

Splits the processor’s data into multiple new Processor objects.

Parameters:: split_fun (Callable[[...], Any]) – A function that takes a row dictionary and index, and returns a key to group the row by.
Returns:: A dictionary where keys are the grouping keys and values are new Processor instances containing the grouped data.
Return type:: dict[Any, Processor]

get_o2o_mapping()¶

Generates a direct 1-to-1 mapping dictionary.

Return type:: dict[str, MapperRepr]

process(mapping, filename_out, params=None, t='list', null_values=None, m2m=False, dry_run=False)¶

Processes the data using a mapping and prepares it for writing.

Parameters:

mapping (dict[str, Callable[[...], Any]]) – The mapping dictionary defining the transformation rules.
filename_out (str) – The path where the output CSV file will be saved.
params (dict[str, Any] | None) – A dictionary of parameters for the odoo-data-flow import command, used when generating the load script.
t (str) – The type of collection to return data in (‘list’ or ‘set’).
null_values (list[Any] | None) – A list of values to be treated as empty.
m2m (bool) – If True, activates special processing for many-to-many data.
dry_run (bool) – If True, prints a sample of the output to the console instead of writing files.

Returns:

A tuple containing the header list and the transformed data.

Return type:

tuple[list[str], list[Any] | set[tuple[Any, …]]]

write_to_file(script_filename, fail=True, append=False, python_exe='python', path='')¶

Generates the .sh script for the import.

Parameters:

script_filename (str) – The path where the shell script will be saved.
fail (bool) – If True, includes a second command with the –fail flag.
append (bool) – If True, appends to the script file instead of overwriting.
python_exe (str) – The python executable to use in the script.
path (str) – The path to prepend to the odoo-data-flow command.

Return type:

None

join_file(filename, master_key, child_key, header_prefix='child', separator=';', encoding='utf-8')¶

Joins data from a secondary file into the processor’s main data.

Parameters:

filename (str) – The path to the secondary file to join.
master_key (str) – The column name in the main data to join on.
child_key (str) – The column name in the secondary data to join on.
header_prefix (str) – A prefix to add to the headers from the child file.
separator (str) – The column separator for the child CSV file.
encoding (str) – The character encoding of the child file.

Return type:

None

Mapper Functions (`lib.mapper`)¶

This module contains all the built-in mapper functions for data transformation.

This module contains a library of mapper functions.

Mappers are the core building blocks for data transformations. Each function in this module is a “mapper factory” - it is a function that you call to configure and return another function, which will then be executed by the Processor for each row of the source data.

odoo_data_flow.lib.mapper.binary(field, path_prefix='', skip=False)¶

Returns a mapper that converts a local file to a base64 string.

Parameters:

field (str) – The source column containing the path to the file.
path_prefix (str) – An optional prefix to prepend to the file path.
skip (bool) – If True, raises SkippingError if the file is not found.

Returns:

A mapper function that returns the base64 encoded string.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.binary_url_map(field, skip=False)¶

Returns a mapper that downloads a file from a URL and converts to base64.

Parameters:

field (str) – The source column containing the URL.
skip (bool) – If True, raises SkippingError if the URL cannot be fetched.

Returns:

A mapper function that returns the base64 encoded string.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.bool_val(field, true_values)¶

Returns a mapper that converts a field value to a boolean ‘1’ or ‘0’.

Parameters:

field (str) – The source column to check.
true_values (list[str]) – A list of strings that should be considered True.

Returns:

A mapper function that returns “1” or “0”.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.concat(separator, *fields, skip=False)¶

Returns a mapper that joins values from multiple fields or static strings.

Parameters:

separator (str) – The string to place between each value.
*fields (Any) – A variable number of source column names or static strings.
skip (bool) – If True, raises SkippingError if the final result is empty.

Returns:

A mapper function that returns the concatenated string.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.concat_field_value_m2m(separator, *fields)¶

(Legacy V9-V12) Specialized concat for attribute value IDs.

Joins each field name with its value (e.g., ‘Color’ + ‘Blue’ -> ‘Color_Blue’), then joins all resulting parts with a comma. This was used to create unique external IDs for product.attribute.value records.

Parameters:

separator (str) – The character to join the field name and value with.
*fields (str) – The attribute columns to process.

Returns:

A mapper function that returns the concatenated string.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.concat_mapper_all(separator, *fields)¶

Returns a mapper that joins values, but only if all values exist.

If any of the values from the specified fields is empty, this mapper returns an empty string.

Parameters:

separator (str) – The string to place between each value.
*fields (Any) – A variable number of source column names or static strings.

Returns:

A mapper function that returns the concatenated string or an empty string.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.cond(field, true_mapper, false_mapper)¶

Returns a mapper that applies one of two mappers based on a condition.

Parameters:

field (str) – The source column to check for a truthy value.
true_mapper (Any) – The mapper to apply if the value in field is truthy.
false_mapper (Any) – The mapper to apply if the value in field is falsy.

Returns:

A mapper function that returns the result of the chosen mapper.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.const(value)¶

Returns a mapper that always provides a constant value.

Parameters:: value (Any)
Return type:: Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.field(col)¶

Returns the column name itself if the column has a value.

This is useful for some dynamic product attribute mappings.

Parameters:: col (str) – The name of the column to check.
Returns:: A mapper function that returns the column name or an empty string.
Return type:: Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2m(prefix, *fields, sep=',')¶

Returns a mapper that creates a comma-separated list of Many2many external IDs.

This mapper has two modes: 1. Multi-column: If multiple fields are provided, it treats the value of

each field as a single ID.

Single-column: If one field is provided, it splits the value of that field by the separator sep.

Parameters:

prefix (str) – The XML ID prefix to apply to each value.
*fields (Any) – One or more source column names.
sep (str) – The separator to use when splitting a single field.

Returns:

A mapper function that returns a comma-separated string of external IDs.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2m_attribute_value(prefix, *fields)¶

(Legacy V9-V12) Creates a list of external IDs for attribute values.

This is a composite mapper for the legacy product attribute workflow.

Parameters:

prefix (str) – The XML ID prefix.
*fields (str) – The attribute columns to process.

Returns:

A mapper that returns a comma-separated string of external IDs.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2m_id_list(prefix, *fields, sep=',')¶

Returns a mapper for creating a list of M2M external IDs.

This is primarily used when creating the related records for a M2M field, such as creating all unique res.partner.category records.

Parameters:

prefix (str) – The XML ID prefix to apply to each value.
*fields (Any) – One or more source fields to read values from.
sep (str) – The separator to use when splitting values.

Returns:

A mapper function that returns a comma-separated string of external IDs.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2m_map(prefix, mapper_func)¶

Returns a mapper that wraps another mapper for Many2many fields.

It takes the comma-separated string result of another mapper and applies the to_m2m formatting to it.

Parameters:

prefix (str) – The XML ID prefix to apply.
mapper_func (Callable[[dict[str, Any], dict[str, Any]], Any]) – The inner mapper function to execute first.

Returns:

A mapper function that returns a formatted m2m external ID list.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2m_template_attribute_value(prefix, *fields)¶

(Modern V13+) Creates a comma-separated list of attribute values.

This mapper concatenates the values of the given fields. This is used for the modern product attribute system where Odoo automatically creates the product.attribute.value records from the raw value names.

It will return an empty string if the template_id is missing from the source line, preventing the creation of orphaned attribute lines.

Parameters:

prefix (str) – (Unused) Kept for backward compatibility.
*fields (Any) – The attribute columns (e.g. ‘Color’, ‘Size’) to get values from.

Returns:

A mapper that returns a comma-separated string of attribute values.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2m_value_list(*fields, sep=',')¶

Returns a mapper that creates a Python list of unique values.

This is used in conjunction with m2m_id_list when creating related records for a M2M field.

Parameters:

*fields (Any) – One or more source fields to read values from.
sep (str) – The separator to use when splitting values.

Returns:

A mapper function that returns a list of strings.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2o(prefix, field, default='', skip=False)¶

Returns a mapper that creates a Many2one external ID from a field’s value.

Parameters:

prefix (str) – The XML ID prefix (e.g., ‘my_module’).
field (str) – The source column containing the value for the ID.
default (str) – The value to return if the source value is empty.
skip (bool) – If True, raises SkippingError if the source value is empty.

Returns:

A mapper function that returns the formatted external ID.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2o_att(prefix, att_list)¶

(Legacy V9-V12) Returns a dictionary of attribute-to-ID mappings.

This is a helper for legacy product attribute workflows where IDs for attribute values were manually constructed.

Parameters:

prefix (str) – The XML ID prefix to use for the attribute value IDs.
att_list (list[str]) – A list of attribute column names to process.

Returns:

A mapper function that returns a dictionary.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2o_att_name(prefix, att_list)¶

Returns a mapper that creates a dictionary of attribute-to-ID mappings.

This is used in legacy product import workflows.

Parameters:

prefix (str) – The XML ID prefix to use for the attribute IDs.
att_list (list[str]) – A list of attribute column names to check for.

Returns:

A mapper function that returns a dictionary.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.m2o_map(prefix, *fields, default='', skip=False)¶

Returns a mapper that creates a Many2one external ID by concatenating fields.

This is useful when the unique identifier for a record is spread across multiple columns.

Parameters:

prefix (str) – The XML ID prefix (e.g., ‘my_module’).
*fields (Any) – A variable number of source column names or static strings to join.
default (str) – The value to return if the final concatenated value is empty.
skip (bool) – If True, raises SkippingError if the final result is empty.

Returns:

A mapper function that returns the formatted external ID.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.map_val(mapping_dict, key_mapper, default='', m2m=False)¶

Returns a mapper that translates a value using a provided dictionary.

Parameters:

mapping_dict (dict[Any, Any]) – The dictionary to use as a translation table.
key_mapper (Any) – A mapper that provides the key to look up.
default (Any) – A default value to return if the key is not found.
m2m (bool) – If True, splits the key by commas and translates each part.

Returns:

A mapper function that returns the translated value.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.num(field, default='0.0')¶

Returns a mapper that standardizes a numeric string.

It replaces all commas with dots.

Parameters:

field (str) – The source column containing the numeric string.
default (str) – The default value to use if the source value is empty.

Returns:

A mapper function that returns the standardized numeric string.

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.record(mapping)¶

Returns a mapper that processes a sub-mapping for a related record.

Used for creating one-to-many records (e.g., sales order lines).

Parameters:: mapping (dict[str, Callable[[dict[str, Any], dict[str, Any]], Any]]) – A mapping dictionary for the related record.
Returns:: A mapper function that returns a dictionary of the processed sub-record.
Return type:: Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.split_file_number(file_nb)¶

Returns a function to split data across a fixed number of chunks.

Parameters:: file_nb (int) – The total number of chunks to create.
Returns:: A function compatible with the Processor.split method.
Return type:: Callable[[dict[str, Any], int], int]

odoo_data_flow.lib.mapper.split_line_number(line_nb)¶

Returns a function to split data into chunks of a specific line count.

Parameters:: line_nb (int) – The number of lines per chunk.
Returns:: A function compatible with the Processor.split method.
Return type:: Callable[[dict[str, Any], int], int]

odoo_data_flow.lib.mapper.to_m2m(prefix, value)¶

Creates a comma-separated list of external IDs .

Creates a comma-separated list of external IDs for a Many2many relationship. It takes a string of comma-separated values, sanitizes each one, and prepends the prefix.

Parameters:

prefix (str) – The XML ID prefix to apply to each value.
value (str) – A single string containing one or more values,
commas. (separated by)

Returns:

A comma-separated string of formatted external IDs.

Return type:

str

odoo_data_flow.lib.mapper.to_m2o(prefix, value, default='')¶

Creates a full external ID for a Many2one relationship.

Creates a full external ID for a Many2one relationship by combining a prefix and a sanitized value.

Parameters:

prefix (str) – The XML ID prefix (e.g., ‘my_module’).
value (Any) – The value to be sanitized and appended to the prefix.
default (str) – The value to return if the input value is empty.

Returns:

The formatted external ID (e.g., ‘my_module.sanitized_value’).

Return type:

str

odoo_data_flow.lib.mapper.val(field, default='', postprocess=<function <lambda>>, skip=False)¶

Returns a mapper that gets a value from a specific field in the row.

Parameters:

field (str)
default (Any)
postprocess (Callable[[...], Any])
skip (bool)

Return type:

Callable[[dict[str, Any], dict[str, Any]], Any]

odoo_data_flow.lib.mapper.val_att(att_list)¶

(Legacy V9-V12) Returns a dictionary of attributes that have a value.

This is a helper for legacy product attribute workflows.

Parameters:: att_list (list[str]) – A list of attribute column names to check.
Returns:: A mapper function that returns a dictionary.
Return type:: Callable[[dict[str, Any], dict[str, Any]], Any]

High-Level Runners¶

These modules contain the high-level functions that are called by the CLI commands.

Importer (`importer`)¶

This module contains the core logic for importing data into Odoo.

odoo_data_flow.importer.run_import(config, filename, model=None, worker=1, batch_size=10, skip=0, fail=False, separator=';', split=None, ignore=None, check=False, context="{'tracking_disable' : True}", o2m=False, encoding='utf-8')¶

Orchestrates the data import process from a CSV file.

Parameters:

config (str) – Path to the connection configuration file.
filename (str) – Path to the source CSV file to import.
model (str | None) – The Odoo model to import data into. If not provided, it’s inferred from the filename.
worker (int) – The number of simultaneous connections to use.
batch_size (int) – The number of records to process in each batch.
skip (int) – The number of initial lines to skip in the source file.
fail (bool) – If True, runs in fail mode, retrying records from the .fail file.
separator (str) – The delimiter used in the CSV file.
split (str | None) – The column name to group records by to avoid concurrent updates.
ignore (str | None) – A comma-separated string of column names to ignore.
check (bool) – If True, checks if records were successfully imported.
context (str) – A string representation of the Odoo context dictionary.
o2m (bool) – If True, enables special handling for one-to-many imports.
encoding (str) – The file encoding of the source file.

Return type:

None

Exporter (`exporter`)¶

This module contains the core logic for exporting data from Odoo.

odoo_data_flow.exporter.run_export(config, filename, model, fields, domain='[]', worker=1, batch_size=10, separator=';', context="{'tracking_disable' : True}", encoding='utf-8')¶

Export runner.

Orchestrates the data export process, writing the output to a CSV file. This function is designed to be called from the main CLI.

Parameters:

config (str)
filename (str)
model (str)
fields (str)
domain (str)
worker (int)
batch_size (int)
separator (str)
context (str)
encoding (str)

Return type:

None

Migrator (`migrator`)¶

Migrate data between two odoo databases.

This module contains the logic for performing a direct, in-memory migration of data from one Odoo instance to another.

odoo_data_flow.migrator.run_migration(config_export, config_import, model, domain='[]', fields=None, mapping=None, export_worker=1, export_batch_size=100, import_worker=1, import_batch_size=10)¶

Performs a server-to-server data migration.

This function chains together the export, transform, and import processes without creating intermediate files.

Parameters:

config_export (str)
config_import (str)
model (str)
domain (str)
fields (list[str] | None)
mapping (dict[str, Callable[[...], Any]] | None)
export_worker (int)
export_batch_size (int)
import_worker (int)
import_batch_size (int)

Return type:

None

API Reference¶

Command-Line Interface (__main__)¶

odoo-data-flow¶

export¶

import¶

migrate¶

path-to-image¶

url-to-image¶

workflow¶

invoice-v9¶

Transformation Processor (lib.transform)¶

Mapper Functions (lib.mapper)¶

High-Level Runners¶

Importer (importer)¶

Exporter (exporter)¶

Migrator (migrator)¶

Command-Line Interface (`main`)¶

Transformation Processor (`lib.transform`)¶

Mapper Functions (`lib.mapper`)¶

Importer (`importer`)¶

Exporter (`exporter`)¶

Migrator (`migrator`)¶