YAML specification¶

The YAML specification defines the simulation or analysis to be executed. The YAML contains the following sections:

definitions: includes motifs, signals, and simulation details - these define conceptually what the user would like to simulate,
instructions: cover the specific simulation or analysis to be run - usually including only technical information on how to perform the task and reference the components defined under definitions for the conceptual information;
output: only HTML format is supported, and doesn’t include any parameters.

The purpose of this page is to list all the YAML specification options.

The overall structure of the YAML specification is the following:

definitions: # mandatory keyword
  motifs:
    my_motif_1: ...
  signals:
    my_sig_1: ...
  simulations:
    my_first_simulation: ...
instructions: # mandatory keyword - at least one instruction has to be specified
  my_instruction_1: # user-defined name of the instruction
    ... # see below for the specification of different instructions
output: # how to present the result after running (the only valid option now is HTML)
  format: HTML

Definitions¶

Simulation¶

SeedMotif¶

Describes motifs by seed, possible gaps, allowed hamming distances, positions that can be changed and what they can be changed to.

Arguments:

seed (str): An amino acid or nucleotide sequence that represents the basic motif seed. The forward slash (/) symbol corresponds to a gap position (only one gap per seed is allowed). All implanted motifs correspond to the seed, or a modified version thereof, as specified in its instantiation strategy. If this argument is set, seed_chain1 and seed_chain2 arguments are not used.
min_gap (int): The minimum gap length, in case the original seed contains a gap.
max_gap (int): The maximum gap length, in case the original seed contains a gap.
hamming_distance_probabilities (dict): A distribution describing the probability of the given seed being modified with a certain number of random mutations, which we refer to as modifications. The keys represent the number of modifications (hamming distance) between the original seed and the implanted motif, and the values represent the probabilities for the respective number of modifications. For example, {0: 0.7, 1: 0.3} means that if signal implanting strategy is used, 30% of the time one position of the seed will be modified, and the remaining 70% of the time the motif will remain unmodified with respect to the seed. The values of hamming_distance_probabilities must sum to 1. If signal-specific receptors are generated with rejection sampling strategy, then any sequence containing the seed with the allowed number of mismatches (non-zero probabilities) will be selected.
position_weights (dict): A distribution across seed positions describing probabilities of modifying each seed position. The keys represent the position in the seed, where counting starts at 0. If the index of a gap is specified in position_weights, it will be removed. The values represent the relative probabilities for modifying each position when it gets selected for modification. For example {0: 0.6, 1: 0, 2: 0.4} means that if signal implanting strategy is used, when a sequence is selected for a modification (as specified in hamming_distance_probabilities), then 60% of the time the amino acid at index 0 is modified, and the remaining 40% of the time the amino acid at index 2. If the values of position_weights do not sum to 1, the remainder will be redistributed over all positions, including those not specified. If signal-specific receptors are generated with rejection sampling strategy, then modifications are only allowed at the seed positions with non-zero probabilities.
alphabet_weights (dict): A distribution across the nucleotide or amino acid alphabet describing the probability of a given nucleotide or amino acid to be used for modification. The keys of the dictionary represent the amino acids and the values are the relative probabilities for choosing this amino acid. If the values of alphabet_weights do not sum to 1, the remainder will be redistributed over all possible amino acids, including those not specified. If signal-specific receptors are generated with rejection sampling strategy, then any sequence containing the seed with modifications having non-zero probabilities will be selected.

YAML specification:

motifs:
    # examples for single chain receptor data
    my_simple_motif: # this will be the identifier of the motif
        seed: AAA # motif is always AAA
    my_gapped_motif:
        seed: AA/A # this motif can be AAA, AA_A, CAA, CA_A, DAA, DA_A, EAA, EA_A
        min_gap: 0
        max_gap: 1
        hamming_distance_probabilities: # it can have a max of 1 substitution
            0: 0.7
            1: 0.3
        position_weights: # note that index 2, the position of the gap, is excluded from position_weights
            0: 1 # only first position can be changed
            1: 0
            3: 0
        alphabet_weights: # the first A can be replaced by C, D or E
            C: 0.4
            D: 0.4
            E: 0.2

PWM¶

Motifs defined by a positional weight matrix and using bionumpy’s PWM internally. For more details on bionumpy’s implementation of PWM, as well as for supported formats, see the documentation at https://bionumpy.github.io/bionumpy/tutorials/position_weight_matrix.html.

Arguments:

file_path: path to the file where the PWM is stored
threshold (float): when matching PWM to a sequence, this is the threshold to consider the sequence as containing the motif

YAML specification:

motifs:
    my_custom_pwm: # this will be the identifier of the motif
        file_path: my_pwm_1.jaspar
        threshold: 2

Signal¶

This class represents the simulated signal. A signal is represented by a list of motifs, and optionally, position weights showing where one of the motifs of the signal can occur in a sequence.

A signal is associated with a metadata label, which is assigned to a receptor or repertoire. For example antigen-specific/disease-associated (receptor) or diseased (repertoire).

Note

IMGT positions

To use sequence position weights, IMGT positions should be explicitly specified as strings, under quotation marks, to allow for all positions to be properly distinguished.

Arguments:

motifs (list): A list of the motifs associated with this signal, either defined by seed or by position weight matrix. Alternatively, it can be a list of a list of motifs, in which case the motifs in the same sublist (max 2 motifs) have to co-occur in the same sequence
sequence_position_weights (dict): a dictionary specifying for each IMGT position in the sequence how likely it is for the signal to be there. If the position is not present in the sequence, the probability of the signal occurring at that position will be redistributed to other positions with probabilities that are not explicitly set to 0 by the user.
v_call (str): V gene with allele if available that has to co-occur with one of the motifs for the signal to exist; can be used in combination with rejection sampling, or full sequence implanting, otherwise ignored; to match in a sequence for rejection sampling, it is checked if this value is contained in the same field of generated sequence;
j_call (str): J gene with allele if available that has to co-occur with one of the motifs for the signal to exist; can be used in combination with rejection sampling, or full sequence implanting, otherwise ignored; to match in a sequence for rejection sampling, it is checked if this value is contained in the same field of generated sequence;
source_file (str): path to the file where the custom signal function is; cannot be combined with the arguments listed above (motifs, v_call, j_call, sequence_position_weights)
is_present_func (str): name of the function from the source_file file that will be used to specify the signal; the function’s signature must be:

def is_present(sequence_aa: str, sequence: str, v_call: str, j_call: str) -> bool:
    # custom implementation where all or some of these arguments can be used

clonal_frequency (dict): clonal frequency in Ligo is simulated through scipy’s zeta distribution function for generating random numbers, with parameters provided under clonal_frequency parameter. If clonal frequency should not be used, this parameter can be None

clonal_frequency:
  a: 2 # shape parameter of the distribution
  loc: 0 # 0 by default but can be used to shift the distribution

YAML specification:

signals:
    my_signal:
        motifs:
            - my_simple_motif
            - my_gapped_motif
        sequence_position_weights:
            '109': 0.5
            '110': 0.5
        v_call: TRBV1
        j_call: TRBJ1
        clonal_frequency:
            a: 2
            loc: 0
    signal_with_custom_func:
        source_file: signal_func.py
        is_present_func: is_signal_present
        clonal_frequency:
            a: 2
            loc: 0

Simulation config¶

Defines all parameters of the simulation.

Arguments:

sim_items (dict): a list of SimConfigItems defining individual units of simulation
is_repertoire (bool): whether the simulation is on a repertoire (person) or sequence/receptor level
paired: if the simulation should output paired data, this parameter should contain a list of a list of sim_item pairs referenced by name that should be combined; if paired data is not needed, then it should be False
sequence_type (str): either amino_acid or nucleotide
simulation_strategy (str): either RejectionSampling or Implanting, see the tutorials for more information on choosing one of these
keep_p_gen_dist (bool): if possible, whether to implement the importance sampling strategy, i.e., keep the distribution of generation probabilities of the sequences the same as provided by the model without any signals
p_gen_bin_count (int): if keep_p_gen_dist is true, how many bins to use to approximate the generation probability distribution
remove_seqs_with_signals (bool): if true, it explicitly controls the proportions of signals in sequences and removes any accidental occurrences
species (str): species that the sequences come from; used to select correct genes to export full length sequences; default is ‘human’
implanting_scaling_factor (int): determines in how many receptors to implant the signal in reach iteration; this is computed as number_of_receptors_needed_for_signal * implanting_scaling_factor; useful when using Implanting simulation strategy in combination with importance sampling, since the generation probability of some receptors with implanted signals might be very rare and those receptors might end up not being kept often with importance sampling; this parameter is only used when keep_p_gen_dist is set to True

YAML specification:

simulations:
  sim1:
    is_repertoire: false
    paired: false
    sequence_type: amino_acid
    simulation_strategy: RejectionSampling
    sim_items:
      sim_item1: # group of sequences with same simulation params
        generative_model:
          chain: beta
          default_model_name: humanTRB
          model_path: null
          type: OLGA
        number_of_examples: 100
        signals:
         signal1: 1

Simulation config item¶

When performing a simulation, one or more simulation config items can be specified. Config items define groups of repertoires or receptors that have the same simulation parameters, such as signals, generative model, clonal frequencies, noise parameters.

Arguments:

signals (dict): signals for the simulation item and the proportion of sequences in the repertoire that will have the given signal. For receptor-level simulation, the proportion will always be 1.
is_noise (bool): indicates whether the implanting should be regarded as noise; if it is True, the signals will be implanted as specified, but the repertoire/receptor in question will have negative class.
generative_model: parameters of the generative model, including its type, path to the model; currently supported models are OLGA and ExperimentalImport
false_positives_prob_in_receptors (float): when performing repertoire level simulation, what percentage of sequences should be false positives
false_negative_prob_in_receptors (float): when performing repertoire level simulation, what percentage of sequences should be false negatives
immune_events (dict): a set of key-value pairs that will be added to the metadata (same values for all data generated in one simulation sim_item) and can be later used as labels
default_clonal_frequency (dict): clonal frequency in Ligo is simulated through scipy’s zeta distribution function for generating random numbers, with parameters provided under default_clonal_frequency parameter. These parameters will be used to assign count values to sequences that do not contain any signals if they are required by the simulation. If clonal frequency shouldn’t be used, this parameter can be None
clonal_frequency: a: 2 # shape parameter of the distribution loc: 0 # 0 by default but can be used to shift the distribution
sequence_len_limits (dict): allows for filtering the generated sequences by length, needs to have parameters min and max specified; if not used, min/max should be -1
sequence_len_limits: min: 4 # keep sequences of length 4 and longer max: -1 # no limit on the max length of the sequences

YAML specification:

simulations: # definitions of simulations should be under key simulations in the definitions part of the specification
    # one simulation with multiple implanting objects, a part of definition section
    my_simulation:
        sim_item1:
            number_of_examples: 10
            seed: null # don't use seed
            receptors_in_repertoire_count: 100
            generative_model:
                chain: beta
                default_model_name: humanTRB
                model_path: null
                type: OLGA
            signals:
                my_signal: 0.25
                my_signal2: 0.01
                my_signal__my_signal2: 0.02 # my_signal and my_signal2 will co-occur in 2% of the receptors in all 10 repertoires
        sim_item2:
            number_of_examples: 5
            receptors_in_repertoire_count: 150
            generative_model:
                chain: beta
                default_model_name: humanTRB
                model_path: null
                type: OLGA
            signals:
                my_signal: 0.75
            default_clonal_frequency:
                a: 2
            sequence_len_limits:
                min: 3

Generative models¶

ExperimentalImport¶

Allows to import existing experimental data and perform annotations and simulations on top of them.

Arguments:

import_format (str): see the list of supported formats under Supported dataset formats
tmp_import_path (str): where to store the imported files
import_params (dict): as defined under the import format selected in the first parameter; for details see Supported dataset formats

YAML specification:

generative_model:
    import_format: AIRR
    tmp_import_path: ./tmp/
    import_params:
        path: path/to/files/
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping AIRR: ligo
            junction: sequence
            junction_aa: sequence_aa
            locus: chain
    type: ExperimentalImport

OLGA¶

This is a wrapper for the OLGA package as described by Sethna et al. 2019 (OLGA package on PyPI or GitHub: https://github.com/statbiophys/OLGA).

Reference:

Zachary Sethna, Yuval Elhanati, Curtis G Callan, Jr, Aleksandra M Walczak, Thierry Mora, OLGA: fast computation of generation probabilities of B- and T-cell receptor amino acid sequences and motifs, Bioinformatics, Volume 35, Issue 17, 1 September 2019, Pages 2974–2981, https://doi.org/10.1093/bioinformatics/btz035

Note:

OLGA generates sequences that correspond to IMGT junction and are used for matching as such. See the https://github.com/statbiophys/OLGA for more details.
Gene names are as provided in OLGA (either in default models or in the user-specified model files). For simulation, one should use gene names in the same format.

Arguments:

model_path (str): if not default model, this parameter should point to a folder where the four OLGA/IGOR format files are stored (could also be inferred from some experimental data)
default_model_name (str): if not using custom models, one of the OLGA default models could be specified here; the value should be the same as it would be passed to command line in OLGA: e.g., humanTRB, human IGH

YAML specification:

generative_model:
    type: OLGA
    model_path: None
    default_model_name: humanTRB

Supported dataset formats¶

These dataset formats can be used in combination with ExperimentalImport generative model.

AIRR¶

Imports data in AIRR format. The AIRR .tsv format is explained here: https://docs.airr-community.org/en/stable/datarep/format.html And the AIRR rearrangement schema can be found here: https://docs.airr-community.org/en/stable/datarep/rearrangements.html

Arguments:

path (str): Either the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): Should be set to False, as we import a SequenceDataset.
import_productive (bool): Whether productive sequences (with value ‘T’ in column productive) should be included in the imported sequences. By default, import_productive is True.
import_with_stop_codon (bool): Whether sequences with stop codons (with value ‘T’ in column stop_codon) should be included in the imported sequences. This only applies if column stop_codon is present. By default, import_with_stop_codon is False.
import_out_of_frame (bool): Whether out of frame sequences (with value ‘F’ in column vj_in_frame) should be included in the imported sequences. This only applies if column vj_in_frame is present. By default, import_out_of_frame is False.
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as AIRR uses the IMGT junction. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): A mapping from AIRR column names to immuneML’s internal data representation. A custom column mapping can be specified here if necessary (for example; adding additional data fields if they are present in the AIRR file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’].. For AIRR, this is by default set to:
junction: sequence junction_aa: sequence_aa locus: chain
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For AIRR format, there is no default column_mapping_synonyms.
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are AIRR column names and values are the names that are internally used in immuneML as metadata fields. These metadata fields can be used as prediction labels for Sequence- and ReceptorDatasets. This parameter can also be used to specify sequence-level metadata columns for RepertoireDatasets, which can be used by reports. To set prediction label metadata for RepertoireDatasets, see metadata_file instead. For AIRR format, there is no default metadata_column_mapping.
separator (str): Column separator, for AIRR this is by default “t”.

YAML specification:

my_airr_dataset:
    format: AIRR
    params:
        path: path/to/files/
        is_repertoire: False
        import_productive: True # whether to include productive sequences in the dataset
        import_with_stop_codon: False # whether to include sequences with stop codon in the dataset
        import_out_of_frame: False # whether to include out of frame sequences in the dataset
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even if the `sequences` column is empty (provided that other fields are as specified here)
        import_empty_aa_sequences: False # remove all sequences with empty `sequence_aa` column
        # Optional fields with AIRR-specific defaults, only change when different behavior is required:
        separator: "\t" # column separator
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping AIRR: immuneML
            junction: sequence
            junction_aa: sequence_aa
            locus: chain

Generic¶

Imports data from any tabular file into a Repertoire-, Sequence- or ReceptorDataset. RepertoireDatasets should be used when making predictions per repertoire, such as predicting a disease state. SequenceDatasets or ReceptorDatasets should be used when predicting values for unpaired (single-chain) and paired immune receptors respectively, like antigen specificity.

This importer works similarly to other importers, but has no predefined default values for which fields are imported, and can therefore be tailored to import data from various different tabular files with headers.

For ReceptorDatasets: this importer assumes the two receptor sequences appear on different lines in the file, and can be paired together by a common sequence identifier. If you instead want to import a ReceptorDataset from a tabular file that contains both receptor chains on one line, see SingleLineReceptor import

Arguments:

path (str): For RepertoireDatasets, this is the path to a directory with files to import. For Sequence- or ReceptorDatasets this path may either be the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset or ReceptorDataset. By default, is_repertoire is set to True.
metadata_file (str): Required for RepertoireDatasets. This parameter specifies the path to the metadata file. This is a csv file with columns filename, subject_id and arbitrary other columns which can be used as labels in instructions. For setting Sequence- or ReceptorDataset metadata, metadata_file is ignored, see metadata_column_mapping instead.
paired (str): Required for Sequence- or ReceptorDatasets. This parameter determines whether to import a SequenceDataset (paired = False) or a ReceptorDataset (paired = True). In a ReceptorDataset, two sequences with chain types specified by receptor_chains are paired together based on a common identifier. This identifier should be mapped to the immuneML field ‘sequence_identifiers’ using the column_mapping.
receptor_chains (str): Required for ReceptorDatasets. Determines which pair of chains to import for each Receptor. Valid values are TRA_TRB, TRG_TRD, IGH_IGL, IGH_IGK.
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means immuneML assumes the IMGT junction (including leading C and trailing Y/F amino acids) is used in the input file, and the first and last amino acids will be removed from the sequences to retrieve the IMGT CDR3 sequence. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): Required for all datasets. A mapping where the keys are the column names in the input file, and the values correspond to the names used in immuneML’s internal data representation. Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’].. At least sequences (nucleotide) or sequence_aas (amino acids) must be specified, but all other fields are optional. A column mapping can look for example like this:
file_column_amino_acids: sequence_aa file_column_v_genes: v_call file_column_j_genes: j_call file_column_frequencies: duplicate_count
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For Generic import, there is no default column_mapping_synonyms.
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are file column names and values are the names that are internally used in immuneML as metadata fields. These metadata fields can be used as prediction labels for Sequence- and ReceptorDatasets. This parameter can also be used to specify sequence-level metadata columns for RepertoireDatasets, which can be used by reports. To set prediction label metadata for RepertoireDatasets, see metadata_file instead. There is no default metadata_column_mapping.
file_column_antigen_specificity: antigen_specificity
columns_to_load (list): Optional; specifies which columns to load from the input file. This may be useful if the input files contain many unused columns. If no value is specified, all columns are loaded.
separator (str): Required parameter. Column separator, for example “t” or “,”. The default value is “t”

YAML specification:

my_generic_dataset:
    format: Generic
    params:
        path: path/to/files/
        is_repertoire: True # whether to import a RepertoireDataset
        metadata_file: path/to/metadata.csv # metadata file for RepertoireDataset
        paired: False # whether to import SequenceDataset (False) or ReceptorDataset (True) when is_repertoire = False
        receptor_chains: TRA_TRB # what chain pair to import for a ReceptorDataset
        separator: "\t" # column separator
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        import_empty_aa_sequences: False # filter out sequences if they don't have sequence_aa set
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping file: immuneML
            file_column_amino_acids: sequence_aas
            file_column_v_genes: v_call
            file_column_j_genes: j_call
            file_column_frequencies: duplicate_count
        metadata_column_mapping: # metadata column mapping file: immuneML
            file_column_antigen_specificity: antigen_specificity
        columns_to_load:  # which subset of columns to load from the file
            - file_column_amino_acids
            - file_column_v_genes
            - file_column_j_genes
            - file_column_frequencies
            - file_column_antigen_specificity

IGoR¶

Imports data generated by IGoR simulations into a Repertoire-, or SequenceDataset. RepertoireDatasets should be used when making predictions per repertoire, such as predicting a disease state. SequenceDatasets should be used when predicting values for unpaired (single-chain) immune receptors, like antigen specificity.

Note that you should run IGoR with the –CDR3 option specified, this tool imports the generated CDR3 files. Sequences with missing anchors are not imported, meaning only sequences with value ‘1’ in the anchors_found column are imported. Nucleotide sequences are automatically translated to amino acid sequences.

Reference: Quentin Marcou, Thierry Mora, Aleksandra M. Walczak ‘High-throughput immune repertoire analysis with IGoR’. Nature Communications, (2018) doi.org/10.1038/s41467-018-02832-w.

Arguments:

path (str): For RepertoireDatasets, this is the path to a directory with IGoR files to import. For Sequence- or ReceptorDatasets this path may either be the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset. By default, is_repertoire is set to True.
metadata_file (str): Required for RepertoireDatasets. This parameter specifies the path to the metadata file. This is a csv file with columns filename, subject_id and arbitrary other columns which can be used as labels in instructions. Only the IGoR files included under the column ‘filename’ are imported into the RepertoireDataset. For setting SequenceDataset metadata, metadata_file is ignored, see metadata_column_mapping instead.
import_with_stop_codon (bool): Whether sequences with stop codons should be included in the imported sequences. By default, import_with_stop_codon is False.
import_out_of_frame (bool): Whether out of frame sequences (with value ‘0’ in column is_inframe) should be included in the imported sequences. By default, import_out_of_frame is False.
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as IGoR uses the IMGT junction. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): A mapping from IGoR column names to immuneML’s internal data representation. A custom column mapping can be specified here if necessary (for example; adding additional data fields if they are present in the IGoR file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’].. For IGoR, this is by default set to:
nt_CDR3: sequences seq_index: sequence_identifiers
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For IGoR format, there is no default column_mapping_synonyms.
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are IGoR column names and values are the names that are internally used in immuneML as metadata fields. These metadata fields can be used as prediction labels for Sequence- and ReceptorDatasets. This parameter can also be used to specify sequence-level metadata columns for RepertoireDatasets, which can be used by reports. To set prediction label metadata for RepertoireDatasets, see metadata_file instead. For IGoR format, there is no default metadata_column_mapping.
separator (str): Column separator, for IGoR this is by default “,”.

YAML specification:

my_igor_dataset:
    format: IGoR
    params:
        path: path/to/files/
        is_repertoire: True # whether to import a RepertoireDataset (True) or a SequenceDataset (False)
        metadata_file: path/to/metadata.csv # metadata file for RepertoireDataset
        metadata_column_mapping: # metadata column mapping IGoR: immuneML for SequenceDataset
            igor_column_name1: metadata_label1
            igor_column_name2: metadata_label2
        import_with_stop_codon: False # whether to include sequences with stop codon in the dataset
        import_out_of_frame: False # whether to include out of frame sequences in the dataset
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        # Optional fields with IGoR-specific defaults, only change when different behavior is required:
        separator: "," # column separator
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping IGoR: immuneML
            nt_CDR3: sequences
            seq_index: sequence_identifiers

IReceptor¶

Imports AIRR datasets retrieved through the iReceptor Gateway into a Repertoire-, Sequence- or ReceptorDataset. The differences between this importer and the AIRR importer are:

This importer takes in a list of .zip files, which must contain one or more AIRR tsv files, and for each AIRR file, a corresponding metadata json file must be present.
This importer does not require a metadata csv file for RepertoireDataset import, it is generated automatically from the metadata json files.

RepertoireDatasets should be used when making predictions per repertoire, such as predicting a disease state. SequenceDatasets or ReceptorDatasets should be used when predicting values for unpaired (single-chain) and paired immune receptors respectively, like antigen specificity.

AIRR rearrangement schema can be found here: https://docs.airr-community.org/en/stable/datarep/rearrangements.html

When importing a ReceptorDataset, the AIRR field cell_id is used to determine the chain pairs.

Arguments:

path (str): This is the path to a directory with .zip files retrieved from the iReceptor Gateway. These .zip files should include AIRR files (with .tsv extension) and corresponding metadata.json files with matching names (e.g., for my_dataset.tsv the corresponding metadata file is called my_dataset-metadata.json). The zip files must use the .zip extension.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset or ReceptorDataset. By default, is_repertoire is set to True.
paired (str): Required for Sequence- or ReceptorDatasets. This parameter determines whether to import a SequenceDataset (paired = False) or a ReceptorDataset (paired = True). In a ReceptorDataset, two sequences with chain types specified by receptor_chains are paired together based on the identifier given in the AIRR column named ‘cell_id’.
receptor_chains (str): Required for ReceptorDatasets. Determines which pair of chains to import for each Receptor. Valid values are TRA_TRB, TRG_TRD, IGH_IGL, IGH_IGK. If receptor_chains is not provided, the chain pair is automatically detected (only one chain pair type allowed per repertoire).
import_productive (bool): Whether productive sequences (with value ‘T’ in column productive) should be included in the imported sequences. By default, import_productive is True.
import_with_stop_codon (bool): Whether sequences with stop codons (with value ‘T’ in column stop_codon) should be included in the imported sequences. This only applies if column stop_codon is present. By default, import_with_stop_codon is False.
import_out_of_frame (bool): Whether out of frame sequences (with value ‘F’ in column vj_in_frame) should be included in the imported sequences. This only applies if column vj_in_frame is present. By default, import_out_of_frame is False.
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as AIRR uses the IMGT junction. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): A mapping from AIRR column names to immuneML’s internal data representation. For AIRR, this is by default set to the values shown in YAML below. A custom column mapping can be specified here if necessary (for example; adding additional data fields if they are present in the AIRR file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’]. A custom column mapping can be specified here if necessary (for example; adding additional data fields if they are present in the AIRR file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’]..
junction: sequences junction_aa: sequence_aas v_call: v_alleles j_call: j_alleles locus: chains duplicate_count: counts sequence_id: sequence_identifiers
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For AIRR format, there is no default column_mapping_synonyms.
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are AIRR column names and values are the names that are internally used in immuneML as metadata fields. These metadata fields can be used as prediction labels for Sequence- and ReceptorDatasets. For AIRR format, there is no default metadata_column_mapping. When importing a RepertoireDataset, the metadata is automatically extracted from the metadata json files.
separator (str): Column separator, for AIRR this is by default “t”.

YAML specification:

my_airr_dataset:
    format: IReceptor
    params:
        path: path/to/zipfiles/
        is_repertoire: True # whether to import a RepertoireDataset
        metadata_column_mapping: # metadata column mapping AIRR: immuneML for Sequence- or ReceptorDatasetDataset
            airr_column_name1: metadata_label1
            airr_column_name2: metadata_label2
        import_productive: True # whether to include productive sequences in the dataset
        import_with_stop_codon: False # whether to include sequences with stop codon in the dataset
        import_out_of_frame: False # whether to include out of frame sequences in the dataset
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even if the `sequences` column is empty (provided that other fields are as specified here)
        import_empty_aa_sequences: False # remove all sequences with empty `sequence_aas` column
        # Optional fields with AIRR-specific defaults, only change when different behavior is required:
        separator: "\t" # column separator
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping AIRR: immuneML
            junction: sequences
            junction_aa: sequence_aas
            v_call: v_alleles
            j_call: j_alleles
            locus: chains
            duplicate_count: counts
            sequence_id: sequence_identifiers

ImmuneML¶

Imports the dataset from the files previously exported by immuneML. It closely resembles the AIRR format but relies on binary representations and is optimized for faster read-in at runtime.

ImmuneMLImport can import any kind of dataset (RepertoireDataset, SequenceDataset, ReceptorDataset).

This format includes:

a dataset file in yaml format with iml_dataset extension with parameters:

name,

identifier,

metadata_file (for repertoire datasets),

metadata_fields (for repertoire datasets),

repertoire_ids (for repertoire datasets)

element_ids (for receptor and sequence datasets),

labels,

a csv metadata file (only for repertoire datasets, should be in the same folder as the iml_dataset file),

data files for different types of data. For repertoire datasets, data files include one binary numpy file per repertoire with sequences and associated information and one metadata yaml file per repertoire with details such as repertoire identifier, disease status, subject id and other similar available information. For sequence and receptor datasets, sequences or receptors respectively, are stored in batches in binary numpy files.

Arguments:

path (str): The path to the previously created dataset file. This file should have an ‘.iml_dataset’ extension. If the path has not been specified, immuneML attempts to load the dataset from a specified metadata file (only for RepertoireDatasets).
metadata_file (str): An optional metadata file for a RepertoireDataset. If specified, the RepertoireDataset metadata will be updated to the newly specified metadata without otherwise changing the Repertoire objects

YAML specification:

my_dataset:
    format: ImmuneML
    params:
        path: path/to/dataset.iml_dataset
        metadata_file: path/to/metadata.csv

ImmunoSEQRearrangement¶

Imports data from Adaptive Biotechnologies immunoSEQ Analyzer rearrangement-level .tsv files into a Repertoire-, or SequenceDataset. RepertoireDatasets should be used when making predictions per repertoire, such as predicting a disease state. SequenceDatasets should be used when predicting values for unpaired (single-chain) immune receptors, like antigen specificity.

The format of the files imported by this importer is described here: https://www.adaptivebiotech.com/wp-content/uploads/2019/07/MRK-00342_immunoSEQ_TechNote_DataExport_WEB_REV.pdf Alternatively, to import sample-level .tsv files, see ImmunoSEQSample import

The only difference between these two importers is which columns they load from the .tsv files.

Arguments:

path (str): For RepertoireDatasets, this is the path to a directory with files to import. For Sequence- or ReceptorDatasets this path may either be the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset. By default, is_repertoire is set to True.
metadata_file (str): Required for RepertoireDatasets. This parameter specifies the path to the metadata file. This is a csv file with columns filename, subject_id and arbitrary other columns which can be used as labels in instructions. Only the files included under the column ‘filename’ are imported into the RepertoireDataset. For setting SequenceDataset metadata, metadata_file is ignored, see metadata_column_mapping instead.
import_productive (bool): Whether productive sequences (with value ‘In’ in column frame_type) should be included in the imported sequences. By default, import_productive is True.
import_with_stop_codon (bool): Whether sequences with stop codons (with value ‘Stop’ in column frame_type) should be included in the imported sequences. By default, import_with_stop_codon is False.
import_out_of_frame (bool): Whether out of frame sequences (with value ‘Out’ in column frame_type) should be included in the imported sequences. By default, import_out_of_frame is False.
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as immunoSEQ files use the IMGT junction. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): A mapping from immunoSEQ column names to immuneML’s internal data representation. For immunoSEQ rearrangement-level files, this is by default set the values shown below in YAML format. A custom column mapping can be specified here if necessary (for example: adding additional data fields if they are present in the file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’]..
rearrangement: sequence amino_acid: sequence_aa v_resolved: v_call j_resolved: j_call templates: duplicate_count locus: chain
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For immunoSEQ rearrangement-level files, this is by default set to:
v_resolved: v_alleles j_resolved: j_alleles
columns_to_load (list): Specifies which subset of columns must be loaded from the file. By default, this is: [rearrangement, v_family, v_gene, v_allele, j_family, j_gene, j_allele, amino_acid, templates, frame_type, locus]
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are immunoSEQ column names and values are the names that are internally used in immuneML as metadata fields. These metadata fields can be used as prediction labels for Sequence- and ReceptorDatasets. This parameter can also be used to specify sequence-level metadata columns for RepertoireDatasets, which can be used by reports. To set prediction label metadata for RepertoireDatasets, see metadata_file instead. For immunoSEQ rearrangement .tsv files, there is no default metadata_column_mapping.
separator (str): Column separator, for ImmunoSEQ files this is by default “t”.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter will typically be False (import only non-empty amino acid sequences)

YAML specification:

my_immunoseq_dataset:
    format: ImmunoSEQRearrangement
    params:
        path: path/to/files/
        is_repertoire: True # whether to import a RepertoireDataset (True) or a SequenceDataset (False)
        metadata_file: path/to/metadata.csv # metadata file for RepertoireDataset
        metadata_column_mapping: # metadata column mapping ImmunoSEQ: immuneML for SequenceDataset
            immunoseq_column_name1: metadata_label1
            immunoseq_column_name2: metadata_label2
        import_productive: True # whether to include productive sequences in the dataset
        import_with_stop_codon: False # whether to include sequences with stop codon in the dataset
        import_out_of_frame: False # whether to include out of frame sequences in the dataset
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        import_empty_aa_sequences: False # filter out sequences if they don't have sequence_aa set
        # Optional fields with ImmunoSEQ rearrangement-specific defaults, only change when different behavior is required:
        separator: "\t" # column separator
        columns_to_load: # subset of columns to load
        - rearrangement
        - v_family
        - v_gene
        - v_resolved
        - j_family
        - j_gene
        - j_resolved
        - amino_acid
        - templates
        - frame_type
        - locus
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping immunoSEQ: immuneML
            rearrangement: sequence
            amino_acid: sequence_aa
            v_resolved: v_call
            j_resolved: j_call
            templates: duplicate_count
            locus: chain

ImmunoSEQSample¶

Imports data from Adaptive Biotechnologies immunoSEQ Analyzer sample-level .tsv files into a Repertoire-, or SequenceDataset. RepertoireDatasets should be used when making predictions per repertoire, such as predicting a disease state. SequenceDatasets should be used when predicting values for unpaired (single-chain) immune receptors, like antigen specificity.

The format of the files imported by this importer is described here in section 3.4.13 https://clients.adaptivebiotech.com/assets/downloads/immunoSEQ_AnalyzerManual.pdf Alternatively, to import rearrangement-level .tsv files, see ImmunoSEQRearrangement import. The only difference between these two importers is which columns they load from the .tsv files.

Arguments:

path (str): For RepertoireDatasets, this is the path to a directory with files to import. For Sequence- or ReceptorDatasets this path may either be the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset. By default, is_repertoire is set to True.
metadata_file (str): Required for RepertoireDatasets. This parameter specifies the path to the metadata file. This is a csv file with columns filename, subject_id and arbitrary other columns which can be used as labels in instructions. Only the files included under the column ‘filename’ are imported into the RepertoireDataset. For setting SequenceDataset metadata, metadata_file is ignored, see metadata_column_mapping instead.
import_productive (bool): Whether productive sequences (with value ‘In’ in column frame_type) should be included in the imported sequences. By default, import_productive is True.
import_with_stop_codon (bool): Whether sequences with stop codons (with value ‘Stop’ in column frame_type) should be included in the imported sequences. By default, import_with_stop_codon is False.
import_out_of_frame (bool): Whether out of frame sequences (with value ‘Out’ in column frame_type) should be included in the imported sequences. By default, import_out_of_frame is False.
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as immunoSEQ files use the IMGT junction. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): A mapping from immunoSEQ column names to immuneML’s internal data representation. For immunoSEQ sample-level files, this is by default set to the values shown bellow in YAML format. A custom column mapping can be specified here if necessary (for example; adding additional data fields if they are present in the file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’]..
nucleotide: sequence aminoAcid: sequence_aa vGeneName: v_call jGeneName: j_call sequenceStatus: frame_type vFamilyName: v_family jFamilyName: j_family vGeneAllele: v_allele jGeneAllele: j_allele count (templates/reads): duplicate_count
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For immunoSEQ sample .tsv files, there is no default column_mapping_synonyms.
columns_to_load (list): Specifies which subset of columns must be loaded from the file. By default, this is: [nucleotide, aminoAcid, count (templates/reads), vFamilyName, vGeneName, vGeneAllele, jFamilyName, jGeneName, jGeneAllele, sequenceStatus]; these are the columns from the original file that will be imported
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are immunoSEQ column names and values are the names that are internally used in immuneML as metadata fields. These metadata fields can be used as prediction labels for Sequence- and ReceptorDatasets. This parameter can also be used to specify sequence-level metadata columns for RepertoireDatasets, which can be used by reports. To set prediction label metadata for RepertoireDatasets, see metadata_file instead. For immunoSEQ sample .tsv files, there is no default metadata_column_mapping.
separator (str): Column separator, for ImmunoSEQ files this is by default “t”.

YAML specification:

my_immunoseq_dataset:
    format: ImmunoSEQSample
    params:
        path: path/to/files/
        is_repertoire: True # whether to import a RepertoireDataset (True) or a SequenceDataset (False)
        metadata_file: path/to/metadata.csv # metadata file for RepertoireDataset
        metadata_column_mapping: # metadata column mapping ImmunoSEQ: immuneML for SequenceDataset
            immunoseq_column_name1: metadata_label1
            immunoseq_column_name2: metadata_label2
        import_productive: True # whether to include productive sequences in the dataset
        import_with_stop_codon: False # whether to include sequences with stop codon in the dataset
        import_out_of_frame: False # whether to include out of frame sequences in the dataset
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        import_empty_aa_sequences: False # filter out sequences if they don't have sequence_aa set
        # Optional fields with ImmunoSEQ sample-specific defaults, only change when different behavior is required:
        separator: "\t" # column separator
        columns_to_load: # subset of columns to load
        - nucleotide
        - aminoAcid
        - count (templates/reads)
        - vFamilyName
        - vGeneName
        - vGeneAllele
        - jFamilyName
        - jGeneName
        - jGeneAllele
        - sequenceStatus
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping immunoSEQ: immuneML
            nucleotide: sequence
            aminoAcid: sequence_aa
            vGeneName: v_call
            jGeneName: j_call
            sequenceStatus: frame_type
            vFamilyName: v_family
            jFamilyName: j_family
            vGeneAllele: v_allele
            jGeneAllele: j_allele
            count (templates/reads): duplicate_count

MiXCR¶

Imports data in MiXCR format into a Repertoire-, or SequenceDataset. RepertoireDatasets should be used when making predictions per repertoire, such as predicting a disease state. SequenceDatasets should be used when predicting values for unpaired (single-chain) immune receptors, like antigen specificity.

Arguments:

path (str): For RepertoireDatasets, this is the path to a directory with MiXCR files to import. For Sequence- or ReceptorDatasets this path may either be the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset. By default, is_repertoire is set to True.
metadata_file (str): Required for RepertoireDatasets. This parameter specifies the path to the metadata file. This is a csv file with columns filename, subject_id and arbitrary other columns which can be used as labels in instructions. Only the MiXCR files included under the column ‘filename’ are imported into the RepertoireDataset. For setting SequenceDataset metadata, metadata_file is ignored, see metadata_column_mapping instead.
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence, such as ‘_’, are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as MiXCR uses IMGT junction as CDR3. Alternatively to importing the CDR3 sequence, other region types can be specified here as well. Valid values are IMGT_CDR3, IMGT_CDR1, IMGT_CDR2, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4.
column_mapping (dict): A mapping from MiXCR column names to immuneML’s internal data representation. The columns that specify the sequences to import are handled by the region_type parameter. A custom column mapping can be specified here if necessary (for example; adding additional data fields if they are present in the MiXCR file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’].. For MiXCR, this is by default set to:
cloneCount: duplicate_count allVHitsWithScore: v_call allJHitsWithScore: j_call
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For MiXCR format, there is no default column_mapping_synonyms.
columns_to_load (list): Specifies which subset of columns must be loaded from the MiXCR file. By default, this is: [cloneCount, allVHitsWithScore, allJHitsWithScore, aaSeqCDR3, nSeqCDR3]
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are MiXCR column names and values are the names that are internally used in immuneML as metadata fields. These metadata fields can be used as prediction labels for Sequence- and ReceptorDatasets. This parameter can also be used to specify sequence-level metadata columns for RepertoireDatasets, which can be used by reports. To set prediction label metadata for RepertoireDatasets, see metadata_file instead. For MiXCR format, there is no default metadata_column_mapping.
separator (str): Column separator, for MiXCR this is by default “t”.

YAML specification:

my_mixcr_dataset:
    format: MiXCR
    params:
        path: path/to/files/
        is_repertoire: True # whether to import a RepertoireDataset (True) or a SequenceDataset (False)
        metadata_file: path/to/metadata.csv # metadata file for RepertoireDataset
        metadata_column_mapping: # metadata column mapping MiXCR: immuneML for SequenceDataset
            mixcrColumnName1: metadata_label1
            mixcrColumnName2: metadata_label2
        region_type: IMGT_CDR3 # what part of the sequence to import
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        import_empty_aa_sequences: False # filter out sequences if they don't have sequence_aa set
        # Optional fields with MiXCR-specific defaults, only change when different behavior is required:
        separator: "\t" # column separator
        columns_to_load: # subset of columns to load, sequence columns are handled by region_type parameter
        - cloneCount
        - allVHitsWithScore
        - allJHitsWithScore
        - aaSeqCDR3
        - nSeqCDR3
        column_mapping: # column mapping MiXCR: immuneML
            cloneCount: duplicate_count
            allVHitsWithScore: v_call
            allJHitsWithScore: j_call

OLGA¶

Imports data generated by OLGA simulations into a Repertoire-, or SequenceDataset. Assumes that the columns in each file correspond to: nucleotide sequences, amino acid sequences, v genes, j genes

Reference: Sethna, Zachary et al. ‘High-throughput immune repertoire analysis with IGoR’. Bioinformatics, (2019) doi.org/10.1093/bioinformatics/btz035.

Arguments:

path (str): For RepertoireDatasets, this is the path to a directory with OLGA files to import. For Sequence- or ReceptorDatasets this path may either be the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset. By default, is_repertoire is set to True.
metadata_file (str): Required for RepertoireDatasets. This parameter specifies the path to the metadata file. This is a csv file with columns filename, subject_id and arbitrary other columns which can be used as labels in instructions. Only the OLGA files included under the column ‘filename’ are imported into the RepertoireDataset. SequenceDataset metadata is currently not supported.
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as OLGA uses the IMGT junction. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
separator (str): Column separator, for OLGA this is by default “t”.
column_mapping (dict): defines which columns to import from olga format: keys are the number of the columns and values are the names of the columns to be mapped to

YAML specification:

my_olga_dataset:
    format: OLGA
    params:
        path: path/to/files/
        is_repertoire: True # whether to import a RepertoireDataset (True) or a SequenceDataset (False)
        metadata_file: path/to/metadata.csv # metadata file for RepertoireDataset
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        import_empty_aa_sequences: False # filter out sequences if they don't have sequence_aa set
        # Optional fields with OLGA-specific defaults, only change when different behavior is required:
        separator: "\t" # column separator
        region_type: IMGT_CDR3 # what part of the sequence to import
        columns_to_load: [0, 1, 2, 3]
        column_mapping:
            0: sequences
            1: sequence_aas
            2: v_genes
            3: j_genes

SingleLineReceptor¶

Imports data from a tabular file (where each line contains a pair of immune receptor sequences) into a ReceptorDataset. If you instead want to import a ReceptorDataset from a tabular file that contains one receptor sequence per line, see Generic import.

Arguments:

path (str): Required parameter. This is the path to a directory with files to import.
receptor_chains (str): Required parameter. Determines which pair of chains to import for each Receptor. Valid values for receptor_chains are: TRA_TRB, TRG_TRD, IGH_IGL, IGH_IGK.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. When IMGT_CDR3 is specified, immuneML assumes the IMGT junction (including leading C and trailing Y/F amino acids) is used in the input file, and the first and last amino acids will be removed from the sequences to retrieve the IMGT CDR3 sequence. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): A mapping where the keys are the column names in the input file, and the values must be mapped to the following fields: <chain>_amino_acid_sequence, <chain>_nucleotide_sequence, <chain>_v_gene, <chain>_j_gene, identifier, epitope. The possible names that can be filled in for <chain> are: ALPHA, BETA, GAMMA, DELTA, HEAVY, LIGHT, KAPPA. Any column namme other than the sequence, v/j genes and identifier will be set as metadata fields to the Receptors, and can subsequently be used as labels in immuneML instructions. For TCR alpha-beta receptor import, a column mapping could for example look like this:
cdr3_a_aa: alpha_amino_acid_sequence cdr3_b_aa: beta_amino_acid_sequence cdr3_a_nucseq: alpha_nucleotide_sequence cdr3_b_nucseq: beta_nucleotide_sequence v_a_gene: alpha_v_call v_b_gene: beta_v_call j_a_gene: alpha_j_call j_b_gene: beta_j_call clone_id: identifier count: duplicate_count epitope: epitope # metadata field
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded.
columns_to_load (list): Optional; specifies which columns to load from the input file. This may be useful if the input files contain many unused columns. If no value is specified, all columns are loaded.
separator (str): Required parameter. Column separator, for example “t” or “,”.
organism (str): The organism that the receptors came from. This will be set as a parameter in the ReceptorDataset object.

YAML specification:

my_receptor_dataset:
    format: SingleLineReceptor
    params:
        path: path/to/files/
        receptor_chains: TRA_TRB # what chain pair to import
        separator: "\t" # column separator
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        import_empty_aa_sequences: False # filter out sequences if they don't have sequence_aa set
        region_type: IMGT_CDR3 # what part of the sequence to import
        columns_to_load: # which subset of columns to load from the file
        - subject
        - epitope
        - count
        - v_a_gene
        - j_a_gene
        - cdr3_a_aa
        - v_b_gene
        - j_b_gene
        - cdr3_b_aa
        - clone_id
        column_mapping: # column mapping file: immuneML
            cdr3_a_aa: alpha_amino_acid_sequence
            cdr3_b_aa: beta_amino_acid_sequence
            cdr3_a_nucseq: alpha_nucleotide_sequence
            cdr3_b_nucseq: beta_nucleotide_sequence
            v_a_gene: alpha_v_call
            v_b_gene: beta_v_call
            j_a_gene: alpha_j_call
            j_b_gene: beta_j_call
            count: duplicate_count
            clone_id: identifier
            epitope: epitope
            organism: mouse

TenxGenomics¶

Imports data from the 10xGenomics Cell Ranger analysis pipeline into a Repertoire-, Sequence- or ReceptorDataset. RepertoireDatasets should be used when making predictions per repertoire, such as predicting a disease state. SequenceDatasets or ReceptorDatasets should be used when predicting values for unpaired (single-chain) and paired immune receptors respectively, like antigen specificity.

The files that should be used as input are named ‘Clonotype consensus annotations (CSV)’, as described here: https://support.10xgenomics.com/single-cell-vdj/software/pipelines/latest/output/annotation#consensus

Note: by default the 10xGenomics field ‘umis’ is used to define the immuneML field counts. If you want to use the 10x Genomics field reads instead, this can be changed in the column_mapping (set reads: counts). Furthermore, the 10xGenomics field clonotype_id is used for the immuneML field cell_id.

Arguments:

path (str): For RepertoireDatasets, this is the path to a directory with 10xGenomics files to import. For Sequence- or ReceptorDatasets this path may either be the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset or ReceptorDataset. By default, is_repertoire is set to True.
metadata_file (str): Required for RepertoireDatasets. This parameter specifies the path to the metadata file. This is a csv file with columns filename, subject_id and arbitrary other columns which can be used as labels in instructions. For setting Sequence- or ReceptorDataset metadata, metadata_file is ignored, see metadata_column_mapping instead.
paired (str): Required for Sequence- or ReceptorDatasets. This parameter determines whether to import a SequenceDataset (paired = False) or a ReceptorDataset (paired = True). In a ReceptorDataset, two sequences with chain types specified by receptor_chains are paired together based on the identifier given in the 10xGenomics column named ‘clonotype_id’.
receptor_chains (str): Required for ReceptorDatasets. Determines which pair of chains to import for each Receptor. Valid values are TRA_TRB, TRG_TRD, IGH_IGL, IGH_IGK. If receptor_chains is not provided, the chain pair is automatically detected (only one chain pair type allowed per repertoire).
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as 10xGenomics uses IMGT junction as CDR3. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): A mapping from 10xGenomics column names to immuneML’s internal data representation. A custom column mapping can be specified here if necessary (for example; adding additional data fields if they are present in the 10xGenomics file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’].. For 10xGenomics, this is by default set to:
cdr3: sequence_aa cdr3_nt: sequence v_gene: v_call j_gene: j_call umis: duplicate_count clonotype_id: cell_id consensus_id: sequence_id
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For 10xGenomics format, there is no default column_mapping_synonyms.
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are 10xGenomics column names and values are the names that are internally used in immuneML as metadata fields. These metadata fields can be used as prediction labels for Sequence- and ReceptorDatasets. This parameter can also be used to specify sequence-level metadata columns for RepertoireDatasets, which can be used by reports. To set prediction label metadata for RepertoireDatasets, see metadata_file instead. For 10xGenomics format, there is no default metadata_column_mapping.
separator (str): Column separator, for 10xGenomics this is by default “,”.

YAML specification:

my_10x_dataset:
    format: 10xGenomics
    params:
        path: path/to/files/
        is_repertoire: True # whether to import a RepertoireDataset
        metadata_file: path/to/metadata.csv # metadata file for RepertoireDataset
        paired: False # whether to import SequenceDataset (False) or ReceptorDataset (True) when is_repertoire = False
        receptor_chains: TRA_TRB # what chain pair to import for a ReceptorDataset
        metadata_column_mapping: # metadata column mapping 10xGenomics: immuneML for SequenceDataset
            tenx_column_name1: metadata_label1
            tenx_column_name2: metadata_label2
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        import_empty_aa_sequences: False # filter out sequences if they don't have sequence_aa set
        # Optional fields with 10xGenomics-specific defaults, only change when different behavior is required:
        separator: "," # column separator
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping 10xGenomics: immuneML
            cdr3: sequence_aa
            cdr3_nt: sequence
            v_gene: v_call
            j_gene: j_call
            umis: duplicate_count
            clonotype_id: cell_id
            consensus_id: sequence_id

VDJdb¶

Imports data in VDJdb format into a Repertoire-, Sequence- or ReceptorDataset. RepertoireDatasets should be used when making predictions per repertoire, such as predicting a disease state. SequenceDatasets or ReceptorDatasets should be used when predicting values for unpaired (single-chain) and paired immune receptors respectively, like antigen specificity.

Arguments:

path (str): For RepertoireDatasets, this is the path to a directory with VDJdb files to import. For Sequence- or ReceptorDatasets this path may either be the path to the file to import, or the path to the folder locating one or multiple files with .tsv, .csv or .txt extensions. By default path is set to the current working directory.
is_repertoire (bool): If True, this imports a RepertoireDataset. If False, it imports a SequenceDataset or ReceptorDataset. By default, is_repertoire is set to True.
metadata_file (str): Required for RepertoireDatasets. This parameter specifies the path to the metadata file. This is a csv file with columns filename, subject_id and arbitrary other columns which can be used as labels in instructions. For setting Sequence- or ReceptorDataset metadata, metadata_file is ignored, see metadata_column_mapping instead.
paired (str): Required for Sequence- or ReceptorDatasets. This parameter determines whether to import a SequenceDataset (paired = False) or a ReceptorDataset (paired = True). In a ReceptorDataset, two sequences with chain types specified by receptor_chains are paired together based on the identifier given in the VDJdb column named ‘complex.id’.
receptor_chains (str): Required for ReceptorDatasets. Determines which pair of chains to import for each Receptor. Valid values are TRA_TRB, TRG_TRD, IGH_IGL, IGH_IGK. If receptor_chains is not provided, the chain pair is automatically detected (only one chain pair type allowed per repertoire).
import_illegal_characters (bool): Whether to import sequences that contain illegal characters, i.e., characters that do not appear in the sequence alphabet (amino acids including stop codon ‘*’, or nucleotides). When set to false, filtering is only applied to the sequence type of interest (when running immuneML in amino acid mode, only entries with illegal characters in the amino acid sequence are removed). By default import_illegal_characters is False.
import_empty_nt_sequences (bool): imports sequences which have an empty nucleotide sequence field; can be True or False. By default, import_empty_nt_sequences is set to True.
import_empty_aa_sequences (bool): imports sequences which have an empty amino acid sequence field; can be True or False; for analysis on amino acid sequences, this parameter should be False (import only non-empty amino acid sequences). By default, import_empty_aa_sequences is set to False.
region_type (str): Which part of the sequence to import. By default, this value is set to IMGT_CDR3. This means the first and last amino acids are removed from the CDR3 sequence, as VDJdb uses IMGT junction as CDR3. Specifying any other value will result in importing the sequences as they are. Valid values are IMGT_CDR1, IMGT_CDR2, IMGT_CDR3, IMGT_FR1, IMGT_FR2, IMGT_FR3, IMGT_FR4, IMGT_JUNCTION, FULL_SEQUENCE.
column_mapping (dict): A mapping from VDJdb column names to immuneML’s internal data representation. A custom column mapping can be specified here if necessary (for example; adding additional data fields if they are present in the VDJdb file, or using alternative column names). Valid immuneML fields that can be specified here are [‘sequence_aa’, ‘sequence’, ‘v_call’, ‘j_call’, ‘chain’, ‘duplicate_count’, ‘frame_type’, ‘sequence_id’, ‘cell_id’].. For VDJdb, this is by default set to:
V: v_call J: j_call CDR3: sequence_aa complex.id: sequence_id Gene: chain
column_mapping_synonyms (dict): This is a column mapping that can be used if a column could have alternative names. The formatting is the same as column_mapping. If some columns specified in column_mapping are not found in the file, the columns specified in column_mapping_synonyms are instead attempted to be loaded. For VDJdb format, there is no default column_mapping_synonyms.
metadata_column_mapping (dict): Specifies metadata for Sequence- and ReceptorDatasets. This should specify a mapping similar to column_mapping where keys are VDJdb column names and values are the names that are internally used in immuneML as metadata fields. This means that epitope, epitope_gene and epitope_species can be used as prediction labels for Sequence- and ReceptorDatasets. This parameter can also be used to specify sequence-level metadata columns for RepertoireDatasets, which can be used by reports. To set prediction label metadata for RepertoireDatasets, see metadata_file instead. For VDJdb format, this parameter is by default set to:
Epitope: epitope Epitope gene: epitope_gene Epitope species: epitope_species
separator (str): Column separator, for VDJdb this is by default “t”.

YAML specification:

my_vdjdb_dataset:
    format: VDJdb
    params:
        path: path/to/files/
        is_repertoire: True # whether to import a RepertoireDataset
        metadata_file: path/to/metadata.csv # metadata file for RepertoireDataset
        paired: False # whether to import SequenceDataset (False) or ReceptorDataset (True) when is_repertoire = False
        receptor_chains: TRA_TRB # what chain pair to import for a ReceptorDataset
        import_illegal_characters: False # remove sequences with illegal characters for the sequence_type being used
        import_empty_nt_sequences: True # keep sequences even though the nucleotide sequence might be empty
        import_empty_aa_sequences: False # filter out sequences if they don't have sequence_aa set
        # Optional fields with VDJdb-specific defaults, only change when different behavior is required:
        separator: "\t" # column separator
        region_type: IMGT_CDR3 # what part of the sequence to import
        column_mapping: # column mapping VDJdb: immuneML
            V: v_call
            J: j_call
            CDR3: sequence_aa
            complex.id: sequence_id
            Gene: chain
        metadata_column_mapping: # metadata column mapping VDJdb: immuneML
            Epitope: epitope
            Epitope gene: epitope_gene
            Epitope species: epitope_species

Instructions¶

FeasibilitySummary¶

FeasibilitySummaryInstruction instruction creates a small synthetic dataset and reports summary metrics to show if the simulation with the given parameters is feasible. The input parameters to this analysis are the name of the simulation (the same that can be used with LigoSim instruction later if feasibility analysis looks acceptable), and the number of sequences to simulate for estimating the feasibility.

The feasibility analysis is performed for each generative model separately as these could differ in the analyses that will be reported.

Arguments:

simulation (str): a name of a simulation object containing a list of SimConfigItem as specified under definitions key; defines how to combine signals with simulated data; specified under definitions
sequence_count (int): how many sequences to generate to estimate feasibility (default value: 100 000)
number_of_processes (int): for the parts of the analysis that are possible to parallelize, how many processes to use

YAML specification:

my_feasibility_summary: # user-defined name of the instruction
    type: FeasibilitySummary # which instruction to execute
    simulation: sim1
    sequence_count: 10000

LigoSim¶

LIgO simulation instruction creates a synthetic dataset from scratch based on the generative model and a set of signals provided by the user.

Arguments:

simulation (str): a name of a simulation object containing a list of SimConfigItem as specified under definitions key; defines how to combine signals with simulated data; specified under definitions
sequence_batch_size (int): how many sequences to generate at once using the generative model before checking for signals and filtering
max_iterations (int): how many iterations are allowed when creating sequences
export_p_gens (bool): whether to compute generation probabilities (if supported by the generative model) for sequences and include them as part of output
number_of_processes (int): determines how many simulation items can be simulated in parallel

YAML specification:

my_simulation_instruction: # user-defined name of the instruction
    type: LIgOSim # which instruction to execute
    simulation: sim1
    sequence_batch_size: 1000
    max_iterations: 1000
    export_p_gens: False
    number_of_processes: 4