Formats

type: text file

plugin: quality-control

type: directory

files:

name	path	format	required
file	stats.tsv	DecontamScoreFormat	True

type: text file

plugin: fragment-insertion

type: directory

files:

name	path	format	required
file	placements.json	PlacementsFormat	True

type: text file

plugin: fragment-insertion

type: directory

files:

name	path	format	required
alignment	aligned-dna-sequences.fasta	AlignedDNAFASTAFormat	True
phylogeny	tree.nwk	NewickFormat	True
raxml_info	raxml-info.txt	RAxMLinfoFormat	True

type: text file

plugin: metadata

type: directory

files:

name	path	format	required
file	artificial-groupings.tsv	ArtificialGroupingFormat	True

type: text file

plugin: fondue

type: directory

files:

name	path	format	required
file	sra-metadata.tsv	SRAMetadataFormat	True

This is a "fake" format only used to store a list of failed SRA IDs, which can be converted to QIIME's metadata and input into any fondue action.

type: text file

plugin: fondue

type: directory

files:

name	path	format	required
file	sra-failed-ids.tsv	SRAFailedIDsFormat	True

This is a format used to store a list of SRA accession IDs (run, study, BioProject, sample and experiment IDs), which can be converted to QIIME's metadata. Artifacts containing of run, study and BioProject IDs can be input into any fondue action.

type: text file

plugin: fondue

type: directory

files:

name	path	format	required
file	ncbi-accession-ids.tsv	NCBIAccessionIDsFormat	True

plugin: sample-classifier

type: directory

files:

name	path	format	required
version_info	sklearn_version.json	JSONFormat	True
sklearn_pipeline	sklearn_pipeline.tar	PickleFormat	True

type: text file

plugin: sample-classifier

type: directory

files:

name	path	format	required
file	outliers.tsv	BooleanSeriesFormat	True

type: text file

plugin: sample-classifier

type: directory

files:

name	path	format	required
file	importance.tsv	ImportanceFormat	True

type: text file

plugin: sample-classifier

type: directory

files:

name	path	format	required
file	predictions.tsv	PredictionsFormat	True

type: text file

plugin: sample-classifier

type: directory

files:

name	path	format	required
file	class_probabilities.tsv	ProbabilitiesFormat	True

plugin: sample-classifier

type: directory

files:

name	path	format	required
file	true_targets.tsv	PredictionsFormat	True

type: text file

plugin: rescript

type: directory

files:

name	path	format	required
file	silva_taxonomy.tsv	SILVATaxonomyFormat	True

type: text file

plugin: rescript

type: directory

files:

name	path	format	required
file	silva_taxmap.tsv	SILVATaxidMapFormat	True

Format for frictionless CSV.

More on this later.

type: text file

Format for the associated metadata for each file in the DataLoaf.

More on this later.

type: text file

plugin: composition

type: directory

files:

name	path	format	required
data_slices	.+\.csv	FrictionlessCSVFileFormat	True
nutrition_facts	datapackage.json	DataPackageSchemaFileFormat	True

Stores the model statistics and optionally the structural zeros table output by the ANCOMBC2 method.

The slices are: - lfc: log-fold change - se: standard error - W: lfc / se (the test statistic) - p: p-value - q: adjusted p-value - diff: differentially abundant boolean (i.e. q < alpha) - diff_robust: robust diff abun boolean (q < alpha AND passed_ss) - passed_ss: whether sensitivity analysis was passed

plugin: composition

type: directory

files:

name	path	format	required
lfc	lfc.jsonl	TableJSONLFileFormat	True
se	se.jsonl	TableJSONLFileFormat	True
W	W.jsonl	TableJSONLFileFormat	True
p	p.jsonl	TableJSONLFileFormat	True
q	q.jsonl	TableJSONLFileFormat	True
diff	diff.jsonl	TableJSONLFileFormat	True
passed_ss	passed_ss.jsonl	TableJSONLFileFormat	True
diff_robust	diff_robust.jsonl	TableJSONLFileFormat	False
structural_zeros	structural-zeros.jsonl	TableJSONLFileFormat	False

type: text file

plugin: deblur

type: directory

files:

name	path	format	required
file	stats.csv	DeblurStatsFmt	True

type: text file

plugin: dada2

type: directory

files:

name	path	format	required
file	stats.tsv	DADA2StatsFormat	True

type: text file

plugin: dada2

type: directory

files:

name	path	format	required
file	Errorstats.tsv	DADA2BaseTransitionStatsFormat	True

type: text file

plugin: vsearch

type: directory

files:

name	path	format	required
file	stats.tsv	UchimeStatsFmt	True

type: text file

plugin: quality-filter

type: directory

files:

name	path	format	required
file	stats.csv	QualityFilterStatsFmt	True

plugin: feature-classifier

type: directory

files:

name	path	format	required
idx1	.+\.ndb	BLASTDBFileFmtV5	True
idx2	.+\.nhr	BLASTDBFileFmtV5	True
idx3	.+\.nin	BLASTDBFileFmtV5	True
idx4	.+\.not	BLASTDBFileFmtV5	True
idx5	.+\.nsq	BLASTDBFileFmtV5	True
idx6	.+\.ntf	BLASTDBFileFmtV5	True
idx7	.+\.nto	BLASTDBFileFmtV5	True
idx8	.+\.njs	BLASTDBFileFmtV5	True

plugin: feature-classifier

type: directory

files:

name	path	format	required
preprocess_params	preprocess_params.json	JSONFormat	True
sklearn_pipeline	sklearn_pipeline.tar	PickleFormat	True

plugin: feature-classifier

type: directory

files:

name	path	format	required
version_info	sklearn_version.json	JSONFormat	True
sklearn_pipeline	sklearn_pipeline.tar	PickleFormat	True

type: text file

plugin: longitudinal

type: directory

files:

name	path	format	required
file	FirstDifferences.tsv	FirstDifferencesFormat	True

plugin: types

type: directory

files:

name	path	format	required
idx1	.+(?<!\.rev)\.1\.bt2l?	Bowtie2IndexFileFormat	True
idx2	.+(?<!\.rev)\.2\.bt2l?	Bowtie2IndexFileFormat	True
ref3	.+\.3\.bt2l?	Bowtie2IndexFileFormat	True
ref4	.+\.4\.bt2l?	Bowtie2IndexFileFormat	True
rev1	.+\.rev\.1\.bt2l?	Bowtie2IndexFileFormat	True
rev2	.+\.rev\.2\.bt2l?	Bowtie2IndexFileFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	distance-matrix.tsv	LSMatFormat	True

Format for a 2+ column TSV file with an expected minimal header.

The only header recognized by this format is:

Feature ID<tab>Taxon

Optionally followed by other arbitrary columns.

This format supports blank lines. The expected header must be the first non-blank line. In addition to the header, there must be at least one line of data.

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	taxonomy.tsv	TSVTaxonomyFormat	True

Format for a 2+ column TSV file without a header.

This format supports comment lines starting with #, and blank lines.

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	taxonomy.tsv	HeaderlessTSVTaxonomyFormat	True

Legacy format for any 2+ column TSV file, with or without a header.

This format has been superseded by taxonomy file formats explicitly with and without headers, TSVTaxonomyFormat and HeaderlessTSVTaxonomyFormat, respectively.

This format remains in place for backwards-compatibility. Transformers are intentionally not hooked up to transform this format into the canonical .qza format (TSVTaxonomyFormat) to prevent users from importing data in this format. Transformers will remain in place to transform this format into in-memory Python objects (e.g. pd.Series) so that existing .qza files can still be loaded and processed.

The only header recognized by this format is:

Feature ID<tab>Taxon

Optionally followed by other arbitrary columns.

If this header isn't present, the format is assumed to be headerless.

This format supports comment lines starting with #, and blank lines.

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	taxonomy.tsv	TaxonomyFormat	True

type: text file

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	dna-sequences.fasta	DNAFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
left_dna_sequences	left-dna-sequences.fasta	DNAFASTAFormat	True
right_dna_sequences	right-dna-sequences.fasta	DNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	aligned-dna-sequences.fasta	AlignedDNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	differentials.tsv	DifferentialFormat	True

type: text file

type: text file

type: text file

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	protein-sequences.fasta	ProteinFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
file	aligned-protein-sequences.fasta	AlignedProteinFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
file	protein-sequences.fasta	MixedCaseProteinFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
file	aligned-protein-sequences.fasta	MixedCaseAlignedProteinFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	rna-sequences.fasta	RNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	aligned-rna-sequences.fasta	AlignedRNAFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
left_rna_sequences	left-rna-sequences.fasta	RNAFASTAFormat	True
right_rna_sequences	right-rna-sequences.fasta	RNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	blast6.tsv	BLAST6Format	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	dna-sequences.fasta	MixedCaseDNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	rna-sequences.fasta	MixedCaseRNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	aligned-dna-sequences.fasta	MixedCaseAlignedDNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	aligned-rna-sequences.fasta	MixedCaseAlignedRNAFASTAFormat	True

Format for a TSV file with information about sequences like length of a feature. The first column contains feature identifiers and is followed by other optional columns.

The file cannot be empty and must have at least two columns.

Validation for additional columns can be added with a semantic validator tied to a property. For example the "validate_seq_char_len" validator for "FeatureData[SequenceCharacteristics % Properties("length")]" adds validation for a numerical column called "length".

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	sequence_characteristics.tsv	SequenceCharacteristicsFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}\.(fa|fasta)$	DNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	mag-to-contigs.json	MAGtoContigsFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	feature-map.json	FeatureMapFormat	True

type: text file

type: binary file

plugin: types

type: directory

files:

name	path	format	required
file	feature-table.biom	BIOMV100Format	True

plugin: types

type: directory

files:

name	path	format	required
file	feature-table.biom	BIOMV210Format	True

plugin: types

type: directory

files:

name	path	format	required
genes	.+\.(fa|fna|fasta)$	DNAFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
proteins	.+\.(fa|faa|fasta)$	ProteinFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
loci	.+\.gff$	GFF3Format	True

plugin: types

type: directory

files:

name	path	format	required
genomes	.+\.(fasta|fa)$	DNAFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
seed_orthologs	.*\..*\.seed_orthologs	OrthologFileFmt	True

plugin: types

type: directory

files:

name	path	format	required
annotations	.+\.annotations	OrthologFileFmt	True

Generic Feature Format Version 3 (GFF3) spec: gff3.md NCBI modifications to the above: https://www.ncbi.nlm.nih.gov/datasets/docs/reference-docs/file-formats/about-ncbi-gff3/

type: text file

plugin: types

type: directory

files:

name	path	format	required
nodes	nodes.dmp	NCBITaxonomyNodesFormat	True
names	names.dmp	NCBITaxonomyNamesFormat	True
index	kaiju_db.+\.fmi	KaijuIndexFormat	True

type: binary file

type: text file

type: text file

type: text file

type: text file

plugin: types

type: directory

files:

name	path	format	required
reports	.+\.report\.(txt|tsv)$	Kraken2ReportFormat	True

plugin: types

type: directory

files:

name	path	format	required
outputs	.+\.output\.(txt|tsv)$	Kraken2OutputFormat	True

plugin: types

type: directory

files:

name	path	format	required
hash	hash.k2d	Kraken2DBFormat	True
opts	opts.k2d	Kraken2DBFormat	True
taxo	taxo.k2d	Kraken2DBFormat	True

plugin: types

type: directory

files:

name	path	format	required
file	report.txt	Kraken2DBReportFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
kmers	database(\d{2,})mers\.kmer_distrib$	BrackenDBFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	metadata.tsv	ImmutableMetadataFormat	True

plugin: types

type: directory

files:

name	path	format	required
file	forward.fastq.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
forward_sequences	forward.fastq.gz	FastqGzFormat	True
reverse_sequences	reverse.fastq.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	reads.fasta	DNAFASTAFormat	True
quality	reads.qual	QualFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	sequences.fastq.gz	FastqGzFormat	True
barcodes	barcodes.fastq.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
file	details.tsv	ErrorCorrectionDetailsFmt	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
sequences	sequences.fastq.gz	FastqGzFormat	True
barcodes	barcodes.fastq.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	Undetermined_S0_L001_R1_001.fastq.gz	FastqGzFormat	True
barcodes	Undetermined_S0_L001_I1_001.fastq.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
forward	forward.fastq.gz	FastqGzFormat	True
reverse	reverse.fastq.gz	FastqGzFormat	True
barcodes	barcodes.fastq.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
forward	Undetermined_S0_L001_R1_001.fastq.gz	FastqGzFormat	True
reverse	Undetermined_S0_L001_R2_001.fastq.gz	FastqGzFormat	True
barcodes	Undetermined_S0_L001_I1_001.fastq.gz	FastqGzFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	ordination.txt	OrdinationFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	ProcrustesStatistics.tsv	ProcrustesStatisticsFmt	True

Mapping of sample identifiers to relative filepaths and read direction.

type: text file

Mapping of sample identifiers to absolute filepaths and read direction.

type: text file

Arbitrary yaml-formatted file.

type: text file

A gzipped fastq file.

type: binary file

plugin: types

type: directory

files:

name	path	format	required
sequences	.+_.+_L[0-9][0-9][0-9]_R[12]_001\.fastq\.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	.+_.+_R[12]_001\.fastq\.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	.+_.+_L[0-9][0-9][0-9]_R[12]_001\.fastq\.gz	FastqGzFormat	True
manifest	MANIFEST	FastqManifestFormat	True
metadata	metadata.yml	YamlFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	.+_.+_L[0-9][0-9][0-9]_R[12]_001\.fastq\.gz	FastqGzFormat	True
manifest	MANIFEST	FastqManifestFormat	True
metadata	metadata.yml	YamlFormat	True

type: text file

type: text file

type: text file

type: text file

type: text file

type: text file

type: text file

type: text file

QIIME 1 demultiplexed FASTA format.

The QIIME 1 demultiplexed FASTA format is the default output format of split_libraries.py and split_libraries_fastq.py. The file output by QIIME 1 is named seqs.fna; this filename is sometimes associated with the file format itself due to its widespread usage in QIIME 1.

The format is documented here: http://qiime.org/documentation/file_formats.html#demultiplexed-sequences

Format details:

- FASTA file with exactly two lines per record: header and sequence. Each sequence must span exactly one line and cannot be split across multiple lines.

- The ID in each header must follow the format <sample-id>_<seq-id>. <sample-id> is the identifier of the sample the sequence belongs to, and <seq-id> is an identifier for the sequence within its sample. In QIIME 1, <seq-id> is typically an incrementing integer starting from zero, but any non-empty value can be used here, as long as the header IDs remain unique throughout the file. Note: <sample-id> may contain sample IDs that contain underscores; the rightmost underscore will used to delimit sample and sequence IDs.

- Descriptions in headers are permitted and ignored.

- Header IDs must be unique within the file.

- Each sequence must be DNA and cannot be empty.

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	seqs.fna	QIIME1DemuxFormat	True

Single-end reads in fastq.gz files where base filename is the sample id

The full file name, minus the extension (.fastq.gz) is the sample id. For example, the sample id for the file: * sample-1.fastq.gz is sample-1 * xyz.fastq.gz is xyz * sample-42_S1_L001_R1_001.fastq.gz is sample-42_S1_L001_R1_001

plugin: types

type: directory

files:

name	path	format	required
sequences	.+\.fastq\.gz	FastqGzFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	.+\.(fa|fasta)$	DNAFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	.+\.(fa|fasta)$	DNAFASTAFormat	True
manifest	MANIFEST	MultiMAGManifestFormat	True

plugin: types

type: directory

files:

name	path	format	required
sequences	[^\.].+_contigs.(fasta|fa)$	DNAFASTAFormat	True

plugin: types

type: directory

files:

name	path	format	required
idx1	.+(?<!\.rev)\.1\.bt2l?	Bowtie2IndexFileFormat	True
idx2	.+(?<!\.rev)\.2\.bt2l?	Bowtie2IndexFileFormat	True
ref3	.+\.3\.bt2l?	Bowtie2IndexFileFormat	True
ref4	.+\.4\.bt2l?	Bowtie2IndexFileFormat	True
rev1	.+\.rev\.1\.bt2l?	Bowtie2IndexFileFormat	True
rev2	.+\.rev\.2\.bt2l?	Bowtie2IndexFileFormat	True

type: binary file

plugin: types

type: directory

files:

name	path	format	required
bams	.+\.bam	BAMFormat	True

plugin: types

type: directory

files:

name	path	format	required
bams	.+\/.+\.bam	BAMFormat	True

type: text file

type: text file

type: text file

type: text file

type: text file

type: text file

type: text file

The <hmmfile>.h3m file contains the profile HMMs and their annotation in a binary format. The <hmmfile>.h3i file is an SSI index for the <hmmfile>.h3m file. The <hmmfile>.h3f file contains precomputed data structures for the fast heuristic filter (the MSV filter). The <hmmfile>.h3p file contains precomputed data structures for the rest of each profile.

plugin: types

type: directory

files:

name	path	format	required
h3m	.*\.hmm\.h3m	ProfileHmmBinaryFileFmt	True
h3i	.*\.hmm\.h3i	ProfileHmmBinaryFileFmt	True
h3f	.*\.hmm\.h3f	ProfileHmmBinaryFileFmt	True
h3p	.*\.hmm\.h3p	ProfileHmmBinaryFileFmt	True

plugin: types

type: directory

files:

name	path	format	required
profile	.*\.hmm	ProteinSingleProfileHmmFileFmt	True

plugin: types

type: directory

files:

name	path	format	required
profiles	.*\.hmm	ProteinMultipleProfileHmmFileFmt	True

plugin: types

type: directory

files:

name	path	format	required
profile	.*\.hmm	DNASingleProfileHmmFileFmt	True

plugin: types

type: directory

files:

name	path	format	required
profiles	.*\.hmm	DNAMultipleProfileHmmFileFmt	True

plugin: types

type: directory

files:

name	path	format	required
profile	.*\.hmm	RNASingleProfileHmmFileFmt	True

plugin: types

type: directory

files:

name	path	format	required
profiles	.*\.hmm	RNAMultipleProfileHmmFileFmt	True

type: text file

type: binary file

plugin: types

type: directory

files:

name	path	format	required
eggnog	eggnog.*db.*	EggnogRefBinFileFmt	True

type: binary file

plugin: types

type: directory

files:

name	path	format	required
file	ref_db.dmnd	DiamondDatabaseFileFmt	True

type: text file

type: text file

type: binary file

plugin: types

type: directory

files:

name	path	format	required
node	nodes.dmp	NCBITaxonomyNodesFormat	True
names	names.dmp	NCBITaxonomyNamesFormat	True
tax_map	prot.accession2taxid.gz	NCBITaxonomyBinaryFileFmt	True

plugin: types

type: directory

files:

name	path	format	required
taxid_info	e5.taxid_info.tsv	EggnogRefTextFileFmt	True
proteins	e5.proteomes.faa	MixedCaseProteinFASTAFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	alpha-diversity.tsv	AlphaDiversityFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	tree.nwk	NewickFormat	True

Format for newline-delimited (ND) JSON file.

type: text file

Format for data resource schema.

type: text file

plugin: types

type: directory

files:

name	path	format	required
data	data.ndjson	NDJSONFileFormat	True
metadata	dataresource.json	DataResourceSchemaFileFormat	True

type: text file

plugin: types

type: directory

files:

name	path	format	required
file	data.table.jsonl	TableJSONLFileFormat	True