New source: SILVA taxonomy

[jplfaria]

Summary

This pull request implements the SILVA taxonomy as an ontology converter. The module converts SILVA small subunit (SSU) taxonomy data into OBO (and OWL) format and is organized in a style similar to the GTDB module.

---

Key Decisions and Implementation Details

• Internal SILVA Taxonomy IDs:

  • As discussed in #1306, the internal SILVA taxonomy ID URLs do not resolve externally.
  • For example, I used URLs such as:

    https://www.arb-silva.de/no_cache/download/archive/current/Exports/taxonomy/10081
    
    with the idea that navigating to:

    https://www.arb-silva.de/no_cache/download/archive/current/Exports/taxonomy/
    
    will allow users to locate the files by internal IDs.
  • Note: I am open to alternative suggestions (e.g. using URLs that resolve directly to bioregistry.io).

• SSU vs. LSU Taxonomy:

  • This module is specific for SILVA small subunit (SSU) taxonomy.
  • The large subunit (LSU) taxonomy is distinct and should be handled separately.
  • The module’s docstring clearly states:

    """Convert SILVA small subunit (ssu) taxonomy to OBO format."""

• Handling ENA Accession Numbers:

  • At the lowest taxonomy level, SILVA uses accession numbers that resolve to ENA entries.
  • Initial Approach:
    • I initially implemented these as cross-references (xrefs) at the genus level (e.g., each ENA accession was added as an xref within the genus term).
    • Example (OBO):

      [Term]
      id: silva.taxon:58060
      name: Angustibacter
      xref: ena.embl:AB234237 ! uncultured bacterium
      xref: ena.embl:AB512285 ! Angustibacter luteus
      is_a: silva.taxon:58059
      property_value: TAXRANK:1000000 TAXRANK:0000005 ! has rank genus
      
  • Revised Approach:
    • I then decided to represent each ENA accession as its own term, with a parent association determined using the taxmap file (see below).
    • Example (OWL snippet):

      <owl:Class rdf:about="https://www.ebi.ac.uk/ena/browser/view/KP324679">
          <rdfs:subClassOf rdf:resource="https://www.arb-silva.de/fileadmin/silva_databases/current/Exports/taxonomy/47493"/>
          <oboInOwl:id>ena.embl:KP324679</oboInOwl:id>
          <rdfs:label>Fanellia korema</rdfs:label>
      </owl:Class>
  • Rank Annotation for ENA Terms:
    • Originally, I was assigning TAXRANK:0000006 (species) to all ENA entries.
    • However, because these ENA entries may represent either species or strain, I decided not to assign any rank to the ENA-derived terms.

• Version Introduced Information:

  • The main SILVA taxonomy file includes a column with the version in which a given taxon ID was introduced.
  • Example in OBO Format:

    [Term]
    id: silva.taxon:58060
    name: Angustibacter
    property_value: version_introduced "138.2"
    ...
    
  • I am open to discussion on whether to include this field by default as I can see it being useful but confusing.

---

Code Organization

• Module-Level Structure:

  • A global constant PREFIX is defined as "silva.taxon".

• Rank Mapping:

  • The dictionary SILVA_RANK_TO_TAXRANK covers all SILVA taxonomic ranks.

• Main Processing Steps:

  1. Main Taxonomy File:

     • Each row is split on ";" (ignoring empty strings).
     • The term’s name is set to the last element (e.g., "Bacteria" or "Actinomycetota"), and the parent is determined by joining all but the last element.

  2. Taxmap File:

     • A new term is created for each ENA accession with prefix ena.embl:.
     • These terms are linked as children of the corresponding main taxonomy term, but no rank is assigned to them.

I welcome any feedback or suggestions on URL handling, inclusion of the version introduced field, or any other aspect of the implementation.

Please let me know if further details are needed.

[codecov]

Codecov Report

Attention: Patch coverage is 35.48387% with 40 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@d75bbc6). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
src/pyobo/sources/silva.py	34.42%	40 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #348   +/-   ##
=======================================
  Coverage        ?   51.96%           
=======================================
  Files           ?      187           
  Lines           ?    12161           
  Branches        ?     1857           
=======================================
  Hits            ?     6319           
  Misses          ?     5603           
  Partials        ?      239           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[cthoyt]

Don't ENA terms represent nucleotide sequences derived from experiments? Can they also represent projects?

From what I understand, they aren't actually themselves representing taxa. Therefore this parent/child relationship doesn't make sense.

The hard work of making a PyOBO source is really understanding what is the relationship SILVA means when it mentions its internal taxonomy and ENA sequences. I can't do this hard work for you in detail, but from a high level it seems like the sequence was derived from an individual of the taxonomy.

Then, there's two options:

1. Find an existing RO relationship that is appropriate for this. Maybe http://purl.obolibrary.org/obo/RO_0001001, even though it's not a perfect ontological fit. Maybe OBI is a better place to look
2. mint an ad-hoc one yourself within the scope of this file, e.g., like in

   pyobo/src/pyobo/sources/clinicaltrials.py

   Line 23 in ada760b

   HAS_INTERVENTION = TypeDef(

If you go the second route, make sure that you do a good job describing what the relationship means (in a concise way)

[jplfaria]

Thank you for your detailed feedback. I completely understand where the hard work lies, and I truly appreciate the guidance you provided. Your suggestions—either reusing an existing RO relationship (like RO_0001001) or minting an ad-hoc one (as in clinicaltrials.py)—are exactly the direction I was hoping for.

I’ll explore those options further. Alternatively, I might start by representing only down to the genus level (as shown in the taxonomy files) until I fully understand the nuances of the lower levels.

Thanks again for steering this work in the right direction!

[cmungall]

I think it's good to surface decision like this higher up, and ideally all of pyobo would be biolink compliant. biolink:has_biological_sequence is the right KG relationship to use.

If you are going to use RO you need to use it consistently with how it's intended and not just pick a label that sounds right.

[cthoyt]

I added an example ad-hoc typedef that you can fill in (or duplicate) for your purposes

[cthoyt]

fixme here

[jplfaria]

Summary of Changes

• New TypeDef Added:
  Introduced HAS_TAXONOMIC_CLASSIFICATION to capture the relationship between an ENA accession (representing a genome sequence) and the taxonomic classification assigned by SILVA.

• Rationale:
  Instead of using a parent/child relationship—which implies a fixed hierarchical level—this new typedef reflects that SILVA can classify sequences to varying levels (often down to genus, but sometimes only to higher ranks). This approach better represents the flexible nature of the taxonomic assignments provided by SILVA.

• Implementation Details:
  The new typedef is defined as follows:

  HAS_TAXONOMIC_CLASSIFICATION = TypeDef(
      reference=default_reference(PREFIX, "has_taxonomic_classification", name="has taxonomic classification"),
      definition="Indicates that the genome sequence represented by an ENA accession is classified under this taxon by SILVA.",
      is_metadata_tag=True,
  )

[jplfaria]

@cthoyt I apologize, I just noticed I did something wrong, I see the warning:

"Merging is blocked 1 review requesting changes by reviewers with write access.
You're not authorized to push to this branch."

I am trying to figure out how to push to the proper branch.

[cthoyt]

@jplfaria are you unable to push to your own branch? The pull request dialog is to inform you that you can't merge the pull request, since it's my repository. That's not an issue for you to worry about

[jplfaria]

@cthoyt thank you for the clarification. I thought that was the case, but my inexperience with git made me think I was doing something wrong.

[cthoyt]

@jplfaria see FIXME comments in recent commits I made

[cthoyt]

we still need to understand the context in what it means for a SILVA taxon to be mapped to an ENA record for a sequence. I am starting to think that this is just as simple as "the sequence was derived from a sample taken from an individual of this species" but please do a deep dive to clarify further

[jplfaria]

The fixes look good to me.

[sierra-moxon]

Would strongly consider reusing a relation from an existing ontology or standard (or working with an existing standard like RO to get a new term added) rather than minting a new one. I can help get a new term added for you, or you can use (as @cmungall states above) a term from Biolink, in particular biolink:has_biological_sequence (https://biolink.github.io/biolink-model/has_biological_sequence/)

[cthoyt]

thanks @sierra-moxon ! This is just what we need. I wish it were easier to find what I needed in Biolink. I've sent biolink/biolink-model#1555 as a plea for helping me and other people who are potential users

On second thought, I am not sure this is what we need, because the description is so vague

[jplfaria]

@cthoyt, I apologize for the delay in responding. I have been buried in deadlines, and things only look to clear in mid-April. At that point, I can deep-dive into this, as I haven't looked into Biolink at all.

[cthoyt]

@jplfaria please don't spend your time looking into biolink at the moment. There's no documentation for the suggested predicates for our use case, meaning that reusing them is not a good course of action. The best way forward is for us to mint our own predicates, but do a 100% job giving detailed context as to what the predicates are for

[cmungall]

In fact @jplfaria will likely be looking at Biolink anyway since the database he is building will be strongly aligned with it!