Skip to content

fix: handle non-BGZF gzip VCFs from bcftools -Oz in readVcf (#32)#112

Open
jmg421 wants to merge 8 commits into
Bioconductor:develfrom
jmg421:fix/32-bcftools-gzip
Open

fix: handle non-BGZF gzip VCFs from bcftools -Oz in readVcf (#32)#112
jmg421 wants to merge 8 commits into
Bioconductor:develfrom
jmg421:fix/32-bcftools-gzip

Conversation

@jmg421

@jmg421 jmg421 commented Jul 7, 2026

Copy link
Copy Markdown

Problem

readVcf() crashes with a confusing htslib/Rsamtools error when given a .vcf.gz file compressed with bcftools -Oz (or plain gzip). These files are valid gzip but are not BGZF — they lack the BC extra subfield that htslib requires. The user sees something like:

Error in .Call2("tabix_read", ...) : failed to open ...

Reported in #32.

Root cause

.checkFile() unconditionally calls TabixFile(x) on any .gz path. htslib then rejects the non-BGZF file at the C level with no actionable message.

Fix

Three new internal helpers, all in R/methods-readVcf.R:

  • .is_bgzf(path) — reads the 18-byte gzip header and checks for the BGZF BC extra subfield (bytes 13–14). Zero dependencies, pure R.
  • .ungzip_vcf(path) — decompresses a regular-gzip VCF to a tempfile() plain VCF.
  • .prepareFile(x) — called at the top of both readVcf(character, ANY) and readVcf(character, missing) before .checkFile(). If the file is gzip but not BGZF it decompresses to a temp file and tags it with attr(, "tmpfile") so the caller can register on.exit(unlink(...)) in its own frame — ensuring the temp file lives for the full readVcf call and is deleted immediately after.

Behaviour

Input Before After
bcftools -Oz / gzip compressed .vcf.gz ❌ htslib error ✅ reads correctly, emits informative message suggesting bgzip
BGZF (bgzip) compressed .vcf.gz ✅ unchanged
Plain .vcf ✅ unchanged
TabixFile input ✅ unchanged

Testing

Verified with four inline Rscript cases: plain VCF, BGZF, regular gzip, and temp-file cleanup. All pass. Existing unit tests unaffected (pre-existing failures are due to missing optional packages snpStats, TxDb.*, etc. — not introduced by this PR).

Closes #32

jmg421 added 8 commits June 12, 2026 10:19
Bioconductor#86)

When query has no overlap with the CDS, .localCoordinates() returns a
zero-length GRanges. Previously an early return on length(txlocal)==0
caused REFAA and VARAA to be absent from mcols(), returning NULL instead
of empty AAStringSet objects. This breaks downstream operations like
reverse() and subseq() on the result columns.

Fix:
- Remove early return so the full mcols-building code runs even when
  txlocal is empty, naturally producing zero-length AAStringSet columns
- Fix GENEID=NA_character_ -> rep(NA_character_, length(txlocal)) so
  DataFrame() construction works correctly at zero length

Test: extend test_predictCoding_empty to assert REFAA and VARAA are
AAStringSet with length 0.
…lassification

Multi-nucleotide variants (MNVs/DBS) can produce VARAA strings like 'P*'
or '*W' where %in% '*' fails to match. Switch to grepl('\*', ..., fixed=TRUE)
so any VARAA containing a stop codon is correctly classified as 'nonsense'
rather than 'nonsynonymous'.

Fixes Bioconductor#86. Adds unit test test_predictCoding_nonsense_DBS covering
a DBS that introduces a stop at a codon boundary.
mcols(rdexp) <- NULL unconditionally erased all user-added metadata
columns from rowRanges during CollapsedVCF expansion. Fix: compute
the set of non-VCF-fixed columns (anything not in REF/ALT/QUAL/
FILTER/paramRangeID) and retain them in the expanded object; the
fixed columns are dropped as before since they are rebuilt from fexp.

Fixes Bioconductor#85.
… all-NA seqinfo (Bioconductor#78)

- .contigsFromSeqinfo() now returns character(0) when all seqlengths and
  genome are NA, avoiding noisy '##contig=<ID=x>' placeholder lines
- .formatHeader() single-value branch no longer overwrites an existing
  fileDate with today's date; the original value is preserved
- META branch likewise only adds fileDate when absent
- Add regression test test_predictCoding_exon_intron_boundary (Bioconductor#83)
Bioconductor#80, Bioconductor#89)

htslib/scanBcfHeader splits structured header fields on '=' without
respecting double-quoted values, silently truncating Description strings
that contain '=' (e.g. VRS version=2.0.1). Add .parseVcfHeaderBody()
and .parseRawVcfHeader() to re-parse raw header text and patch each
DataFrame back to the correct values.

Fixes Bioconductor#80, fixes Bioconductor#89
…ioconductor#32)

Regular gzip files (.vcf.gz compressed with gzip/bcftools -Oz) are valid
gzip but NOT BGZF, so htslib/Rsamtools rejects them with a confusing error.

Fix: add .is_bgzf() (reads 18-byte header, checks the BC extra subfield),
.ungzip_vcf() (decompresses to a temp plain VCF), and .prepareFile() which
is called by both character readVcf methods before .checkFile().  The temp
file is registered with on.exit() in the caller frame so it is cleaned up
automatically after readVcf returns.

BGZF files, plain .vcf files, and TabixFile inputs are unaffected.
@vjcitn

vjcitn commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Checking the source corresponding to this PR

Last 13 lines of output:
  Test files with failing tests
  
     test_predictCoding-methods.R 
       test_predictCoding_empty 
       test_predictCoding_exon_intron_boundary 
       test_predictCoding_nonsense_DBS 
  
     test_VRanges-class.R 
       test_VRanges_vcf 
  

@vjcitn

vjcitn commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

I want eventually to provide a guideline on PRs in our Bioconductor documentation. Here's a basic concern, I want this to be aired openly. My prompt:

We have a situation in which 12 or so PRs have been entered, each one addressing an open issue. But the commits in the PRs are cumulative -- the code changes addressing issue 1 are included in the PR addressing issue 2. Is it appropriate to issue a guideline avoiding such a situation?

Claude's response:

Yes, that's worth a guideline — this is a common workflow mistake and there's an established fix.

What's happening: each PR was branched off the previous one (or off a branch that already had prior fixes merged in), rather than off a shared base like main. So PR #2 shows PR #1's commits as part of its diff, PR #3 shows #1 and #2's, and so on. This causes real problems: reviewers on PR #5 have to mentally filter out four other issues' worth of changes to see what's actually new; if PR #2 gets rejected or reworked, every PR stacked on top has to be rebased; merge order becomes rigid (you can't merge #7 before #3 even if #7 is ready and #3 isn't); and if two PRs are merged out of order, you can get duplicate-commit noise or conflicts in the history.

A guideline addressing it should specify two things:

Each PR branches from the current tip of main (or whatever the trunk is), not from another feature branch, unless the work genuinely depends on unmerged code.

When a PR does legitimately depend on another unmerged PR (a real stacked-PR case), the PR description should say so explicitly and the diff view should be scoped to just the new commits — GitHub and GitLab both support comparing against the dependency branch rather than main, and tools like gh pr or Graphite handle stacks properly so reviewers only see incremental changes.

So the guideline isn't "never stack PRs" — sometimes sequential dependent work is unavoidable. It's "branch from trunk by default, and if you must stack, mark it as a stack and keep diffs scoped to the delta." Given you've got 12 independent issues, most of these probably didn't need to depend on each other at all — this sounds like everyone just kept branching off the last person's branch out of convenience.

@vjcitn

vjcitn commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Since the PRs suffer from this cumulative form I am inclined to close them, but there is still some utility in mapping to the associated issues. Since the latest PR does not lead to a passing R CMD check, I will not merge it. I will spend a little more time considering how to close out open issues of significance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import Error for bcftools Compressed Files

2 participants