Skip to content

BUG: Count Tc and Tw in the advance used for space detection - #4227

Merged
stefan6419846 merged 4 commits into
py-pdf:mainfrom
Pranjalitem189:fix-char-word-spacing
Nov 20, 2026
Merged

stefan6419846 merged 4 commits into
py-pdf:mainfrom
Pranjalitem189:fix-char-word-spacing

Conversation

@Pranjalitem189

@Pranjalitem189 Pranjalitem189 commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem

extract_text() inserts spaces inside words when a PDF uses word or character spacing.
With the PDF from the issue, WORD NAME is extracted as WO RD NAME.

Cause

A space is inserted when the next text starts more than half a space after the
calculated end of the previous text. That calculation ignored Tc (no handler
existed) and Tw (stored in space_scale, but never used).

In the issue's PDF, a space is shown with 2.664 Tw. Those 2.664 pt were not
counted, so the gap before the following Td looked like 2.68 pt, which is above
the 1.5 pt threshold.

Changes

  • Handle Tc and store the actual Tw value. Both are saved and restored with q/Q.
  • get_text_operands also returns the number of glyphs and of single-byte code 32 characters.
  • Add Tc * glyphs + Tw * spaces to the width used for the space check.
  • As defined in the PDF specification, Tw only applies to the single-byte code 32,
    so composite fonts do not get word spacing.

This fixes the cause instead of raising the threshold (the 1.05 factor suggested
in #4067), so the existing threshold stays unchanged.

Tests

  • New test_character_and_word_spacing_count_towards_the_advance covering Tw, Tc,
    and a real gap that has to keep its space. The first two cases fail without the fix.
  • Updated the cache tests for the new return values of get_text_operands.

Closes #3219

Character and word spacing were ignored, so text shown with them got
a synthetic space, e.g. "WO RD" instead of "WORD".
@codecov

codecov Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.26%. Comparing base (0da9460) to head (b4c46db).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4227   +/-   ##
=======================================
  Coverage   99.25%   99.26%           
=======================================
  Files          59       59           
  Lines       12425    12438   +13     
  Branches     2368     2371    +3     
=======================================
+ Hits        12333    12346   +13     
  Misses         38       38           
  Partials       54       54           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread pypdf/_text_extraction/__init__.py Resolved
Comment thread tests/test_text_extraction.py Resolved
Pranjalitem189 and others added 3 commits November 18, 2026 00:18
Horizontal scaling now applies to the whole advance, as defined in the
PDF specification. Character and word spacing are counted per character
code instead of per decoded character, so two-byte codes forming a
surrogate pair are no longer merged, and word spacing applies to the
single-byte code 32 in multi-byte encodings like Shift-JIS.

Also use meaningful test IDs.
@stefan6419846
stefan6419846 merged commit 68ab616 into py-pdf:main Nov 20, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Erroneous Whitespace in Text Extraction

2 participants