Repository navigation
Conversation
On a page with an embedded image and OCR enabled, the page text was rebuilt
from raw page.chars joined with "", which glued adjacent columns together
("Customer NameVendor Name Ltd"). Use page.extract_text_lines() instead, which
keeps inter-word spacing. Line tops are still the Y positions used to
interleave the image OCR blocks.
Adds a regression test and updates the pdf_complex_layout snapshot, which had
the glued form ("ItemQuantity", "Widget A5") as its expected output.
Author
|
@microsoft-github-policy-service agree |
San-Hsien (SanHsien)
added a commit
to SanHsien/markitdown
that referenced
this pull request
Sep 30, 2026
…issue microsoft#2567 Adopt microsoft#2462, microsoft#2464, microsoft#2466, microsoft#2467, microsoft#2469, microsoft#2476 and microsoft#2477 via cherry-pick; record remaining items as adoption pending or not applicable. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2565
On a PDF page that contains an embedded image and has OCR enabled, the page text was rebuilt from raw
page.charsjoined with"". Chars carry no separator between adjacent columns, so columns were glued together (Customer NameVendor Name Ltd).This switches that block to
page.extract_text_lines(strip=True, return_chars=False), which keeps the inter-word spacing. Each line'stopis still the Y position used to interleave the image OCR blocks, so the interleaving is unchanged.Tests
test_pdf_page_with_image_keeps_spaces_between_columns. It builds a page with two columns and a small image in memory (PyMuPDF is already a dependency), asserts the columns are not glued, and that the image is still OCR'd and interleaved. It fails onmain.test_pdf_complex_layout. Its snapshot encoded the bug: in that PDFItem(x 72–95) andQuantity(x 216–261) sit about 120 pt apart on one line, and were expected asItemQuantity;Widget A5becomesWidget A 5. Only those two tokens change. Headings, ordering and the OCR block position are identical.pytestinpackages/markitdown-ocr: 109 passed (108 before plus the new test), Python 3.11.black --checkis clean on the two touched files.Notes
page.extract_text()split on newlines,y = i * 10) is gone. With no chars,extract_text_lines()returns[], and the old fallback produced one empty text item that the loop already skips (if item["text"]).convert. Fix OCR fallback and image placement #2276 re-indents the same block into atry/except; happy to rebase on whichever lands first.