Question 1

How do I fix DOCX tables being moved to the end of extracted text when python-docx is installed?

Accepted Answer

This happens when the parser reads `document.paragraphs` first and `document.tables` second, which loses their interleaved order in the Word body. Upgrade `book-to-skill` to the release containing the DOCX order fix; the corrected parser iterates body blocks in document order, typically with `document.iter_inner_content()`, handling `Paragraph` and `Table` objects as they appear. On older `python-docx` versions without ordered block iteration, it falls back to the stdlib ZIP/XML parser. As an immediate workaround, force the fallback by running `pip uninstall python-docx` in your virtual environment, or call `extract_docx_with_zipfile("document.docx")` directly. Verify that a table appearing before Chapter 2 is extracted before the `Chapter 2` heading.

Question 2

How do I fix pytest showing 5 skipped tests for bs4 and trafilatura in CI?

Accepted Answer

The tests are skipped because they use `pytest.importorskip` to require optional packages. Install the test-only dependencies in a dedicated CI job:

```yaml
optional-deps:
  name: test (optional deps installed)
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-python@v5
      with:
        python-version: "3.12"
    - run: pip install pytest beautifulsoup4 trafilatura
    - run: pytest tests/ -q
```

Keep the existing dependency-free job unchanged. To verify locally:

```bash
pip install pytest beautifulsoup4 trafilatura
python3 -m pytest tests/ -q
```

With both packages installed, the result changes from `698 passed, 5 skipped` to `703 passed, 0 skipped`. The affected tests are the `bs4`-gated test in `tests/test_html_block_boundaries.py` and four `trafilatura`-gated tests in `tests/test_html_boilerplate_extraction.py`.

Question 3

Why is chapter_headings_sample missing for each source in metadata.json?

Accepted Answer

The CLI metadata projection in `book_to_skill/utils.py` copied `chapters_detected` and `chapters_method` but omitted the per-source `chapter_headings_sample`; only the consolidated top-level sample was serialized. Upgrade to a build containing PR #245, commit `b27360ccfc2f92e0a6571dac03e32389a6e4cfe4`:

```sh
pip install git+https://github.com/virgiliojr94/book-to-skill.git@b27360ccfc2f92e0a6571dac03e32389a6e4cfe4
```

Then rerun extraction, for example:

```sh
BOOK_SKILL_WORKDIR="$PWD/repro-output" python3 scripts/extract.py alpha.md beta.md --mode text --install-missing no
```

If patching version 1.4.0 locally, add the existing extraction result to each source object in `main()` next to the other chapter fields, using the variable that holds `extract_single_file()` output:

```python
"chapter_headings_sample": source.get("chapter_headings_sample", []),
```

Do not recompute the sample or assign the consolidated top-level sample, because that can attach the first source's headings to later sources. Empty lists and Unicode headings must be preserved. Validate with `pytest -q` and `ruff check .`.

Question 4

How to fix the stdlib EPUB fallback silently skipping chapters when OPF manifest hrefs contain fragments or percent-encoded filenames?

Accepted Answer

Upgrade to a release containing the PR `fix(epub): resolve encoded manifest hrefs`; the bug is present on commit `f08d343`. When patching the fallback, do not pass the raw OPF `href` to `ZipFile.read()`. First resolve it relative to the OPF directory, remove query/fragment components, and decode percent escapes:

```python
import posixpath
from urllib.parse import urlsplit, unquote

def manifest_member_name(opf_path, href):
    href_path = urlsplit(href).path  # drops ?query and #fragment
    href_path = unquote(href_path)  # Chapter%201.xhtml -> Chapter 1.xhtml
    return posixpath.normpath(posixpath.join(posixpath.dirname(opf_path), href_path))
```

Then call `zf.read(manifest_member_name(opf_path, href))`. XML parsers such as ElementTree already decode XML entities in attribute values. Thus `chapter.xhtml#section-2` resolves to `chapter.xhtml`, and `Text/Chapter%201.xhtml` from `OEBPS/content.opf` resolves to `OEBPS/Text/Chapter 1.xhtml`.

Question 5

How do I fix invisible Unicode Default_Ignorable characters such as Mongolian FVS, Khmer inherent vowels, and Shorthand format controls not being stripped or detected?

Accepted Answer

Patch `book_to_skill/sanitize.py::is_invisible_codepoint()` to include the complete missing set of assigned `Default_Ignorable_Code_Point=Yes` code points: `U+17B4`, `U+17B5`, `U+180B`, `U+180C`, `U+180D`, `U+1BCA0`, `U+1BCA1`, `U+1BCA2`, and `U+1BCA3`. For example, add their scalar values to the shared invisible-code-point set: `0x17B4, 0x17B5, 0x180B, 0x180C, 0x180D, 0x1BCA0, 0x1BCA1, 0x1BCA2, 0x1BCA3`. Do not maintain a separate scanner list: `tools/scan_generated_skill.py` imports the same predicate, so updating it makes extraction strip these characters and the scanner report the same set consistently. The issue is verified on Python 3.9.5 with Unicode 13.0.0; all nine code points are already assigned there. After patching, rerun the test suite and regenerate/rescan affected skills.

book-to-skill

Overview

README Preview

FAQ (5)

同类型项目

hermes-agent

firecrawl

go

markitdown