Q: Why does GLiNER2 entity extraction miss entities after the first 512 words of a long document?

Versions before the fix silently truncated each item's text at `max_seq_length` (512 words by default) for entity, relation and structured extraction on GLiNER2 models (`fastino/gliner2-base-v1`, `fastino/gliner2-large-v1`), so content past that window never reached the model. Fix: upgrade to a build containing PR #552 (commit `689e4d808cbd66008500da8f58d8fa6e71510b8f`). After the fix, overlong items fail with an explicit per-item `INPUT_TOO_LONG` error instead of a silent partial read; rejected items are excluded from model calls and report zero input-token usage, while valid siblings in the same batch keep their results and original positions. Whole-document extraction via overlapping windows is still an unmerged enhancement (#649) — until it lands, split long documents into chunks that fit the word/subword window before calling `POST /v1/extract/{model}`.

Q: How to fix duplicate NER entities and 'structured extraction property must be a string' errors after upgrading to gliner2 2.0.0?

gliner2 2.0.0 changed outputs for `fastino/gliner2-base-v1` and `fastino/gliner2-large-v1` in two ways: (1) NER dedup key changed from `text.lower()` to `(text.lower(), start, end)`, so every mention of a repeated entity is returned instead of only the first (entity counts can jump e.g. 21 → 100 on long texts); (2) structured extraction is rescored and some fields now return `null`, which the SIE `GLiNER2Adapter` rejects with `GLiNER2 structured extraction property 'person' must be a string`, failing the whole request. Workaround: pin gliner2 to the 1.x line (`gliner2>=1.3.1,<2`, i.e. 1.3.2) — this is what the default bundle and root lock do. Note: the opt-in entity LoRA adapter requires exactly gliner2 1.3.2, so it breaks under 2.x. If you must run 2.0.0, deduplicate entities by lowercased surface form yourself to restore 1.x behavior, and handle `null` structured fields (omit the property or emit a per-item error) instead of failing the request. Migration of these models to 2.x is still pending a decision on mention policy, null handling, and re-baselining.

Q: How to fix NaN/Inf output from torch.nn.functional.linear when loading safetensors weights via mmap on macOS arm64 (Apple Silicon)?

Cause: on the affected macOS arm64 stack (e.g. torch 2.9.1, Python 3.12), memory-mapped safetensors tensors can start at unaligned addresses (data_ptr % 64 != 0, because the tensor's file offset isn't 4/64-byte aligned). Feeding such a mapped weight into `torch.nn.functional.linear` produces nonfinite (NaN/Inf) output even though weight, input and bias are all finite. Workaround: force an aligned in-memory copy of the weight before use — `weight = weight.clone()` — or load the checkpoint with `safetensors.torch.load_file("model.safetensors")` instead of mmap views via `safe_open` / lazy tensor frameworks. After cloning, the byte-identical weight gives 0 nonfinite values and matches a float64 NumPy reference (max abs error ~5.2e-6). To check whether you're affected, inspect `tensor.data_ptr() % 64`: a nonzero value on a mapped tensor flags the risk.

Q: Why is GLiFormer inference slow on short texts even though the GPU is mostly idle, and how do I fix it?

On short inputs (40-140 tokens) GLiFormer's 47 ms forward is CPU-bound by kernel launches: it fires ~1,270 small CUDA kernels with ~71 stream synchronizations (31 from `.item()` calls in the task heads), while the GPU is busy only ~8 ms. The fix is CUDA graph replay, implemented in PR #796 following the GLiClass approach from #367: the static part of the forward (DeBERTa backbone with fixed padded shapes) is recorded once and replayed as a CUDA graph, while the task heads with data-dependent control flow stay eager and read the graph's static output buffers. CUDA graphs are opt-in and OFF by default — enable the 'exact shapes' mode, which passes pretrained parity checks with bit-identical outputs to eager execution. Bucketed (padded-length) mode is rejected/unavailable because it failed the fixed numerical tolerance limits. Memory use is capped by a per-model budget, one recording runs at a time, recording is skipped under memory pressure, and graph memory is released on model unload.

Question 1

Why do Docker image builds install newer package versions than uv.lock pins, and how to make pip installs reproducible?

Accepted Answer

This happens when image builds run `pip install -r` with ranged specifiers (e.g., `gliner>=0.2.26,<1`): pip resolves the newest matching release at build time, which shadows the exact version pinned in `uv.lock`, so two builds of the same commit can ship different, untested versions. In sie_server this was fixed by PRs #653 and #802: bundle images no longer resolve ranges during the build. Instead they install a saved dependency closure from 15 checked-in target locks covering all 11 released Linux amd64 images and the 4 supported CPU arm64 combinations. Packages keep their root `uv.lock` pins unless an explicit, documented bundle exception applies; CI rejects stale or inconsistent locks, and image builds verify and retain the installed `/app/bundle-libs` inventory (`pip freeze`) so any deployed version can be audited. To fix this in your own project: (1) export locked versions with `uv export --frozen --no-hashes` and pass them to pip as a constraints file (`pip install -c constraints.txt -r requirements.txt`) so locked packages keep their pinned versions, (2) or maintain per-bundle lock files with exact versions checked into the repo, and (3) record the resolved `pip freeze` of the installed target directory inside the image.

Question 2

Why does GLiNER2 entity extraction miss entities after the first 512 words of a long document?

Accepted Answer

Versions before the fix silently truncated each item's text at `max_seq_length` (512 words by default) for entity, relation and structured extraction on GLiNER2 models (`fastino/gliner2-base-v1`, `fastino/gliner2-large-v1`), so content past that window never reached the model. Fix: upgrade to a build containing PR #552 (commit `689e4d808cbd66008500da8f58d8fa6e71510b8f`). After the fix, overlong items fail with an explicit per-item `INPUT_TOO_LONG` error instead of a silent partial read; rejected items are excluded from model calls and report zero input-token usage, while valid siblings in the same batch keep their results and original positions. Whole-document extraction via overlapping windows is still an unmerged enhancement (#649) — until it lands, split long documents into chunks that fit the word/subword window before calling `POST /v1/extract/{model}`.

Question 3

How to fix duplicate NER entities and 'structured extraction property must be a string' errors after upgrading to gliner2 2.0.0?

Accepted Answer

gliner2 2.0.0 changed outputs for `fastino/gliner2-base-v1` and `fastino/gliner2-large-v1` in two ways: (1) NER dedup key changed from `text.lower()` to `(text.lower(), start, end)`, so every mention of a repeated entity is returned instead of only the first (entity counts can jump e.g. 21 → 100 on long texts); (2) structured extraction is rescored and some fields now return `null`, which the SIE `GLiNER2Adapter` rejects with `GLiNER2 structured extraction property 'person' must be a string`, failing the whole request. Workaround: pin gliner2 to the 1.x line (`gliner2>=1.3.1,<2`, i.e. 1.3.2) — this is what the default bundle and root lock do. Note: the opt-in entity LoRA adapter requires exactly gliner2 1.3.2, so it breaks under 2.x. If you must run 2.0.0, deduplicate entities by lowercased surface form yourself to restore 1.x behavior, and handle `null` structured fields (omit the property or emit a per-item error) instead of failing the request. Migration of these models to 2.x is still pending a decision on mention policy, null handling, and re-baselining.

Question 4

How to fix NaN/Inf output from torch.nn.functional.linear when loading safetensors weights via mmap on macOS arm64 (Apple Silicon)?

Accepted Answer

Cause: on the affected macOS arm64 stack (e.g. torch 2.9.1, Python 3.12), memory-mapped safetensors tensors can start at unaligned addresses (data_ptr % 64 != 0, because the tensor's file offset isn't 4/64-byte aligned). Feeding such a mapped weight into `torch.nn.functional.linear` produces nonfinite (NaN/Inf) output even though weight, input and bias are all finite. Workaround: force an aligned in-memory copy of the weight before use — `weight = weight.clone()` — or load the checkpoint with `safetensors.torch.load_file("model.safetensors")` instead of mmap views via `safe_open` / lazy tensor frameworks. After cloning, the byte-identical weight gives 0 nonfinite values and matches a float64 NumPy reference (max abs error ~5.2e-6). To check whether you're affected, inspect `tensor.data_ptr() % 64`: a nonzero value on a mapped tensor flags the risk.

Question 5

Why is GLiFormer inference slow on short texts even though the GPU is mostly idle, and how do I fix it?

Accepted Answer

On short inputs (40-140 tokens) GLiFormer's 47 ms forward is CPU-bound by kernel launches: it fires ~1,270 small CUDA kernels with ~71 stream synchronizations (31 from `.item()` calls in the task heads), while the GPU is busy only ~8 ms. The fix is CUDA graph replay, implemented in PR #796 following the GLiClass approach from #367: the static part of the forward (DeBERTa backbone with fixed padded shapes) is recorded once and replayed as a CUDA graph, while the task heads with data-dependent control flow stay eager and read the graph's static output buffers. CUDA graphs are opt-in and OFF by default — enable the 'exact shapes' mode, which passes pretrained parity checks with bit-identical outputs to eager execution. Bucketed (padded-length) mode is rejected/unavailable because it failed the fixed numerical tolerance limits. Memory use is capped by a per-model budget, one recording runs at a time, recording is skipped under memory pressure, and graph memory is released on model unload.

sie

Core Features

What It Can't Do

Use Cases

Tags

Getting Started

Similar Projects